Door-to-door canvassing is effective in mobilizing turnout and support for a ballot proposal. Methodological concerns related to precinct-level randomization are discussed.

Arceneaux, Kevin. 2005. "Using Cluster Randomized Field Experiments to Study Voting Behavior." The Science of Voter Mobilization. Special Editors Donald P. Green and Alan S. Gerber. The Annals of the American Academy of Political and Social Science, vol. 601: 169 - 179.

As part of a broader discussion on the use of cluster random assignment in randomized field experiments, Kevin Arceneaux presents the results of a voter mobilization experiment conducted in Kansas City, Missouri. In the article, Arceneaux discusses the practice of randomly assigning aggregate groups such as voter precincts to control and treatment groups as opposed to individual level randomization and analysis. He argues that such a precinct-level approach is advantageous, particularly in the case of door-to-door canvassing experiments, because it minimizes contact between members of control and treatment groups and can potentially increase the geographic area that canvassers are able to cover. To illustrate, Arceneaux presents the design, method and results of a cluster randomized experiment carried out in a GOTV initiative encouraging voters in African American precincts of Kansas City to support a local ballot measure to fund provision of bus service. The campaign was organized by a community group, The Association for Community Organizations for Reform Now (ACORN), and results suggest that door-to-door canvassing was effective in mobilizing turnout and support for the ballot initiative.

Electoral Context: A budget crisis forced the Kansas City Transit Authority to place a proposal on the November 2003 special municipal election ballot requesting a 3/8 percent increase in the city's sales tax to increase revenue. If the ballot measure were not approved, bus service would have to be curtailed. The ACORN, a community group that advocates on behalf of low-income families, organized a voter mobilization campaign in support of the tax that featured door-to-door canvassing. There were three other city questions on the ballot along with a contest for a state senator position.

Subject Population: ACORN selected twenty-eight primarily African American precincts to be included in the study. Many individuals in the precincts studied (both treatment and control) relied heavily on buses for transportation.

Randomization Procedure: Random assignment was performed at the precinct level. Half (fourteen) of the precincts selected by ACORN were randomly assigned to be canvassed, while the other half were assigned to the control group.

Treatment: Relying on a pool of seventy-five predominately African American canvassers between eighteen and twenty-five years old, ACORN carried out a month-long intensive campaign to educate voters about the proposed sales tax proposal and encourage them to vote. Canvassers walked the treatment precincts twice. The first walk through the precincts focused on education. Door-knock scripts were aimed at persuading voters to vote for the proposed sales tax increase rather than face serious cuts in bus service. 84.2 percent of those contacted in the first walk expressed support for the proposal. The second walk was conducted a week out from the election and focused on voter mobilization. Contacts from the first walk who expressed support for the tax increase were reminded about the sales tax proposal and encouraged to vote.

Findings: Arceneaux presents the findings of individual level turnout effects of the GOTV campaign as well as the precinct-level differentials in the number of votes to support the transportation measure. OLS regression estimates an Intent-to-Treat effect that suggests assignment to the treatment group resulted in a 4.4 percentage point increase in the likelihood of turnout (robust standard error: 2.5 percentage points). When covariates for past turnout are incorporated, OLS estimates a 5.3 percentage point increase in the likelihood of turnout among subjects in the treatment group (standard error: 1.7 percentage points). Two stage least squares regression incorporates the contact rate and estimates a 7.0 percentage point treatment effect (robust standard error: 3.9 percentage points) and an 8.5 percentage point effect when covariates for past turnout are included (standard error: 2.7 percentage points).

Arceneaux also conducts analysis at the precinct level which effectively reduces the number of observations from the number of individuals contacted to the number of precincts (twenty-eight). Because electoral outcomes are reported by precinct, Arceneaux can also assess whether ACORN successfully convinced citizens to vote for the sales tax proposal by calculating the marginal difference between yes and no precinct-level votes for the treatment and control groups and comparing them. Combining estimates from the turnout Intent-to-Treat effect and the marginal vote difference ITT effect allows researchers to estimate the extent to which a campaign mobilizes supporters. The turnout Intent-to-Treat effect with covariates is 5.3 percentage points (standard error: 1.7 percentage points), and the marginal vote difference Intent-to-Treat effect is 0.9 percent (standard error: 0.9 percent) suggesting that 78 percent of the new voters mobilized by ACORN supported the ballot proposition. In concrete terms, this analysis shows that ACORN mobilized 261 people to vote (4,933 targeted individuals times 5.3 percent ITT turnout effect). Consequently, the campaign netted 146 yes votes.