Partisan phone calls and mail, administered in different combinations. Neither the phone calls nor the mail used independently or together resulted in significant treatment effects.

Cardy, Emily Arthur. 2005. "An Experimental Field Study of the GOTV and Persuasion Effects of Partisan Direct Mail and Phone Calls." The Science of Voter Mobilization. Special Editors Donald P. Green and Alan S. Gerber. The Annals of the American Academy of Political and Social Science. vol. 601: 28-40.

Emily Arthur Cardy presents results from a field experiment examining mobilization and persuasion effects of partisan direct mail and phone calls. The experiment was conducted during a state's 2002 gubernatorial election. The study finds that neither partisan direct mail nor partisan phone calls used independently or together, garner significant GOTV or persuasion effects.

Electoral Context: The election in this study was a 2002 state gubernatorial primary, which was an unexpectedly close race between two candidates. The election was the most expensive primary in the state's history, with a total of nearly $30 million spent on the campaigns. The issue of abortion was prominent in both campaigns, particularly in the weeks preceding the election. The campaign was sponsored by an organization considered to be a leader in pro-choice politics that sought to gain public support for the only pro-choice candidate in the race.

Subject Population: Only voters registered in the parties of the two candidates examined in the experiment were targeted. Approximately 800,000 subjects were identified as pro-choice voters by a national foundation through live calls from which the direct mail firm obtained the relevant data. A communications firm placed a second round of calls to assess degrees of support for pro-choice abortion policies. Only voters identified as "strongly pro-choice" were included in the final sample lists. Another list of potential pro-choice voters was assembled by estimating probabilities of individual support for pro-choice policies through a statistical probability model applied to demographic information obtained from calls and voting records. Voters with a minimum of 70 percent estimated likelihood of voting pro-choice were included in the sample. Finally, one hundred names from each of the top two levels of support identified through phone-calls and direct mail data as well as one hundred names from the modeled sample were assigned to each of four treatment groups and a control group. These 1,500 subjects were further interviewed and folded into the larger sample. There were between 1,940 and 2,022 voters in each treatment group and the control group, totaling 9,980 registered Democrats. The meticulous targeting process ensured that individuals in the sample had been contacted previously and a degree of certainty that the question of abortion rights was important to each individual in the sample.

Randomization Procedure: Individual subjects were randomly assigned to one of four treatment groups or the control group.

Treatment: Treatment groups included "phone only," "mail only," "phone and mail," and "intensive phone and mail" The direct mail and phone calls were managed by a communications firm specializing in direct mail and fundraising and a major phone consulting firm specializing in live phone communications. Voters in the "phone only" treatment received two persuasive calls from the pro-choice leader. Both phone scripts explained how the candidates differed on abortion issues. The first call focused on persuasion and the second on mobilization. The phone calls were made on a schedule of fifteen days and seven days before the election. Contact rates were estimated to be approximately 50 percent. The "mail only" group received two pieces of persuasion mail from the pro-choice leader, again explaining the differences between the candidates on abortion issues and encouraging voters to vote. The first two mailers were explicitly persuasive with an implicit GOTV message. Other pieces of mail had explicit persuasive GOTV messages. The "mail and phone" group received both pieces of mail and both phone calls. The voters who received the "intensive phone and mail" treatment received two phone calls and five pieces of mail. After the election, three hundred people in each treatment group and the control group were interviewed in a survey to assess how effectively the campaigns had persuaded voters.

Findings: With regard to mobilizing turnout, only the phone and mail treatment group and intensive treatment group produced small positive, statistically insignificant intent-to-treat effects of 1.2 percentage points. Even after controlling for voter history, none of the treatments produced statistically significant effects. With regard to the persuasion effects of the treatments, results of the post-election survey indicate that the phone and intensive treatment groups garnered lower percentages of support for the pro-choice candidate than did the control group. The mail and mail and phone treatment groups garnered higher percentages of support for the pro-choice candidate than did the control, indicating that these treatments had a persuasive effect on nonvoters. Voters in all of the treatment groups exhibited higher percentages of support for the pro-choice candidate than did the voters in the control group except for the intensive treatment group, which was equal to the control. With a 6.3 percentage point increase in support for the candidate, the phone group had the largest effect, followed by mail with a 5 percentage point increase, and then phone and mail, which is estimated to have produced a 1.7 percentage point increase. A chi-square test of the differences of support between subjects in the control and treatment groups indicates that the results are not statistically significant. The survey produces stronger results than does analysis of the voter records suggesting measurement error in the self-reported persuasive and mobilizing effects of the campaign.