Nonpartisan commercial phone bank messages may be more effective than partisan messages. Results may differ across parties and electoral contexts.

Panagopoulos, Costas. 2006. "Partisan and Nonpartisan Message Content and Voter Mobilization." Unpublished Manuscript. Institution for Social and Policy Studies, Yale University.

Costas Panagopoulos reports the results of a randomized field experiment conducted during the November 2005 municipal elections in Albany, New York, and designed to compare the mobilization impact of partisan versus nonpartisan messages delivered via commercial phone banks. There has been considerable contestation about the external validity of previous studies along these lines, and critics have argued that the failure to detect mobilization effects from calls delivered by commercial phone banks is a consequence of nonpartisan message content. In response, this experiment incorporated a series of considerations designed to boost both internal and external validity in order to test this supposition directly. The results indicate that nonpartisan messages may be more effective at mobilization than partisan messages. The results also suggest this effect may differ across parties and electoral contexts. Findings are not statistically significant. The author concludes that neither partisan nor nonpartisan phone bank initiatives are effective.

Electoral Context: The 2005 general election in Albany, NY featured contests for mayor and members of the city council as well as county clerk, president of the council, treasurer, city court judge and for members of the board of education. There were also two statewide proposals on the ballot. The 2005 election was a relatively low salience, off-year municipal election.

Subject Population: The subject population consisted of a random sample of 24,308 voters registered as either Republicans or Democrats.

Randomization Procedure: The list of randomly selected registered voters was stratified and matched by previous voting history and individual voters were randomly assigned to one of two treatment groups or the control group.

Treatment: A total 2,486 voters (672 Republicans and 1,814 Democrats) were assigned to a treatment group that received a partisan appeal encouraging them to vote on Election Day. The phone scripts were developed in collaboration with leading communications professionals from both major parties. Registered Republicans received a Republican message, and registered Democrats received a Democratic message. A total of 2,472 voters (651 Democrats and 1,821 Democrats) were randomly assigned to a treatment group that received a nonpartisan GOTV appeal. The remaining 19,350 voters in the sample (5,349 Republicans and 14,001 Democrats) were assigned to the control group and received no message. Phone calls were conducted by a professional commercial phone bank establishment on November 6 and 7, 2005.

59.1% of subjects assigned to the partisan treatment group and 60.4% of subjects assigned to receive the nonpartisan message were successfully contacted.

Findings: Estimates of Panagopoulos' model imply that turnout rate in the sample that was exposed to nonpartisan phone messages was higher (55.3%) than both the control group (54.8%) and the group that received partisan messages (54.4%). Neither partisan nor nonpartisan messages delivered via high-quality commercial phone calls exerted an effect that was statistically distinguishable from zero. The estimated intent-to-treat effect of nonpartisan calls was 0.5 percentage points (standard error: 1.1 percentage points). The estimated intent-to-treat effect of partisan calls on turnout was -0.4 percentage points (standard error: 1.1 percentage points.) Considering contact rates, treatment-on-treated effects are similarly small and statistically insignificant. The estimated treatment on treated effect of partisan calls was -0.6 percentage points (standard error: 1.8 percentage points) and for nonpartisan calls, the treatment-on-treated effect was estimated to be 0.8 percentage points (standard error: 1.8 percentage points). Results by party imply inconclusively that partisan messages may boost participation amongst Republicans more so than nonpartisan messages; turnout among Republicans assigned to the partisan treatment group was higher (54.9%) than both the nonpartisan treatment group (53.5%) and the control group (53.6%). Conversely, nonpartisan appeals appear to stimulate turnout among Democrats more so than partisan messages; Democratic turnout among voters assigned to the nonpartisan treatment group was 0.8 percentage points higher than turnout in the control group (56.0% to 55.2% respectively), while only 54.2% of voters assigned to the partisan treatment group voted. These findings suggest the effects of partisan and nonpartisan messages may differ across parties and electoral contexts. The conclusions remain speculative because the effects and interactions fail to achieve significance at conventional levels.