---
title: 'PS 813: Problem Set 2'
author: "YOUR NAME"
date: "February 4, 2026"
output:
  html_document:
    df_print: paged
code:
  execute: 
    - r
    - python
embed-resources: true
---
  
```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
# Useful packages
library(estimatr)
library(tidyverse)
library(haven)
```


> Please upload your solutions as a .html file saved as "Yourlastname_Yourfirstinitial_pset2.html"). In addition, an electronic copy of your .qmd file (saved as "Yourlastname_Yourfirstinitial_pset2.qmd") must be submitted to the course website at the same time. We should be able to run your code without error messages. In order to receive credit, homework submissions must be substantially started and all work must be shown.

# Problem 1

Can information campaigns persuade voters in backsliding democracies to oppose further consolidation of power? Baysan (2022) analyzes a field experiment conducted in the run-up to the 2017 Turkish Constitutional Referendum. The referendum was brought by the governing AKP party and would abolish the country's parliamentary system in favor of a presidential system, dramatically weakening checks on the executive. During the campaign prior to the referendum, the largest opposition party conducted a field experiment involving a door-to-door information campaign in Izmir, the third largest province in Turkey by population.

Canvassers were randomly assigned to neighborhoods. Each neighborhood could be assigned to one of two treatment conditions: a treatment condition that provided voters with information on how the proposed reforms would reduce checks and balances (authoritarian script) and a treatment condition that provided information about the executive's poor performance with respect to past outcomes on the economy and national security (economy script). Neighborhoods that were not selected to receive either of the two informational campaigns are the control group. 

Observations were collected at the level of the ballot box (3,992 total) with multiple ballot boxes located in a given neighborhood. Treatment was assigned via blocked cluster randomization. Specifically, treatment was assigned completely at random to neighborhoods within four strata based on the average neighborhood vote share for the opposition party in 2015. In each stratum, 25 neighborhoods received one of the two treatments - 12 received the "economy" script and 13 received the "authoritarian" script. The remainder received control (113 neighborhoods in strata 1,2,3 and 112 in stratum 4).

The citation for the paper is 

> Baysan, Ceren. "Persistent polarizing effects of persuasion: Experimental evidence from turkey." American Economic Review 112, no. 11 (2022): 3528-3546.

The code below will read in and subset the data 

```{r, message = F, warning = F}
referendum <- read_dta("data/for_analysis_allelections_ballot.dta") %>% 
  filter(!is.na(no_pct_ballot))
```

Below are some of the columns that you may need to complete the problem:

- `nn` - Neighborhood identifier
- `treat` - Whether a neighborhood received either one of the two canvassing treatments
- `script_name` - Door-to-door canvassing script received by the neighborhood ("authoritarian" or "economy" if treated, blank if control)
- `chp_pct_n_avg_quart` - Randomization strata - Quartile of 2015 average neighborhood vote share for the opposition party (1, 2, 3, 4)
- `no_ballot` - Number of voters at the ballot box that voted No in the referendum
- `valid_ballot` - Number of valid votes cast at the ballot box for the referendum.
- `no_pct_ballot` - Percent of voters at the ballot box that voted No in the referendum.
- `num_registered_voters_genn` - Number of registered voters, November 2015

## Part A:

Start by aggregating the data to the neighborhood level. Generate a data frame in which each row is a neighborhood. The columns should include at a minimum: an indicator for whether a neighborhood is treated, the treatment script received by a neighborhood, the quartile of 2015 opposition vote share, and the share of valid votes in the neighborhood that voted "No" in the referendum.

Use a simple difference-in-means estimator to estimate the neighborhood-level average treatment effect of receiving the "economy" canvassing treatment compared to receiving neither of the treatments on a neighborhood's "No" vote share in the referendum. Provide a point estimate and 95% confidence interval.

## Part B:

Now use a stratified difference-in-means estimator to estimate the the neighborhood-level average treatment effect of receiving the "economy" canvassing treatment compared to receiving neither treatment on a neighborhood's "No" vote share in the referendum, stratifying on the 2015 opposition vote share quartile. Provide a point estimate and 95% confidence interval.

Compare your results and discuss the reasons for any differences you observe. Are both estimators unbiased for the ATE? Which estimator is preferable?

## Part C:

Estimate the conditional average treatment effect of the "economy" canvassing treatment (compared to no canvassing) among neighborhoods in the first quartile of opposition vote share (the least pro-opposition neighborhoods). Report a point estimate and 95% confidence interval. Then estimate the difference in conditional average treatment effects between the first and second quartiles of opposition vote share. Report a point estimate and 95% confidence interval for the difference. Interpret your results. Do you conclude that there is evidence for effect modification driven by a neighborhood's prior level of support for the opposition?

## Part D:

Now return to the ballot box level dataset. Construct a randomization test/permutation test for the sharp null of no treatment effect of the "economy" canvassing treatment compared to no canvassing on ballot-box level "No" vote share. Use the absolute difference-in-means as your test statistic. Your permutation test should replicate the randomization procedure that was used to conduct the experiment. Set the seed to `53706` and use `5000` monte carlo iterations. 

Report the p-value. Under an $\alpha = .05$ threshold, would we reject the sharp null?

# Problem 2

Do international election monitors reduce the incidence of electoral fraud? [Hyde (2007)](https://www.aeaweb.org/articles?id=10.1257/000282803321946921) studies the 2003 presidential election in Armenia, an election that took place during a period where the incumbent ruling party headed by President Robert Kocharian had consolidated power and often behaved in ways that were considered undemocratic.

The full citation for this paper is

> Hyde, Susan D. "The observer effect in international politics: Evidence from a natural experiment." *World Politics* 60.1 (2007): 37-63.

At the time of the election, OSCE/ODIHR election monitors reported widespread electoral irregularities that favored the incumbent party such as ballot-box stuffing (pp. 47). However, we do not necessarily know whether these irregularities would have been worse in the absence of monitors. Notably, not all polling stations were monitored -- the OSCE/ODIHR mission could only send observers to some of the polling stations in the country. Since in the context of this election only the incumbent party would have the capacity to carry out significant election fraud, Hyde examines whether the presence of election observers from the OSCE/ODIHR mission at polling stations in Armenia reduced the incumbent party's vote share at that polling station.

For the purposes of this problem, you will be using the `armenia2003.dta` dataset

The R code below will read in this data (which is stored in the STATA .dta format)
```{r, echo=T, message=F}
### Hyde (2007) Armenia dataset
armenia <- read_dta("data/armenia2003.dta")
```

This dataset consists of 1764 observations polling-station-level election results from the 2003 Armenian election made available by the Armenian Central Election Commission. The election took place over two rounds with an initial round having a large number of candidates and a second, run-off election, between Kocharian and the second-place vote-getter, Karen Demirchyan. We will focus on monitoring and voting in the first round.  The specific columns you will need are:

- `kocharian` - Round 1 vote share for the incumbent (Kocharian)
- `mon_voting` - Whether the polling station was monitored in round 1 of the election
- `turnout` - Proportion of registered voters who voted in Round 1
- `totalvoters` - Total number of registered voters recorded for the polling station
- `total` - Total number of votes cast in Round 1
- `urban` - Indicator for whether the polling place was in an urban area (0 = rural, 1 = urban)
- `nearNagorno` - Indicator for whether the polling place is near the Nagorno-Karabakh region (0 = no, 1 = yes)

## Part A

Hyde describes the study as a "natural experiment," stating: 

> "I learned from conversations with staff and participants in the OSCE observation mission to Armenia that the method used to assign observers to polling stations was functionally equivalent to random assignment. This permits the use of natural experimental design. Although the OSCE/ODIHR mission did not assign observers using a random numbers table or its equivalent, the method would have been highly unlikely to produce a list of assigned polling stations that were systematically different from the polling stations that observers were not assigned to visit. Each team's assigned list was selected arbitrarily from a complete list of polling stations." (p. 48)

For the purposes of this part, assume election monitors were assigned as the author describes - in a manner "functionally equivalent to random assignment." Estimate the average treatment effect of election monitoring on incumbent vote share in round 1 using an appropriate estimator. Provide a 95\% asymptotic confidence interval using the Neyman variance estimator and interpret your results. Can we reject the null of no average treatment effect at the $\alpha = 0.05$ level? 

## Part B

Using a balance test on three pre-treatment covariates: the total number of registered voters, whether the polling place was located in an urban area and whether the polling place was near the Nagorno-Karabakh region, assess whether you believe the author's argument that election monitors were assigned in a manner "functionally equivalent to random assignment."

## Part C

Divide the sample into five strata based on the total number of registered voters at each polling station (`totalvoters`): 

|Stratum|Total Registered Voters|
|-------|-----------------------|
|Tiny| `totalvoters` < 430|
|Small| 430 $\le$ `totalvoters` < 1192|
|Medium| 1192 $\le$ `totalvoters` < 1628|
|Large| 1628 $\le$ `totalvoters` < 1879|
|Huge | 1879 $\le$ `totalvoters` |

Estimate the average treatment effect of election monitoring in round 1 on incumbent vote share using a stratified difference-in-means estimator, stratifying on the total number of registered voters. Provide a 95\% asymptotic confidence interval and interpret your results. Can we reject the null of no average treatment effect at the $\alpha = 0.05$ level? Compare your answer to the unadjusted estimate from Part A and discuss why they may differ.

## Part D

In Table 4 of the paper, Hyde uses an estimator for the average treatment effect of a polling place receiving election monitors in round 1 on the incumbent's vote share in round 1 *conditional* on the total number of votes cast in the election (`total`). Will this approach be unbiased for the average treatment effect of election monitors on the incumbent's vote share if we believe that one of the mechanisms through which election monitoring operates is by reducing the incidence of ballot-stuffing (which inflates the number of "cast" votes in the election)?  Why or why not?

# Problem 3

A popular experimental design in political science is the **conjoint** experiment. This is a factorial choice experiment where respondents are exposed to a set of randomly generated profiles (e.g. of candidates for office) and asked to rate them or select which they favor more. In this experimental design, **multiple** treatments are randomized and assigned to each response task. For example, a profile in a candidate choice experiment where respondents are asked to rate a hypothetical political candidate may have four attributes (treatments) such as gender, age, political positions, and career background each with multiple levels.

The quantity of interest is the average treatment effect of assigning one level of one attribute compared to assigning another level of that attribute (with the averaging taking place across respondents and across all the other attributes in the design, see [Hainmueller, Hopkins and Yamamoto (2014)](https://www.cambridge.org/core/journals/political-analysis/article/causal-inference-in-conjoint-analysis-understanding-multidimensional-choices-via-stated-preference-experiments/414DA03BAA2ACE060FFE005F53EFF8C8) for more on how to interpret this quantity which they call the "average marginal component effect" or AMCE - here we'll just refer to this as a kind of ATE).

We will analyze the data from the immigration conjoint experiment in [Hainmueller, Hopkins and Yamamoto (2014)](https://www.cambridge.org/core/journals/political-analysis/article/causal-inference-in-conjoint-analysis-understanding-multidimensional-choices-via-stated-preference-experiments/414DA03BAA2ACE060FFE005F53EFF8C8) which was designed to assess Americans' attitudes towards different forms of immigration. In this design, respondents in the United States were exposed to profiles of hypothetical immigrants and asked whether they would be willing to admit them into the country.

The code below will read this dataset into R

```{r, warning=F, message=F}
immigrationconjoint <- read_csv("data/immigrationconjoint.csv")
```

Profiles varied on nine different attributes:

- `Education`
- `Gender`
- `Country of Origin`
- `Reason for Application`
- `Job`
- `Job Experience`
- `Job Plans`
- `Prior Entry`
- `Language Skills`

The variable `Chosen_Immigrant` denotes whether the respondent preferred that immigrant profile for admission.

Hint: You will find the `table()` function helpful for this problem.

# Part A

Some attributes were assigned completely at random. For example, `Job Plans`. Using a difference-in-means estimator, provide a point estimate for the average treatment effect of being assigned a profile having `no plans to look for work` relative to a profile that has a `contract with employer` on the probability that profile is preferred. Is this estimator unbiased for the ATE? Explain why or why not.

# Part B

Examine the joint distribution of `Country of Origin` and `Reason for Application`? Are these two treatments independent of one another? Explain why or why not. What does this tell you about the treatment assignment process?

# Part C 

Consider now the two treatments `Education` and `Job` and inspect their joint distribution. Using a simple difference-in-means estimator, provide a point estimate for the average treatment effect of being assigned a profile having a `college degree` relative to a profile that has a `high school` degree. Is this estimator unbiased for the ATE? Explain why or why not.

# Part D 

Construct an appropriate estimator for the ATE of a profile having a `college degree` relative to a profile having a `high school` degree under the assumption that `Education` is conditionally randomized given `Job`. Provide a point estimate and explain the rationale for how you constructed this estimator.
