Syllabus

Course Overview

Questions of cause and effect are central to the study of political science and to the social sciences more broadly. But making inferences about causation from empirical data is a significant challenge. Critically, there is no simple, assumption-free process for learning about a causal relationship from the data alone. Causal inference requires researchers to make assumptions about the underlying data generating process in order to identify and estimate causal effects. The goal of this course is to provide students with a structured statistical framework for articulating the assumptions behind causal research designs and for estimating effects using quantitative data.

The course begins by introducing the counterfactual framework of causal inference as a way of defining causal quantities of interest such as the “average treatment effect.” It then proceeds to illustrate a variety of different designs for identifying and estimating these quantities. We will start with the most basic experimental designs and progress to more complex experimental and observational methods. For each approach, we will discuss the necessary assumptions that a researcher needs to make about the process that generated the data, how to assess whether these assumptions are reasonable, how to interpret the quantity being estimated and ultimately how to conduct the analysis.

This course will involve a combination of lectures and problem sets. Problem sets will contain a mixture of both theoretical and applied questions and serve to reinforce key concepts and allow students to assess their progress and understanding throughout the course. Assignments will involve analysis of data using the programming language. This is a free and open source language for statistical computing that is used extensively for data analysis in many fields. Prior experience with the fundamentals of programming is required.

Prerequisites

This course is the second in the political science graduate methodology sequence. Completing the introductory course prior to this sequence should prepare you for the material in this class. We will rely on some background knowledge of core concepts in probability, statistics and inference as well as experience with statistical programming in . However, there are no strict, specific course pre-requisites as many different disciplines and departments offer introductory statistics classes that cover the relevant material.

In general, you should have had some introduction to probability theory and should be familiar with concepts like the properties of random variables (especially expectation and variance), estimands and estimators, and statistical inference. Familiarity with linear regression is also a plus, but we will be reviewing it during the relevant week.

Please contact the instructor if you are interested in enrolling but are unsure of the requirements.

Logistics

  • Lectures: Mondays/Wednesdays from 9:30am-10:45am

You should attend lectures regularly as they comprise a significant element of the course instruction. Lecture materials will be posted on the course website.

  • Discussion Forum: We will be using Ed as our primary course discussion platform. If you are enrolled in the class on Canvas, you should already have access to the Ed board for this class. Please use this to post questions about the readings/lecture material as well as about the problem sets.

  • Course Materials: Lecture materials, problem sets and tutorial code will be posted on the course website. Problem set solutions will be posted after the due date on Canvas. Links to readings can be found on the course website organized by week.

Textbooks

The course will involve readings from a variety of different textbooks and published papers. The class will not require the purchase of a single, specific, text and all excerpts from textbooks are available online (either directly or through library resources). However, you may wish to obtain some of these texts to use as a personal reference and they may be valuable to you in the future.

Grading

Students’ final grades are based on four components:

Problem Sets (25%)

Students will complete a total of five problem sets throughout the semester. Problem sets will cover roughly a two-week period of course material. A complete schedule of the assignments can be found on the Assignments page.

The goal of the problem sets is to encourage exploration of the material and to provide you with a clear and credible means of assessing your understanding and progress through the course. As such, problem sets are designed to be challenging and we expect students to find some questions difficult.

Problem sets will be graded on a (+/✓/-) scale:

  • + : Complete and near-perfect work
  • ✓ : Generally good work with clear effort shown but with notable errors
  • - : Significantly incomplete work with major conceptual errors and little effort shown

Collaboration Policy

We strongly encourage collaboration between students on the problem sets and highly recommend that students discuss problems with each other either in person or via the discussion forum. However, each student is expected to submit their own write-up of the answers and any relevant code.

Office Hours and Online Discussion

Students should feel free to discuss any questions about the problem sets with the teaching staff during sections and office hours. We also strongly encourage students to post questions about both the problem sets and the assigned readings on the course discussion board and respond to other students’ questions. Responding to other students’ questions will contribute to your participation grade.

Submission Guidelines

Problem sets will be distributed as HTML and Quarto files (.qmd). You should submit your answers and any relevant R code in the same format: including the Quarto file (.qmd extension) and a corresponding rendered .html file as your submission. You will be submitting your problem sets via Gradescope.

In-person Midterm (30%)

We will have an in-person midterm examination on Wednesday, March 3rd, 2027. This exam will cover the material in the first half of the course (experiments and selection-on-observables) The exam will take the form of a standard pen-and-paper timed examination involving both theory and practical analysis of sample code and results.

In-person Final (35%)

The final exam will take place in person during exams week. The exam date and time are TBA. Room information will be made available nearer to the end of the semester. The exam will take the same form as the midterm, but will be slightly longer given the additional time available during the exam period.

Participation (10%)

We expect students to take an active role in learning in both lecture and section. Engagement with the teaching staff by asking and answering questions will contribute to this grade as will interaction on the discussion board.

Computing

This course will use the R programming language. This is a free and open source programming language that is available for nearly all computing platforms. You should download and install it from https://www.r-project.org.

Unless you have strong preferences for a specific coding environment, we recommend that you use the free RStudio Desktop Integrated Development Environment (IDE) which you can download from https://rstudio.com/products/rstudio/download/#download.

In addition to base R, we will be frequently using data management and processing tools found in the tidyverse set of packages along with basic graphics and visualization using ggplot2.

See the Resources page for additional information.

Policy on Generative Large Language Models

Large Language Models (LLMs) continue to have an immense impact on the educational field. Over the last several years, we have seen striking growth in the capabilities of these models and it is clear that they will remain a permanent and inextricable presence in all of our lives. This very course website was built from a LaTeX syllabus and another course website with considerable assistance from Anthropic’s series of Claude models. Having taught research methods courses every year since 2020, I have seen first-hand how these tools are rapidly reshaping how students engage with the material - for better and for worse.

The last year in particular has seen a massive expansion of agentic LLM tools into research workflows, with the attendant explosion of “research productivity” in the form of papers being posted on pre-print servers and submitted to journals. Whether this is of any actual benefit to science remains unclear and it is certainly creating challenges for the discipline as a whole in sifting through and curating the research product. I generally think this is an issue of volume and not in kind. Scholars were perfectly capable of generating work that has the appearance of science without any of the substance far before LLMs came on the scene. The task of training honest and careful scientists remains very much the same, but the ubiquity of these tools requires developing a mental model of what they can accomplish for you as well as a mental model for the scientific process itself (beyond treating it as a simple game of publication counting).

First and foremost, it is not necessary to use LLMs at all for this course. If you personally dislike the tools, you are not forced in any way to use them.

For me, the most positive case for agentic LLMs is that they are a kind of “universal interface” in natural language that makes it easier for users to get a computer to do what they want simply by articulating the task in English. The actual text output of the LLM is not the most important part, it is rather its ability to call tools on your computer, interpret the output and decide on follow-up steps given some general guidance. This replicates the sort of think-decide-act loop that we would consider to be the role of a typical knowledge worker. In essence, every graduate student can now have their own team of RAs for a comparatively low monthly payment to Anthropic or OpenAI. And the improvements in open models mean that the best case for the near future is that it becomes cheaper to run these tools potentially even on local hardware.

For students who have not had much experience interfacing with computers through text, agentic LLM interfaces are extremely promising. I would encourage you to read through Kieran Healy’s “Plain Text Social Science” if you are unfamiliar with these forms of interaction with your computer (e.g. using a command line shell, version control via git, document typesetting via markdown). You don’t need to go so far as to make Emacs or Vim your day-to-day text editor (frankly, I personally still like having some graphical IDE), but you should understand these ways of interacting with your computer to make it do things. LLM harnesses (like Claude Code or OpenAI’s Codex) are great here because they allow you to use these very powerful command line tools without needing to memorize a bunch of specific syntax or comb through a bunch of configuration files.

At a high level, I think you should use LLMs in research to do things with your computer - especially those things that are frustrating and benefit from automation while having little pedagogical value in doing yourself. As an example, I recently had an LLM fork an existing repository for a poster template in Typst and re-work it to use UW-Madison branding. This is something useful to me that would take way too much time doing manually. It’s not something that I’m trying to learn from necessarily and the output can be easily checked for correctness. Viewing LLMs as a means of doing suggests that your main mode of interface with them should probably be through a command line interface to some sort of model harness rather than through an online chat window.

Of course the main thing that you will typically have an LLM harness do is write code to accomplish some task for your research (e.g. implement an estimator). Here, I think it’s worth making an important distinction between coding and software engineering. LLMs demonstrably work great for taking something expressed in natural language and implementing it in runnable code. They both have been trained on large volumes of code and are excellent translators. Additionally, the built-in evaluation loops in agentic systems allow the code’s outputs to be evaluated on correctness with respect to some clearly defined criteria. You’ll notice many agents will just write unit tests unprompted to verify a function’s implementation (and revise if behavior is unexpected). Where there remains a lot of uncertainty is in how good these tools are at software engineering. Certainly, many developers are experimenting with using teams of LLM agents to not just write code but to architect a project and engineer solutions to problems. Given the fact that software engineer employment has not entirely cratered, my sense is that there are still some major limitations to full automation and that having humans in the loop is essential. Because the cost of generating text is so low for language models, pure LLM projects often end up over-engineered and can lose a sense of focus from continual feature creep. Humans provide a valuable source of friction and also of a kind of deep memory that keeps the model in line.

But the work we do as scientists is typically not software development, although we may benefit from incorporating practices from the field into our workflows. I have previously argued that the thing we lack in science as distinct from engineering is that we do not have a “try-check-fail-repeat” evaluation loop where the “correctness” of the method and implementation can be assessed from the “correctness” of the output. I would back down from this somewhat - we very often evaluate the implementation of our models by checking against simulated data where the truth is known. Indeed, ask any LLM to implement an estimator from scratch and it will probably include a monte carlo simulation to verify it works correctly. But I do think that the entirety of a scientific task cannot be reduced to a series of unit tests and evaluations. You need to know what your code is doing to the data that you are working with and for that you will need to inspect it as a human even if you do use LLMs as assistants to generate it. One of the places where I think LLMs often go off-the-rails is not necessarily in doing something incorrectly, but in choosing to do something entirely correct but also entirely inappropriate for the task.

I like Paul Goldsmith-Pinkham’s framing of empirical research as having an “O-Ring” production function (referencing Kremer (1993)). A very tiny error can completely break an empirical result - everything needs to go right. I have found this to be the case from doing many replications of other empirical papers and you will likely encounter the same phenomenon when completing the problem sets and in your own research. To find these errors, you need a clear mental model of the research process and the sorts of errors that can occur. That is one of the biggest things that you will learn from a research methods sequence. And the fail cases for LLMs are, in my experience, a lot weirder than the usual fail cases for humans.

Part of the goal of the methods sequence is to teach you how to code for research. This involves understanding the mechanics of how statistical analyses are implemented in software and, at a basic level, understanding how to use your computer to control and manipulate inputs to obtain the desired outputs. It certainly is not necessary that you fully understand low-level programming concepts like pointers, memory management, processor architecture - you’re not computer science students. Instead, we train you in a higher level programming language (like R), which abstracts from the complexities of computer hardware. But you still need to understand how you are interacting with your data - and that means thinking to some extent in code. As such, I do not think you should use LLMs for tasks where the process is designed to teach you how to code - such as the problem sets. In practice, I find that students delegate far too much to the model and spend insufficient time understanding the mechanics of what the code is doing. This makes it actually quite difficult to meaningfully debug outputs and understand how to diagnose errors. This problem has existed in the discipline even before the rise of LLMs. Researchers rely too much on packaged implementations of estimators, do not understand what they are actually doing, and mis-implement and/or mis-interpret the results. So in my view, it can be incredibly valuable to implement statistical methods in code by yourself in order to fully understand what these techniques do and how they work!

Another common LLM use is as a personal tutor. As mentioned earlier, I think the chat interface is the least interesting part of the LLM and using it purely in chat mode is not the most effective way to use these tools to learn. I think they are best used as translators that interface directly with a text that you are reading. A common thing that I will do is give the model a fairly dense methods paper and ask it to re-articulate it in notation that I am more familiar with or in a context that is easier for me to think about. For autodidacts, these can be extremely useful tools for expanding your base of knowledge and building your own mental models given what you already know.

However, I would not use LLMs as a complete substitute for your colleagues and for the teaching staff. One of the unfortunate consequences of LLM-proliferation is that students don’t post on discussion boards as often as they used to - even in graduate classes. I feel that this is ultimately detrimental to the sort of community-building and professionalization that this class is designed for. Additionally, one of the benefits of asking the teaching staff is that they are familiar enough with the topic and the context that they can infer a lot of what is unstated or implied by your question and better tailor the response. We will provide relevant context that is specific to the political science discipline. I have found that asking very open-ended questions to many of the recent models can lead to them providing too much information that can be difficult to contextualize if you do not already have an intuition for what is and is not relevant.

For search and discovery of new research papers and materials, I think we are at a point where all of search has a language model component to it and you should probably just use the best ones. There’s no difference in using an LLM vs. just Googling something except in the latter case you’re going to get a cheap/low-quality model. I would avoid them for synthesizing and summarizing literatures even if they do well at this since part of professionalization requires building your own personal understanding of how authors relate to one another in a given literature. Unlike the rest of the internet, academia is still a link(citation)-based culture and it is worth leveraging that researcher-provided context to guide your reading. Additionally, these authors are real human beings who you will meet at conferences - it’s worth understanding who they are conversing with in order to better understand the shape of the discipline. At a minimum, you’ll at least remember their names. However, I do think LLMs can be very good at unearthing surprisingly related work in fields that you are not familiar with. As long as the models have web access (and they should), you can always have them verify the sources (and access them yourself).

I draw the most severe line against LLM use in generating text for writing that is meant to come from you - that is, when communicating your research findings to other scholars. I’m fine with you having LLMs do things for you, but you should not have them think for you and you certainly should not have them speak for you. Academic writing is about communicating your own thinking and reasoning to other human beings and I consider passing LLM-generated text as your own writing to be a fundamentally antisocial act that is hostile to the very nature of scientific community. Moreover, the more advanced the models get, the more their default voice becomes utterly grating and insufferable to read (Claude models in particular). LLM-generated text is detectable - both by newer attribution models (Pangram seems to have a very low false positive rate) and just by other scholars. LLM-generated text designed to communicate scientific results is even more detectable in my view - it prioritizes the strangest things, includes odd asides and frankly just lacks a coherent model of its audience and what it needs to say to them. Every time I have experimented with using them in generating things like summaries or presentation slides, I have to re-write everything from scratch. They are far more trouble than they’re worth. While I understand that some students want to use these tools to adjust style rather than substance, I think that distinction is often harder to draw than it may seem and the consequences of even stylistic adjustment is that you are flattening your own voice into a bland slurry.

Lastly, any LLM policy needs to consider its feasibility. It is clear to me that any restrictions on LLM use aside from restricted, in-class evaluations are fundamentally unenforceable. Therefore, with respect to the problem sets, students are permitted to use LLMs in whatever capacity they see fit. I have attempted to design the problem sets such that they contain “out-of-distribution” challenges (e.g. a replication of an existing paper that concludes contrary to the original result) and otherwise general “traps” that try to evaluate deep substantive knowledge of the problem. Over the last two years, I have found on (e.g. take home exams) that LLM-using students produce mediocre but not completely terrible results. Nevertheless, they do make mistakes (and often behave in ways that could be described as “not wrong, just strange”) and it is clear to me which students use them to their detriment. Perhaps this will change in the next year or two - such is the nature of this field. Indeed, my decision to move entirely to in-person assessment was driven by the observation that although take-home exams still provided some variation among students, that variation was dramatically lower than the in-class exams.

In the end, you should be doing your homework and you should not be using LLMs to reduce the amount of time that you are spending on homework. This is precisely how you end up doing poorly on the exams and recent research seems to support this conclusion.

Accommodations and Accessibility

The University of Wisconsin–Madison supports the right of all enrolled students to a full and equal educational opportunity. The Americans with Disabilities Act (ADA), Wisconsin State Statute (36.12), and UW–Madison policy (Faculty Document 1071) require that students with disabilities be reasonably accommodated in instruction and campus life. Reasonable accommodations for students with disabilities is a shared faculty and student responsibility.

Students are expected to inform faculty of their need for instructional accommodations by the end of the third week of the semester, or as soon as possible after a disability has been incurred or recognized.

I will work either directly with you or in coordination with the McBurney Disability Resource Center to identify and provide reasonable instructional accommodations. Once you are approved for accommodations by the McBurney Center, please be sure to make the relevant selections in McBurney Connect. When I have received your Student Accommodation Letter, I will send you a follow-up e-mail to connect and discuss how the accommodations will be implemented for this course. Disability information, including instructional accommodations as part of a student’s educational record, is confidential and protected under FERPA.

North Hall Accessibility

The Political Science department is located in North Hall, the oldest building on campus. Due to its age, this building is not accessible to individuals with mobility disabilities and does not have an elevator or accessible restroom. The department is committed to equal opportunity for all students to attend office hours, advising, and other department-related events. Please contact me if North Hall presents a disability-related barrier to you, and the department will gladly work to ensure access.

Acknowledgments

This course is indebted to the many wonderful and generous scholars who have developed causal inference curricula in political science departments throughout the world and who have made their course materials available to the public. In particular, I thank Matthew Blackwell, Brandon Stewart, Molly Roberts, Kosuke Imai, Teppei Yamamoto, Jens Hainmueller, Adam Glynn, Gary King, Justin Grimmer, and Edward Kennedy whose lecture notes and syllabi have been immensely valuable in the creation of this course. Special thanks to Molly Offer-Westort, Andy Eggers and Bobby Gulotty who helped in the development of this course in its earlier incarnation as PLSC 30600 at the University of Chicago. I also thank the previous teaching assistants of this course: Arthur Yu, Oscar Cuadros, Zikai Li, Cindy Wang, and Junda Li.

Lastly, thanks to Andrew Heiss and Matt Blackwell for their Quarto website templates, which I have extensively borrowed from in designing this course site.