Syllabus

Course Overview

Statistical models provide a structure for the analysis of data. Often many scientific questions revolve around drawing statistical inferences about some parameter, such as a regression coefficient. Alternatively, models provide a structured way for generating predictions on new or out-of-sample data. Understanding the fundamentals of how to define, estimate and validate a statistical model is essential to the process of quantitative empirical research.

This course is part of the second year of the quantitative methods sequence and builds on the material covered in the first year. It will introduce students to likelihood, Bayesian and semi-/non-parametric approaches to statistical inference. The overarching framework of this class is model-based inference for description and prediction – a complement to the design-based framework of causal inference. However, we will also discuss how to incorporate predictive models into a causal inference workflow, particularly when adjusting for confounding in selection-on-observables designs and searching for treatment effect heterogeneity. Students will learn both the theory behind these estimation approaches and the computational methods used to implement these estimators, starting from Newton-Raphson approaches for finding maxima in simple parametric models to Markov Chain Monte Carlo (MCMC) and (variational) Expectation-Maximization (EM) methods for bayesian models, concluding with a brief introduction to stochastic gradient descent methods for modern large-data settings. All approaches will be demonstrated in the statistical programming language. They will also be exposed to modern machine learning methods for flexible estimation of regression models – techniques such as penalized regression, random forests and kernel methods/gaussian processes among others. Applied examples will be drawn from across the political science literature, with a particular focus on the analysis of large survey data (e.g. the American National Election Survey (ANES), the Cooperative Election Survey (CES), the European Social Survey (ESS)).

In general, this course aims to provide students with a broad overview of the history and the present of statistical modelling in political science as well as an understanding of where the discipline is likely to move in the future.

This course will involve a combination of lectures and problem sets as well as a final replication project developed through the course of the semester. Lectures will focus on introducing the core theoretical concepts being taught in this course as well as providing illustrations through worked applied examples. Problem sets will contain a mixture of both theoretical and applied questions and serve to reinforce key concepts and allow students to assess their progress and understanding throughout the course. Two in-person examinations will take place during class days.

Assignments will involve analysis of data using the programming language. This is a free and open source language for statistical computing that is used extensively for data analysis in many fields. Prior experience with the fundamentals of programming is required. Assignments will be written and distributed using the Quarto publishing system – the most recent extension of the R Markdown notebook interface.

Prerequisites

This course assumes that you have both a background in the core concepts of probability, statistics and inference as well as prior exposure to linear regression models. Completing the first two courses in the political science graduate methodology sequence should prepare you for the material in this class. However, there are no strict, specific course pre-requisites as many different disciplines and departments offer introductory statistics classes that cover the relevant material. Prior exposure to causal inference from a counterfactual/potential outcomes framework is helpful but not strictly required. Some weeks will assume familiarity with quantities like the (conditional) average treatment effect as well as the identifying assumptions underpinning experiments and selection-on-observables designs.

If you are unsure of whether you meet the requirements, skim/read through the first six chapters of Regression and Other Stories, one of the books being used by this course. You should find most of the concepts behind the material relatively familiar, aside from the references to Bayesian models (which will be covered in this course).

Please contact the instructor at strezhnev@wisc.edu if you are interested in enrolling but are unsure of the requirements.

Logistics

  • Lectures: Mondays/Wednesdays from 9:30am-10:45am in Sterling Hall 1339

You should attend lectures regularly as they comprise a significant element of the course instruction. Lecture materials will be posted on the course website.

  • Discussion Forum: We will be using Slack as our primary course discussion platform. You should have received an invitation to this Slack channel from me. Please e-mail me if you have not. You should use the channel to chat with your fellow students about the readings/lecture material and the problem sets as well as to contact the instructional staff with direct questions.

  • Course Materials: Lecture materials and problem sets will be posted on the course website. Problem set solutions will be posted after the due date on Canvas. Links to readings can be found on the Schedule page organized by week.

Textbooks

The course will involve readings from a variety of different textbook chapters and published papers. The class will not require the purchase of any textbook as they are available online either directly from the author’s website or via the University library. However, you may wish to obtain a paper copy for your own personal use or reference.

Three textbooks from which many readings will be drawn are:

A complete list of textbooks and supplementary references can be found on the Resources page.

Grading

Students’ final grades are based on four components:

Problem Sets (20%)

Students will complete a total of four problem sets throughout the semester. Problem sets will primarily cover topics from the lectures for that week and the previous week. A complete schedule of the assignments can be found on the Assignments page.

The goal of the problem sets is to encourage exploration of the material and to provide you with a clear and credible means of assessing your understanding and progress through the course. As such, problem sets are designed to be challenging and I expect students to find some questions difficult.

Problem sets will be graded on a (+/✓/-) scale:

  • + : Complete and near-perfect work
  • ✓ : Generally good work with clear effort shown but with notable errors
  • - : Significantly incomplete work with major conceptual errors and little effort shown

Collaboration Policy

I strongly encourage collaboration between students on the problem sets and highly recommend that students discuss problems with each other either in person or via the discussion forum. However, each student is expected to submit their own write-up of the answers and any relevant code.

Office Hours and Online Discussion

Students should feel free to discuss any questions about the problem sets with me during class and during office hours. I also strongly encourage students to post questions about both the problem sets and the readings on the course discussion board and respond to other students’ questions. Responding to other students’ questions will contribute to your participation grade.

Submission Guidelines

Problem sets will be distributed as HTML and Quarto files (.qmd). You should submit your answers and any relevant R code in the same format: including the Quarto file (.qmd extension) and a corresponding rendered .html file as your submission. Quarto builds on the earlier R Markdown project, which allows you to combine the text formatting syntax of the Markdown markup language with the ability to embed and execute chunks of code directly into a text document. This allows you to present your code, graphical output, and discussion/write-up all in the same document. I highly recommend that you edit the distributed Quarto assignment file for each problem set directly to make organization easier.

In-person Midterms (40%)

We will have two in-person midterm examinations on Wednesday, October 7th, 2026 and on Wednesday, November 4th, 2026. Each exam will take the form of a standard pen-and-paper timed examination involving both theory and practical analysis of sample code and results.

Replication project (30%)

Students will work in pairs to complete a replication project and write up their results in the form of a poster to be presented at the Models, Experiments and Data (MEAD) workshop poster session on December 4th.

You should first read the two articles by Gary King on the nature and purpose of the replication assignment:

The assignment consists of five steps:

  1. Identify a published article in political science that is of interest to you and which implements some form of statistical analysis.
  2. Obtain the original dataset and analysis code.
  3. Replicate the core results of interest.
  4. Propose and write the analysis code for your modification or extension of the original analysis.
  5. Write up your results in the form of an academic poster.

There are many directions that you could choose to go in your replication and I will work with you to identify promising avenues for improving on earlier work. For example, our class will spend time covering flexible methods for regression adjustment. You may wish to replicate a selection-on-observables paper that uses a typical additive linear regression adjustment using these techniques appropriately to see if the results change (that is, if the original results failed to sufficiently adjust for the presented confounders). You might also propose additional confounders not addressed in the original study. Feel free to bring in materials and concepts learned from previous courses in the sequence such as 813. You will also likely need to read ahead or read additional papers in order to come up with a promising direction for the project. That is one of the goals of the replication project: to encourage you to read broadly and learn about quantitative methods in the context of papers that you find interesting.

You will complete steps 1-3 and submit a short memo to me demonstrating that you have identified a paper and replicated the result(s) of interest (or that replication fails!). The memo should also outline your plans for extending or modifying the analysis, though you do not need to have written the code for this or implemented it yet. This memo is ungraded, but it will ensure that you are completing the project at a reasonable pace and allow me to guide you in the right direction or identify any potential issues that you might encounter. The memo is due Wednesday, October 21

Thanks to the work of many scholars over the last several decades, obtaining replication data for most published work is relatively straightforward - you will often find the archives hosted directly by the journal as part of their replication policy or on a centralized repository like Harvard’s Dataverse. Computational reproducibility is also very straightforward and complete replication failures are very rare for modern papers. One thing to note, however, is that you may have to convert the analysis code over to your preferred language - for example, from Stata to R. This is the sort of task that LLMs are ideal for in my view.

You will submit your complete analysis code along with your poster and present your poster in person at the MEAD poster session. I should be able to generate all figures and results presented in the poster from your provided code. If you want to use this as an opportunity to learn how to work Quarto, I would encourage you to make it possible to generate the poster itself from your replication code, but this is not required.

Participation (10%)

I expect students to take an active role in learning in lecture. Engagement with the teaching staff by asking and answering questions will contribute to this grade as will interaction on the Slack channel.

Computing

This course will use the R programming language. This is a free and open source programming language that is available for nearly all computing platforms. You should download and install it from https://www.r-project.org.

Unless you have strong preferences for a specific coding environment, I also highly recommend that you use either the RStudio Desktop Integrated Development Environment (IDE) or the Positron IDE. Both are designed to give you a simple and easy-to-use environment for editing code and are built with data science in mind. These IDEs also make it very easy to write and compile Quarto documents: the format in which the problem sets will be distributed. Personally, I still like how RStudio handles the editing and previewing of these documents, but recent updates to Positron have made it a very reasonable alternative and I expect to migrate over completely in the coming months/years.

In addition to base R, we will be frequently using data management and processing tools found in the tidyverse set of packages along with basic graphics and visualization using ggplot2.

The course will also introduce the Stan language and software for specifying and estimating Bayesian models. Stan is written in C++ but has bindings for a variety of programming languages. We will use two interfaces for Stan in R: RStan and brms. Running Stan on your local machine will require that you have the correct libraries installed to allow you to compile C++ source code into an executable. See the installation guide for more details on the requirements for your particular operating system.

See the Resources page for additional information.

Policy on Generative Large Language Models

Large Language Models (LLMs) continue to have an immense impact on the educational field. Over the last several years, we have seen striking growth in the capabilities of these models and it is clear that they will remain a permanent and inextricable presence in all of our lives. This very course website was built from a LaTeX syllabus and another course website with considerable assistance from Anthropic’s series of Claude models. Having taught research methods courses every year since 2020, I have seen first-hand how these tools are rapidly reshaping how students engage with the material - for better and for worse.

The last year in particular has seen a massive expansion of agentic LLM tools into research workflows, with the attendant explosion of “research productivity” in the form of papers being posted on pre-print servers and submitted to journals. Whether this is of any actual benefit to science remains unclear and it is certainly creating challenges for the discipline as a whole in sifting through and curating the research product. I generally think this is an issue of volume and not in kind. Scholars were perfectly capable of generating work that has the appearance of science without any of the substance far before LLMs came on the scene. The task of training honest and careful scientists remains very much the same, but the ubiquity of these tools requires developing a mental model of what they can accomplish for you as well as a mental model for the scientific process itself (beyond treating it as a simple game of publication counting).

First and foremost, it is not necessary to use LLMs at all for this course. If you personally dislike the tools, you are not forced in any way to use them.

For me, the most positive case for agentic LLMs is that they are a kind of “universal interface” in natural language that makes it easier for users to get a computer to do what they want simply by articulating the task in English. The actual text output of the LLM is not the most important part, it is rather its ability to call tools on your computer, interpret the output and decide on follow-up steps given some general guidance. This replicates the sort of think-decide-act loop that we would consider to be the role of a typical knowledge worker. In essence, every graduate student can now have their own team of RAs for a comparatively low monthly payment to Anthropic or OpenAI. And the improvements in open models mean that the best case for the near future is that it becomes cheaper to run these tools potentially even on local hardware.

For students who have not had much experience interfacing with computers through text, agentic LLM interfaces are extremely promising. I would encourage you to read through Kieran Healy’s “Plain Text Social Science” if you are unfamiliar with these forms of interaction with your computer (e.g. using a command line shell, version control via git, document typesetting via markdown). You don’t need to go so far as to make Emacs or Vim your day-to-day text editor (frankly, I personally still like having some graphical IDE), but you should understand these ways of interacting with your computer to make it do things. LLM harnesses (like Claude Code or OpenAI’s Codex) are great here because they allow you to use these very powerful command line tools without needing to memorize a bunch of specific syntax or comb through a bunch of configuration files.

At a high level, I think you should use LLMs in research to do things with your computer - especially those things that are frustrating and benefit from automation while having little pedagogical value in doing yourself. As an example, I recently had an LLM fork an existing repository for a poster template in Typst and re-work it to use UW-Madison branding. This is something useful to me that would take way too much time doing manually. It’s not something that I’m trying to learn from necessarily and the output can be easily checked for correctness. Viewing LLMs as a means of doing suggests that your main mode of interface with them should probably be through a command line interface to some sort of model harness rather than through an online chat window.

Of course the main thing that you will typically have an LLM harness do is write code to accomplish some task for your research (e.g. implement an estimator). Here, I think it’s worth making an important distinction between coding and software engineering. LLMs demonstrably work great for taking something expressed in natural language and implementing it in runnable code. They both have been trained on large volumes of code and are excellent translators. Additionally, the built-in evaluation loops in agentic systems allow the code’s outputs to be evaluated on correctness with respect to some clearly defined criteria. You’ll notice many agents will just write unit tests unprompted to verify a function’s implementation (and revise if behavior is unexpected). Where there remains a lot of uncertainty is in how good these tools are at software engineering. Certainly, many developers are experimenting with using teams of LLM agents to not just write code but to architect a project and engineer solutions to problems. Given the fact that software engineer employment has not entirely cratered, my sense is that there are still some major limitations to full automation and that having humans in the loop is essential. Because the cost of generating text is so low for language models, pure LLM projects often end up over-engineered and can lose a sense of focus from continual feature creep. Humans provide a valuable source of friction and also of a kind of deep memory that keeps the model in line.

But the work we do as scientists is typically not software development, although we may benefit from incorporating practices from the field into our workflows. I have previously argued that the thing we lack in science as distinct from engineering is that we do not have a “try-check-fail-repeat” evaluation loop where the “correctness” of the method and implementation can be assessed from the “correctness” of the output. I would back down from this somewhat - we very often evaluate the implementation of our models by checking against simulated data where the truth is known. Indeed, ask any LLM to implement an estimator from scratch and it will probably include a monte carlo simulation to verify it works correctly. But I do think that the entirety of a scientific task cannot be reduced to a series of unit tests and evaluations. You need to know what your code is doing to the data that you are working with and for that you will need to inspect it as a human even if you do use LLMs as assistants to generate it. One of the places where I think LLMs often go off-the-rails is not necessarily in doing something incorrectly, but in choosing to do something entirely correct but also entirely inappropriate for the task.

I like Paul Goldsmith-Pinkham’s framing of empirical research as having an “O-Ring” production function (referencing Kremer (1993)). A very tiny error can completely break an empirical result - everything needs to go right. I have found this to be the case from doing many replications of other empirical papers and you will likely encounter the same phenomenon when completing the problem sets and potentially in doing your replication project. To find these errors, you need a clear mental model of the research process and the sorts of errors that can occur. That is one of the biggest things that you will learn from a research methods sequence. And the fail cases for LLMs are, in my experience, a lot weirder than the usual fail cases for humans.

Part of the goal of the methods sequence is to teach you how to code for research. This involves understanding the mechanics of how statistical analyses are implemented in software and, at a basic level, understanding how to use your computer to control and manipulate inputs to obtain the desired outputs. It certainly is not necessary that you fully understand low-level programming concepts like pointers, memory management, processor architecture - you’re not computer science students. Instead, we train you in a higher level programming language (like R), which abstracts from the complexities of computer hardware. But you still need to understand how you are interacting with your data - and that means thinking to some extent in code. As such, I do not think you should use LLMs for tasks where the process is designed to teach you how to code - such as the problem sets. In practice, I find that students delegate far too much to the model and spend insufficient time understanding the mechanics of what the code is doing. This makes it actually quite difficult to meaningfully debug outputs and understand how to diagnose errors. This problem has existed in the discipline even before the rise of LLMs. Researchers rely too much on packaged implementations of estimators, do not understand what they are actually doing, and mis-implement and/or mis-interpret the results. So in my view, it can be incredibly valuable to implement statistical methods in code by yourself in order to fully understand what these techniques do and how they work!

Another common LLM use is as a personal tutor. As mentioned earlier, I think the chat interface is the least interesting part of the LLM and using it purely in chat mode is not the most effective way to use these tools to learn. I think they are best used as translators that interface directly with a text that you are reading. A common thing that I will do is give the model a fairly dense methods paper and ask it to re-articulate it in notation that I am more familiar with or in a context that is easier for me to think about. For autodidacts, these can be extremely useful tools for expanding your base of knowledge and building your own mental models given what you already know.

However, I would not use LLMs as a complete substitute for your colleagues and for the teaching staff. One of the unfortunate consequences of LLM-proliferation is that students don’t post on discussion boards as often as they used to - even in graduate classes. I feel that this is ultimately detrimental to the sort of community-building and professionalization that this class is designed for. Additionally, one of the benefits of asking the teaching staff is that they are familiar enough with the topic and the context that they can infer a lot of what is unstated or implied by your question and better tailor the response. We will provide relevant context that is specific to the political science discipline. I have found that asking very open-ended questions to many of the recent models can lead to them providing too much information that can be difficult to contextualize if you do not already have an intuition for what is and is not relevant.

For search and discovery of new research papers and materials, I think we are at a point where all of search has a language model component to it and you should probably just use the best ones. There’s no difference in using an LLM vs. just Googling something except in the latter case you’re going to get a cheap/low-quality model. I would avoid them for synthesizing and summarizing literatures even if they do well at this since part of professionalization requires building your own personal understanding of how authors relate to one another in a given literature. Unlike the rest of the internet, academia is still a link(citation)-based culture and it is worth leveraging that researcher-provided context to guide your reading. Additionally, these authors are real human beings who you will meet at conferences - it’s worth understanding who they are conversing with in order to better understand the shape of the discipline. At a minimum, you’ll at least remember their names. However, I do think LLMs can be very good at unearthing surprisingly related work in fields that you are not familiar with. As long as the models have web access (and they should), you can always have them verify the sources (and access them yourself).

I draw the most severe line against LLM use in generating text for writing that is meant to come from you - that is, when communicating your research findings to other scholars. I’m fine with you having LLMs do things for you, but you should not have them think for you and you certainly should not have them speak for you. Academic writing is about communicating your own thinking and reasoning to other human beings and I consider passing LLM-generated text as your own writing to be a fundamentally antisocial act that is hostile to the very nature of scientific community. Moreover, the more advanced the models get, the more their default voice becomes utterly grating and insufferable to read (Claude models in particular). LLM-generated text is detectable - both by newer attribution models (Pangram seems to have a very low false positive rate) and just by other scholars. LLM-generated text designed to communicate scientific results is even more detectable in my view - it prioritizes the strangest things, includes odd asides and frankly just lacks a coherent model of its audience and what it needs to say to them. Every time I have experimented with using them in generating things like summaries or presentation slides, I have to re-write everything from scratch. They are far more trouble than they’re worth. While I understand that some students want to use these tools to adjust style rather than substance, I think that distinction is often harder to draw than it may seem and the consequences of even stylistic adjustment is that you are flattening your own voice into a bland slurry.

Lastly, any LLM policy needs to consider its feasibility. It is clear to me that any restrictions on LLM use aside from restricted, in-class evaluations are fundamentally unenforceable. Therefore, with respect to the problem sets, students are permitted to use LLMs in whatever capacity they see fit. I have attempted to design the problem sets such that they contain “out-of-distribution” challenges (e.g. a replication of an existing paper that concludes contrary to the original result) and otherwise general “traps” that try to evaluate deep substantive knowledge of the problem. Over the last two years, I have found on (e.g. take home exams) that LLM-using students produce mediocre but not completely terrible results. Nevertheless, they do make mistakes (and often behave in ways that could be described as “not wrong, just strange”) and it is clear to me which students use them to their detriment. Perhaps this will change in the next year or two - such is the nature of this field. Indeed, my decision to move entirely to in-person assessment was driven by the observation that although take-home exams still provided some variation among students, that variation was dramatically lower than the in-class exams.

In the end, you should be doing your homework and you should not be using LLMs to reduce the amount of time that you are spending on homework. This is precisely how you end up doing poorly on the exams and recent research seems to support this conclusion. As far as the replication project goes, I am much more open to LLM use for coding, especially when exploring and working with the unfamiliar codebase of the paper that you choose to replicate. However, the final poster should be entirely human-authored by you and you will be evaluated on your ability to communicate your results through both the poster and in-person at the December 4th poster session.

Accommodations and Accessibility

The University of Wisconsin–Madison supports the right of all enrolled students to a full and equal educational opportunity. The Americans with Disabilities Act (ADA), Wisconsin State Statute (36.12), and UW–Madison policy (Faculty Document 1071) require that students with disabilities be reasonably accommodated in instruction and campus life. Reasonable accommodations for students with disabilities is a shared faculty and student responsibility.

Students are expected to inform faculty of their need for instructional accommodations by the end of the third week of the semester, or as soon as possible after a disability has been incurred or recognized.

I will work either directly with you or in coordination with the McBurney Disability Resource Center to identify and provide reasonable instructional accommodations. Once you are approved for accommodations by the McBurney Center, please be sure to make the relevant selections in McBurney Connect. When I have received your Student Accommodation Letter, I will send you a follow-up e-mail to connect and discuss how the accommodations will be implemented for this course. Disability information, including instructional accommodations as part of a student’s educational record, is confidential and protected under FERPA.

North Hall Accessibility

The Political Science department is located in North Hall, the oldest building on campus. Due to its age, this building is not accessible to individuals with mobility disabilities and does not have an elevator or accessible restroom. The department is committed to equal opportunity for all students to attend office hours, advising, and other department-related events. Please contact me if North Hall presents a disability-related barrier to you, and the department will gladly work to ensure access.

Acknowledgments

This course is indebted to the many wonderful and generous scholars who have developed statistical methods curricula in political science departments throughout the world and who have made their course materials available to the public. This course in particular has been heavily inspired by Gov 2001 and Gov 2003 at Harvard University as well as Quant III at MIT. In particular, I thank Matthew Blackwell, Brandon Stewart, Erin Hartman, Molly Roberts, Kosuke Imai, Teppei Yamamoto, Jens Hainmueller, Adam Glynn, Gary King, Justin Grimmer, and In Song Kim whose lecture notes and syllabi have been immensely valuable in the creation of this course.

Lastly, thanks to Andrew Heiss and Matt Blackwell for their Quarto website templates, which I have extensively borrowed from in designing this course site.