Syllabus

Course Overview

Statistics is at the heart of modern social science. This course is the first in the graduate quantitative methods sequence in the Department of Political Science at the University of Wisconsin-Madison. It begins with the foundations of probability theory and builds up through statistical inference to linear regression. Alongside the statistical material, we will develop the programming skills that play an essential role in the practice of empirical research.

The goal of the sequence is to enable you to produce quality quantitative research and to engage critically with the work of others in the discipline. We start by developing a common language for reasoning about uncertainty: probability, random variables, expectation, and the behavior of sample quantities as samples grow large. We then turn to inference – how we use a sample to make statements about a population with an adequate accounting of uncertainty. The second half of the course develops linear regression as a tool for describing conditional relationships. We will conclude with a brief preview of the principles of causal inference to be developed in greater detail in the second course, PS813. Throughout, the emphasis is on conceptual understanding and the development of your internal mental model of the scientific process rather than on memorizing static recipes and flowcharts for conducting research.

This course will involve a combination of lectures, a weekly section, and problem sets. Lectures will introduce the core theoretical concepts and illustrate them with worked examples. Section will focus on implementation and on building the practical computing skills needed to carry out the analyses discussed in lecture. Problem sets will contain a mixture of both theoretical and applied questions and serve to reinforce key concepts and allow students to assess their progress and understanding throughout the course.

Assignments will involve analysis of data using the programming language. This is a free and open source language for statistical computing that is used extensively for data analysis in many fields. No prior programming experience is assumed – you will learn to program in this course if you do not already know how. Assignments will be written and distributed using the Quarto publishing system – the most recent extension of the R Markdown notebook interface.

Course Goals

By the end of the semester, you should be able to:

  1. Critically read, interpret and replicate the quantitative content of many articles in the quantitative social sciences.
  2. Understand how to reason about properties of estimators: bias, variance, large-sample behavior
  3. Conduct, interpret, and communicate results from an analysis using multiple regression.
  4. Write clean, reusable, and reliable code.
  5. Feel empowered working with data.

The methods sequence is designed to get you to a point where you can teach yourself new statistical methods by reading the literature. We cannot teach you every technique you will need over the course of your career, but we can prepare you to learn them on your own. Beyond the sequence, I strongly encourage you to participate in the broader statistical life at UW-Madison, including the Models, Experiments and Data (MEAD) workshop.

Prerequisites

This is an introductory course in the political science department’s statistical methods sequence and assumes no prior background in statistics. It will assume familiarity with mathematical fundamentals, especially algebra and the basics of calculus: derivatives and integrals. All of the relevant pre-requisite material is covered in the summer math camp provided by the department.

Informally, and considerably more importantly, the most essential prerequisite is a willingness to work hard on unfamiliar material. Learning statistics and programming is a lot like learning a new language: fluency and comfort come from daily practice and consistent effort rather than from cramming. Please contact me at strezhnev@wisc.edu if you are interested in enrolling but are unsure of the requirements.

Logistics

  • Lectures: Tuesdays/Wednesdays, 3:30PM-4:45PM, Ogg Room (North Hall 422).

  • Section: Fridays, 9:30AM-10:20AM, Ogg Room (North Hall 422).

You should attend lectures regularly as they comprise a significant element of the course instruction. Lecture materials will be posted on the course website. Lectures typically focus on the statistical material while the weekly sections emphasize applications and computational skills. Each section will typically begin with questions and review, and the remainder will be devoted to short guided exercises.

  • Discussion Forum: We will be using Slack as our primary course discussion platform. Please reach out to the course staff for an invitation to the methods sequence Slack channel if you are not already on it. Please use this to post questions about the readings/lecture material as well as about the problem sets. We also recommend using the channel to contact teaching staff.

  • Course Materials: Lecture materials, problem sets and tutorial code will be posted on the course website. Problem set solutions will be posted after the due date on Canvas. Links to readings can be found on the Schedule page organized by week.

Textbooks

The primary textbook for this class is Aronow and Miller (2019) Foundations of Agnostic Statistics. It is available online through the UW-Madison Library. However, you may also wish to purchase a physical copy as it is an excellent reference for the core probability and inference material that we will be covering.

  • Aronow, P. M., and Benjamin T. Miller. 2019. Foundations of Agnostic Statistics. Cambridge: Cambridge University Press.

Additional readings will be provided in the form of links to articles and online textbook chapters. Additional supplementary references including recommended guides to programming, can be found on the Resources page.

Grading

Students’ final grades are based on four components:

Problem Sets (20%)

Students will complete a total of five problem sets throughout the semester. Problem sets cover roughly a two-week block of course material. A complete schedule of the assignments can be found on the Assignments page.

The goal of the problem sets is to encourage exploration of the material and to provide you with a clear and credible means of assessing your understanding and progress through the course. As such, problem sets are designed to be challenging and we expect students to find some questions difficult.

Problem sets will be graded on a (+/✓/-) scale:

  • + : Complete and near-perfect work
  • ✓ : Generally good work with clear effort shown but with notable errors
  • - : Significantly incomplete work with major conceptual errors and little effort shown

Collaboration Policy

We strongly encourage collaboration between students on the problem sets and highly recommend that students discuss problems with each other either in person or via the discussion forum. However, each student is expected to submit their own write-up of the answers and any relevant code.

Office Hours and Online Discussion

Students should feel free to discuss any questions about the problem sets with the teaching staff during sections and office hours. We also strongly encourage students to post questions about both the problem sets and the assigned readings on the course discussion board and respond to other students’ questions. Responding to other students’ questions will contribute to your participation grade.

Submission Guidelines

Problem sets will be distributed as HTMl and Quarto files (.qmd). You should submit your answers and any relevant R code in the same format: including the Quarto file (.qmd extension) and a corresponding rendered .html file as your submission. I highly recommend editing the provided Quarto file and providing your work in line with the questions. You will be submitting your problem sets via Gradescope.

Two in-person Midterms (20% each)

We will have two in-person midterm examinations that will take place during the class period. The first will take place on Wednesday, October 7 and cover the first five weeks of probability theory. The second will take place on Wednesday, November 18 and cover statistical inference and linear regression. The exams will take the form of a standard pen-and-paper timed examination involving both theory and practical analysis of sample code and results.

Replication project (30%)

Students will work in pairs to complete a replication project and write up their results in the form of a poster to be presented at the Models, Experiments and Data (MEAD) workshop poster session on December 4th.

You should first read the two articles by Gary King on the nature and purpose of the replication assignment:

The assignment consists of five steps:

  1. Identify a published article in political science that is of interest to you and which implements some form of statistical analysis.
  2. Obtain the original dataset and analysis code.
  3. Replicate the core results of interest.
  4. Propose and write the analysis code for your modification or extension of the original analysis.
  5. Write up your results in the form of an academic poster.

There are many directions that you could choose to go in your replication and I will work with you to identify promising avenues for building on earlier work. The goal of the project is to give you an opportunity to play around with real data and in a substantive setting that you find personally interesting. I hope that the replication project encourages you to read broadly and try to learn about things beyond what is covered in the class.

You will complete steps 1-3 and submit a short memo to me demonstrating that you have identified a paper and replicated the result(s) of interest (or that replication fails!). The memo should also outline your plans for extending or modifying the analysis, though you do not need to have written the code for this or implemented it yet. This memo is ungraded, but it will ensure that you are completing the project at a reasonable pace and allow me to guide you in the right direction or identify any potential issues that you might encounter. The memo is due Wednesday, October 21

Thanks to the work of many scholars over the last several decades, obtaining replication data for most published work is relatively straightforward - you will often find the archives hosted directly by the journal as part of their replication policy or on a centralized repository like Harvard’s Dataverse. Computational reproducibility is also very straightforward and complete replication failures are very rare for modern papers. One thing to note, however, is that you may have to convert the analysis code over to your preferred language - for example, from Stata to R. This is the sort of task that LLMs are ideal for in my view.

You will submit your complete analysis code along with your poster and present your poster in person at the MEAD poster session. I should be able to generate all figures and results presented in the poster from your provided code. If you want to use this as an opportunity to learn how to work Quarto, I would encourage you to make it possible to generate the poster itself from your replication code, but this is not required.

Participation (10%)

We expect students to take an active role in learning in both lecture and section. Engagement with the teaching staff by asking and answering questions will contribute to this grade as will interaction on the Slack. Additionally, bonus credit can be earned by attending talks from external speakers at the MEAD workshop.

Computing

This course will use the R programming language. This is a free and open source programming language that is available for nearly all computing platforms. You should download and install it from https://www.r-project.org.

Unless you have strong preferences for a specific coding environment, we recommend that you use the free RStudio Desktop Integrated Development Environment (IDE) which you can download from https://rstudio.com/products/rstudio/download/#download.

In addition to base R, we will be frequently using data management and processing tools found in the tidyverse set of packages along with basic graphics and visualization using ggplot2.

See the Resources page for additional information.

Policy on Generative Large Language Models

Large Language Models (LLMs) continue to have an immense impact on the educational field. Over the last several years, we have seen striking growth in the capabilities of these models and it is clear that they will remain a permanent and inextricable presence in all of our lives. This very course website was built from a LaTeX syllabus and another course website with considerable assistance from Anthropic’s series of Claude models. Having taught research methods courses every year since 2020, I have seen first-hand how these tools are rapidly reshaping how students engage with the material - for better and for worse.

The last year in particular has seen a massive expansion of agentic LLM tools into research workflows, with the attendant explosion of “research productivity” in the form of papers being posted on pre-print servers and submitted to journals. Whether this is of any actual benefit to science remains unclear and it is certainly creating challenges for the discipline as a whole in sifting through and curating the research product. I generally think this is an issue of volume and not in kind. Scholars were perfectly capable of generating work that has the appearance of science without any of the substance far before LLMs came on the scene. The task of training honest and careful scientists remains very much the same, but the ubiquity of these tools requires developing a mental model of what they can accomplish for you as well as a mental model for the scientific process itself (beyond treating it as a simple game of publication counting).

First and foremost, it is not necessary to use LLMs at all for this course. If you personally dislike the tools, you are not forced in any way to use them.

For me, the most positive case for agentic LLMs is that they are a kind of “universal interface” in natural language that makes it easier for users to get a computer to do what they want simply by articulating the task in English. The actual text output of the LLM is not the most important part, it is rather its ability to call tools on your computer, interpret the output and decide on follow-up steps given some general guidance. This replicates the sort of think-decide-act loop that we would consider to be the role of a typical knowledge worker. In essence, every graduate student can now have their own team of RAs for a comparatively low monthly payment to Anthropic or OpenAI. And the improvements in open models mean that the best case for the near future is that it becomes cheaper to run these tools potentially even on local hardware.

For students who have not had much experience interfacing with computers through text, agentic LLM interfaces are extremely promising. I would encourage you to read through Kieran Healy’s “Plain Text Social Science” if you are unfamiliar with these forms of interaction with your computer (e.g. using a command line shell, version control via git, document typesetting via markdown). You don’t need to go so far as to make Emacs or Vim your day-to-day text editor (frankly, I personally still like having some graphical IDE), but you should understand these ways of interacting with your computer to make it do things. LLM harnesses (like Claude Code or OpenAI’s Codex) are great here because they allow you to interface via the command line without needing to memorize a bunch of command line syntax and sit through combing through configuration files.

At a high level, I think you should use LLMs in research to do things with your computer - especially those things that are frustrating and benefit from automation while having little pedagogical value in doing yourself. As an example, I recently had an LLM fork an existing repository for a poster template in Typst and re-work it to use UW-Madison branding. This is something useful to me that would take way too much time doing manually. It’s not something that I’m trying to learn from necessarily and the output can be easily checked for correctness. Viewing LLMs as a means of doing suggests that your main mode of interface with them should probably be through a command line interface to some sort of model harness rather than through an online chat window.

Of course the main thing that you will typically have an LLM harness do is write code to do something for your research (e.g. implement an estimator). Here, I think it’s worth making an important distinction between coding and software engineering. LLMs demonstrably work great for taking something expressed in natural language and implementing it in runnable code. They both have been trained on large volumes of code and are excellent translators. Additionally, the built-in evaluation loops in agentic systems allow the code’s outputs to be evaluated on correctness with respect to some clearly defined criteria. You’ll notice many agents will just write unit tests unprompted to verify a function’s implementation (and revise if behavior is unexpected). Where there remains a lot of uncertainty is in how good these tools are at software engineering. Certainly, many developers are experimenting with using teams of LLM agents to not just write code but to architect a project and engineer and evaluate a solution. My sense given the fact that software engineer employment has not entirely cratered is that there are severe limitations to full automation and that having humans in the loop is essential. Because the cost of generating text is so low for language models, pure LLM projects often end up over-engineered and can lose a sense of focus from continual feature creep. Humans provide a valuable source of friction and also of a kind of deep memory that keeps the model in line.

But the work we do as scientists is typically not software development, although we may benefit from incorporating practices from the field in our workflows. I have previously argued that the thing we lack in science as distinct from engineering is that we do not have a “try-check-fail-repeat” evaluation loop where the “correctness” of the method and implementation can be assessed from the “correctness” of the output. I would back down from this somewhat - we very often evaluate the implementation of our models by checking against simulated data where the truth is known. Indeed, ask any LLM to implement an estimator from scratch and it will probably include a monte carlo simulation to verify it works correctly. But I do think that the entirety of a scientific task cannot be reduced to a series of unit tests and evaluations. You need to know what your code is doing to the data that you are working with and for that you will need to inspect it as a human even if you do use LLMs as assistants to generate it. One of the places where I think LLMs often go off-the-rails is not necessarily in doing something incorrectly, but in choosing to do something entirely correct but also entirely inappropriate for the task.

I like Paul Goldsmith-Pinkham’s framing of empirical research as having an “O-Ring” production function (referencing Kremer (1993)). A very tiny error can completely break an empirical result - everything needs to go right. I have found this to be the case from doing many replications of other empirical papers and you will likely encounter the same phenomenon when completing the problem sets and potentially in doing your replication project. To find these errors, you need a clear mental model of the research process and the sorts of errors that can occur. That is one of the biggest things that you will learn from a research methods sequence. And the fail cases for LLMs are, in my experience, a lot weirder than the usual fail cases for humans.

Part of the goal of the methods sequence is to teach you how to code for research. This involves understanding the mechanics of how statistical analyses are implemented in software and, at a basic level, understanding how to use your computer to control and manipulate inputs to obtain the desired outputs. It certainly is not necessary that you fully understand low-level programming concepts like pointers, memory management, processor architecture - you’re not computer science students. Instead, we train you in a higher level programming language (like R), which abstracts from the complexities of computer hardware. But you still need to understand how you are interacting with your data - and that means thinking to some extent in code. As such, I do not think you should use LLMs for tasks where the process is designed to teach you how to code - such as the problem sets. In practice, I find that students delegate far too much to the model and spend insufficient time understanding the mechanics of what the code is doing. This makes it actually quite difficult to meaningfully debug outputs and understand how to diagnose errors. This problem has existed in the discipline even before the rise of LLMs. Researchers rely too much on packaged implementations of estimators, do not understand what they are actually doing, and mis-implement and/or mis-interpret the results. So in my view, it can be incredibly valuable to implement statistical methods in code by yourself in order to fully understand what these techniques do and how they work!

Another common LLM use is as a personal tutor. As mentioned earlier, I think the chat interface is the least interesting part of the LLM and using it purely in chat mode is not the most effective way to use these tools to learn. I think they are best used as translators that interface directly with a text that you are reading. A common thing that I will do is give the model a fairly dense methods paper and ask it to re-articulate it in notation that I am more familiar with or in a context that is easier for me to think about. For autodidacts, these can be extremely useful tools for expanding your base of knowledge and building your own mental models given what you already know.

However, I would not use LLMs as a complete substitute for your colleagues and for the teaching staff. One of the unfortunate consequences of LLM-proliferation is that students don’t post on discussion boards as often as they used to - even in graduate classes. I feel that this is ultimately detrimental to the sort of community-building and professionalization that this class is designed for. Additionally, one of the benefits of asking the teaching staff is that they are familiar enough with the topic and the context that they can infer a lot of what is unstated or implied by your question and better tailor the response. We will provide relevant context that is specific to the political science discipline. I have found that asking very open-ended questions to many of the recent models can lead to them providing too much information that can be difficult to contextualize if you do not already have an intuition for what is and is not relevant.

For search and discovery of new research papers and materials, I think we are at a point where all of search has a language model component to it and you should probably just use the best ones. There’s no difference in using an LLM vs. just Googling something except in the latter case you’re going to get a cheap/low-quality model. I would avoid them for synthesizing and summarizing literatures even if they do well at this since part of professionalization requires building your own personal understanding of how authors relate to one another in a given literature. Unlike the rest of the internet, academia is still a link(citation)-based culture and it is worth leveraging that researcher-provided context to guide your reading. Additionally, these authors are real human beings who you will meet at conferences - it’s worth understanding who they are conversing with in order to better understand the shape of the discipline. At a minimum, you’ll at least remember their names. However, I do think LLMs can be very good at unearthing surprisingly related work in fields that you are not familiar with. As long as the models have web access (and they should), you can always have them verify the sources (and access them yourself).

I draw the most severe line against LLM use in generating text for writing that is meant to come from you - that is, when communicating your research findings to other scholars. I’m fine with you having LLMs do things for you, but you should not have them think for you and you certainly should not have them speak for you. Academic writing is about communicating your own thinking and reasoning to other human beings and I consider passing LLM-generated text as your own writing to be a fundamentally antisocial act that is hostile to the very nature of scientific community. Moreover, the more advanced the models get, the more their default voice becomes utterly grating and insufferable to read (Claude models in particular). LLM-generated text is detectable - both by newer attribution models (Pangram seems to have a very low false positive rate) and just by other scholars. LLM-generated text designed to communicate scientific results is even more detectable in my view - it prioritizes the strangest things, includes odd asides and frankly just lacks a coherent model of its audience and what it needs to say to them. Every time I have experimented with using them in generating things like summaries or presentation slides, I have to re-write everything from scratch. They are far more trouble than they’re worth. While I understand that some students want to use these tools to adjust style rather than substance, I think that distinction is often harder to draw than it may seem and the consequences of even stylistic adjustment is that you are flattening your own voice into a bland slurry.

Lastly, any LLM policy needs to consider its feasibility. It is clear to me that any restrictions on LLM use aside from restricted, in-class evaluations are fundamentally unenforceable. Therefore, with respect to the problem sets, students are permitted to use LLMs in whatever capacity they see fit. I have attempted to design the problem sets such that they contain “out-of-distribution” challenges (e.g. a replication of an existing paper that concludes contrary to the original result) and otherwise general “traps” that try to evaluate deep substantive knowledge of the problem. Over the last two years, I have found on (e.g. take home exams) that LLM-using students produce mediocre but not completely terrible results. Nevertheless, they do make mistakes (and often behave in ways that could be described as “not wrong, just strange”) and it is clear to me which students use them to their detriment. Perhaps this will change in the next year or two - such is the nature of this field. Indeed, my decision to move entirely to in-person assessment was driven by the observation that although take-home exams still provided some variation among students, that variation was dramatically lower than the in-class exams.

In the end, you should be doing your homework and you should not be using LLMs to reduce the amount of time that you are spending on homework. This is precisely how you end up doing poorly on the exams and recent research seems to support this conclusion. As far as the replication project goes, I am much more open to LLM use for coding, especially when exploring and working with the unfamiliar codebase of the paper that you choose to replicate. However, the final poster should be entirely human-authored by you and you will be evaluated on your ability to communicate your results through both the poster and in-person at the December 4th poster session.

Accommodations and Accessibility

The University of Wisconsin–Madison supports the right of all enrolled students to a full and equal educational opportunity. The Americans with Disabilities Act (ADA), Wisconsin State Statute (36.12), and UW–Madison policy (Faculty Document 1071) require that students with disabilities be reasonably accommodated in instruction and campus life. Reasonable accommodations for students with disabilities is a shared faculty and student responsibility.

Students are expected to inform faculty of their need for instructional accommodations by the end of the third week of the semester, or as soon as possible after a disability has been incurred or recognized.

I will work either directly with you or in coordination with the McBurney Disability Resource Center to identify and provide reasonable instructional accommodations. Once you are approved for accommodations by the McBurney Center, please be sure to make the relevant selections in McBurney Connect. When I have received your Student Accommodation Letter, I will send you a follow-up e-mail to connect and discuss how the accommodations will be implemented for this course. Disability information, including instructional accommodations as part of a student’s educational record, is confidential and protected under FERPA.

North Hall Accessibility

The Political Science department is located in North Hall, the oldest building on campus. Due to its age, this building is not accessible to individuals with mobility disabilities and does not have an elevator or accessible restroom. The department is committed to equal opportunity for all students to attend office hours, advising, and other department-related events. Please contact me if North Hall presents a disability-related barrier to you, and the department will gladly work to ensure access.

Acknowledgments

This course is indebted to the many wonderful and generous scholars in political methodology, econometrics and sociology who have made their materials available to the public. In particular, I thank Adeline Lo, whose previous version of PS812 forms the core of this syllabus and its schedule, as well as Matthew Blackwell, P Aronow, Adam Glynn, Justin Grimmer, Jens Hainmueller, Erin Hartman, Chad Hazlett, Kosuke Imai, Gary King, Dean Knox, Kevin Quinn, Molly Roberts, Molly Offer-Westort, Brandon Stewart, and Teppei Yamamoto

Lastly, thanks to Andrew Heiss and Matt Blackwell for their Quarto website templates, which I have extensively borrowed from in designing this course site.