Schedule
A schedule of topics and readings is provided below. Each week will cover a single topic or group of topics. You should treat the readings as a reference and as a more detailed exposition of the topics discussed in lecture. Consult the readings when you want to know more or want a slightly different approach to explaining a particular topic.
Monday lectures will typically introduce the core theory for the week’s topic while Wednesday lectures will go into greater detail and work through applications. You should make sure to review the readings prior to that week’s lectures with an aim towards completing them before Wednesday’s lecture.
All hyperlinks to papers are either to the published version of the paper (if published) or to the working paper. You should use your university library access to obtain the full version if the article is not open access. Likewise, textbook readings will be to the free/open version of the book (if available) or to the UW-Madison university library eBook.
There is no class during the first week of instruction due to APSA and Labor Day (September 2 and 7). The course begins on Wednesday, September 9.
Week 1: Course Introduction and Review
Wednesday, September 9
Topics:
- What is a statistical model and what is it good for?
- Review of regression
Readings:
- Review: Chapters 1-7; Gelman, Andrew, Jennifer Hill, and Aki Vehtari. 2020. Regression and Other Stories. Cambridge: Cambridge University Press.
Problem Set 1 assigned Wednesday, September 9 – due Wednesday, September 23.
Week 2: Introduction to Likelihood Inference
Monday, September 14 & Wednesday, September 16
Topics:
- What is a “parametric” model?
- The likelihood function
- Maximum likelihood estimation
Readings:
- Chapter 5 (“Parametric Models”); Aronow, P. M. and Benjamin T. Miller. 2019. Foundations of Agnostic Statistics. Cambridge: Cambridge University Press.
- Chapter 6 (“Likelihood Inference”); Evans, Michael J., and Jeffrey S. Rosenthal. 2010. Probability and Statistics: The Science of Uncertainty, 2nd ed. New York: W.H. Freeman.
Week 3: Generalized Linear Models
Monday, September 21 & Wednesday, September 23
Topics:
- Properties of maximum likelihood estimators
- The generalized linear model framework
- Binary outcome models
Readings:
- Chapters 13-14; Gelman, Andrew, Jennifer Hill, and Aki Vehtari. 2020. Regression and Other Stories. Cambridge: Cambridge University Press.
Problem Set 1 due Wednesday, September 23. Problem Set 2 assigned Wednesday, September 23 – due Wednesday, October 7.
Week 4: More Likelihood Models
Monday, September 28 & Wednesday, September 30
Topics:
- Duration models (parametric, semi-parametric (Cox) and non-parametric approaches)
- Event count models (Poisson regression and quasi-MLE methods)
Readings:
- Chapter 15; Gelman, Andrew, Jennifer Hill, and Aki Vehtari. 2020. Regression and Other Stories. Cambridge: Cambridge University Press.
- Clark, T. G., M. J. Bradburn, S. B. Love, and D. G. Altman. 2003. “Survival Analysis Part I: Basic Concepts and First Analyses.” British Journal of Cancer 89 (2): 232-238.
- Stensrud, Mats J., and Miguel A. Hernán. 2020. “Why Test for Proportional Hazards?” JAMA 323 (14): 1401-1402.
- Chapter 8; Wooldridge, Jeffrey M. 1999. “Quasi-Likelihood Methods for Count Data.” In Handbook of Applied Econometrics, vol. 2, 352-406.
Week 5: Bayesian Inference
Monday, October 5
Topics:
- Principles of posterior inference
- How to write a Bayesian model
- Quantities of interest: posterior mode, posterior mean, credible intervals
Readings:
- Chapter 9; Gelman, Andrew, Jennifer Hill, and Aki Vehtari. 2020. Regression and Other Stories. Cambridge: Cambridge University Press.
- Chapters 10-11; Gelman, Andrew, John Carlin, Hal Stern, David Dunson, Aki Vehtari, and Donald Rubin. 2013. Bayesian Data Analysis. 3rd ed. Boca Raton: Chapman and Hall/CRC.
Problem Set 2 due Wednesday, October 7. Problem Set 3 assigned Wednesday, October 7 – due Wednesday, October 21.
Midterm 1 In-Class, Wednesday, October 7th
Week 6: Estimating Bayesian Models
Monday, October 12 & Wednesday, October 14
Topics:
- Analytically tractable solutions through conjugate priors
- Markov Chain Monte Carlo methods
- Introduction to MCMC in Stan
Readings:
- Chapters 3 and 9; McElreath, Richard. 2020. Statistical Rethinking: A Bayesian Course with Examples in R and Stan. 2nd ed. Boca Raton: Chapman and Hall/CRC.
Week 7: Multilevel Regression Models
Monday, October 19 & Wednesday, October 21
Topics:
- Partial pooling through priors
- Methods for model validation
- Multilevel regression and post-stratification (MRP)
Readings:
- Chapter 11 and Appendix A; Gelman, Andrew, Jennifer Hill, and Aki Vehtari. 2020. Regression and Other Stories. Cambridge: Cambridge University Press.
- Chapter 15; Gelman, Andrew, John Carlin, Hal Stern, David Dunson, Aki Vehtari, and Donald Rubin. 2013. Bayesian Data Analysis. 3rd ed. Boca Raton: Chapman and Hall/CRC.
- Park, David K., Andrew Gelman, and Joseph Bafumi. 2004. “Bayesian Multilevel Estimation with Poststratification: State-Level Estimates from National Polls.” Political Analysis 12 (4): 375-385.
Problem Set 3 due Wednesday, October 21. Problem Set 4 assigned Wednesday, October 21 – due Wednesday, November 4.
Replication project memo due Wednesday, October 21
Week 8: Mixture Models and the EM Algorithm
Monday, October 26 & Wednesday, October 28
Topics:
- Exploratory data analysis and clustering models
- MLE and MAP estimation via the “Expectation-Maximization” algorithm
Readings:
- Chapters 13 and 22; Gelman, Andrew, John Carlin, Hal Stern, David Dunson, Aki Vehtari, and Donald Rubin. 2013. Bayesian Data Analysis. 3rd ed. Boca Raton: Chapman and Hall/CRC.
Week 9: Item Response Theory and Ideal Point Models
Monday, November 2
Topics:
- Latent trait models
- “Ideal point” models in legislative voting
Readings:
- Martin, Andrew D., and Kevin M. Quinn. 2002. “Dynamic Ideal Point Estimation via Markov Chain Monte Carlo for the U.S. Supreme Court, 1953-1999.” Political Analysis 10 (2): 134-153.
- Clinton, Joshua, Simon Jackman, and Douglas Rivers. 2004. “The Statistical Analysis of Roll Call Data.” American Political Science Review 98 (2): 355-370.
- Imai, Kosuke, James Lo, and Jonathan Olmsted. 2016. “Fast Estimation of Ideal Points with Massive Data.” American Political Science Review 110 (4): 631-656.
Problem Set 4 due Wednesday, November 4.
Midterm 2 In-Class, Wednesday, November 4th
Week 10: Regularization and Model Selection
Monday, November 9 & Wednesday, November 11
Topics:
- Variable selection and penalized regression (Ridge, LASSO)
- Cross-fitting and out-of-sample validation
Readings:
- Chapters 6-7; James, Gareth, Daniela Witten, Trevor Hastie, and Robert Tibshirani. 2013. An Introduction to Statistical Learning. New York: Springer.
Week 11: Flexible Regression – Trees and Forests
Monday, November 16 & Wednesday, November 18
Topics:
- Regression trees
- Aggregation via sums of trees + regularization via BART
- Bagging, boosting, random forests
Readings:
- Chapter 8; James, Gareth, Daniela Witten, Trevor Hastie, and Robert Tibshirani. 2013. An Introduction to Statistical Learning. New York: Springer.
- Chapters 9-10 and 15; Hastie, Trevor, Robert Tibshirani, and Jerome Friedman. 2009. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. 2nd ed. New York: Springer.
- Hill, Jennifer, Antonio Linero, and Jared Murray. 2020. “Bayesian Additive Regression Trees: A Review and Look Forward.” Annual Review of Statistics and Its Application 7: 251-278.
Week 12: Flexible Regression – Kernels and Gaussian Processes
Monday, November 23
Thanksgiving recess runs November 26-29. Class meets as usual on Monday this week. Wednesday is cancelled.
Topics:
- Infinite dimensional function spaces
- Kernel ridge regression
- Gaussian process regression
Readings:
- Hainmueller, Jens, and Chad Hazlett. 2014. “Kernel Regularized Least Squares: Reducing Misspecification Bias with a Flexible and Interpretable Machine Learning Approach.” Political Analysis 22 (2): 143-168.
- Chapters 2 and 4; Rasmussen, Carl Edward, and Christopher K. I. Williams. 2006. Gaussian Processes for Machine Learning. Cambridge, MA: MIT Press.
Week 13: Causal Inference with Flexible Regression
Monday, November 30 & Wednesday, December 2
Topics:
- Efficient semiparametric estimation of statistical functionals
- Influence functions
- “Doubly-robust”/“Neyman-orthogonal” estimators
Readings:
- Fisher, Aaron, and Edward H. Kennedy. 2021. “Visually Communicating and Teaching Intuition for Influence Functions.” The American Statistician 75 (2): 162-172.
- Kennedy, Edward H. 2024. “Semiparametric Doubly Robust Targeted Double Machine Learning: A Review.” In Handbook of Statistical Methods for Precision Medicine, 207-236.
Poster Session Friday, December 4th
Week 14: Working with “Big” Datasets
Monday, December 7 & Wednesday, December 9
Topics:
- Estimation with out-of-memory data
- Stochastic gradient descent
- Lossless compression in regression
Readings:
- Toulis, Panos, and Edoardo M. Airoldi. 2015. “Scalable Estimation Strategies Based on Stochastic Approximations: Classical Results and New Insights.” Statistics and Computing 25 (4): 781-795.
- Wong, Jeffrey, Eskil Forsell, Randall A. Lewis, Tobias Mao, and Matthew Wardrop. 2021. “You Only Compress Once: Optimal Data Compression for Estimating Linear Models.” arXiv preprint arXiv:2102.11297.