Week 4: Joint and Conditional Distributions

Week overview

• Last week
  • Random variables
  • Distributions: CDFs, PMFs and PDFs
  • Expectations and variances
• This week
  • Joint distributions
  • Conditional distributions
  • Conditional expectations
  • Covariances and correlations

Joint distribution functions

• We’ve so far talked about the joint probabilities of events \(P(A \cap B)\)

  • And their conditional probabilities: \(P(A \mid B)\)

• Let’s extend these ideas to talk about random variables

• We can define a joint cumulative distribution function that takes values of multiple variables as inputs

  • For two random variables, \(X\) and \(Y\)

    \[F_{XY}(x, y) = P(X \le x, Y \le y) \]

  • We can generalize to many random variables

    \[F_{X_1, \dotsc X_n}(x_1, \dotsc x_n) = P(X_1 \le x_1, \dotsc X_n \le x_n)\]

Joint PMFs

• For discrete random variables, we can also talk about the joint probability that \(X\) takes on a particular value \(x\) and \(Y\) takes on a particular value \(y\)

• We define the joint probability mass function (PMF) in a similar way as before

  • For two discrete random variables, \(X\) and \(Y\)

    \[p_{XY}(x, y) = P(X = x, Y = y) \]

  • Generalizing to many random variables

    \[p_{X_1, \dotsc X_n}(x_1, \dotsc x_n) = P(X_1 = x_1, \dotsc X_n = x_n) \]

Joint PDFs

• As before, with continuous random variables \(X\) and \(Y\), the probability that \(X\) or \(Y\) take on specific values \(x\) and \(y\) is zero.

  • We cannot work with a probability mass function \(P(X = x, Y = y)\)

• Instead, we’ll define a joint density function as the derivative of the joint CDF with respect to each of the variables

  • For two continuous random variables, \(X\) and \(Y\)

    \[f_{XY}(x, y) = \frac{\partial^2 F_{XY}(x,y)}{\partial x \partial y} \]

  • Again, generalizing to many random variables

    \[f_{X_1, \dotsc X_n}(x_1, \dotsc x_n) = \frac{\partial^n F_{X_1, \dotsc X_n}(x_1, \dotsc x_n)}{\partial x_1 \dotsc \partial x_n} \]

Covariance and correlation

• When we have multiple random variables, we may want a summary of how the two quantities vary together.

  • In the same way we wanted a summary of central tendency (expectation) or variability (variance)

• The covariance between two random variables \(X\) and \(Y\) is defined as

  \[ \Cov(X, Y) = \mathbb{E}[(X - \mathbb{E}[X])(Y - \mathbb{E}[Y])] \]

• We can factor and apply linearity of expectations to get another way of writing the covariance

  \[ \begin{aligned} \Cov(X, Y) &= \mathbb{E}[(X - \mathbb{E}[X])(Y - \mathbb{E}[Y])] \\ &= \mathbb{E}\big[XY - X\,\mathbb{E}[Y] - \mathbb{E}[X]\,Y + \mathbb{E}[X]\,\mathbb{E}[Y]\big] \\ &= \mathbb{E}[XY] - \mathbb{E}[X]\,\mathbb{E}[Y] - \mathbb{E}[X]\,\mathbb{E}[Y] + \mathbb{E}[X]\,\mathbb{E}[Y] \\ &= \mathbb{E}[XY] - \mathbb{E}[X]\,\mathbb{E}[Y] \end{aligned} \]

Properties of covariance

• We can pull out constants out of the covariance. Let \(a\) and \(b\) be constants and \(X\) and \(Y\) random variables:

  \[ \Cov(aX, bY) = ab\Cov(X, Y) \]

• Adding a constant doesn’t change the covariance (just as it doesn’t change the variance)

  \[ \Cov(X + a, Y + b) = \Cov(X, Y) \]

• The covariance of any random variable with a constant is \(0\)

  \[ \Cov(X, a) = 0 \]

• Covariances have the distributive property

  \[ \Cov(X + Z, Y) = \Cov(X, Y) + \Cov(Z, Y) \]

Covariance and correlation

• Covariance is scale dependent - the units in which \(X\) and \(Y\) are measured matter for the overall magnitude of the covariance.

  • Sometimes we want a scale free measure of association as a summary.

• The correlation is constructed by normalizing the covariance between \(X\) and \(Y\) by their respective standard deviations

  \[\rho_{X,Y} = \Corr(X, Y) = \frac{\Cov(X, Y)}{\sqrt{\Var(X) \Var(Y)}} = \frac{\Cov(X, Y)}{\sqrt{\Var(X)} \sqrt{\Var(Y)}} \]

• The correlation coefficient lies between \([-1, 1]\)

  • These bounds come from an application of the Cauchy-Schwarz inequality!

    \[\Cov(X, Y)^2 \le \Var(X) \Var(Y)\]

Properties of variance

• The covariance appears when we want to take the variance of a sum or difference in two random variables.

• The variance of the sum of two random variables is:

  \[ \Var(X + Y) = \Var(X) + \Var(Y) + 2\Cov(X, Y) \]

• The variance of the difference of two random variables is:

  \[ \Var(X - Y) = \Var(X) + \Var(Y) - 2\Cov(X, Y) \]

• Intuitively:

  • If I add two random variables together, I’m going to get something noisier unless \(X\) tends to be high when \(Y\) is low (and vice-versa)
  • If I subtract two random variables, I’m also going to get something noisier unless \(X\) is high when \(Y\) is high (and vice-versa)

Conditional distributions

• We often want to ask how the distribution of one variable \(Y\) changes if we were to set \(X\) to a particular value

  • As we did before with conditional probabilities, we can define the conditional distribution of \(Y \mid X\)

• For discrete random variables, the definition is exactly what we had before: joint over marginal

  \[p_{Y \mid X}(y \mid x) = P(Y = y \mid X = x) = \frac{p_{XY}(x, y)}{p_X(x)} \]

• For continuous random variables it’s a bit tricky (since \(X = x\) has probability zero), but we use the same approach to define a conditional density

  \[f_{Y \mid X}(y \mid x) = \frac{f_{XY}(x, y)}{f_X(x)} \]

• Think of conditioning on \(X = x\) as conditioning on a tiny window around \(x\): that window appears in the numerator and denominator and cancels, leaving the ratio of densities.

Factoring joint PMFs/PDFs

• As we did before with the probabilities of intersections of events, we can factor joint PMFs and PDFs into the product of conditionals

• For two discrete random variables \(X\) and \(Y\)

  \[ p_{XY}(x, y) = p_{Y \mid X}(y \mid x) \cdot p_{X}(x) = p_{X \mid Y}(x \mid y) \cdot p_Y(y)\]

• For two continuous random variables \(X\) and \(Y\)

  \[ f_{XY}(x, y) = f_{Y \mid X}(y \mid x) \cdot f_{X}(x) = f_{X \mid Y}(x \mid y) \cdot f_Y(y)\]

Factoring joint PMFs/PDFs

• This generalizes to many random variables:

  • For discrete random variables \(X_1, \dotsc, X_N\)

    \[ \begin{aligned} p_{X_1, \dotsc, X_N}(x_1, \dotsc, x_N) &= p_{X_1}(x_1) \cdot p_{X_2 \mid X_1}(x_2 \mid x_1) \cdot p_{X_3 \mid X_1, X_2}(x_3 \mid x_1, x_2) \\ &\quad \cdots p_{X_N \mid X_1, \dotsc, X_{N-1}}(x_N \mid x_1, \dotsc, x_{N-1}) \\ &= \prod_{i=1}^{N} p_{X_i \mid X_1, \dotsc, X_{i-1}}(x_i \mid x_1, \dotsc, x_{i-1}) \end{aligned} \]

• For continuous random variables \(X_1, \dotsc, X_N\)

  \[ \begin{aligned} f_{X_1, \dotsc, X_N}(x_1, \dotsc, x_N) &= f_{X_1}(x_1) \cdot f_{X_2 \mid X_1}(x_2 \mid x_1) \cdot f_{X_3 \mid X_1, X_2}(x_3 \mid x_1, x_2) \\ &\quad \cdots f_{X_N \mid X_1, \dotsc, X_{N-1}}(x_N \mid x_1, \dotsc, x_{N-1}) \\ &= \prod_{i=1}^{N} f_{X_i \mid X_1, \dotsc, X_{i-1}}(x_i \mid x_1, \dotsc, x_{i-1}) \end{aligned} \]

Marginalizing joint distributions

• We can recover a marginal distribution from a joint distribution by summing/integrating over whichever variable we want to get rid of

• To get the intuition, we can factor the joint density into a conditional and the marginal we care about

  \[f_Y(y) = \int_{-\infty}^{\infty} f_{XY}(x, y) dx = \int_{-\infty}^{\infty} f_{X \mid Y}(x \mid y) f_Y(y) dx \]

• Pull \(f_Y(y)\) out of the integral (it doesn’t depend on \(x\))

  \[f_Y(y) = f_Y(y) \int_{-\infty}^{\infty} f_{X \mid Y}(x \mid y) dx \]

• And all densities integrate to \(1\) (including conditional densities)

  \[f_Y(y) = f_Y(y) \times 1\]

Independence

• Two random variables \(X\) and \(Y\) are independent if and only if their joint CDF factors into the product of their marginal CDFs

  • Essentially the same definition as before for events!

  \[ X \indep Y \iff F_{XY}(x, y) = F_X(x) \, F_Y(y) \quad \text{for all } x, y \]

• This also implies that their joint PMFs/PDFs factor as well

  \[ p_{XY}(x, y) = p_X(x) \, p_Y(y) \qquad f_{XY}(x, y) = f_X(x) \, f_Y(y) \]

• Conditioning on one doesn’t change the distribution of the other

  • The marginal and conditional distributions are the same

  \[ p_{Y \mid X}(y \mid x) = p_Y(y) \qquad p_{X \mid Y}(x \mid y) = p_{X}(x)\] \[ f_{Y \mid X}(y \mid x) = f_Y(y) \qquad f_{X \mid Y}(x \mid y) = f_{X}(x)\]

Independence and Covariance

• Independence implies zero covariance

• First, let’s write \(\mathbb{E}[XY]\) as a double sum over the joint PMF using the law of the unconscious statistician.

  \[ \mathbb{E}[XY] = \sum_{x} \sum_{y} xy \cdot p_{XY}(x, y) \]

• Independence means the joint distribution factors into the marginals

  \[ \mathbb{E}[XY] = \sum_{x} \sum_{y} xy \cdot p_{X}(x) p_{Y}(y) \]

• \(x p_X(x)\) factors out of our sum over \(y\)

  \[ \mathbb{E}[XY] = \sum_{x}x \cdot p_{X}(x) \sum_{y} y \cdot p_{Y}(y) \]

Independence and Covariance

• Plugging in our definitions of expected value

  \[ \begin{aligned} \mathbb{E}[XY] &= \sum_{x} x \cdot p_{X}(x) \sum_{y} y \cdot p_{Y}(y) \\ &= \sum_{x} x \cdot p_{X}(x) \, \mathbb{E}[Y] \\ &= \mathbb{E}[Y] \sum_{x} x \cdot p_{X}(x) \\ &= \mathbb{E}[X] \, \mathbb{E}[Y] \end{aligned} \]

• So the covariance is zero

  \[ \Cov(X, Y) = \mathbb{E}[XY] - \mathbb{E}[X] \, \mathbb{E}[Y] = \mathbb{E}[X] \, \mathbb{E}[Y] - \mathbb{E}[X] \, \mathbb{E}[Y] = 0 \]

Conditional expectations

• We often want a summary of the conditional distribution of \(Y\) given a particular value of \(X\).

  • What is a good “prediction” for \(Y\) if I know \(X\)?

• The regression function or conditional expectation function (CEF) is one such quantity

  \[ \mu(x) = \mathbb{E}[Y \mid X = x] = \begin{cases} \displaystyle \sum_{y} y \cdot p_{Y \mid X}(y \mid x) & \text{discrete } Y \\[1em] \displaystyle \int_{-\infty}^{\infty} y \cdot f_{Y \mid X}(y \mid x) \, dy & \text{continuous } Y \end{cases} \]

• The CEF is a function of \(X\)

  • I give it a value of \(X\), it outputs me what the expected value of \(Y\) is for that value of \(X\)

• Learning about CEFs is at the heart of many social science questions!

Law of Iterated Expectations

• We can recover the marginal expectation \(\mathbb{E}[Y]\) just by applying the expectation operator to \(\mathbb{E}[Y \mid X]\)

  • But what distribution are we taking that expectation over?
  • The distribution of \(X\). Remember \(\mathbb{E}[Y \mid X]\) is a function of \(X\)

• Applying the law of the unconscious statistician

  \[ \mathbb{E}[Y] = \mathbb{E}\big[\mathbb{E}[Y \mid X]\big] = \begin{cases} \displaystyle \sum_{x} \mathbb{E}[Y \mid X = x] \cdot p_X(x) & \text{discrete } X \\[1em] \displaystyle \int_{-\infty}^{\infty} \mathbb{E}[Y \mid X = x] \cdot f_X(x) \, dx & \text{continuous } X \end{cases} \]

• This is known as the law of total expectation or law of iterated expectation

Law of Iterated Expectations

• Let’s complete the proof for the discrete case. Substitute in the definition of conditional expectation

  \[ \mathbb{E}\big[\mathbb{E}[Y \mid X]\big] = \sum_{x} \sum_{y} y \cdot p_{Y \mid X}(y \mid x) \cdot p_X(x) \]

• That conditional times marginal is just the joint distribution of \(X\) and \(Y\)

  \[ \mathbb{E}\big[\mathbb{E}[Y \mid X]\big] = \sum_{x} \sum_{y} y \cdot p_{XY}(x,y) \]

• Pull out the \(y\) from the sum over \(x\) (remember, we can swap order of summation) \[ \mathbb{E}\big[\mathbb{E}[Y \mid X]\big] = \sum_{y} y \sum_{x} p_{XY}(x,y) \]

Law of Iterated Expectations

• The sum over \(x\) is just marginalizing out \(x\) from the joint distribution of \(X\) and \(Y\) \[ \begin{aligned} \mathbb{E}\big[\mathbb{E}[Y \mid X]\big] &= \sum_{y} y \sum_{x} p_{XY}(x,y) \\ &= \sum_{y} y \sum_{x} p_{X \mid Y}(x \mid y) \cdot p_Y(y) \\ &= \sum_{y} y \cdot p_Y(y) \underbrace{\sum_{x} p_{X \mid Y}(x \mid y)}_{= 1} \\ &= \sum_{y} y \cdot p_Y(y) = \mathbb{E}[Y] \end{aligned} \]

Law of Iterated Expectations

Example: Survey weighting to recalled vote

• We want to predict the 2026 midterm elections.
  • Problem: Surveys aren’t representative and we don’t have a perfect way of dealing with non-response bias
• A newly popular strategy used by some polling firms is to weight by recalled vote
  • Imagine that the voters in 2026 are similar to 2024
  • We have a known 2024 benchmark: the observed Trump-Harris vote totals.
  • Calculate average vote choice conditional on recalled vote and average using the known 2024 distribution

Example: Weighting to recalled vote

• Recent example: The Economist/YouGov poll (September 25–28, 2026) asks respondents

  • \(X\): the respondent’s recalled 2024 vote: Trump or Harris
  • \(Y_D\): 1 if they intend to vote for the Democratic House candidate in 2026, 0 otherwise
  • \(Y_R\): 1 if they intend to vote for the Republican House candidate in 2026, 0 otherwise

• For each party’s share:

  \[ \begin{aligned} \mathbb{E}[Y_D] &= \mathbb{E}[Y_D \mid \text{Trump}] \, P(\text{Trump}) + \mathbb{E}[Y_D \mid \text{Harris}] \, P(\text{Harris}) \\ \mathbb{E}[Y_R] &= \mathbb{E}[Y_R \mid \text{Trump}] \, P(\text{Trump}) + \mathbb{E}[Y_R \mid \text{Harris}] \, P(\text{Harris}) \end{aligned} \]

Example: Weighting to recalled vote

• The Economist/YouGov crosstabs report:

  2024 vote (\(x\))	\(\mathbb{E}[Y_D \mid X = x]\)	\(\mathbb{E}[Y_R \mid X = x]\)	\(P(X = x)\), raw sample	\(P(X = x)\), 2024 result
  Trump	0.07	0.80	0.48	0.51
  Harris	0.90	0.01	0.52	0.49

• There are differences in how well each party is holding their voters in 2026

  • 0.90 of Harris voters back the Democratic house candiate, but only 0.80 of Trump voters back the Republican house candidate
  • Trump voters defect to the Democratic candidate (0.07) far more than Harris voters defect to the Republican (0.01)

• But the raw sample over-represents Harris voters (0.52 vs. 0.49 in the actual result)

Example: Weighting to recalled vote

• Averaging over the 2024 result

  \[ \begin{aligned} \mathbb{E}[Y_D] &= 0.07 (0.51) + 0.90 (0.49) = 0.477 \\ \mathbb{E}[Y_R] &= 0.80 (0.51) + 0.01 (0.49) = 0.413 \end{aligned} \]

• The margin, \(\mathbb{E}[Y_D] - \mathbb{E}[Y_R]\): the Democratic candidate leads by 6 points

  • The actual poll has it at \(45\%\) to \(37\%\) because they’re also including 2024 non-voters who are leaning Democratic.

• Survey weighting is just the art of figuring out which distribution of \(X\) to average over.

  • In Week 13, we’ll talk about how to actually do estimation

The mean minimizes mean squared error

• Why do we focus so much on the expected value when thinking about prediction - why is it our “best guess”?
  • It turns out we can formalize the notion of “best guess” with respect to a particular error: the mean squared error (MSE)
• Let \(c\) be our single guess. The mean squared error between \(Y\) and \(c\) is defined as \(\mathbb{E}[(Y - c)^2]\)
  • We want to find the value of \(c\) that minimizes this quantity.

The mean minimizes mean squared error

• Add and subtract \(\mathbb{E}[Y]\) and expand the square

  \[ \begin{aligned} \mathbb{E}[(Y - c)^2] &= \mathbb{E}\big[\big((Y - \mathbb{E}[Y]) + (\mathbb{E}[Y] - c)\big)^2\big] \\ &= \mathbb{E}[(Y - \mathbb{E}[Y])^2] + 2(\mathbb{E}[Y] - c)\,\mathbb{E}[Y - \mathbb{E}[Y]] + (\mathbb{E}[Y] - c)^2 \\ &= \Var(Y) + (\mathbb{E}[Y] - c)^2 \end{aligned} \]

• The middle term drops out because \(\mathbb{E}[Y - \mathbb{E}[Y]] = 0\)

• \(\Var(Y)\) doesn’t depend on \(c\), and \((\mathbb{E}[Y] - c)^2 \ge 0\), so the MSE is smallest at \(c = \mathbb{E}[Y]\)

  • The best constant prediction with respect to MSE is the mean, and the smallest achievable MSE is the variance

The CEF minimizes the MSE

• Now, instead of a single constant \(c\), we can plug in a function of \(X\), \(g(X)\)

  • We can show that the function \(g(X)\) that does the best job is the CEF.

• The MSE between \(Y\) and our function is \(\mathbb{E}[(Y - g(X))^2]\) and we’re searching over the space of all possible functions \(g()\)

• We’ll use a similar trick from before: add and subtract the CEF \(\mu(X) = \mathbb{E}[Y \mid X]\) and expand the square:

  \[ \begin{aligned} \mathbb{E}[(Y - g(X))^2] &= \mathbb{E}[(Y - \mu(X))^2] + 2\,\mathbb{E}[(Y - \mu(X))(\mu(X) - g(X))] \\ &\quad + \mathbb{E}[(\mu(X) - g(X))^2] \end{aligned} \]

The CEF minimizes the MSE

• The middle term is zero: by the law of total expectation, condition on \(X = x\) first. Given \(X = x\), \(\mu(x) - g(x)\) is just a constant

  \[ \mathbb{E}[(Y - \mu(x))(\mu(x) - g(x)) \mid X = x] = (\mu(x) - g(x))\underbrace{\big(\mathbb{E}[Y \mid X = x] - \mu(x)\big)}_{=0} = 0 \]

• So for any \(g\)

  \[ \mathbb{E}[(Y - g(X))^2] = \mathbb{E}[(Y - \mu(X))^2] + \underbrace{\mathbb{E}[(\mu(X) - g(X))^2]}_{\ge 0} \ge \mathbb{E}[(Y - \mu(X))^2] \]

• The CEF is the best predictor of \(Y\) given \(X\) in MSE.

The best linear predictor

• The CEF can take any shape, which makes it hard to summarize when \(X\) is continuous or high-dimensional.

  • Often we have hypotheses about a rate of change in an outcome as an independent variable increases or decreases
  • We want to estimate a single number to summarize an (average) rate of change.

• What if we restrict \(g(X)\) to be a line, \(g(X) = a + bX\)?

• The best linear predictor (BLP) is the line with the smallest MSE

  \[ (\alpha, \beta) = \arg\min_{a,\, b} \ \mathbb{E}\big[(Y - a - bX)^2\big] \]

• We are searching over a smaller space of options

  \[ \mathbb{E}\big[(Y - \mu(X))^2\big] \le \mathbb{E}\big[(Y - \alpha - \beta X)^2\big] \]

The best linear predictor

• Let’s solve this optimization problem by taking the derivatives with respect to \(a\) and \(b\) and solving for \(0\)

• Derivative passes inside the expectation. Then applying the chain rule…

  \[ \begin{aligned} \frac{\partial}{\partial a} \mathbb{E}\big[(Y - a - bX)^2\big] &= \mathbb{E}\big[2(Y - a - bX) \cdot (-1)\big] = -2\,\mathbb{E}[Y - a - bX] \\ \frac{\partial}{\partial b} \mathbb{E}\big[(Y - a - bX)^2\big] &= \mathbb{E}\big[2(Y - a - bX) \cdot (-X)\big] = -2\,\mathbb{E}\big[X(Y - a - bX)\big] \end{aligned} \]

• Set both to \(0\) at \((a, b) = (\alpha, \beta)\) and divide both sides by \(-2\)

  \[ \mathbb{E}[Y - \alpha - \beta X] = 0 \qquad \mathbb{E}\big[X(Y - \alpha - \beta X)\big] = 0 \]

The best linear predictor

• This gives the first-order conditions for the optimization problem

  \[ \mathbb{E}[Y - \alpha - \beta X] = 0 \qquad \mathbb{E}\big[X(Y - \alpha - \beta X)\big] = 0 \]

• The first gives the intercept: \(\alpha = \mathbb{E}[Y] - \beta\,\mathbb{E}[X]\)

• Plug it into the second

  \[ \begin{aligned} 0 &= \mathbb{E}[XY] - \alpha\,\mathbb{E}[X] - \beta\,\mathbb{E}[X^2] \\ &= \mathbb{E}[XY] - \mathbb{E}[X]\,\mathbb{E}[Y] - \beta\big(\mathbb{E}[X^2] - \mathbb{E}[X]^2\big) \\ &= \Cov(X, Y) - \beta \Var(X) \end{aligned} \]

• So the BLP is

  \[ \beta = \frac{\Cov(X, Y)}{\Var(X)} \qquad \alpha = \mathbb{E}[Y] - \beta\,\mathbb{E}[X] \]

The best linear predictor

• Recall that for any function \(g\)

  \[ \mathbb{E}[(Y - g(X))^2] = \mathbb{E}[(Y - \mu(X))^2] + \mathbb{E}[(\mu(X) - g(X))^2] \]

• The first term doesn’t depend on \(g\). So among lines, minimizing the MSE for \(Y\) is the same as minimizing the distance to the CEF

  \[ (\alpha, \beta) = \arg\min_{a,\, b} \ \mathbb{E}\big[(\mu(X) - a - bX)^2\big] \]

  • The BLP is the best linear approximation to the CEF
  • If the CEF is linear, the BLP is the CEF

• Like the CEF, the BLP is a feature of the joint distribution: an estimand

• For our Clinton vote share example: \(\beta \approx -0.46\).

  • Each additional increase in county % non-Hispanic white is associated with about half a point reduction in Clinton vote share

The BLP is a summary

Bonus: The law of total variance

• We can also decompose the variance of \(Y\) by conditioning on \(X\)

  \[ \Var(Y) = \underbrace{\mathbb{E}\big[\Var(Y \mid X)\big]}_{\text{within}} + \underbrace{\Var\big(\mathbb{E}[Y \mid X]\big)}_{\text{between}} \]

• Within: the average spread of \(Y\) around the CEF inside each slice of \(X\)

• Between: how much the CEF itself moves as \(X\) changes. The part of the variation in \(Y\) “explained” by \(X\)

• In the recalled vote example (\(Y_D\) is binary, so \(\Var(Y_D \mid X = x) = \mu(x)(1 - \mu(x))\))

  • Within: 0.077, between: 0.172, total: \(\Var(Y_D) =\) 0.249
  • Recalled 2024 vote accounts for about 69% of the variance in 2026 House vote preference

Bonus: The law of total variance

• Start from \(\Var(Y \mid X) = \mathbb{E}[Y^2 \mid X] - \mathbb{E}[Y \mid X]^2\) and take expectations of each piece, using the law of iterated expectations

  \[ \begin{aligned} \mathbb{E}\big[\Var(Y \mid X)\big] &= \mathbb{E}\big[\mathbb{E}[Y^2 \mid X]\big] - \mathbb{E}\big[\mathbb{E}[Y \mid X]^2\big] = \mathbb{E}[Y^2] - \mathbb{E}\big[\mathbb{E}[Y \mid X]^2\big] \\ \Var\big(\mathbb{E}[Y \mid X]\big) &= \mathbb{E}\big[\mathbb{E}[Y \mid X]^2\big] - \mathbb{E}\big[\mathbb{E}[Y \mid X]\big]^2 = \mathbb{E}\big[\mathbb{E}[Y \mid X]^2\big] - \mathbb{E}[Y]^2 \end{aligned} \]

• Add the two lines: the \(\mathbb{E}\big[\mathbb{E}[Y \mid X]^2\big]\) terms cancel

  \[ \mathbb{E}\big[\Var(Y \mid X)\big] + \Var\big(\mathbb{E}[Y \mid X]\big) = \mathbb{E}[Y^2] - \mathbb{E}[Y]^2 = \Var(Y) \]

• Since both pieces are non-negative, conditioning never adds variance: \(\Var\big(\mathbb{E}[Y \mid X]\big) \le \Var(Y)\)

Next week

• What is your estimand?
  • How should we connect theory to empirical questions?
  • Thinking of estimands as features of distributions.
  • Preview of descriptive versus causal estimands
• Midterm exam
  • Probability questions in the context of actual research settings.
  • Focus on properties of conditional probability, properties of expectations.