Applied Math Seminar: Evan Rosenman (CMC)

Title:  Recalibration of Predicted Probabilities Using the “Logit Shift”: Why Does It Work, and When Can It Be Expected to Work Well?

Abstract: In the context of election analysis, researchers frequently face the “recalibration problem.” That is: they must reconcile individual-level vote probabilities, modeled prior to the election, with vote totals observed in each precinct once the election has taken place. Making these adjustments such that the probabilities match known aggregates, researchers can obtain better-calibrated estimates of key quantities such as vote preferences among subgroups of the electorate defined by race, age, and gender.

We provide theoretical grounding for one of the most commonly used recalibration strategies, known colloquially as the “logit shift.” The logit shift is a heuristic adjustment, in which a constant correction on the logit scale is found, such that aggregated predictions match observed totals.

We show that the logit shift offers a fast and accurate approximation to a principled, but computationally impractical adjustment strategy: computing the posterior probabilities, conditional on the observed totals. After deriving analytical bounds on the quality of the approximation, we illustrate its accuracy using Monte Carlo simulations. We also discuss scenarios in which the logit shift is less effective at recalibrating predictions: when the totals are available only for highly heterogeneous populations, and when the original predictions correctly capture the mean of true individual probabilities, but fail to capture the shape of their distribution.