We discovered an issue with how our primary model was making state-by-state and district-by-district forecasts. Specifically, the model was not properly calculating the demographic regressions that we use as a complement to the polls.
This article isn't interesting because of the specific issue that FiveThirtyEight identified or how it was impacting their results. In fact the article itself is...kind of boring. So why link to it?
I think it's fascinating that this article was published at all. This is a well-known journalistic organization publishing a correction based on the functioning of a predictive model, and going deep (in public!) about what exactly the issue was and how it was caused. I personally haven't seen anything like this before. It's a testament to the unique culture at FiveThirtyEight and the uniquely data-driven bent of their readers that they have a space where this type of article can find its way to the front page.
Predictive models underly much of our understanding of the world today, and govern our lives in increasingly important ways. This type of transparency around results (and incorrect results) isn't just a technology challenge (model explainability), it's a cultural one. This type of transparency should be recognized and rewarded.Read more...