How Big Data Triumphs Over Gut InstinctsO
However you may feel about the outcome of Tuesday's Presidential election, one thing is clear:
Big Data triumphed over Gut Instincts.
This is not a post about policy disagreements or who had the best vision for America's future. This is a post about the ongoing rise of Big Data and the People who embrace it.
It's also a post about the difficulty of objective analysis when the information you're analyzing is contrary to your beliefs and desires.
The lead-up to the election brought prognosticators out of the woodwork. Countless pundits and "experts" weighed on on what they believed would happen. For the most part, they fit into two camps:
- Talking Heads
- Stat Heads
Talking Heads could be found on any number of cable news shows. They based their electoral predictions on their gut instincts. On flighty ideas like "momentum," and their opinion on "the mood of the country." One well-respected pundit claimed that the "vibrations were right" for a Romney win, whatever that even means. They might throw a few numbers out there, like unemployment figures or the debt, but they'd rarely look to opinion polls. Unless of course they could cherry pick polls in their favor.
Stat Heads could mainly be found on the internet. They based their electoral predictions not on gut feelings or nebulous concepts, but on unbiased analysis of hard data. They didn't ignore the polls, they studied them and ran them through models. Their analysis delivered probabilities that matched up extraordinarily well with the election's actual results.
What was clear to a clear-minded outsider was that the Talking Heads almost exclusively made predictions that matched their desired outcome. They weren't conducting well-thought out analyses so much as they were trying desperately to convince the world that what they wanted to believe was actually the truth.
To be clear, this happened on both sides of the aisle. While folks like Dick Morris were the worst of the Talking Heads, this sort of thing happened with the left wing pundits of the world after polls starting coming out post-Debate #1.
Watching all of this reminded me of my post on the baseball MVP race, which pits Triple Crown winner Miguel Cabrera against once-in-a-lifetime rookie Mike Trout. As I see it, the talking heads who were ranting and raving about nebulous gut feelings and "momentum" while ignoring key data are no different than the dinosaurs of baseball who a player on his RBI total while scoffing at advanced statistics like WAR and OPS.
What I wonder is, will this sort of non-sense continue in future elections? Now, I obviously don't think Talking Heads are going to just fall off the face of the Earth. The voting public at large is much less informed about election issues than a big time baseball fan is about his sport. But, could we perhaps see a hybrid of sorts going forward?
In Nate Silver's book, The Signal and the Noise, he dives into the post-Moneyball era of baseball and points out something incredibly interesting. While stats and deep data analysis is paramount to a modern club's success, it has yet to kill off scouting. In fact, at teams like the Oakland A's, scouting is more important than ever. Only through a hybrid of soft and hard analysis can teams consistently and reliably forecast players' expected abilities and contributions. And only through that combination can teams consistently build out a winning ball club.
My question for WSO is, will we see this sort of hybrid approach play a bigger role in political coverage going forward? Or are we permanently doomed to overblown horse-race reporting that ignores the data and fails to report on the issues?