In June of 2014, then House Majority Leader Eric Cantor lost his reelection bid in the Republican primary in spite of being heavily favored. Polls showed him with a double digit lead days before the election. That was the first major signal that political pollsters needed to change their approach. Unfortunately, due to a variety of issues, they kicked the can down the road.
After the surprise of 2016 and further tumult around the 2020 election, it’s time for the industry to take a hard look at itself and make adjustments in the interest of fairness and accuracy. To fix political polling, the industry needs to focus on four aspects of the process:
- Research objective
- Survey design
- Use and promotion
Any research needs to reflect back on what you are trying to learn. With political polls, the goal should be to understand the pulse of the voter to help educate candidates on where they stand. The goal should not be to identify the winner. Why? The intent of quantitative research (really any research) isn’t to provide definitive answers. Rather, its intent is to provide information about the current environment to help make better decisions. In the case of polling – it helps determine if a candidate’s strategy needs to shift to accommodate the voting public.
Political polling is used by candidates to understand where they stand in the public eye. Polls need to be more focused on understanding what the public thinks as opposed to definitively identifying a winner. When they attempt to do this, recent history shows polls are inaccurate and can prove misleading. Even to the point of potentially discouraging some people from voting. (i.e. My candidate doesn’t stand a chance. Why go vote?)
Polls often simplify what is a surprisingly complex issue. If the election were held today, who would you vote for? The catch is, the election isn’t being held that day. Respondents aren’t in the voting booth. Views change, votes swing. Especially in high profile campaigns.
Take Jamal and Jane (names changed, a real example), a husband and wife who talked about their plans to vote in the presidential election. Just prior to the first debate between Trump and Biden, Jamal was uncertain. He wanted to see how Biden performed because he was concerned with his age and ability to manage the office. Following the second debate, he was comfortable voting for Biden. Jane did not like Biden, was hesitant about Trump’s demeanor but liked his policies. She was going to vote for Trump. On Election Day, Jamal votes for Biden, Jane voted for neither candidate. “I couldn’t bring myself to vote for either of them.”
In a poll, Jane would have been counted as a Trump voter because polling questions typically force a decision. Instead, polls should ask preference using a scale rather than an absolute approach to questioning.
Take a recent Wall Street Journal NBC News poll and how they approached acceptance of a COVID-19 vaccine. While others asked yes or no questions about taking a vaccine, the WSJ/NBC poll approached it this way:
When a vaccine for the coronavirus is available, which best describes what you will do?
- Wait until it’s been available for a while (50%)
- Get the vaccine as soon as possible (20%)
- Not willing to get the vaccine (17%)
- Only willing to get the vaccine if required (10%)
- Unsure (3%)
This approach recognizes that decisions are not made in absolutes. The same can be said about voting preferences. A better approach to identifying candidate preferences would be:
If the election were held today, who would you vote for?
- Definitely will vote for Candidate A
- Probably will vote for Candidate A
- Probably will vote for Candidate B
- Definitely will vote for Candidate B
This approach provides us not only with the strength of each candidate’s backing, but also the likelihood that some voters are still in play. It could also help minimize the reluctance of some voters to express their true voting intent to pollsters.
Normally a question like this would allow for a midpoint. You might also allow an ‘out’ where respondents could say they won’t vote for either candidate. But our need to know how people are currently thinking suggests a forced preference is more appropriate. All of these design issues assume the poll avoids biased questioning techniques that drive results in a preferred direction.
Here’s the dirtiest secret that everyone in the polling industry knows, but people outside the industry don’t want to hear about. There is no such thing as a truly representative sample anymore. The days of everyone answering the telephone are 40 years past.
Some polls rely on online surveys conducted with paid panelists. These panels are reliable and have improved in quality, yet are far from representative. Some political pollsters make phone calls. Each sampling approach leaves out critical elements of the population.
Are rural voters fully represented in online polls? How about older Americans who are more likely to vote than their younger counterparts? Online polling tends to skew higher income, younger, female and more educated. Telephone data collection, even when cell phone sample is included, skews older. As a result, adjustments are made to accommodate the differences. The more you adjust (weight) the data, the greater the risk there is to providing potentially misleading insight.
The best approach would be to rely on mixed data collection using both phone and online panels. This is more expensive and takes more time but would allow for a blended and balanced respondent base that comes closer to being representative of the broader population.
Use and Promotion
The most difficult thing to do is communicate the polling data properly. The news media plays a significant role in how we learn about and absorb these polls. They have a great responsibility to be unbiased and present reliable information. (Just ask the Chicago Daily Tribune about what happens when you get it wrong after the paper incorrectly announced Thomas Dewey’s victory over President Harry Truman on their front page.)
Presenting credible polling data to the public – that stays away from identifying a likely winner before a vote is even held – will help increase trust in the news industry at a time when the term ‘fake news’ has become part of our lexicon.
For the past several years following major elections, there is inevitable finger pointing, arguing and hand wringing about how the polls were wrong. Then the industry and news media fall back on the same approach and are surprised when credibility is questioned. It’s time for a new approach that recognizes that the role of polls should be to provide information and insight into what voters are thinking and not to tell voters (and candidates) who is likely to win before a vote is held.