As the 2012 presidential election is upon us, we’ve seen plenty of poll data on the status of the race. These polls are part of American political tradition dating back to 1824 when The Harrisburg Pennsylvanian conducted a local poll showing Andrew Jackson leading John Quincy Adams in the the contest for President.
Fast forward 188 years to today where there are hundreds of polls with new data being released daily. Problem is, most consumers aren’t aware of the details that can make or break the validity of a political poll. The media adds to the confusion by often reporting the results only with little explanation as to the methods used to conduct the poll.
In an effort to eliminate the confusion behind political polls, I have compiled a simple list of five simple questions you should ask yourself when evaluating the validity of any political poll.
1. Who conducted and funded the poll?
This is perhaps the most important question to ask yourself when evaluating the validity of a poll. It is important to remember that political polls are conducted for a reason. Knowing who is funding and conducting the poll will help you identify any potential biases. Reputable polls will offer this provide you with the information you need to evaluate the validity of the results, including who conducted the research and who paid the bill. The best advice is determining validity based on who conducted or funded a poll is to simply use commons sense. Polls funded by news outlets are more reputable than those by political candidates or special interest groups.
2. Who was polled?
The audience used as the sample is crucial to understanding the context of poll results. Consider whether the sample drawn from all U.S. adults, or was it from a specific region, state or city? Perhaps the sample was made up of members of a certain group or association. Pollsters also tend qualify their samples using terms such as “likely voters” or “undecided voters.” If this is the case, be sure find out how the polling administrators came to the determination that these individuals were indeed qualified to be apart of that group. The most important thing to remember is to make sure the administrators found the most appropriate audience for their poll.
3. What was the sample size and margin of error?
The margin of error in a poll is a statistical formulation of how well the sample reflects the general population. The margin of error can be made smaller by polling more people. A poll involving only a few hundred people may have a large margin of error, and a poll with thousands of people will have a small margin of error. Political polls by professional polling agencies are designed to poll just enough people to get a somewhat small margin of error (usually two or three percent). To poll more people would be expensive, and to poll fewer people would mean less accurate results. This is important because the results of a poll may tell one story without the margin of error, and another with it. For example, if candidate one is leading the polls against candidate two by 51 percent to 49 percent, one might think this bodes well for candidate one. However, if there is a three percent margin of error, then there is a substantial chance that the population as a whole supports candidate one as little as 48 percent, and candidate two as much as 52 percent. A more accurate description of the race is that the two candidates are neck in neck.
4. When was the poll conducted in relation to the publication of its results?
Timing is everything in political polls. Interpretation of the results is dependent on when the poll was conducted relative to key events that might impact opinions. Pay special attention to the timeframe between collection, publishing and the events in between that might influence results. Avoid polls that are more than a month old and make sure the time between data collection and publication is short.
5. How was the data gathered?
Polls are usually conducted either in-person, online, by telephone or mail. Most national polls are conducted via telephone using random number dialing technology. Each of these methods has their own pitfalls and subsequent biases, but no method is perfect. Be wary of polls where data was collected in shopping malls, in stores or on the sidewalks and remember that reputable polls will always offer information as to how the data was collected.
There you have it. Five easy questions that will provide a solid foundation for evaluating the validity of political polls. Have some advice of your own? Please add them below in the comments section.