Polls shape headlines, strategy and public debate. But the quality of polls varies enormously, and a number with a decimal point is not automatically accurate. Whether you are commissioning research or reading someone else’s, these are the questions to ask.
1. Who was surveyed, and how were they chosen?
A credible poll clearly describes its population, such as all adults, registered voters or likely voters, and how respondents were selected. Self-selected online votes are not polls in the research sense.
2. How many people responded?
Larger samples reduce random error, but size alone does not guarantee quality. A well-designed sample of several hundred can be more reliable than a poorly designed sample of many thousands.
3. How were the interviews done?
Phone, online panels, text and mixed approaches each have strengths and weaknesses. Good pollsters disclose the mode and explain how they reach people who are harder to contact.
4. Were the results weighted?
Weighting adjusts the sample to match the population on characteristics such as age, gender, education and region. Ask what the results were weighted to and why.
5. What exactly was asked?
Question wording and order can change results. Credible pollsters publish the full question text so readers can judge for themselves whether it was neutral.
6. What is the margin of error, and what does it mean?
The margin of error describes sampling uncertainty only. It does not account for problems with wording, coverage or who chose not to respond. Small differences inside the margin are not meaningful.
A simple checklist
- Sponsor and pollster are named
- Population, dates and sample size are stated
- Method and weighting are disclosed
- Full question wording is available
- Results are reported with appropriate caution
Polls that meet these standards deserve attention. Polls that do not should be treated with care, however striking the headline.
Need help with this? Talk to Creative Contact Consulting Group about your situation.