Question Design
How the wording, order, and framing of poll questions can dramatically affect the results, and how to detect biased questions.
Words Matter Enormously
Small changes in question wording can produce dramatically different results. A classic experiment by researchers Rugg and later Schuman and Presser found that about 62% of Americans supported 'not allowing' public speeches against democracy, but only about 46% supported 'forbidding' such speeches, even though the two phrases mean the same thing. The word 'forbid' sounds more authoritarian, triggering resistance. That is a 16-point swing produced entirely by one word.
In political polling, framing effects are pervasive. Asking 'Do you support the Affordable Care Act?' produces different results than 'Do you support Obamacare?', even though they are the same law; polls in the 2010s repeatedly found several points of difference. Questions that mention 'government spending' poll worse than those describing the same programs as 'investment.' Pollsters must navigate these effects carefully, and partisan polls exploit them deliberately.
