20 Questions A Journalist Should Ask About Poll Results - continued
11. What is the sampling error for the poll results?
Interviews with a scientific sample of 1,000 adults can accurately reflect the opinions of nearly 210 million American adults. That means interviews attempted with all 210 million adults—if such were possible—would give approximately the same results as a well-conducted survey based on 1,000 interviews.
What happens if another carefully done poll of 1,000 adults gives slightly different results from the first survey? Neither of the polls is "wrong." This range of possible results is called the error due to sampling, often called the margin of error.
This is not an "error" in the sense of making a mistake. Rather, it is a measure of the possible range of approximation in the results because a sample was used.
Pollsters express the degree of the certainty of results based on a sample as a "confidence level." This means a sample is likely to be within so many points of the results one would have gotten if an interview were attempted with the entire target population. Most polls are usually reported using the 95% confidence level.
Thus, for example, a "3 percentage point margin of error" in a national poll means that if the attempt were made to interview every adult in the nation with the same questions in the same way at the same time as the poll was taken, the poll's answers would fall within plus or minus 3 percentage points of the complete count's results 95% of the time.
This does not address the issue of whether people cooperate with the survey, or if the questions are understood, or if any other methodological issue exists. The sampling error is only the portion of the potential error in a survey introduced by using a sample rather than interviewing the entire population. Sampling error tells us nothing about the refusals or those consistently unavailable for interview; it also tells us nothing about the biasing effects of a particular question wording or the bias a particular interviewer may inject into the interview situation. It also applies only to scientific surveys.
Remember that the sampling error margin applies to each figure in the results—it is at least 3 percentage points plus or minus for each one in our example. Thus, in a poll question matching two candidates for President, both figures are subject to sampling error.