20 Questions A Journalist Should Ask About Poll Results - continued
8. When was the poll done?
Events have a dramatic impact on poll results. Your interpretation of a poll should depend on when it was conducted relative to key events. Even the freshest poll results can be overtaken by events. The President may have given a stirring speech to the nation, pictures of abuse of prisoners by the military may have been broadcast, the stock market may have crashed or an oil tanker may have sunk, spilling millions of gallons of crude on beautiful beaches.
Poll results that are several weeks or months old may be perfectly valid, but events may have erased any newsworthy relationship to current public opinion.
9. How were the interviews conducted?
There are four main possibilities: in person, by telephone, online or by mail. Most surveys are conducted by telephone, with the calls made by interviewers from a central location. However, some surveys are still conducted by sending interviewers into people's homes to conduct the interviews.
Some surveys are conducted by mail. In scientific polls, the pollster picks the people to receive the mail questionnaires. The respondent fills out the questionnaire and returns it.
Mail surveys can be excellent sources of information, but it takes weeks to do a mail survey, meaning that the results cannot be as timely as a telephone survey. And mail surveys can be subject to other kinds of errors, particularly extremely low response rates. In many mail surveys, many more people fail to participate than do. This makes the results suspect.
Surveys done in shopping malls, in stores or on the sidewalk may have their uses for their sponsors, but publishing the results in the media is not among them. These approaches may yield interesting human-interest stories, but they should never be treated as if they represent public opinion.
Advances in computer technology have allowed the development of computerized interviewing systems that dial the phone, play taped questions to a respondent and then record answers the person gives by punching numbers on the telephone keypad. Such surveys may have a variety of significant problems, including uncontrolled selection of respondents within the household, the ability of young children to complete the survey and poor response rates. These problems disqualify this type of survey unless the journalist knows that the survey has proper respondent selection, verifiable age screening, and reasonable response rates.