20 Questions A Journalist Should Ask About Poll Results - continued
10. What about polls on the Internet or World Wide Web?
The explosive growth of the Internet and the World Wide Web has given rise to an equally explosive growth in various types of online polls and surveys.
Online surveys can be scientific if the samples are drawn in the right way. Some online surveys start with a scientific national random sample and recruit participants while others just take anyone who volunteers. Online surveys need to be carefully evaluated before use.
Several methods have been developed to sample the opinions of those who have online access. The fundamental rules of sampling still apply online: the pollster must select those who are asked to participate in the survey in a random fashion. In those cases where the population of interest has nearly universal Internet access or where the pollster has carefully recruited from the entire population, online polls are candidates for reporting.
However, even a survey that accurately sampled all those who have access to the Internet would still fall short of a poll of all Americans, as about one in three adults do not have Internet access.
But many Internet polls are simply the latest variation on the pseudo-polls that have existed for many years. Whether the effort is a click-on Web survey, a dial-in poll or a mail-in survey, the results should be ignored and not reported. All these pseudo-polls suffer from the same problem: the respondents are self-selected. The individuals choose themselves to take part in the poll—there is no pollster choosing the respondents to be interviewed.
Remember, the purpose of a poll is to draw conclusions about the population, not about the sample. In these pseudo-polls, there is no way to project the results to any larger group. Any similarity between the results of a pseudo-poll and a scientific survey is pure chance.
Clicking on your candidate's button in the "voting booth" on a Web site may drive up the numbers for your candidate in a presidential horse-race poll online. For most such efforts, no effort is made to pick the respondents, to limit users from voting multiple times or to reach out for people who might not normally visit the Web site.
The dial-in or click-in polls may be fine for deciding who should win on American Idol or which music video is the MTV Video of the Week. The opinions expressed may be real, but in sum the numbers are just entertainment. There is no way to tell who actually called in, how old they are, or how many times each person called.
Never be fooled by the number of responses. In some cases a few people call in thousands of times. Even if 500,000 calls are tallied, no one has any real knowledge of what the results mean. If big numbers impress you, remember that the Literary Digest's non-scientific sample of 2,000,000 people said Landon would beat Roosevelt in the 1936 Presidential election.
Mail-in coupon polls are just as bad. In this case, the magazine or newspaper includes a coupon to be returned with the answers to the questions. Again, there is no way to know who responded and how many times each person did.
Another variation on the pseudo-poll comes as part of a fund-raising effort. An organization sends out a letter with a survey form attached to a large list of people, asking for opinions and for the respondent to send money to support the organization or pay for tabulating the survey. The questions are often loaded and the results of such an effort are always meaningless.
This technique is used by a wide variety of organizations from political parties and special-interest groups to charitable organizations. Again, if the poll in question is part of a fund-raising pitch, pitch it—in the wastebasket.