Questions - Heart of All Polls - continued
Avoiding problem questions
- Pre-testing is an important way to discover and correct problem questions
- Split samples--asking half the respondents version #1 and half the respondents version #2 of a question to test for the effects of different question wording
- Use a self-administered, rather than an interviewer-administered survey instrument for topics that explore personal or socially unacceptable behavior, e.g.,
- Illegal activity
- Embarrassing situations
- Condom use
- Drug use
- Number of sex partners
- Unplanned or unwanted pregnancies
- Sexually transmitted diseases
Types of questions
Because survey questions can serve many different functions, they come in a wide variety of forms:
- Filter - e.g., asking respondents if they know who Mr. X is before asking their opinion of him
- Closed ended - respondents select from response categories, e.g., which of the following do you favor?
- Open-ended - respondents are not offered response categories, e.g., what do you think is the most important problem facing the nation?
- Measure factual or objective data - e.g., how often do you go out to eat at a restaurant?
- Measure subjective states - e.g., how strongly do you support change?
- Estimate magnitude - e.g., how big a problem do you think that is?
- Rank ratings - e.g., what is your favorite college football team?
When they agree to take part in a survey people understand that they are expected to respond to questions. Refusing to answer questions would be impolite, and saying "I don't know" or "I've never given that any thought" can be embarrassing. Better just to make up something. Pollsters refer to these as non-attitudes, or doorstep opinions. The respondent has never given a moment's thought to the issue, but now that a question has been asked will give some kind of response. It is easier that way. Besides, who is to say that is not really the respondent's opinion? One way to determine whether respondents ever fabricate answers is to ask their opinion about a fictitious person. When that has been done, pollsters have found that rather than admit they have never heard of the person, many respondents solemnly declare that they do or do not like him.
A method commonly used to avoid this problem as much as possible is the screening question. Studies have shown that most people will take advantage of an escape route if they are provided one. Instead of asking what a respondent thinks of Senator Snort, the interviewer first asks the respondent if he or she has ever heard of Senator Snort. Only those who have heard of the good senator are then asked their opinion of him. For issues that require the respondent to have a certain level of knowledge about people or policies, screening questions can be effective for reducing the number of doorstep opinions.