Monday, Nov 3
Presser, S., Mick P. Couper, J.T. Lessler, E. Martin, J. Martin, J.M. Rothgeb, and Eleanor Singer. 2004. "Methods for Testing and Evaluating Survey Questions." Public Opinion Quarterly, 68(1): 109-130.
A growing awareness of conventional pretesting's drawbacks has led to two interrelated changes. First, there has been a subtle shift in the goals of testing, from an exclusive focus on identifying and fixing overt problems experienced by interviewers and respondents to a broader concern for improving data quality so that measurements meet a survey's objectives. Second, new testing methods have been developed or adapted from other uses. The development of these methods raises issues of how they might best be used in combination, as well as whether they in fact lead to improvements in survey measurement. Four general recommendations seem particularly important to us for advancing questionnaire testing and evaluation. These involve: 1. the connection between problem identification and measurement error, 2. the impact of testing methods on survey costs, 3. the role of basic research and theory in guiding the repair of question flaws, and 4. the development of a data base to facilitate cumulative knowledge.