Promoting uniform question understanding in today's and tomorrow's surveys

Survey respondents misunderstand questions more frequently than one might expect but, current methods for collecting data make it hard to detect and correct misunderstanding. The conventional practice has been to leave the interpretation of questions up to respondents; interviewers react to requests for clarification with nondirective probes like "Let me repeat the question." The current article reviews a research program that has explored alternatives to standardized wording, in which interviewers and web survey systems can define survey concepts as needed as a way to assure uniform comprehension across respondents. One problem is that many respondents fail to recognize that their understanding is not aligned with the survey sponsors' and so do not ask for clarification - a problem that, we argue, is more serious in the survey response task than other tasks in which information is exchanged. Using today's survey techniques (telephone and face-to-face interviews, web surveys) it is possible to increase respondents' sensitivity to their own misunderstanding, increasing their requests for clarification; and, based on respondents' verbal and visual cues of comprehension difficulty, it is possible to intervene to correct misunderstanding. This approach can be extended in surveys of the future by incorporating mature speech recognition capabilities, modeling respondent uncertainty about question meaning so that when clarification is needed it can be provided automatically, and developing interface agents when appropriate. By evaluating simulated versions of these technologies in the near term researchers will be better able exploit them as they become available.

Keywords: