Question Difficulty and Respondents' Cognitive Ability: The Impact on Data Quality

There is increasing evidence that reductions in cognitive functioning can be negatively related to the quality of survey data. Research also indicates that the accuracy and completeness of reports decrease with increasing question difficulty. In the present paper the interaction of question difficulty with respondents' cognitive ability is investigated. It was expected that respondents with relatively low information processing ability are less able than those with higher ability to provide complete and accurate answers when responding to particularly difficul questions. The number of "don't know"-responses in a survey conducted with people over 70 years of age was used as an indicator of reduced data quality. The data were analyzed as a function of question difficulty and respondents' cognitive ability. The findings demonstrate the expected interaction: Respondents lower in cognitive ability were particularly likely to answer "don't know" to difficult questions. Respondents higher in cognitive ability were less affected by variations in question difficulty. The selective loss of data due to failures to respond may bias surveys findings because it leads to an underrepresentation of responses of lower cognitive ability respondents in the data. Implications for conducting surveys with lower cognitive ability respondents are discussed.