Monday, April 21
Grant Miller: Managerial Incentives in Public Service Delivery
Lee, Sunghee, Nancy A. Mathiowetz, and Roger Tourangeau. 2007. "Measuring Disability in Surveys: Consistency Over Time and Across Respondents." Journal of Official Statistics, 23(2): 163--184.
We conducted an experiment that compared different versions of a set of disability questions (including the questions included on the Census 2000 Long Form). Disability items are prone to a number of methodological problems, including inconsistency over time and self-proxy differences. Our experimental versions were designed to reduce these problems. The different versions constituted attempts to simplify the questions in various ways; in addition, some of the questions used a five-point response scale rather than the yes-no format commonly used in survey items on disability. We also varied whether the data were collected from the sample persons themselves or from a proxy from the same household. We administered the questions in a telephone interview to a national sample of households that included at least two adults 40 years old or older. In each cooperating household, we selected two adults in this age range and interviewed one of them about themselves and the other sample adult; we attempted to interview the sample households (again by telephone) a second time two weeks after the initial interview. We found that the wording of the disability questions had a major effect on the apparent prevalence of disability in this population, but, despite our efforts to simplify it, the wording of the questions had little effect on the consistency of responses across interviews or on self-proxy differences. Answers were more consistent across interviews when the same person answered the questions both times – whether that person was a self-respondent or a proxy. Only about two-thirds of those classified as having a disability in the first interview were classified as having a disability in the second. Disability is a complex concept and different respondents may have different views about whether a given person has a disability. As a result, changes in respondents may lead to changes in the disability classification of the target person. On the other hand, changing the wording to simplify the judgment seemed to have only modest effects on consistency across interviews or consistency across respondents.