Home > Publications . Search All . Browse All . Country . Browse PSC Pubs . PSC Report Series

PSC In The News

RSS Feed icon

COSSA makes 10 suggestions to next Administration for supporting and using social science research

Thompson says US prison population is 'staggeringly high' at about 1.5 million, despite 2% drop for 2015

Levy et al. find Michigan's Medicaid expansion boosted state's economy while increasing number of insured

More News

Highlights

2017 PAA Annual Meeting, April 27-29, Chicago

NIH funding opportunity: Etiology of Health Disparities and Health Advantages among Immigrant Populations (R01 and R21), open Jan 2017

Russell Sage 2017 Summer Institute in Computational Social Science, June 18-July 1. Application deadline Feb 17.

Russell Sage 2-week workshop on social science genomics, June 11-23, 2017, Santa Barbara

More Highlights

Next Brown Bag

Mon, Jan 23, 2017 at noon:
Decline of cash assistance and child well-being, Luke Shaefer

Sources of Error in Cognitive Interviews

Archived Abstract of Former PSC Researcher

Conrad, Frederick G., and Johnny Blair. 2009. "Sources of Error in Cognitive Interviews." Public Opinion Quarterly, 73(1): 32-55.

Cognitive interviewing is used to identify problems in questionnaires under development by asking a small number of pretest participants to verbally report their thinking while answering the draft questions. Just as responses in production interviews include measurement error, so the detection of problems in cognitive interviews can include error. In the current study, we examine error in the problem detection of both cognitive interviewers evaluating their own interviews and independent judges listening to the full set of interviews. The cognitive interviewers were instructed to probe for additional information in one of two ways: the Conditional Probe group was instructed to probe only about what respondents had explicitly reported; the Discretionary Probe group was instructed to probe whenever they felt it appropriate. Agreement about problems was surprisingly low overall, but differed by interviewing technique. The Conditional Probe interviewers uncovered fewer potential problems but with higher inter-judge reliability than did the Discretionary Probe interviewers. These differences in reliability were related to the type of probes. When interviewers in either group probed beyond the content of respondents' verbal reports, they were prone to believe that the respondent had experienced a problem when the majority of judges did not believe this to be the case (false alarms). Despite generally poor performance at the level of individual verbal reports, judges reached relatively consistent conclusions across the interviews about which questions most needed repair. Some practical measures may improve the conclusions drawn from cognitive interviews but the quality of the findings is limited by the content of the verbal reports.

DOI:10.1093/poq/nfp013 (Full Text)

Country of focus: United States of America.

Browse | Search : All Pubs | Next