Population Heterogeneity and Causal Inference

Population heterogeneity is ubiquitous in social science research. The very objective of social science is not to discover abstract and universal laws but to understand population heterogeneity. Due to population heterogeneity, causal inference with observational data in social science is impossible without strong assumptions. There are two potential sources of bias. The first is bias in unobserved pretreatment factors affecting the outcome even in the absence of treatment. The second is bias due to heterogeneity in treatment effects. In this paper, I show how “composition bias” due to population heterogeneity arises when treatment propensity is systematically associated with heterogeneous treatment effects. Of particular interest is the way in which composition bias, a form of selection bias, arises even under the classic assumption of ignorability, as I demonstrate with a simple simulation example.