Using a variable to select cases shrinks its measured causal effect
Evidence
Many of the independent variables are used to define one population or another. These include (1) democracy (i.e., regime type), (2) trade, (3) major powers, and (4) contiguity. For example, major power status is a core part of the definition of opportunity political relevance. Since we use major power status to define relevant dyads we might expect to see this factor be less important in an analysis of opportunity relevance dyads. In fact, this appears to be the case: both the coefficient and the significance levels are much lower in the opportunity relevant population. To appreciate the substantive differences between the coefficient estimates in the two models, consider how a major power dyad compares to a baseline dyad composed of nondemocratic, unallied, noncontiguous minor powers with mean values for all other variables. Using the coefficients from the model based on all dyads, the estimated probability of a dispute for a major power dyad with all of the other attributes of the baseline dyad is more than six times higher than the baseline dyad. In contrast, using coefficients from the model estimated from only the opportunity relevant dyads, a major power dyad is only two times more likely to experience a dispute compared to the baseline. — Gary Goertz, Social Science Concepts, 1549
Categorization
- XREF: Parallels Hall’s argument in Essential Essays that theory and data are constituted together — the measurement instrument shapes the object it measures.
- DEEPTHINKING: The sixfold-to-twofold collapse quantifies the tautology trap: any factor built into population definition returns a diluted coefficient, so reported effect sizes systematically depend on selection logic.