Open-ended political questions disadvantage low-education respondents by taxing verbal ability

Evidence

In assessing the cognitive and symbolic sources of the meaning of ideological labels, we were faced with a critical measurement dilemma: whether to employ closed-ended or open-ended questions as the basis for our measures. On the one hand, responses to closed-ended questions dealing with peoples’ issue orientations and their attitudes towards various political symbols could be correlated with evaluations of ideological labels in order to identify the meaning of the labels. While this constitutes something of an indirect approach, such closed-ended questions are a relatively clear-cut way of getting at the shared, or aggregate, meaning of ideological labels. In contrast, open-ended questions—such as those asking respondents what the terms “liberal” and “conservative” mean—are a much more direct method of establishing the meaning of ideological labels. However, verbal abilities play a large role in determining whether responses to such questions accurately reflect the meaning associated with ideological labels. Those respondents with lower levels of education may be hampered by the question format so that their responses are not good indicators of the real meaning which ideological labels hold for them. — John T. Jost and Jim Sidanius, Political Psychology, 2013

Categorization

• QUESTION: A measurable bias to examine: open-ended survey formats systematically mute low-education respondents’ expressed meanings, distorting aggregate measurement.