A question design can distinguish genuine cynicism from vocabulary ignorance in survey respondents

Evidence

In a 1960 reinterview of the original sample whose 1956 responses had been assigned to our levels of conceptualization, we therefore asked in the context of the differences in “what the parties stand for,” “Would you say that either one of the parties is more conservative or more liberal than the other?” (It was the first time we had ever introduced these terms in our interviewing of this sample.) If the answer was affirmative, we asked which party seemed the more conservative and then, “What do you have in mind when you say that the Republicans (Democrats) are more conservative than the Democrats (Republicans)?” When the respondent said that he did not see differences of this kind between the two parties, we were anxious to distinguish between those who were actually cynical about meaningful party differences and those who took this route to avoid admitting that they did not know what the terms signified. We therefore went on to ask this group, “Do you think that people generally consider the Democrats or the Republicans more conservative, or wouldn’t you want to guess about that?” At this point, we were willing to assume that if a person had no idea of the rather standard assumptions, he probably had no idea of what the terms meant; and indeed, those who did try to guess which party other people thought more conservative made a very poor showing when we went on to ask them (paralleling our “meaning” question for the first group), “What do people have in mind when they say that the Republicans (Democrats) are more conservative than the Democrats (Republicans)?” … The responses were classified in a code inspired by the original work on levels of conceptualization, although it was considerably more detailed. — John T. Jost and Jim Sidanius, Political Psychology, 1905

Categorization

• QUESTION: This probe technique for separating ‘genuinely cynical’ from ‘doesn’t know the terms’ is a reusable methodological trick worth recalling for any survey or interview design.
• XREF: Connects to the ladder-of-abstraction and measurement-design concerns in earlier political-psychology and survey-methods readings about detecting response styles.
• SEED: An essay could spin this into a general lesson: how questioning design discriminates real beliefs from mere vocabulary gaps, applicable far beyond political polling.