Measuring populist attitudes – identifying a core

A central question in contemporary populism research is to what extent people hold so-called populist attitudes. In light of this question, various scales have been proposed to measure populist attitudes (see here or Chapters 6 and 7 here for an overview). While the scales differ in various aspects, what they have in common is that, in one way or another, they draw on trailblazing measurement studies that were conducted in the USA or the Netherlands using a similar battery of survey items assumed to measure populist attitudes (see first six items in Table 1, below). At the moment of writing, the scale devised in the Dutch study remains perhaps the most influential battery of items to measure populist attitudes and examine relationships between populism and broader political behaviour. And while such studies are insightful and enrich the discipline, the integrity of their results depends on having an appropriate measurement of populist attitudes.

Table 1: Survey items of the populist attitudes scale

items Question wording
populism_1 The politicians in Congress need to follow the will of the people.
populism_2 The people, not the politicians, should make our most important policy decisions.
populism_3 The political differences between the people and the elite are larger than the differences among the people.
populism_4 I would rather be represented by an ordinary citizen than an experienced politician.
populism_5 Politicians (elected officials) talk too much and take too little action.
populism_6 What people call “compromise” in politics is really just selling out on one’s principles.
populism_7 The particular interests of the political class negatively affect the welfare of the people
populism_8 Politicians always end up agreeing when it comes to protecting their privileges.

 

In a recent article in Politics, we argue that the measurement and empirical evaluation of both the individual items in particular and the scale as a whole have yet to receive sufficient attention despite some efforts being made in other studies. With that crucial caveat in mind, we use a cross-national sample (9 countries, n = 18,368) and apply Item Response Theory (IRT) to the most commonly used populism scale in order to evaluate whether that scale (1) accurately extracts people’s populist attitudes from self-reported answers, and (2) provides a pan-European measurement for the European context. This exercise allows us to make three crucial observations that are relevant for (populism) scholars and beyond.

First, the IRT analysis of the traditional 6-item scale shows that all six items have difficulties in capturing high and extreme values of an aggregate populist attitudes instrument. Some of these items, like the ‘elite-procrastination’ item (item 5 in Table 1), perform notably better than others. While the brilliant study by Castanho Silva et al. (2019) confirms this is not uncommon across a wide variety of populism scales and samples, this observation presents a challenge for the findings and measurement of populist attitudes to the extent that most research in this area focuses precisely on the most -and least- populist. Hence, suggesting further scale development is required for accurate inferences.

With that in mind, we propose two additional items to formulate an 8-item scale (items 7 and 8 in Table 1). While our analysis shows the new items clearly outperform some of the traditional items, the addition of these new items gauging the perceived conflict between ‘the people’ and ‘the elite’ does not expand the extent to which we accurately capture the full range of the latent populist construct. For that reason, in a final step, we set out to fit a more parsimonious measurement through the refinement of the 8-item scale that can serve as the basis for future studies. By combining Occam’s razor and psychometric principles, we deduce a core measure comprised of 3 high-performing items covering the hypothesised theoretical concept and consisting of one item from the traditional 6-item scale (item 1 in Table 1) and the two novel items (items 7 and 8 in Table 1).

Overall, the intention of this exercise is not necessarily to provide a ‘minimal’ populist instrument that comprises no more than three items. Rather, we wish to highlight some of the areas for improvement of existing measures and propose a foundation (at least in a pan-European context) that researchers can build on and use as a baseline for further advancement of this field.

Steven M. Van Hauwaert

Steven M. Van Hauwaert

Steven M. Van Hauwaert is the principal investigator of the Global Public Opinions Project (GPOP). His primary research interests are in the fields of comparative political behaviour and public opinion, as well as populism and political extremism. His recent academic contributions have appeared in Acta Politica, Comparative European Politics, Electoral Studies, the European Journal of Political Research, European Societies and the Journal of European Integration. He is also an associate editor of the ECPR journal Political Research Exchange (PRX).

Christian H. Schimpf

Christian H. Schimpf

Christian H. Schimpf is a Ph.D. candidate in political science at the Graduate School of Economics and Social Sciences (University of Mannheim). He is also a data processing specialist with the Comparative Study of Electoral Systems (CSES) Secretariat at the GESIS Leibniz Institute for the Social Sciences, Mannheim. His research interests include comparative political behaviour, populism, and public opinion. His research has been published, among others, in Political Studies and Local Government Studies

Flavio Azevedo

Flavio Azevedo

Flavio Azevedo is a Ph.D. candidate in Political Science at the Center for Comparative Politics at Cologne University, Germany. He uses comparative and psychometric approaches to investigate the dispositional and situational processes underlying ideological subscription and populist proclivities. In 2018, he visited Professor Jost at New Your University on a Fulbright research scholarship. Previously, he also made research visits at Sciences Po, Paris; Vrije Universiteit Amsterdam; and Kent University.

Discussion

1 comment
  1. Interesting argument. The question is, thought methodologically sound, whether these items are measuring populism…
    – The politicians in Congress need to follow the will of the people.
    – The particular interests of the political class negatively affect the welfare of the people
    – Politicians always end up agreeing when it comes to protecting their privileges.

    On a theoretical level, I think that the original 6-items are still outperforming any other measurement (I have seen so far).

Leave a Comment

Your email address will not be published. Required fields are marked *