Politics is a journal of the Political Studies Association. We aim to highlight key articles as well as bring you editorial news.

Hindsight bias in expert surveys: How democratic crises influence retrospective evaluations

While the past few decades have seen a global expansion of democracy, more recently many academic studies have warned of democratic backsliding. In some cases, seemingly consolidated democracies have begun to resemble authoritarian regimes, while elsewhere liberal democratic institutions such as freedom of the press have been systematically undermined.

Academics, policymakers, journalists and a range of other actors often use indices to measure the quality of democracy across time and space. Some of these indices have been criticized on methodological grounds for apparent political bias and/or a lack of transparency. Due to these well-known limitations, academic studies of the quality of democracy increasingly rely on indices derived from expert assessments—that is, the judgments of other academics, journalists, and political practitioners.

How reliable are experts when it comes to assessing the quality of democracy? Although indices based on expert assessments represent an improvement over previous measures, they are based on the assumption that expert judgments are objective and accurate. However, we show that experts remain vulnerable to errors in reasoning that may impede the impartiality and reliability of their judgments. In particular, we find that expert judgments may be influenced by the very phenomenon that many index users are trying to understand: democratic crises.

In our recent article in Politics, we examine one particular form of expert bias: hindsight bias. Hindsight bias refers to the tendency to retrospectively exaggerate one’s foresight of an event—in this case, a crisis of democracy. We argue that hindsight bias is particularly insidious when applied to retrospective evaluations of democratic crises due to the difficulty of separating our evaluations of the pre-crisis period from the knowledge that a crisis occurred.

Complicating matters further, political memory is often ideologically polarized, as interpretations of past events may be embedded in current political debates that seek to (re)frame past conflicts. Although this sort of ideological bias differs from hindsight bias, the two problems are related. While we cannot alter past events, the way in which these events are (re-)interpreted in the present, and their significance in relation to events that are currently unfolding, is constantly evolving. We suggest that crises of democracy, by their very nature, are particularly vulnerable to shifting/competing interpretations in this way.

When it comes to expert assessments of democratic crises, we find evidence of considerable disagreement. In spite of the fact that their evaluations paint widely different pictures, however, experts remain highly confident in their individual judgments.

We argue that this poses a potential problem for those who use expert surveys to study democratic breakdowns. Our findings suggest that experts’ knowledge of a crisis colours their evaluations, and, more importantly, experts may not be fully aware that this is happening.


In the Journal

From the Editors