Researchers tested to see whether politicians changed their prior beliefs in the face of overwhelming evidence, and the results were pretty depressing.
The study, published in the British Journal of Political Science, set out to answer two questions: Does evidence help politicians make informed decisions, even if it challenges their beliefs? And does providing more evidence increase the likelihood of politicians accepting that evidence?
Julian Christensen, a political science researcher at Aarhus University in Denmark, wasn’t particularly surprised to find that a politician’s prior belief affects how they interpret factual information—that much has been shown before. But he was taken back when he saw that politicians doubled down when they were given even more evidence that goes against their prior beliefs. “That was a surprise for us,” he says, adding, “It’s also kind of depressing.”
For the study, Christensen and a group of Danish researchers enrolled 954 Danish politicians from local government to participate in a series of experiments. At the heart of the study was a particularly thorny political debate: The role the private sector should play in the delivery of public services. While some politicians maintain that the public sector is the best supplier of public services, others argue that private contractors deliver services more efficiently. The politicians were randomly assigned to different groups.
In a series of experiments, politicians were given raw data and asked to evaluate a private or public service. The researchers used three examples; a school, road maintenance, and a rehabilitation centre. In the first experiment, researchers were asked to look at surveys of parent satisfaction from two schools—a private school and a public school. In one group, parents were more satisfied with the service provided in the private school, while in the other, parents were more satisfied with the public school. There was also a placebo group, where participants were asked which school was providing the better service, school A or B (there was no mention of it being a private or public school). Researchers carried out the same experiment with another group of politicians, but this time on something less contentious in policy making—road maintenance. Christensen point out that during both experiments, the “information is quite unambiguous.”
Despite this, the findings showed that when politicians were asked to evaluate the information, around 84% to 98% of politicians that received information that fit their prior belief interpreted the information they were given correctly. However, politicians who received information at odds with their beliefs correctly interpreted the information correctly only 38% to 61% of the time.
For the third experiment, researchers provided politicians with a table that showed the number of patients who had cruciate ligament injury who went on to receive treatment at either a private or public rehabilitation centre. Depending on which group the politician was put in, there were either more patients fully recovered from the public rehabilitation center or vice versa. Looking at the data, the politician were then asked which service performed better. Researchers then provided even more information on the rehabilitation centres; showing data on how patients recovering from ankle-stabilizing operation, fracture in the elbow, wrist operation, and shoulder arthroscopy. This time, the findings showed that politicians were more likely to double down on their previously held belief when presented with mounting evidence to the contrary.
This wasn’t just an issue that affected the right or the left. The tendency to interpret evidence so it fits with previously held beliefs was true of politicians across the political spectrum. That said, Christensen found “people with more extreme views are more affected,” as “it’s more important for them to defend their attitudes.”
While the results were illuminating, they may have been influenced by the fact that participants were told the evidence they were being given was fake. “For ethical reasons, and to avoid confusing the suppliers with real organizations, we made it clear in the introductory text to the experiments that the information was fictitious,” researchers note in the study. Given this, it’s possible, of course, that the politicians participating in the study may have been skeptical of the data presented to them.
Are politicians any different from the general population? Christensen was keen to stress that “we see the same basic pattern among citizens as well.” When researchers ran the same experiment with over 1,000 ordinary Danish citizens, the findings were similar. “Politicians are human beings” Christensen says, and just like every other person, “they are effected by their attitudes when they approach factual information.”