Evaluating ‘inoculation’ theory in the fight against online mis and disinformation

Abstract Information: With the proliferation of digital communications, mis and disinformation have become problems that cut across every significant social issue, from climate to public health to gender equality. Inoculation theory - the idea that you can guard against the impact of misinformation by sharing compelling counter narratives - has in turn gained an increasing amount of attention. But how well does it work?

This ignite session shares the results of an evaluation of inoculation campaigns against climate misinformation in Australia, the US, and Brazil. The campaigns were conducted through social media and involved circulating compelling counter-narratives. In Australia the campaign related to the link between net-zero and the cost of living, in Brazil we focused on greenwashing by large agricultural companies.
The evaluation aimed to untangle the impact of actively circulating mis-information narratives from that of counter-narratives, on audience opinions. We used an experimental design to analyse whether it is more impactful to see a counter narrative before or after the harmful narrative, compared to a control group. This generated important findings for designing anti-misinformation campaigns.

This short session will present an overview of the evaluation that was conducted to assess the impact of these campaigns, with the methodology and our learning about inoculating audiences from misinformation.

Relevance Statement: This ignite session will share the results of an evaluation into campaigns aiming to inoculate vulnerable audiences against harmful climate misinformation narratives in Australia, the US and Brazil. These were social media campaigns aiming to promote contextually relevant climate related narratives. In the US and Australia, we focused on net-zero and the cost of living. In Brazil, we focused on greenwashing by big agricultural companies.

This evaluation fits into a broader area of enquiry about how we can best address the harmful online misinformation, and which tactics are most suitable to fight it. While the World Health Organisation recognised the infodemic in 2020 as a key contributor to the COVID-19 pandemic, approaches to fighting misinformation and evaluating these are relatively nascent. This evaluation aimed to untangle the impact of actively circulating misinformation narratives from the impact of counter-narratives on audience opinions, and analyse whether it is more impactful to see the counter narrative before the harmful narrative.

To assess this, we used panel surveys to compare attitudinal differences, related to how respondents felt about climate related issues in one of four conditions.

Participants were randomly allocated to one of the following:
Only the misinformation content
Misinformation then a counter narrative
A counter narrative the misinformation
No content at all.

We then conducted regression analysis to create a robust comparison, and understand whether different demographic groups reacted differently within each condition. As such, this evaluation generated important learning around how inoculation can be most effective, when fighting misinformation. It also provides a replicable blue-print for comparing the impact of harmful content pieces circulating online to progressive counternarratives. It is therefore a timely contribution to the field of evaluation, in this burgeoning research area.