Markus Wagner

This study introduces a 'generative debunking' framework that integrates climate contrarian claim classification (CARDS) and fallacy detection (FLICC) into an LLM prompting pipeline, so that a model takes a climate myth as input and produces a debunking that follows the fact-myth-fallacy-fact ('truth sandwich') structure; the authors pair three prompting strategies of increasing complexity with GPT-4, Palm2, and Mixtral, and four authors (including one climate misinformation expert) rate 60 debunkings for 20 myths on fact, fallacy, and structure, finding that GPT-4 with a simple prompt and Mixtral with a structured prompt perform relatively well, that fallacy explanations score higher than facts, and that non-expert annotators show poor agreement with the expert on fact quality.

This study introduces a 'generative debunking' framework that integrates climate contrarian claim classification (CARDS) and fallacy detection (FLICC) into an LLM prompting pipeline, so that a model takes a climate myth as input and produces a debunking that follows the fact-myth-fallacy-fact ('truth sandwich') structure; the authors pair three prompting strategies of increasing complexity with GPT-4, Palm2, and Mixtral, and four authors (including one climate misinformation expert) rate 60 debunkings for 20 myths on fact, fallacy, and structure, finding that GPT-4 with a simple prompt and Mixtral with a structured prompt perform relatively well, that fallacy explanations score higher than facts, and that non-expert annotators show poor agreement with the expert on fact quality.