Shifting Opinion of Humans: Language Models and Persuasion

Anthropic, the creators behind the ChatGPT competitor Claude AI, conducted a study to measure the persuasiveness of language models. This inquiry is pivotal given the broad application of AI for purposes such as promoting political causes, selling products, or spreading information and disinformation.

The researchers at Anthropic employed various strategies with hundreds of participants, evaluating their agreement with statements before and after presenting them with AI-generated content. This content ranged in approach from making compelling cases and role-playing as experts, to logical reasoning and using deceptive methods. One example was the attempt to persuade readers of the necessity for corporations to disclose their climate impact.

After these sessions, the study compared the levels of agreement before and after exposure to AI-crafted arguments against those triggered by human-written texts, though not necessarily by persuasion experts. The results indicated that Anthropic's top model is "roughly as persuasive as humans." This finding may not be surprising to some, given the human-like quality of responses from advanced AI chatbots. It's becoming increasingly difficult (almost impossible) to distinguish between texts written by machines and those by humans with the most sophisticated models.

However, the critical issue arises when AI models achieve or even surpass human-like persuasiveness. They could be utilized on a massive scale for various agendas. While ChatGPT incorporates ethical guidelines, and so does Opus, there exist open-source language models without such restrictions. These models have the capacity to generate text that can sway readers towards any cause, armed with a myriad of arguments.

This situation presents an incredible dilemma: on one hand, we have companies with proprietary "black box" language models, operating under their own set of guidelines. On the other hand, open-source models offer visibility into the data and mechanics behind the AI, providing an opportunity for scrutiny and understanding. Yet, these open-source alternatives also bear the potential to bypass ethical considerations.