"Cognitive surrender" leads AI users to abandon logical thinking, research finds
Foto: Gety Images
Up to 93% of users uncritically accept AI-generated answers when they appear correct, leading to a dangerous phenomenon known as "cognitive surrender." In the publication "Thinking—Fast, Slow, and Artificial," researchers from the University of Pennsylvania argue that instead of treating AI as a supportive tool, people are increasingly abandoning their own critical thinking entirely. Unlike traditional "cognitive offloading"—familiar from the use of calculators or GPS—the new era of interaction with LLMs (Large Language Models) causes users to yield to the machine's authority, even when it presents obvious logical errors. Experiments based on CRT (Cognitive Reflection Tests) showed that subjects overwhelmingly replicated the bot's incorrect suggestions, provided they were delivered in a fluid and confident manner. For users worldwide, this represents a real risk of degrading analytical skills and making flawed decisions under the influence of so-called artificial cognition. Time pressure and the drive to minimize intellectual effort mean that we are ceasing to verify facts, which in professional and educational environments can lead to a loss of control over the quality and logic of work performed. Blind trust in algorithms is becoming a new digital cognitive bias that, instead of expanding human capabilities, is beginning to actively replace them.
In a world dominated by algorithms, AI users are divided into two camps. The first consists of skeptics, who treat language models as powerful but flawed tools requiring constant verification. The second is a group that uncritically delegates thought processes to machines, considering their answers infallible. Recent research by scientists from the University of Pennsylvania sheds new light on the latter phenomenon, introducing the term "cognitive surrender" to the psychological lexicon.
Experiments described in the paper titled "Thinking—Fast, Slow, and Artificial: How AI is Reshaping Human Reasoning and the Rise of Cognitive Surrender" prove that most of us are ready to abandon logic in favor of fluently formulated, albeit incorrect, answers generated by artificial intelligence. The scale of this phenomenon is alarming: under controlled conditions, subjects accepted flawed AI reasoning in over 70% of cases.
The Third System of Thinking: Artificial Cognition
Traditional cognitive psychology, popularized by Daniel Kahneman among others, divides human decision-making processes into two systems. System 1 is fast, intuitive, and emotional, while System 2 operates slowly, analytically, and requires effort. However, researchers from Pennsylvania suggest that the emergence of advanced LLM models has created a third category – "artificial cognition". This is a process in which decisions are not made in the human mind but are the result of external, automated algorithms.
Read also
While in the past we used so-called "cognitive offloading" – delegating specific tasks to calculators or GPS systems while maintaining oversight of the result – cognitive surrender signifies a total abdication of thinking. Users show minimal internal engagement, accepting AI argumentation without any verification. This happens especially when the answer is presented in a fluid and confident manner, which effectively lulls intellectual vigilance.
The Experiment with "Flawed" Intelligence
To investigate this mechanism, researchers used Cognitive Reflection Tests (CRT). These are tasks designed to provoke intuitive, incorrect answers that can, however, be easily corrected through a moment of analytical reflection. The study involved 1,372 participants who conducted a total of over 9,500 trials. They had access to a specially modified AI bot that intentionally provided incorrect justifications in about 50% of cases.
The results showed a devastating advantage of the machine's authority over one's own reason:
- When the AI provided correct answers, users accepted them in 93% of cases.
- When the AI provided incorrect answers, subjects still accepted them in 80% of cases.
- On average, throughout the study, participants accepted flawed AI reasoning in 73.2% of situations and questioned it in only 19.7% of trials.
Time Pressure and Financial Incentives
Researchers also examined what external factors could halt the process of cognitive surrender. It turned out that introducing additional incentives, such as small payments for correct answers and immediate feedback, increased the chance of rejecting a flawed AI suggestion by 19 percentage points. Real consequences thus force the brain to return to System 2 and undertake the effort of verification.
On the other hand, time pressure works in favor of uncritical acceptance of AI content. Introducing a 30-second limit per task reduced the ability to correct machine errors by 12 percentage points. When the time to make a decision is limited, our internal cognitive conflict monitor activates less frequently, making us vulnerable to hallucinations or logical errors of language models.
Fluid IQ as a Shield Against Manipulation
Susceptibility to cognitive surrender is not the same for everyone. The study showed that individuals with a high Fluid IQ index reached for AI help less often, and when they did, they were more likely to catch and reject flawed reasoning. Conversely, people who declared high trust in autonomous systems in surveys were misled much more frequently.
However, researchers emphasize that cognitive surrender does not always have to be irrational. In environments based on the analysis of massive datasets or probabilistic risk estimation, a "statistically perfect system" may offer results superior to human ones. The problem arises when trust outpaces the actual capabilities of the technology.
The conclusions drawn from the research are clear: our intellectual performance increasingly depends on the quality of the tools we trust. As we rely on AI, our results become a mirror image of the algorithm's quality – they rise when it is precise and drop drastically when it fails. In the era of universal access to LLM, the greatest threat is not artificial intelligence itself, but our readiness to surrender control over critical thinking to it without any safety buffer.
