Generative AI and the Perception of Cognitive Decline
Recent headlines suggest that generative AI tools, including large language models (LLMs), might be experiencing ‘cognitive decline.’ This raises intriguing questions, especially given the human tendency to anthropomorphize AI. Analyzing these claims requires a careful examination of both human cognition and the technical underpinnings of AI development.
Understanding Human Cognitive Decline
To accurately assess claims about AI cognitive decline, we must first establish a clear understanding of human cognitive decline. The American Psychological Association (APA) defines cognitive decline as: “Reduction in one or more cognitive abilities, such as memory, awareness, judgment, and mental acuity, across the adult lifespan.” Cognitive decline is a natural part of aging, but a severe decline can indicate underlying diseases. The symptoms of cognitive decline vary, but may include diminished mental processing speed, forgetfulness, and difficulty maintaining a train of thought.
It is reasonable to expect that individuals, as they age, may experience cognitive decline. This decline is not uniform, with differences occurring depending on individual factors and the specific cognitive abilities measured.
Generative AI and LLMs: A Primer
Generative AI, including LLMs, are developed through extensive data training. This process involves feeding the AI vast amounts of existing text from the internet, and using mathematical and computational pattern-matching techniques. These models are tuned and released for public use, often appearing fluent in natural language and capable of human-like dialogue. The field of AI is fast-evolving; new techniques and optimizations are constantly being developed. This often means AI developers will start over, with a newer program and better inputs, rather than simply building off of existing products. This can complicate evaluations of their capabilities and performance.
If I release two LLM versions – version 1.0 and version 2.0 – it’s highly probable that version 2.0 was built from scratch, with improvements based on observations of version 1.0, but built on a fundamentally new project. Version 2.0 will likely perform better regardless of whether or not version 1.0 had some level of ‘cognitive decline’ from initial release until its replacement. This is crucial context when assessing claims of ‘decline’ in AI.
Examining Claims of AI Cognitive Decline
Several recent headlines assert that AI is showing signs of cognitive decline, mirroring the condition in humans. These claims are often based on studies that compare the performance of different generations of AI models.
For example, a study entitled “Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis” by Roy Dayan, Benjamin Uliel, and Gal Koplewitz, BMJ, December 19, 2024 found that “’ Older’ large language model versions scored lower than their ‘younger’ versions, as is often the case with human participants, showing cognitive decline seemingly comparable to neurodegenerative processes in the human brain (we take ‘older’ in this context to mean a version released further in the past).” This comparison is problematic for the reasons outlined earlier.
The Flawed Analogy
Comparing different versions of an AI model to human cognitive decline can be misleading. A newer model is generally expected to outperform an older one due to technological advancements. If version 1.0 underperforms when compared to version 2.0, that does not indicate that version 1.0 has declined over time. If anything, it indicates that the newer version possesses improved efficiency and improved overall results, and will always appear ‘better’ in retrospect.
Other Considerations: AI Updates and the Right to Forget
The same test administered to a human at age 40 and then again at age 80 will likely render different results as a result of natural aging and biological change. In the case of AI, if version 1.0’s capabilities are measured, the measurements are likely to remain constant over time. An AI maker might introduce minor updates to version 1.0. The fact that the results of tests administered to version 2.0 are superior to those of 1.0 does not denote decline.
Various factors can impair the performance of an AI model. For example, poorly screened data incorporated in augmented training may provide data replete with inaccuracies and errors. These flawed data can lead to flawed results which will make the AI’s capabilities lower than before. This, indeed, may be considered a decline, although the model itself would never be the same.
Another factor that can influence model performance is the ‘right to forget.’ This refers to the practice of removing data from an AI model, in response to concerns of personal privacy. Deleting useful information from an AI model can decrease its performance. Thus, caution must be taken to avoid impairing the model’s functionality.
Some of the latest artificially intelligent programs are devised for self-improvement. They are designed to fix themselves, but this ability can have pitfalls. AI can potentially undercut itself, which certainly is a decline and could happen with no human intervention.
One popular trend in AI training is to use synthetic data. Research suggests that if synthetic data is used, the AI model can potentially weaken, in a phenomenon called catastrophic model collapse. The AI is made weaker, and less viable. The use of synthetic data is another way that AI’s capabilities can decline.
Critical Thinking and the Future of AI
In the words of Mark Twain, reports of AI’s cognitive decline might be greatly exaggerated. Claims about AI should always be assessed with a critical eye. The field of AI is constantly evolving, so it’s essential to understand the nuances of model development and compare claims with the available evidence. Only then can we begin to understand how this technology can be used for the good of humanity.
