Max Planck Study Challenges AI's Cognitive Abilities
A recent study by the Max Planck Institute, titled 'The Illusion of Thinking', has revealed intriguing insights into the capabilities of Large Language Models (LLMs) and their smaller counterparts, Lightweight Language Models (LRMs). The study, published in 2023, challenges the notion of AI's cognitive abilities and raises questions about the current state of the field.
The study found that while LRMs outperform LLMs for moderately complex tasks, they fail completely at high complexity. This is a significant revelation, as it suggests that the current approach to scaling up models may not be the best strategy for improving AI's problem-solving capabilities. Interestingly, Apple's research paper, which can be accessed online, echoes these findings, observing an 'accuracy collapse' in GenAI models when faced with higher-complexity problems.
The study also highlights the limitations of LRMs. They struggle to generalize problem-solving capabilities and have a scaling limit in reasoning effort. This means that as the complexity of tasks increases, LRMs' performance plateaus or even declines. Apple's AI models, despite their advancements, are not immune to these limitations, according to AI thought leader Ethan Mollick, who argues that Apple's AI is not leading the AI race.
Even the most advanced GenAI models currently lack the cognitive abilities to handle highly complex tasks. While standard LLMs perform better than LRMs for simple tasks, the challenge lies in bridging the gap for complex problems. Mollick believes that AI's impact is significant despite these limitations, and new techniques may further advance the field in the future. The findings from the Max Planck Institute's study serve as a reminder that while AI has made remarkable strides, there is still much work to be done to achieve true artificial intelligence.
