ChatGPT's language skills put to the test: Examining the model's language comprehension abilities
In a groundbreaking study, psycholinguist Michael Vitevitch from the Speech-Language-Hearing Department at the University of Kansas has found that ChatGPT, a popular AI model, processes language in a manner distinct from human cognition[^1^][^3^]. The study, titled "What nonsense reveals about ChatGPT's understanding of language," aimed to identify where AI can complement human linguistic strengths rather than mimicking human cognition[^3^].
To test ChatGPT's language processing capabilities, Vitevitch used "nonwords"—meaningless sounds or letter combinations—and archaic terms. In the study, ChatGPT correctly defined 36 out of 52 archaic terms, acknowledged uncertainty for 11, drew from other languages for 3, and made up definitions for 2[^3^].
One interesting observation was that ChatGPT did not respond with English words that sounded like the given Spanish words. Instead, it used words from other languages[^3^]. This suggests that ChatGPT excels at pattern recognition but not necessarily in the same way humans do.
In the realm of inventing new words for modern concepts, ChatGPT demonstrated its ability to create novel words, a trait it shares with humans in the form of 'sniglets'—words that don't exist, such as 'carperpetuation' for a thread that doesn't get sucked up by a vacuum cleaner[^3^]. However, Vitevitch found that ChatGPT "hallucinated" on a few definitions[^3^].
Vitevitch believes that AI should be engineered to provide a safety net for tasks where humans struggle. He argues that AI and humans process language differently, and it is acceptable for them to do so[^1^][^3^]. The findings of the study are published in PLOS One[^3^].
The study was originally published by Cosmos, shedding light on the unique ways in which AI processes language[^3^]. As we continue to interact with AI models like ChatGPT, understanding these differences can help us better utilise these tools and appreciate their capabilities.
[^1^]: KU News, 2025-06-18 [^3^]: Cosmos Magazine, 2025-06-28
In the study, ChatGPT demonstrated its ability to create novel words, a trait it shares with humans in the form of 'sniglets', showcasing its artificial-intelligence-driven capacity for inventing new words for modern concepts. On the other hand, when faced with uncertainty, ChatGPT draws from other languages to form responses, indicating a reliance on technology for language pattern recognition rather than mimicking human cognition.
