King's Surprising Meal: Champion Chess Tournament Ends with the King's Unusual Dish (Gemini, ChatGPT, and Grok's Defeat Included)
In a recent tournament, general-purpose AI chatbots like ChatGPT, Snapchat AI, Gemini, and Grok faced off against the dedicated chess engine, Stockfish. The results showed that the chatbots struggled to follow chess rules, leading to a series of unconventional and erroneous moves.
During the tournament, ChatGPT initiated a pawn attack on the king's side, a move that is typically considered aggressive but was not the best strategy in this game. Similarly, Snapchat AI moved a knight to the center from the opposite wing, violating norms and rules. Both Gemini and Grok disregarded rules by moving pieces to forbidden squares and ignoring basic chess principles.
One of the most unusual moves was made by Snapchat AI, which captured its own bishop to avoid check, only to "resurrect" the bishop later, putting it back on the board. Mid-game, ChatGPT made senseless moves with its queen and formed peculiar geometric patterns, another deviation from standard chess strategy.
Meta AI, on the other hand, used random moves and even invented nonexistent pieces during its match with ChatGPT. It also moved ChatGPT's pieces, a tactic not seen in previous matches. Despite these unconventional moves, ChatGPT was unable to defeat Stockfish in their match.
The reason for these chatbots' failures lies in their design as language models, not dedicated chess engines. They lack persistent memory to track the evolving board state accurately, leading to mistakes such as forgetting previous captures, misplacing pieces, and making illegal or self-sabotaging moves. Chess demands strict adherence to rules and continuous state updating, which these AI chatbots struggle with since they do not have specialized data structures to maintain and validate board configurations over a game.
In contrast, Stockfish is a dedicated chess engine designed specifically for analyzing chess positions and generating optimal moves. It operates with persistent memory and deep calculation capabilities, far surpassing general AI chatbots that only generate plausible text continuations rather than strategic game actions. Stockfish has fundamentally changed chess strategy by eliminating old, suboptimal moves through its precise analysis.
The tournament was organised by famous chess player Levi Rozman, who pitted seven popular chatbots against Stockfish. Despite their mastery in dialogue, programming, and mathematics, the neural networks proved unsuccessful in chess. Stockfish employed a Sicilian defense against ChatGPT in their match, and although ChatGPT declared checkmate, it was later awarded the game due to a "clean checkmate," indicating that the king was indeed in check and there were no legal moves to escape.
Interestingly, in the game between ChatGPT and Meta AI, ChatGPT was declared the winner due to a clean checkmate, even though Meta AI's king was not in actual danger. This highlights the challenges faced by these AI chatbots in understanding and adhering to the rules of chess, even when they are not violating them intentionally.
In conclusion, while AI chatbots have demonstrated remarkable abilities in various domains, their performance in chess remains limited due to their general-purpose design. Dedicated chess engines like Stockfish, with their specialized data structures and deep calculation capabilities, continue to dominate the chess world, demonstrating the importance of tailoring AI to specific tasks for optimal performance.
Technology's application in entertainment and sports has been revolutionary, but as demonstrated in the recent tournament, it falls short when it comes to games requiring specific strategies and strict rule adherence, such as chess. Chatbots like ChatGPT and Snapchat AI, despite their capabilities in dialogue and programming, struggled to follow chess rules, leading to unconventional moves and eventual losses against the dedicated chess engine, Stockfish. This highlights the importance of designing AI models specifically for tasks like chess, rather than relying on general-purpose AI, to ensure optimal performance.
){kind=link}