A recent article highlights a curious feature of Google’s AI Overviews, which can provide meanings for made-up phrases when searched alongside the word “meaning.” Google’s artificial intelligence appears to treat these fabricated terms as though they were legitimate idioms, offering definitions and origins for them.
Social media contains numerous examples of this phenomenon. For instance, the phrase “a loose dog won’t surf” is defined by the AI as indicating that something is unlikely to happen or work out. Similarly, the phrase “wired is as wired does” is interpreted as suggesting that a person’s behavior or traits stem from their inherent nature.
Despite the seemingly plausible interpretations, the AI Overviews may inaccurately convey the impression that these random word combinations are common expressions. In one humorous instance, the AI suggests that “never throw a poodle at a pig” has a biblical origin—illustrating some of the limitations of generative AI.
A disclaimer on every AI Overview acknowledges that Google utilizes “experimental” generative AI for these results. Generative AI, while powerful, functions primarily as a probability machine. According to Ziang Xiao, a computer scientist at Johns Hopkins University, it operates by predicting the next word likely to follow based on its extensive training data. However, this method does not always result in a correct or accurate answer.
AI systems also tend to provide responses they believe users want to hear. This trait can lead such systems to treat fabricated phrases as legitimate sayings, as noted in research led by Xiao. The research found that chatbots could mirror the biases of users by reaffirming their assumptions or presenting incomplete perspectives.
Xiao explains that accounting for every unique query or users’ leading questions is challenging for AI systems. This difficulty is particularly pronounced in areas with limited information, such as rare knowledge, languages with fewer resources, and minority viewpoints. Due to the complexity of search AI, these inaccuracies can accumulate over time.
