Has a dog ever played in the NBA? Google’s AI thinks so.
What are some ways to keep cheese from sliding off pizza? Google recommends adding 1/8 cup of non-toxic glue, citing a sarcastic Reddit post from 2013.
This Tweet is currently unavailable. It might be loading or has been removed.
It’s been just over a week since Google unleashed its AI Overviews to all US users and reports of outlandish responses are emerging on social media. Confusing or false responses from AI are a known issue, but they are now on full display via the most popular search engine.
“People are responding very positively to [AI Overviews]. It’s one of the most positive changes I’ve seen in search,” Google CEO Sundar Pichai tells The Verge. People are more engaged with it than with traditional, link-based search results, as measured by a higher click-through rate, according to Pichai, who says Google can’t be expected to have perfect results every time.
In another example, Google’s AI Overview produced some interesting statistics when asked which US president went to the University of Wisconsin-Madison. The answer is none; Harvard is the most popular higher-education destination for future presidents. But the response from Gemini makes it look like more than half a dozen earned multiple degrees from the school, many after they’d died. It turns out Gemini pulled from a tongue-in-cheek page on the UW-Madison alumni page that lists “alumni with presidential names.”
This Tweet is currently unavailable. It might be loading or has been removed.
In another social media post, Google Gemini misidentified a poisonous mushroom, saying it was a common button mushroom.
How does Google decide what information to surface in each AI-powered response? In a perfect world, it would scan through its training data and surface only well-researched or widely agreed-upon information (“common knowledge”). However, in the example about gluing cheese on pizza, it seems to prefer quoting a joke on Reddit than nothing at all.
Google signed a content-licensing deal with Reddit in February to use its data in AI products, as did OpenAI this month for ChatGPT. A growing list of websites and news publications have also signed content-licensing deals for AI products, including The Wall Street Journal, Financial Times, Stack Overflow, and more.
Now that we know Google’s AI is vulnerable to surfacing any ol’ piece of information, people are attempting to “poison” its results. One user added multiple awards to his own website, which promptly appeared in Google’s summary.
Recommended by Our Editors
This Tweet is currently unavailable. It might be loading or has been removed.
Google and OpenAI are racing to bring more AI features to the web, including new voice assistants and photo-based capabilities. In his interview with The Verge, Pichai argues that “LLMs aren’t necessarily the best approach to get at factuality.
“Hallucination is still an unsolved problem. In some ways it’s an inherent feature,” he adds. “It’s what makes these models creative.”
Meanwhile, Meta’s AI lead says he’s thinking beyond LLMs, which he calls “very limited.”
Everything Announced at Google I/OGet Our Best Stories!
Sign up for What’s New Now to get our top stories delivered to your inbox every morning.
This newsletter may contain advertising, deals, or affiliate links. Subscribing to a newsletter indicates your consent to our Terms of Use and Privacy Policy. You may unsubscribe from the newsletters at any time.
