On Monday, Google announced its AI chatbot Bard — a rival to OpenAI’s ChatGPT that’s due to become “more widely available to the public in the coming weeks.” But the bot isn’t off to a great start, with experts noting that Bard made a factual error in its very first demo.

A GIF shared by Google shows Bard answering the question: “What new discoveries from the James Webb Space Telescope can I tell my 9 year old about?” Bard offers three bullet points in return, including one that states that the telescope “took the very first pictures of a planet outside of our own solar system.”

However, a number of astronomers on Twitter pointed out that this is incorrect and that the first image of an exoplanet was taken in 2004 — as stated here on NASA’s website.

“Not to be a ~well, actually~ jerk, and I’m sure Bard will be impressive, but for the record: JWST did not take ‘the very first image of a planet outside our solar system,’” tweeted astrophysicist Grant Tremblay.

Bruce Macintosh, director of University of California Observatories at UC Santa Cruz, also pointed out the mistake. “Speaking as someone who imaged an exoplanet 14 years before JWST was launched, it feels like you should find a better example?” he tweeted.

(The mistake and the experts’ objections were first spotted by Reuters and New Scientist.)

In a follow-up tweet, Tremblay added: “I do love and appreciate that one of the most powerful companies on the planet is using a JWST search to advertise their LLM. Awesome! But ChatGPT etc., while spooky impressive, are often *very confidently* wrong. Will be interesting to see a future where LLMs self error check.”

As Tremblay notes, a major problem for AI chatbots like ChatGPT and Bard is their tendency to confidently state incorrect information as fact. The systems frequently “hallucinate” — that is, make up information — because they are essentially autocomplete systems.

Rather than querying a database of proven facts to answer questions, they are trained on huge corpora of text and analyze patterns to determine which word follows the next in any given sentence. In other words, they are probabilistic, not deterministic — a trait that has led one prominent AI professor to label them “bullshit generators.”

Of course, the internet is already full of false and misleading information, but the issue is compounded by Microsoft and Google’s desire to use these tools as search engines. There, the chatbots’ answers take on the authority of a would-be all-knowing machine.

Microsoft, which demoed its new AI-powered Bing search engine yesterday, has tried to preempt these issues by placing liability on the user. “Bing is powered by AI, so surprises and mistakes are possible,” says the company’s disclaimer. “Make sure to check the facts, and share feedback so we can learn and improve!”

A spokesperson for Google, Jane Park, gave The Verge this statement: “This highlights the importance of a rigorous testing process, something that we’re kicking off this week with our Trusted Tester program. We’ll combine external feedback with our own internal testing to make sure Bard’s responses meet a high bar for quality, safety and groundedness in real-world information.”

Related:

Update, Wednesday February 8th, 11:03AM ET: The story has been updated with comment from Google.