Hallucinations (of LLMs)

What is a hallucination

It is a term used to describe a situation when an AI model generates false, misleading or made-up information that is presented as fact, despite having no basis whatsoever in its training data or the context provided.

Why do models hallucinate

The model doesn't have any “knowledge” and instead relies on probability and predicting the next word (token) in a sequence. Sometimes the most linguistically probable sequence is false information.

However, once it has generated that answer, it is kept in the context, so every following answer will build on it and our model will dig deeper and deeper into its lie.

How do I know the AI is lying?

Because hallucinations are often very convincing, it is key to take a critical approach to every piece of generated information, especially when it comes to facts, dates, statistics, quotes and references.

Besides having to approach answers critically, here are a few simple methods that can help a bit:

1. The tomato method - add at the end of the prompt “If you don't know the answer to this question, write tomato”. It sometimes happens that we then get the word tomato dropped in the middle of the answer, which already indicates that we should verify that answer.
2. Ask 2 or 3 similar questions in other chats. Especially if our question concerns a niche, new or little-known topic.
3. We can ask for sources and quotes, but only if we plan to check them. It's precisely in the details that hallucinations hide, and the deeper we go into them without verification, the greater the chance of them.