TopolearnAnything you publish, send to customers or act on needs a habit of checking, or a model's mistake becomes your mistake.
A hallucination is when a model states something false as if it were true: a made-up quote, a book that does not exist, a wrong date, an invented link or legal case. It happens because the model generates plausible-sounding text; it has no built-in step that checks a fact against a source.
Hallucinations are most likely when:
- the question is about something rare, very specific or recent (after the knowledge cutoff, the date its training text ends),
- you ask for exact numbers, citations, quotes or URLs,
- the prompt pushes the model to answer even when it does not know.
Other limits worth knowing:
- Models can be weak at precise arithmetic and counting (like letters in a word) unless they can use a calculator or code.
- They tend to agree with the person asking, so a leading question ("Isn't it true that...") invites a yes.
- They can sound equally confident whether right or wrong, so confidence tells you nothing.
What reduces hallucinations:
- Give the facts. Paste the document, email or data and ask the model to answer only from it. Answering from supplied text is called grounding.
- Allow "I don't know". Say explicitly that it should say so when the information is missing.
- Ask for quotes. Have it quote the sentence it relied on, so you can check quickly.
- Check what matters. Verify names, numbers, links and anything legal, medical or financial before using it.
These techniques reduce the problem; none removes it completely. Treat model output like a draft from a fast but sometimes careless assistant.
Example
Answer the question using only the email below.
If the email does not contain the answer, reply exactly: "Not in the email."
Quote the sentence you used.
Email: """Hi team, the launch moves to 14 March. Budget is unchanged. - Dana"""
Question: What is the new budget?
Answer: Not in the email. (It says the budget is unchanged but gives no amount.)Remember
- Hallucinations are confident, plausible, false statements
- Risk is highest for specific facts, numbers, citations and recent events
- Ground the model in supplied text, allow 'I don't know', and verify what matters