Definition
Hallucination is a broad label for generated content that departs from its evidential basis. It can involve adding a fact the source never states or producing an answer that conflicts with the source. In research operations, those failure types should be distinguished because finding a supporting quotation and checking its meaning require different review steps.
Not every bad company record is a hallucination. A faithful quotation about the wrong company is an identity error. A correct historical statement presented as current is a date or scope error. A classification rule applied incorrectly is another distinct problem. Calling all failures hallucination can hide the actual cause and encourage a model change where identity checks, retention, or question definitions need repair instead.
In origination work
Review answers against retained material and classify the reason for rejection. Check company identity, source date, quotation accuracy, and whether the text supports the chosen value. Keep an unanswered state available so the model is not required to fill every field. Docket's Audit removes unsupported findings and leaves gaps visible. A complete-looking dossier is not preferable to a partial record when apparent completeness depends on inventing revenue mix, ownership, qualifications, or transaction intentions that the evidence cannot establish.
What the term covers
In natural-language generation, hallucination refers to output that is fluent but not supported by the input or by fact. Surveys of the literature distinguish content that contradicts the source from content that cannot be verified from it, and describe causes in training data, model design and decoding. The US National Institute of Standards and Technology uses the term confabulation in its generative-AI risk profile and treats it as a distinct risk to be managed.
For research on companies the symptoms are concrete: an invented founding year, a plausible but nonexistent subsidiary, a licence number that matches no record, a quotation that appears on no page. These errors are dangerous precisely because they are plausible. A fabricated detail in a well-written summary reads the same as a true one.
Measurement and mitigation
Evaluation methods break long outputs into atomic claims and check each against a knowledge source, producing a score for the proportion supported. Work along these lines shows that long generated text contains many individual statements, and that errors accumulate with length. The corollary is that shorter, structured outputs, each field backed by a quotation, are easier to verify than free-form summaries.
Sampling many answers and checking their agreement is another technique studied in the literature, and consistent answers tend to be more reliable than inconsistent ones, though agreement does not guarantee truth. The more robust mitigation is architectural: give the model sources, require a supporting quotation, verify the quotation mechanically, and then have a separate reviewer re-read the retained material with no ability to fetch anything new.
Policy for acquisition research
A rule that works: no value without a verbatim quotation from a retained page; no quotation, no value. Values that fail the check are struck, and the field is left visibly open. Gaps are acceptable; invented facts are not. This also sets expectations with clients: a report with open fields is the system working, where a report with no gaps may be hiding errors.
Test continuously. Maintain a labelled set of companies with known facts, run the system on it regularly, and count unsupported statements. Publish internal error rates and change the process when they rise. Treat model upgrades as changes requiring re-testing, as behaviour on this dimension can shift.
Related terms
- Grounding — Basing generated research answers on supplied evidence while still checking that the evidence supports each answer.
- Identity resolution — Establishing which real business a candidate record, name, domain, or source actually refers to.
- Screening rubric — A defined set of questions, accepted answers, evidence requirements, and decision rules for assessing candidates.
Further reading
- Diagnose unsupported research findings
- How the research agents work
- Grounding
- Retained source
- Evidence standards
- Analysts versus AI agents
- Article: benchmarking AI agents for acquisition research
Sources and further reading
- Survey of Hallucination in Natural Language Generation, arXiv (2022). Taxonomy of how generated text departs from its source, and of the metrics that try to catch it.
- FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation, arXiv (2023). Scores generated text one atomic claim at a time, which is the only granularity at which 'is this true' is answerable.
- AI Risk Management Framework: Generative AI Profile, National Institute of Standards and Technology (2024). Names confabulation as a distinct generative-AI risk and lists practices for managing it.
- Self-Consistency Improves Chain of Thought Reasoning in Language Models, arXiv (2022). Sampling several answers and taking the agreement beats taking one. The cheap version of a second reader.
- Measuring Attribution in Natural Language Generation Models, arXiv (2021). Defines Attributable to Identified Sources (AIS): whether a statement is actually supported by the document cited for it.
To apply these distinctions to your own acquisition criteria, discuss your research brief or inspect the demo platform.