By admin 
When Large Language Models (LLMs) produce inaccurate or nonsensical outputs, a phenomenon commonly referred to as "hallucination," the immediate assumption by developers and engineering teams is that the model simply lacks the necessary factual information. This perceived deficiency is typically diagnosed as a knowledge gap, leading to standard engineering responses: scaling up model size, expanding training datasets, or developing intricate retrieval architectures to inject external information. However, a groundbreaking new study by researchers at Google Research and Technion challenges this prevailing paradigm, revealing that the issue is often not one of missing knowledge, but rather a failure in the model’s ability to access information it has already parametrically encoded.
The comprehensive study, published on arXiv (https://arxiv.org/abs/2602.14080), demonstrates through extensive experimentation that state-of-the-art models, including prominent frontier models like GPT-5 and Gemini-3, actually encode an astonishing 95-98% of the facts tested within their parameters. This finding dramatically shifts the understanding of LLM limitations, indicating that for a significant portion of factual inaccuracies, the primary bottleneck is not the absence of information but its efficient retrieval during the generation process. This insight has profound implications for the development of more reliable AI applications, suggesting that optimizing recall mechanisms at inference time can yield substantial improvements without necessarily demanding larger models or an over-reliance on external databases.
Knowledge Profiling: A New Lens for Measuring Model Understanding
To precisely map this critical gap between knowledge storage and knowledge retrieval, the researchers propose a fundamental shift in evaluation methodology. Instead of solely relying on question-level accuracy, they advocate for a move towards "fact-level profiling." This innovative approach moves beyond a simple binary assessment of whether an LLM answers an isolated prompt correctly or incorrectly. Fact-level profiling involves rigorously testing a single underlying piece of information across a diverse range of conditions. This allows for a nuanced understanding of whether a fact is truly stored within the model’s parameters, its accessibility from various angles and phrasings, and the computational effort required to retrieve it.
This sophisticated framework introduces a crucial distinction between a fact being parametrically "encoded" and a model "knowing" that fact. A model is considered to have encoded a fact if it can accurately reproduce it when presented with its original training context. Conversely, a model knows a fact if it can reliably answer questions pertaining to that fact across a multitude of varied phrasings and directions of inquiry. The researchers articulate this distinction powerfully: "Encoding and recall failures are indistinguishable under accuracy metrics, yet they imply different limitations and solutions. Encoding failures call for pre-training interventions, such as scaling model size or data coverage. Recall failures suggest post-training interventions that often improve how models utilize what they already encode."
To illustrate this concept, the study uses a simple yet illustrative example: the fact that the band Oasis played their first gig at the Boardwalk club. By analyzing how LLMs process this information under different experimental conditions, the research categorizes knowledge into five distinct profiles, providing a granular view of model capabilities. These profiles help to visualize the spectrum from complete ignorance to robust, accessible knowledge.
Scaling Illusions, Long-Tail Knowledge, and Tip-of-the-Tongue Recoveries
The researchers’ extensive evaluation involved 13 different LLMs, analyzing over 4 million responses to a meticulously curated benchmark called WikiProfile. This benchmark is comprised of 2,150 facts extracted from Wikipedia, designed to test each fact across a wide spectrum of formats, from exact context completion to multiple-choice verification.
The findings for frontier models are particularly striking. For models like GPT-5 and Gemini-3, the encoding of information is approaching saturation, with 95-98% of tested facts successfully encoded. However, a significant disconnect emerges: these models still fail to directly recall 26-34% of these encoded facts without employing additional computational effort. This phenomenon is akin to the human experience of being on the "tip of the tongue," where the information is present but momentarily inaccessible.
Crucially, the study reveals that "inference-time thinking," a mechanism that provides models with extra computational steps, acts as a vital recovery tool. This additional processing successfully retrieves 40-65% of the encoded facts that models initially fail to recall directly. This suggests that LLMs, much like humans, can benefit from deliberate cognitive effort to access stored information.
The pervasive belief that simply scaling up model size is the panacea for LLM inaccuracies is directly challenged by this research. The study highlights that companies frequently misdiagnose recall failures as encoding deficiencies, leading them to invest heavily in fine-tuning larger internal models – an expensive and ultimately misdirected architectural choice. Nitay Calderon, a Research Scientist at Google, elaborates on this point: "When facts come out wrong, the go-to move is to scale, meaning train a larger model or add more data. Both are expensive, and if the facts are already encoded, neither helps."
An illustrative example comes from the scaling of the Gemma3 model. While increasing its parameters from 1 billion to 27 billion significantly reduced encoding failures from 85% to 23%, effectively filling the "empty shelves" of knowledge, it paradoxically increased the proportion of recall failures. The share of facts that became inaccessible without additional processing peaked at 40%. This suggests that scaling primarily addresses the storage problem, not the access problem. As models accumulate vast amounts of knowledge, a larger pool of information can become trapped in an "encoded but inaccessible" state, shifting the dominant source of model errors from missing data to failed recall.
The researchers posit that "recall is tightly coupled to the conditions under which facts were learned, degrading when queries diverge from training-time patterns." This underscores the profound impact of query formulation on a model’s ability to unlock stored answers. Even when facts are encoded at rates similar to popular ones, the study identifies a substantial recall gap for "long-tail" or less frequent facts, exceeding 25% for frontier models.
Furthermore, the research highlights challenges in generating answers to reverse questions. For instance, a model might readily state that Oasis played their first gig at the Boardwalk club but struggle to identify who played at that venue. Yet, when presented with the same information in a multiple-choice format, the model demonstrates knowledge of the correct answer. The researchers interpret these phenomena not as limitations in memorization or bidirectional encoding, but as recall failures. They state, "Whereas these failures are often interpreted as limitations of memorization or bidirectional encoding, our results suggest a different picture: rare facts are often encoded but inaccessible, and reverse facts can be recognized even when they cannot be generated. This reframes both phenomena as recall failures rather than ‘missing knowledge.’"
The Return on Investment of "Thinking" and Practical Guidance for Developers
The remarkably high encoding rates observed in frontier models necessitate a strategic reorientation for developers regarding factuality and pipeline architecture.
A primary takeaway is the imperative to "Don’t treat every factual failure as a retrieval problem." The default response to LLM hallucinations in enterprise settings often involves deploying Retrieval-Augmented Generation (RAG), scaling vector databases, or ingesting more domain-specific documents. While RAG is an effective solution for incorporating fresh or proprietary data, employing it as a universal fix for hallucinations introduces unnecessary latency and costs for facts that the model already possesses in its parametric memory. Calderon emphasizes this point: "A lot of what teams solve with RAG are facts the model can already answer from memory, so you’re paying extra latency and per-call cost for nothing. If a fact is truly missing, RAG can be the right fix. But if the fact is encoded and the model just can’t recall it, RAG and scaling the model only add cost on top of the real problem."
Secondly, developers are advised to "Use inference-time reasoning selectively." The study found that "thinking" capabilities successfully recovered 40-65% of encoded facts that models initially failed to recall directly. However, since only an estimated 10-20% of facts truly require this deeper reasoning, applying it universally can be a significant drain on compute budgets. The central challenge lies in dynamically routing queries, as current models lack the self-awareness to reliably predict when they are about to err. Calderon explains, "To use the compute well, the model has to sense ahead of time that a plain answer is about to fail, so it can escalate before answering. That self-awareness is its own skill, and today’s models aren’t reliably good at it." This metacognitive bottleneck is a key area of ongoing research, driving the development of frameworks like "faithful uncertainty," which aim to equip LLMs with the ability to accurately gauge their confidence and trigger more profound reasoning processes instead of resorting to hallucinations.
A third practical recommendation is to "Deploy generate-then-verify pipelines." Given that LLMs demonstrate superior capability in recognizing facts (verification) compared to generating them from scratch, developers can architect systems that involve a model generating a response, followed by an explicit prompt to reflect upon and verify its own claims. Calderon notes, "Since recognizing a correct answer is easier than generating one, a verify pass over the model’s own output could catch mistakes that plain generation misses and add some factual improvement on top."
Furthermore, the research strongly advocates for "Test semantic access, not just benchmark accuracy." Standard accuracy metrics can obscure the underlying capabilities of a model. Evaluation sets should be designed to probe the same underlying fact through various phrasings, contexts, and directions to gain a genuine understanding of what a model truly knows versus what it can reliably access.
Finally, developers should "Leverage query reformulation and retries." The context-dependent nature of recall means that prompt framing critically influences success. Techniques such as altering the structure of a prompt, generating relevant intermediate context, or instructing the model to develop a reasoning chain before providing an answer are legitimate and effective mechanisms for surfacing information that direct prompts might miss.
Limitations and Practical Takeaways for Enterprise AI
While the findings of this study are highly significant, it is important to acknowledge certain limitations. The WikiProfile benchmark, by its nature, relies on encyclopedic facts from Wikipedia. Consequently, these findings may not perfectly generalize to proprietary or highly specialized enterprise domains. A model’s capacity to store and recall a niche internal company metric might differ significantly from its handling of publicly available encyclopedic data.
The cost of fully profiling a frontier model on the WikiProfile suite is estimated at approximately $500. However, developers can substantially reduce this expense by omitting multiple-choice variants or employing fewer response samples per question.
The WikiProfile benchmark is accessible on Hugging Face (https://huggingface.co/datasets/google/WikiProfile), enabling teams to evaluate their own systems. The benchmark includes the exact prompts used in its creation, allowing enterprise data engineering teams to replicate the pipeline on their internal corpora. This enables them to diagnose whether their bespoke agents are suffering from missing data or, more critically, from issues with accessing existing knowledge. However, organizations should temper their expectations when transitioning from encyclopedic data to more specialized domains. As Calderon cautions, "The pipeline is built to be applied on a new corpus, and we provide all the prompts we used. The one thing to expect: on Wikipedia it was mostly a recall problem. Domain-specific facts may genuinely not be encoded in the model."
Ultimately, this research marks a pivotal shift in how we understand and address LLM factuality. It levels the playing field for enterprise AI stacks, particularly for companies that do not develop models from scratch. Calderon concludes, "For companies that don’t build models from scratch, this is good news. Pre-training is hugely expensive and out of reach for most, but the levers that matter now are not: post-training can help with little data and few steps, and inference-time tools like thinking, verify steps, and retrieval are already what most teams use." This paradigm shift empowers developers to focus on optimizing the usage of knowledge that models already possess, leading to more efficient, cost-effective, and reliable AI applications.
This story was updated to include remarks from Google.
