Predictable Confabulations: How Model Size and Data Frequency Govern LLM Factual Recall

Paper: 2605.18732 Authors: Matthew L. Smith, Jonathan P. Shock, Samuel T. Segun, Iyiola E. Olatunji, Tegawendé F. Bissyandé Categories: cs.CL, cs.AI, cs.LG

The Gap

We know scaling laws predict aggregate LLM performance — more parameters, better benchmarks. But when you ask GPT-4 about a specific fact, why does it nail some and hallucinate others? Existing scaling laws treat the model as a black box averaging over all tasks. They don’t explain why the same model recalls “Einstein’s 1905 papers” flawlessly but fabricates citations for obscure 1987 conference proceedings.

Prior work (Carlini et al., Kandpal et al.) showed training data frequency matters for memorization. Separately, we know bigger models store more facts. But no one had quantified how these two variables — model size and topic frequency in training data — jointly determine factual recall quality. The gap: no predictive model linking architectural scale and data composition to fact-level performance.

Problem: Why do LLMs recall some facts but hallucinate others?
   |
   v
Hypothesis: Recall = f(model_size, data_frequency)
   |
   v
Method: Test 38 models on 8,900 scholarly references
        Measure: parameter count + topic frequency in training corpus
   |
   v
Evidence: Sigmoid fit in log-linear space
          R^2 = 0.60 across families, 0.74-0.94 within families
   |
   v
Conclusion: Signal-to-noise ratio gates recall
            Signal ~ frequency, Noise floor ~ 1/capacity

The Increment

One sentence: Before this paper, we knew bigger models and more training examples help separately; after, we have a quantitative law predicting factual recall from the interaction of model capacity and concept frequency in training data.

Core Mechanism

The authors treat factual recall as a signal detection problem. Each fact in the training corpus generates a “signal” proportional to how often it appears. The model’s internal representation space has a “noise floor” that decreases as parameter count grows — more capacity means cleaner separation between stored concepts.

They tested 38 models (7M to 405B parameters) from families like Llama, Qwen, Gemma, and Phi on 8,900 scholarly references spanning physics, math, CS, and biology. For each reference, they measured two things: (1) how many parameters the model has, and (2) how frequently the reference’s topic appears in a proxy training corpus (arXiv papers). They used an automated citation verification system to score recall quality on a 0-1 scale.

The key finding: recall quality follows a sigmoid curve when plotted against log(params) + β·log(frequency). The sigmoid shape means there’s a threshold — below it, the model mostly hallucinates; above it, recall becomes reliable. The log-linear combination captures the multiplicative interaction: doubling parameters has the same effect as increasing topic frequency by a factor of 2^(1/β).

Training Data               Model Internals              Output
--------------             -------------------          --------
Topic appears              Concept vector               Recall
N times                    embedded in                  quality
    |                      parameter space              score
    v                            |                         ^
Frequency                  Signal strength               |
signal                     = log(N)                      |
    |                            |                        |
    +----------> Combined -----> SNR -------> Sigmoid -->+
    |            with model      threshold    activation
    |            capacity                     function
    v                            ^
log(params)                Noise floor
increases                  = -log(params)
capacity

Think of the model as a radio receiver trying to pick up stations. Each fact is a radio station broadcasting at a power proportional to how often it appeared in training (frequency = signal strength). The model’s parameter count determines the quality of its antenna and noise-canceling circuitry (capacity = noise suppression). A weak station (rare fact) needs a really good receiver (large model) to come through clearly. A strong station (common fact) can be heard even on a cheap radio (small model). But there’s a threshold: below a certain signal-to-noise ratio, you get static (hallucinations); above it, you get clear audio (accurate recall). The sigmoid curve is the transition from static to clarity as you turn up either the broadcast power or the receiver quality.

Key Concepts

• Superposition in neural networks: Imagine a library where books are stored not on individual shelves but as holograms — each hologram contains fragments of many books overlapping in the same physical space. When you shine light through the hologram at the right angle (query the network), one book’s image emerges clearly while others fade into noise. Neural networks store concepts this way: a single parameter participates in representing many different facts simultaneously. The “right angle” is the activation pattern that isolates one concept from the superposition. This works because most concepts are only needed rarely, so their representations can overlap without constant interference. But if two concepts are too similar or the storage is too crowded (small model, many facts), they interfere destructively — you get a blurry image (hallucination) instead of a clear one (correct recall).

• Signal-to-noise ratio (SNR) as a gating mechanism: In communication theory, SNR determines whether a message gets through. Below a threshold, the receiver can’t distinguish signal from noise — it might as well be guessing. Above the threshold, error rates drop exponentially. The authors propose LLM recall works the same way: each fact has a signal strength (training frequency) and the model has a noise floor (inversely proportional to capacity). When signal/noise > threshold, the model reliably recalls the fact. When signal/noise < threshold, it confabulates. The sigmoid shape in their data is the hallmark of this threshold behavior — it’s not a gradual improvement but a phase transition from unreliable to reliable recall.

Framework Shift

Before (aggregate scaling laws):        After (this paper):

Performance                              Recall Quality
    ^                                        ^
    |     /                                  |         ___---
    |    /                                   |      _--
    |   /                                    |    _-
    |  /                                     |  _-
    | /                                      | /
    |/                                       |/___________
    +----------> Model Size                  +----------> log(params) + β·log(freq)
    
Single curve for all tasks               Fact-specific curves
No role for data composition             Data frequency is load-bearing

One sentence: From treating models as uniform black boxes that improve monotonically with scale, to recognizing factual recall as a threshold phenomenon gated by the interaction of architectural capacity and training data composition.

Expert Assessment

Problem choice: This is a real gap. Scaling laws have been phenomenally useful for predicting aggregate performance, but practitioners care about reliability on specific facts — especially in domains like medicine or law where hallucinations are costly. The choice to focus on scholarly references is smart: they’re verifiable, span difficulty levels, and have measurable frequency in training corpora. The problem sits at the intersection of interpretability (why do models fail?) and practical deployment (when can we trust them?).

Method maturity: The approach is elegant but not novel — it’s applying signal detection theory to LLM behavior. The real contribution is empirical: they actually measured topic frequency in a proxy corpus and tested across 38 models. The sigmoid fit is clean, but the 60% variance explained across families suggests other factors matter (architecture differences, tokenization, training dynamics). The within-family R² of 74-94% is more convincing. One concern: they use arXiv as a proxy for training data, but models are trained on web scrapes with different distributions. The correlation might hold because arXiv frequency correlates with general web frequency, not because it’s the actual training data.

Experimental integrity: Baselines are fair — they compare models from different families at different scales. The automated citation verification is a strength (no human annotation bias), but also a limitation (it can’t catch subtle errors like correct author, wrong year). The 8,900 references are well-distributed across fields, but all are scholarly citations — does this generalize to other fact types (historical events, product specs)? The paper doesn’t test that. The sigmoid fit is convincing, but they don’t show residuals or test alternative functional forms rigorously. A Bayesian model comparison would strengthen the claim that sigmoid-in-log-space is the right functional form.

Writing quality: The abstract and introduction are crisp. The methods section is dense but complete. The results section leans too heavily on figures — the text should walk through the key numbers more explicitly. The discussion connects to superposition theory, which is intellectually satisfying, but the mechanistic link is speculative (they don’t probe model internals to confirm superposition). The paper would benefit from a limitations section acknowledging the proxy corpus issue and the narrow fact domain. The related work section undersells prior work on memorization (Carlini, Kandpal) — this paper is more incremental than the framing suggests.

Verdict: weak accept — Solid empirical contribution with a clean quantitative result, but the mechanistic story is underdeveloped and the generalization beyond scholarly citations is untested.

Takeaways

For practitioners: If you’re deploying an LLM in a domain where factual accuracy matters, this paper gives you a heuristic for risk assessment. Facts that are rare in training data need much larger models to recall reliably. You can estimate risk by checking how often your domain appears in web corpora (use Common Crawl statistics as a proxy). If you’re stuck with a smaller model, focus on facts that are well-represented in training data or use retrieval augmentation for rare facts.

For researchers: The sigmoid-in-log-space functional form is a testable prediction. If you’re working on interpretability, check whether this pattern holds for other fact types (not just citations). If you’re training models, this suggests a data curation strategy: for facts you care about, ensure they appear above the frequency threshold for your model size. The SNR framing also suggests a path to better prompting: if you can increase “signal” by providing context that primes the relevant concept, you might push marginal facts over the recall threshold.

For theorists: The connection to superposition is tantalizing but needs mechanistic validation. Can you probe model internals to show that rare facts have noisier representations? Does the noise floor actually scale as 1/params, or is that an approximation? The sigmoid shape suggests a phase transition — is there a statistical mechanics analogy that predicts the transition point from first principles?