HALP: Catching Hallucinations Before They Happen

Paper: 2603.05465 Authors: Sai Akhil Kogilathota, Sripadha Vallabha E G, Luzhe Sun, Jiawei Zhou Categories: cs.CV

The Gap

Vision-language models confidently describe things that aren’t there. Show them a photo of a dog, they might tell you about a cat. Ask about a street scene, they’ll invent pedestrians. Current hallucination detection methods—like CHAIR, POPE, or self-consistency checks—all share a fatal flaw: they wait until after the model has generated text. By then, you’ve already paid the computational cost of decoding, and the damage is done. Some approaches try to fix hallucinations mid-generation using contrastive decoding or attention manipulation, but these still require partial generation and add overhead to every token.

The core question nobody had answered: can you tell if a model is about to hallucinate before it outputs anything?

Problem: VLMs hallucinate + detection happens post-generation
   |
   v
Assumption: Hallucination risk is encoded in pre-generation states
   |
   v
Method: Train probes on internal representations (visual/vision-token/query-token)
   |
   v
Evidence: 0.93 AUROC on multiple models without decoding
   |
   v
Conclusion: Pre-generation detection is viable + layer/modality varies by architecture

The Increment

One sentence: Before this paper, you had to generate text to detect hallucinations; after it, you can predict hallucination risk from a single forward pass before any decoding.

Core Mechanism

HALP extracts three types of internal representations from vision-language models and trains lightweight classifiers on them. First, visual-only features come straight from the vision encoder before any text interaction—these are the raw visual embeddings. Second, vision-token representations are extracted from within the text decoder after visual information has been projected into the language model’s space. Third, query-token representations capture the fused state right before generation begins, where the model has processed both the image and the input query.

For each representation type, the authors extract features at multiple layers (early, middle, late) and train simple probes—logistic regression or small MLPs—to classify whether the model will hallucinate on a given input. The training data comes from existing hallucination benchmarks where ground truth labels indicate whether a model’s response contains fabricated content.

Image + Query
    |
    v
[Vision Encoder] -----> (i) Visual-only features
    |                        |
    v                        v
[Projection] -----> (ii) Vision-token repr.    [Probe]
    |                        |                     |
    v                        v                     v
[Text Decoder] ---> (iii) Query-token repr. --> Hallucination
    |                                            prediction
    v
[Generation] <--- (we never reach here)

Think of it like airport security screening. Traditional methods are like checking passengers after they’ve boarded the plane—you catch problems, but the flight is already delayed. HALP is the metal detector at the entrance: it scans internal “signals” (representations) before anyone gets on board (before generation). The visual-only features are like checking someone’s ID photo—pure visual information. Vision-token representations are like checking their boarding pass after it’s been stamped—visual info that’s been processed into the system’s format. Query-token representations are like the final security checkpoint where they’ve shown both ID and boarding pass and are about to enter the gate—the last moment before the irreversible action (generation) begins. The probe is the security officer who’s learned to spot suspicious patterns in these signals.

Key Concepts

• Internal Representations: When a vision-language model processes an image and query, it doesn’t jump straight to generating text. Instead, it builds up layers of internal understanding—numerical vectors that encode what the model “thinks” about the input at different stages. Early layers might capture basic visual features like edges and colors. Middle layers might recognize objects. Late layers integrate visual and textual context to prepare for generation. These representations are like the model’s internal notes before it commits to an answer. The key insight: if the model is confused or uncertain (hallucination-prone), that confusion shows up in these internal states before any text appears.

• Probing: Instead of trying to interpret what these internal representations mean directly, you train a simple classifier—a probe—to predict outcomes from them. It’s like training a dog to sniff for drugs without teaching it chemistry. You show the probe thousands of examples: “when the internal state looks like this, the model hallucinated; when it looks like that, it didn’t.” The probe learns patterns that correlate with hallucination risk. Crucially, probes are lightweight (logistic regression or tiny neural nets), so they add minimal overhead. They’re diagnostic tools, not part of the main model.

• Pre-generation Detection: Traditional approaches are reactive—they analyze generated text for inconsistencies or check it against the image. Pre-generation detection is predictive—it assesses risk before committing to generation. Imagine a chess player who can tell they’re about to blunder before moving the piece, versus one who only realizes after the move is made. The practical win: you can abort generation early, route the query to a more reliable model, or switch to a safer decoding strategy—all before wasting compute on a doomed output.

Framework Shift

Before (mainstream approach):        After (this paper):

Image + Query                        Image + Query
    |                                    |
    v                                    v
[VLM generates text]                 [Single forward pass]
    |                                    |
    v                                    v
"The cat is on the mat"              [Extract internal repr.]
    |                                    |
    v                                    v
[Post-hoc detection]                 [Probe predicts risk]
    |                                    |
    v                                    v
"Wait, there's no cat!"              "High hallucination risk"
    |                                    |
    v                                    v
(too late, already generated)        (abort or reroute)

From post-mortem analysis to pre-flight risk assessment, the core shift is moving the detection boundary from after generation to before it.

Expert Assessment

Problem choice: This is a real gap with practical stakes. Hallucination detection is a bottleneck for deploying VLMs in high-stakes domains (medical imaging, autonomous vehicles, accessibility tools). Post-generation detection is expensive and doesn’t prevent bad outputs from being generated in the first place. The timing is right—VLMs are mature enough that people care about reliability, not just capability.

Method maturity: The approach is refreshingly simple—train probes on internal states. No architectural changes, no complex decoding schemes. The insight is clever: hallucination risk should be detectable in the model’s “uncertainty” before it commits. However, the paper doesn’t deeply investigate why certain layers work better for certain architectures. The probe design is basic (logistic regression, small MLPs), which is fine for a proof of concept but leaves room for more sophisticated uncertainty quantification methods.

Experimental integrity: The evaluation spans eight models and multiple benchmarks (POPE, CHAIR, MMHal-Bench), which is solid coverage. The AUROC numbers are strong (0.93 on some models), but there’s a catch: performance varies wildly by architecture (0.79 on Qwen2.5-VL vs 0.93 on Gemma-3). The paper acknowledges this but doesn’t fully explain it—are some architectures inherently more “readable” for probing? The baselines are fair (comparing against post-generation methods), but I’d want to see ablations on probe complexity and training data size. One red flag: no discussion of adversarial robustness—can you fool the probe by crafting inputs that look safe internally but still hallucinate?

Writing quality: The abstract and intro are crisp. The method section is clear but could use more intuition about why different representation types matter. The results section is thorough but repetitive—tables could be condensed. The biggest missed opportunity: no failure analysis. When does the probe fail? What kinds of hallucinations slip through? A single figure showing failure modes would make this paper twice as useful.

Verdict: weak accept — Solid empirical contribution with practical value, but lacks depth in explaining architectural variance and failure modes.

Takeaways

The transferable idea: internal representations encode uncertainty before it manifests in output. This applies beyond hallucination detection. You could probe for toxicity risk before generating offensive text, factual errors before answering questions, or reasoning failures before solving math problems. The key is identifying the right layer and modality for your architecture—this paper shows it’s not universal (late query tokens work for most models, but not all).

Concrete technique: if you’re deploying a VLM in production, train a lightweight probe on your specific model and task. Collect a dataset of (input, did_it_hallucinate) pairs, extract internal states at multiple layers, and train a classifier. Use it as a gatekeeper: high-risk queries get routed to a more expensive but reliable model, or trigger human review. The computational overhead is minimal (one forward pass + probe inference), and you avoid generating garbage in the first place.

The architectural insight: where hallucination risk lives varies by model design. If your VLM has a strong vision encoder, visual-only features might be predictive. If it relies heavily on cross-modal fusion, query tokens are your best bet. Don’t assume one size fits all—profile your model.