A Neutrino Model Knew More Than It Was Using

Paper: 2608.26090 Authors: Raphaël Bonnet-Guerrini, Johann Ioannou-Nikolaides, Inar Timiryasov, Vincenzo Piuri Categories: astro-ph.HE, cs.AI, cs.LG, hep-ex

The Gap

Mechanistic interpretability has largely been developed on language models, where its practitioners search for latent features corresponding to concepts they can articulate. Particle physics offers a setting where the concepts are already formalised — energy, direction, quality, background nuisance — which makes it a natural place to ask a sharper question: does a scientific foundation model’s internal representation contain physical structure that its task head is not using?

That question has an unusual shape. In interpretability work the interesting failure is usually that a concept is absent, or entangled, or spuriously encoded. Here the failure of interest is one of underuse: a representation that encodes validated physics, feeding a head that ignores most of it. If that is happening, the model is discarding information it demonstrably possesses, and knowing it gives you a concrete engineering opportunity rather than a curiosity.

Getting to that claim requires clearing a high bar first. Any latent found in a representation can be a coincidental correlate; separating a validated atlas of concepts from an artefact requires a validation protocol strict enough to be falsifiable.

   SETTING: a scientific foundation model

   language models: concepts are the thing you are
                    trying to articulate
   particle physics: concepts are ALREADY FORMALISED
                     (energy, direction, quality, nuisance)
        |
        +-- makes a sharper question askable
        |
        v
   [THE QUESTION]
     does the internal representation encode validated
     physical structure that the TASK HEAD does not use?
        |
        v
   usual interpretability failure: concept absent,
                                   entangled, spurious
   failure of interest here:       UNDERUSE
        |
        v
   [BAR TO CLEAR] any latent could be a coincidental
                  correlate -> need a strict,
                  falsifiable validation protocol

The Increment

One sentence: Before this paper, it was unknown whether a scientific foundation model’s representation contained physical structure its task head ignored; after it, a strictly validated concept atlas shows the direction head barely uses it, and a new head built on that atlas cuts median angular resolution from 20.2 to 3.2 degrees.

Core Mechanism

The model under study is a neutrino foundation model pretrained on IceCube data and fine-tuned for direction reconstruction. Sparse autoencoders are applied to its representations — the same dictionary-learning approach used in language model interpretability, chosen because it decomposes a dense representation into a sparse set of features that can be examined individually.

The validation protocol is the part that makes the atlas trustworthy, and it has three components that each close a different loophole:

• Held-out tests — features must generalise beyond the data they were found on.
• Matched nuisance controls — features are checked against matched conditions where the supposed physical driver is absent, so a feature that merely tracks a nuisance variable is caught.
• Replication across independent dictionary trainings — the same concept must reappear when the dictionary is learned from scratch. This is the strongest of the three: a latent that only appears under one particular training run is an artefact of that run, not a property of the representation.

What the protocol yields is a validated atlas of physical concepts in the model representation — established structure, not candidate structure.

Then comes the pivot, and it is a causal result rather than a correlational one. Causal interventions show that the direction head barely draws on this atlas. The head producing the model’s actual output is not using the physics that the representation validates. This is the underuse finding, and it is established by intervention: perturbing the concepts in the atlas does not much move the head, which is why “barely draws on” is a mechanism claim rather than an observation about correlation.

The constructive response follows directly from the diagnosis. Motivated by this underused information, the authors train an uncertainty head on the same event-level representation to predict the model’s angular reconstruction error. And the new head behaves differently: unlike the direction head, it depends causally on quality and brightness features from the atlas. So the atlas is not decoration — a task built on the same representation can be made to use it.

The payoff is a number a physicist can act on. At 20% selection efficiency, this interpretable estimator improves the median angular resolution from 20.2° to 3.2°. The estimator is a quality filter: keep the best 20% of events by predicted uncertainty, and the surviving set has a median resolution six times better. It is interpretable, because it works by reading named physical features rather than an opaque score.

   PIPELINE

   [ neutrino foundation model, IceCube pretrain,
     fine-tuned for direction reconstruction ]
                    |
                    v
   SPARSE AUTOENCODERS -> candidate features
                    |
                    v
   VALIDATION PROTOCOL (three loopholes closed)
     [1] held-out tests                  (generalise?)
     [2] matched nuisance controls       (real driver?)
     [3] replication across independent
         dictionary trainings            (not a run artefact?)
                    |
                    v
   VALIDATED ATLAS of physical concepts
                    |
                    v
   CAUSAL INTERVENTION: the DIRECTION HEAD
     barely draws on the atlas
     -> the model encodes physics its own output ignores
                    |
                    v
   BUILD an UNCERTAINTY HEAD on the same representation
     -> it DOES depend causally on quality / brightness
        features from the atlas
                    |
                    v
   at 20% selection efficiency:
     median angular resolution  20.2 deg  ->  3.2 deg

Think of it as a hospital discovering that its best scanner was being read by a technician who only looked at one corner of the image. The scanner’s raw output contains density, texture and edge information that a careful reader would use; nobody doubted the machine. The problem is in the reading. Two things follow, and the paper does both. First, demonstrate the underuse causally — show that annotating features in the image does not change the technician’s verdict, which proves they are not looking there. Second, build a second reader that does use those annotations, and show the diagnostic quality improves. And note the validation discipline: before claiming a feature is “density”, you would check it on held-out scans, against a control where the supposed cause is absent, and confirm it reappears when the annotation procedure is redone from scratch.

Key Concepts

• Underuse as an interpretability finding: a representation containing validated structure that the task head ignores. It differs from the usual failure modes — absence, entanglement, spurious encoding — and it is immediately actionable, because the missing capability is already present in the model.
• A three-part validation protocol: held-out tests, matched nuisance controls, and replication across independent dictionary trainings. Replication is the strongest element, since it distinguishes a property of the representation from an artefact of one dictionary’s training run.
• Causal intervention to establish use: perturbing atlas concepts and observing the effect on the head’s output. This is what upgrades “the head correlates with the atlas” into “the head depends on it”, and the distinction is the difference between a diagnosis and a coincidence.
• Interpretable uncertainty estimation: predicting reconstruction error from named physical features rather than an opaque score. It is what makes the efficiency gain usable in analysis, since a physicist can inspect why an event was rejected.

Framework Shift

Before (evaluate the output head):
  foundation model -> direction head -> reconstruction
  question asked: how accurate is the output?
  representation treated as an internal detail

After (evaluate what the representation contains
       and whether the head uses it):
  sparse autoencoders -> validated atlas of concepts
  causal intervention -> direction head BARELY uses it
  -> build an uncertainty head that DOES
  -> 20% selection efficiency: 20.2 deg -> 3.2 deg

From asking how accurate a model’s output is, to asking what its representation knows and whether the output path uses it, the core shift is that a model can hold validated structure it fails to exploit, and finding that structure is a route to new capability rather than a description of existing capability.

Expert Assessment

Problem choice: Excellent, and a genuinely novel application. Bringing mechanistic interpretability into particle physics is interesting partly because the domain’s concepts are already formalised, which removes the fuzziest part of interpretability work and lets the question be sharp: does the model know physics it is not using? Framing the finding as underuse rather than absence is what makes it constructive.

Method maturity: The validation protocol is the methodological substance and it is well designed — three independent checks, each aimed at a distinct way a latent could be spurious, with replication across independent dictionary trainings doing the heaviest lifting. The pivot from diagnosis to construction is the strongest part of the paper: training an uncertainty head that depends on the atlas causally, and demonstrating it improves a quantity physicists care about, closes the loop in a way that a purely diagnostic paper would not.

Experimental integrity: The causal intervention is the right instrument for the central claim, and the reported improvement is dramatic and concrete. The honest reading is that the direction-head finding and the uncertainty-head gain are separate claims: the first is about underuse, the second about a new task that happens to benefit. The paper connects them by motivation, and the connection would be tighter if it showed that the direction head’s underuse is what limits it, rather than that a different head can exploit the atlas independently.

Writing quality: The structure — atlas, validation, causal underuse, constructive use — is clear and the result is stated in units the domain uses. Because this is a first application of the technique to the field, a short exposition of what a sparse-autoencoder feature means for a physics reader would widen the audience considerably, as would one example of an atlas feature described in physical terms.

Verdict: strong accept — a well-validated interpretability result with a constructive payoff, and a convincing demonstration that a scientific foundation model was underusing validated physics it already encoded.

Takeaways

• Ask what a representation contains, not only how accurate the output is. Here the model encoded validated physics its own head barely used, and the gap was the opportunity.
• Validate latents with replication across independent dictionary trainings. A concept that reappears under a fresh dictionary is a property of the representation; one that does not is a property of the training run.
• Establish use causally. Perturbing the concept is what separates “the head depends on this” from “the head is correlated with this”, and only the former justifies building on it.
• Build new heads on existing representations before retraining. Training a different task head on an unchanged backbone captured the unused information at a fraction of the cost of a new model.