Cross-Sample Prediction Churn: The Hidden Instability in Scientific ML

Paper: 2605.13826 Authors: Gordan Prastalo, Kevin Maik Jablonka Categories: cs.LG, cond-mat.mtrl-sci, physics.chem-ph

The Gap

Scientific ML benchmarks report test accuracy. They don’t report whether your model’s predictions would survive if you’d drawn a different training set from the same distribution. Train two classifiers on independent bootstraps of the same data — they’ll agree on aggregate accuracy to within 1-4 percentage points, but disagree on the predicted class for 8-22% of test molecules. The field has been optimizing for a metric (accuracy) while ignoring the stability of individual predictions.

Existing uncertainty methods (deep ensembles, MC dropout, stochastic weight averaging) all operate on the parameter side — they vary model weights while keeping training data fixed. None of them address the question: would this prediction hold if I’d sampled different training examples?

Problem: Models agree on accuracy, disagree on predictions
   |
   v
Hypothesis: Parameter-side methods can't see data-side instability
   |
   v
Method: Twin-bootstrap (two nets, two data samples, consistency loss)
   |
   v
Evidence: 45% churn reduction beyond bagging at matched compute
   |
   v
Conclusion: Data-side variation matters; needs its own metric

The Increment

One sentence: Before this paper, scientific ML treated prediction disagreement as noise; after, it’s a measurable quantity with dedicated reduction methods.

Core Mechanism

Twin-bootstrap trains two neural networks simultaneously, each on an independent bootstrap sample of the training data. During training, the networks see different subsets of examples, but they’re forced to agree on their predictions through a symmetric KL divergence loss added to the standard classification loss.

The architecture is simple: two identical networks with separate parameters. Each forward pass processes a batch through both networks. Network A sees bootstrap sample 1, network B sees bootstrap sample 2. The total loss has three terms: classification loss for A, classification loss for B, and a consistency term that penalizes disagreement between A’s and B’s predicted probability distributions.

At test time, you average the predictions from both networks. The key insight is that by explicitly training for agreement across different data samples, you get models that are less sensitive to which specific training examples they happened to see.

Training:
  Bootstrap 1 --> Network A --> Predictions A --\
                                                  \
                                                   >-- sym-KL loss
                                                  /
  Bootstrap 2 --> Network B --> Predictions B --/
                    |                |
                    v                v
              Loss_A (CE)      Loss_B (CE)
                    |                |
                    \----------------/
                            |
                            v
                  Total = Loss_A + Loss_B + lambda * sym-KL

Inference:
  Test sample --> Network A --\
                               >-- Average --> Final prediction
  Test sample --> Network B --/

Think of it like training two apprentices who study different textbooks but must defend their answers to each other. Each apprentice (network) learns from different examples (bootstrap sample), but they’re graded not just on getting the right answer, but also on whether they agree with each other. When they disagree, both get penalized, forcing them to converge on predictions that are robust to which specific examples they studied. The disagreement penalty acts like peer review during training — it filters out predictions that are artifacts of particular training samples.

Key Concepts

• Cross-sample prediction churn: Train two models on independent bootstraps of the same dataset. They’ll have similar test accuracy, but they’ll disagree on the predicted class for a significant fraction of test examples. That disagreement rate is the churn. It measures how much your predictions depend on the random draw of training data, not model randomness. If you train on Monday’s bootstrap and predict “toxic” for molecule X, then train on Tuesday’s bootstrap and predict “safe” for the same molecule X, that’s churn. It’s not about confidence intervals or epistemic uncertainty — it’s about whether the same input gets the same output when you resample training data.

• Parameter-side vs data-side methods: Parameter-side methods (ensembles, dropout, weight averaging) vary model parameters while keeping training data fixed. They answer: “Given this training set, how uncertain is my model?” Data-side methods vary training data while tracking prediction stability. They answer: “If I’d sampled different training examples, would I get the same prediction?” The paper shows these are orthogonal dimensions. Deep ensembles reduce parameter uncertainty but don’t touch data-side churn. Bagging reduces churn but doesn’t help with parameter uncertainty. You need both.

• Symmetric KL divergence: Standard KL divergence KL(P || Q) measures how much information you lose when approximating distribution P with distribution Q. It’s asymmetric — KL(P || Q) ≠ KL(Q || P). Symmetric KL is just the sum: KL(P || Q) + KL(Q || P). In twin-bootstrap, P is network A’s prediction, Q is network B’s prediction. The symmetric version penalizes both networks equally for disagreement, so neither network can “dominate” the other. It’s a mutual consistency constraint, not a teacher-student setup.

Framework Shift

Before (standard training):          After (twin-bootstrap):

  Training data                        Training data
       |                                    |
       v                              +-----+-----+
  Single model                        |           |
       |                              v           v
       v                         Bootstrap 1  Bootstrap 2
  Test predictions                     |           |
                                       v           v
                                   Network A   Network B
                                       |           |
                                       +-----+-----+
                                             |
                                             v
                                    Consistency loss
                                             |
                                             v
                                      Test predictions
                                      (averaged)

Focus: Accuracy                      Focus: Accuracy + Stability
Metric: Test error                   Metric: Test error + Churn rate

From single-model optimization to dual-model co-training, the core shift is treating prediction stability as a first-class objective alongside accuracy.

Expert Assessment

Problem choice: This is a real gap. The field has been reporting accuracy as if it’s the only thing that matters, but in scientific applications (drug discovery, materials design), you care whether a prediction is robust to data sampling. The authors didn’t manufacture this problem — they measured it across 9 chemistry benchmarks and found 8-22% churn rates. That’s not noise; that’s a systematic issue.

Method maturity: Twin-bootstrap is elegant but not groundbreaking. It’s essentially bagging with a consistency loss. The insight is solid: if you want models to agree across data samples, train them to agree. The execution is straightforward — no architectural tricks, no hyperparameter hell. The comparison to parameter-side methods is the real contribution, showing they’re solving different problems. One concern: the method requires 2x training compute. The authors compare to bagging-K=2 at matched compute, which is fair, but practitioners might balk at doubling training time.

Experimental integrity: Baselines are fair. They compare to deep ensembles, MC dropout, SWA, and bagging across 9 datasets. The churn metric is well-defined and consistently measured. One weakness: all experiments are on molecular property prediction. Does this generalize to other scientific domains (genomics, climate, physics)? The paper doesn’t say. The ablation on lambda (consistency loss weight) is minimal — just one plot showing sensitivity. More analysis of when twin-bootstrap helps vs hurts would strengthen the claims.

Writing quality: The abstract and introduction are crisp. The method section is clear. The results section is dense with tables but lacks intuition about *why twin-bootstrap works better than bagging. The discussion of computational cost is buried in the appendix — it should be front and center since 2x compute is a real barrier. The paper would benefit from a failure case analysis: when does twin-bootstrap not help?

Verdict: weak accept — Identifies a real problem, proposes a simple solution, demonstrates effectiveness across multiple benchmarks. The method isn’t novel enough for a strong accept, but the problem framing and the data-side vs parameter-side distinction are valuable contributions. The field needs this metric.

Takeaways

1. Add churn to your benchmark reports: If you’re publishing scientific ML results, report cross-sample prediction churn alongside test accuracy. Train two models on independent bootstraps, measure disagreement rate on test set. It’s cheap to compute and reveals instability that accuracy hides.

2. Consistency losses for stability: When you care about prediction stability, add a consistency term between models trained on different data samples. The symmetric KL formulation is simple and effective. This transfers beyond chemistry — any domain where predictions inform decisions (medical diagnosis, financial risk, infrastructure monitoring).

3. Parameter-side and data-side are orthogonal: Deep ensembles and bagging solve different problems. If you want both low uncertainty and low churn, you need both. Don’t assume one subsumes the other.

4. Bootstrap as a stability diagnostic: Even if you don’t use twin-bootstrap, train models on multiple bootstraps and check prediction agreement. High churn is a red flag that your model is overfitting to specific training examples rather than learning robust patterns.