When Physics Priors Meet Scale: AllScAIP's Data-Driven Path to Long-Range Molecular Interactions

Paper: 2603.06567 Authors: Eric Qu, Brandon M. Wood, Aditi S. Krishnapriyan, Zachary W. Ulissi Categories: cs.LG, cond-mat.mtrl-sci, cs.CE, physics.chem-ph, q-bio.QM

The Gap

Machine-learned interatomic potentials (MLIPs) have gotten good at predicting how atoms interact in small systems. Models like MACE, NequIP, and SchNet bake in physics knowledge—equivariance, message passing on local neighborhoods, explicit electrostatic terms—to work efficiently with limited data. But when you scale up to biomolecules or electrolytes with hundreds of atoms, these models hit a wall: long-range interactions (think Coulomb forces between distant charged groups) don’t fit neatly into local message-passing schemes. The field’s workaround? Bolt on explicit physics terms or specialized components to handle long-range effects. This creates a dependency: you need domain experts to design the right physics priors for each new system type.

The core tension is this: physics priors help when data is scarce, but do they become a bottleneck when you have massive datasets? No one had systematically tested whether pure data-driven attention could replace hand-crafted long-range components at scale.

Problem: Local message passing fails for long-range interactions
                    |
                    v
Assumption: Maybe all-to-all attention + scale > physics priors
                    |
                    v
Method: AllScAIP with full attention, train on 100M samples
                    |
                    v
Evidence: SOTA on molecules, stable MD, ablations show priors hurt at scale
                    |
                    v
Conclusion: Data-driven attention scales better than physics shortcuts

The Increment

One sentence: Before, you needed physics experts to design long-range interaction terms for each molecular system; after, you can throw data and attention at the problem and let the model figure it out.

Core Mechanism

AllScAIP has three main components working in sequence. First, a local message-passing layer processes each atom’s immediate neighborhood (within a cutoff radius), building up representations that capture short-range chemistry. Second, an all-to-all attention layer lets every atom attend to every other atom in the system, regardless of distance—this is where long-range interactions get learned. Third, an energy head aggregates these representations into a total energy prediction, with forces derived by taking gradients.

The data flow looks like this: atomic positions and species go in, get embedded, pass through alternating local and global layers, then output energy. Forces come for free via automatic differentiation. The key operation in the attention layer is standard transformer attention, but applied to 3D atomic coordinates with careful handling of equivariance (the model’s predictions shouldn’t change if you rotate the molecule).

Input: Atomic positions + species
         |
         v
    [Embedding]
         |
         v
    [Local MP] ---> captures short-range chemistry
         |
         v
    [All-to-all] -> every atom sees every atom
    [Attention]     (long-range interactions)
         |
         v
    [Energy Head] ---> sum to total energy
         |
         v
    Forces = -gradient(Energy)

Think of AllScAIP like a city’s communication network. The local message-passing is your neighborhood gossip—fast, efficient, covers 90% of daily needs. The all-to-all attention is the internet backbone—expensive to run, but essential for coordinating across distant parts of the city. In small towns (small molecules, little data), you can get by with just neighborhood gossip plus a few dedicated phone lines (physics priors) to key locations. But in a megacity (large systems, massive data), you need the full internet backbone, and those dedicated phone lines actually create bottlenecks because they’re too rigid. The model learns to route information through the attention network more flexibly than any hand-designed physics term could.

Key Concepts

• All-to-all attention for atoms: Imagine you’re at a party trying to predict who’ll leave first. If you only talk to people standing near you (local message passing), you miss that someone across the room just got a text from their babysitter. All-to-all attention means everyone can whisper to everyone else simultaneously. For atoms, this means a charged oxygen on one end of a protein can directly influence a charged nitrogen 50 angstroms away, without information having to hop through intermediate atoms. The cost? O(N²) computation where N is atom count. The benefit? You actually capture the physics correctly instead of hoping local hops approximate it.

• Inductive bias vs. scale trade-off: An inductive bias is a shortcut you give the model based on domain knowledge. It’s like teaching a kid “living things need water” before they learn biology—helps them learn faster with fewer examples. In MLIPs, equivariance (predictions don’t change if you rotate the molecule) is a strong inductive bias. The paper’s key finding: these shortcuts help when you have 10K training examples, but hurt when you have 100M. Why? Because the shortcut constrains the model’s expressiveness. With enough data, a flexible model can learn the pattern AND the exceptions, while the shortcut-laden model is stuck with the approximation you hard-coded.

• Energy conservation in learned potentials: In physics, energy is conserved—you can’t create or destroy it. For a learned model, this means if you predict energy at different atomic configurations, the forces (negative gradient of energy) must be consistent. If your model says moving atom A right decreases energy by 5 units, but moving it left increases energy by 6 units, you’ve violated conservation and your molecular dynamics simulation will explode. AllScAIP ensures this by computing forces as exact gradients of the energy prediction, not as separate outputs. It’s the difference between a GPS that calculates your speed from position changes (consistent) versus one that guesses speed independently (will drift and give nonsense directions).

Framework Shift

Before (mainstream approach):        After (this paper):

Physics priors everywhere:           Minimal priors, maximal data:

[Equivariance] [Cutoff radius]      [Basic equivariance]
      |              |                        |
      v              v                        v
[Message Pass] [Explicit E-static]      [Local MP]
      |              |                        |
      +------+-------+                        v
             |                          [All-to-all]
             v                          [Attention]
        [Energy]                             |
                                             v
                                        [Energy]

Rigid structure,                     Flexible structure,
works with 10K samples               needs 100M samples

From hand-crafted physics components to learned attention patterns, the core shift is trusting data over domain expertise when you have enough of it.

Expert Assessment

Problem choice: This is a real gap, not manufactured. Long-range interactions are a known pain point in computational chemistry, and the field’s reliance on explicit physics terms is a genuine limitation for generalization. The timing is right—datasets have finally reached the scale where this question becomes testable.

Method maturity: It’s more “obvious in hindsight” than clever. All-to-all attention is the straightforward solution everyone avoided because of computational cost. The insight is empirical: showing that scale flips the conventional wisdom about inductive biases. That said, there’s no magic here—they’re throwing compute at the problem. The ablation studies are thorough, which elevates this from a brute-force exercise to a useful investigation.

Experimental integrity: Baselines are fair, covering the major MLIP architectures. The OMol25, OMat24, and OC20 benchmarks are standard. The MD simulation validation (density, heat of vaporization) is crucial—many papers skip this and get away with just force/energy MAE, which can be misleading. One yellow flag: the paper doesn’t deeply explore failure modes or where physics priors still win. The “priors hurt at scale” claim is strong but could use more nuance.

Writing quality: The abstract and intro are crisp. The method section is adequate but could better explain why certain design choices matter (e.g., why this specific attention mechanism over alternatives). The ablation section is the paper’s strength—clear, systematic, tells a coherent story. The related work section feels perfunctory, missing opportunities to position this work in the broader “scale vs. priors” debate happening across ML.

Verdict: weak accept — Solid empirical work that challenges conventional wisdom with thorough experiments, though the method itself is straightforward application of existing techniques at scale.

Takeaways

The transferable idea: when you have 10-100x more data than typical for your domain, revisit every inductive bias you’ve baked in. What helped with small data might be limiting you now. Specifically, test whether flexible, data-hungry architectures (like full attention) outperform domain-specific constraints.

Concretely, if you’re working on any scientific ML problem where people use message passing on graphs (drug discovery, materials, climate modeling), try this: train a baseline with all the domain priors, then train a version with minimal priors but 10x the parameters and data. If the second wins, you’ve found a scaling path. If the first still wins, your domain genuinely needs those priors (or you need even more data).

The other steal: their energy-conserving force prediction via gradients is a pattern worth copying. Anytime you’re predicting related quantities (energy/forces, position/velocity, probability/log-probability), predict the fundamental one and derive the others. Prevents inconsistencies that break downstream applications.