Neural Scaling Laws for Boosted Jet Tagging

Paper: 2602.15781 Authors: Matthias Vigl, Nicole Hartman, Michael Kagan, Lukas Heinrich Categories: hep-ex, cs.LG, hep-ph, physics.data-an

Abstract

This paper establishes the first systematic investigation of neural scaling laws for high energy physics, specifically focusing on boosted jet classification using the JetClass dataset. The authors demonstrate that the principles driving Large Language Model success—scaling compute through increased model capacity and dataset size—apply equally to particle physics tasks. They derive compute-optimal scaling laws that reveal consistent performance limits achievable through increased computational resources, quantify the benefits of data repetition in simulation-constrained environments, and show how feature representation choices fundamentally affect both scaling behavior and asymptotic performance ceilings.

Key Contributions

• Compute-Optimal Scaling Laws: Derived power-law relationships between compute budget $C$, model parameters $N$, and dataset size $D$ for jet tagging, following the form $L(C) = L_{\infty} + \alpha C^{-\beta}$ where $L_{\infty}$ represents the irreducible error floor
• Data Repetition Analysis: Quantified how repeated training on expensive simulation data affects scaling, introducing an effective dataset size $D_{\text{eff}}$ that accounts for diminishing returns from repetition
• Feature Representation Impact: Demonstrated that low-level particle features (4-vectors) achieve better asymptotic limits than high-level engineered features, with performance gaps widening at scale
• Particle Multiplicity Effects: Showed that models trained on higher particle multiplicities exhibit improved scaling coefficients and lower performance floors

Methodology and Experimental Design

The research employs ParticleNet architectures trained on the JetClass dataset, which contains 100 million simulated jets across 10 classification categories. The authors systematically vary three key dimensions:

Compute Budget Scaling: Models range from 10¹⁸ to 10²¹ FLOPs, spanning four orders of magnitude. For each compute budget, they optimize the allocation between model size and training tokens following Chinchilla-style scaling principles.

Feature Representations: Three input modalities are compared:

• High-level features (16 engineered variables)
• Particle 4-vectors (energy-momentum representations)
• Particle 4-vectors with additional substructure variables

Data Repetition Regimes: Training extends from single-epoch to 100-epoch scenarios, modeling the reality of expensive HEP simulations where generating new data is computationally prohibitive.

The scaling law formulation follows:

$L(N, D) = L_{\infty} + \frac{\alpha_N}{N^{\beta_N}} + \frac{\alpha_D}{D^{\beta_D}}$

where the compute-optimal trajectory satisfies $N \propto C^a$ and $D \propto C^b$ with $a + b = 1$.

Results and Scaling Behavior

The empirical results reveal several critical insights:

Power-Law Scaling Holds: Across all feature representations and particle multiplicities, performance follows predictable power laws with compute. The exponent $\beta$ typically ranges from 0.05 to 0.15, indicating that each order of magnitude increase in compute yields consistent fractional improvements.

Asymptotic Limits Vary by Representation: The irreducible error $L_{\infty}$ differs substantially:

• High-level features: $L_{\infty} \approx 0.45$
• Particle 4-vectors: $L_{\infty} \approx 0.35$
• 4-vectors + substructure: $L_{\infty} \approx 0.32$

This demonstrates that feature expressiveness directly determines the performance ceiling, regardless of compute invested.

Data Repetition Provides Effective Size Gains: When training data is repeated $R$ times, the effective dataset size follows $D_{\text{eff}} = D \cdot R^{\gamma}$ where $\gamma \approx 0.6$. This means 100 epochs on 1M samples performs comparably to 16 epochs on 10M samples—a crucial finding for simulation-limited domains.

Particle Multiplicity Matters: Models trained on jets with 128 particles consistently outperform those trained on 64 particles, with both better scaling coefficients and lower asymptotic limits. This suggests that preserving fine-grained particle information improves both sample efficiency and ultimate performance.

Implications for High Energy Physics

This work has profound implications for how the HEP community should approach machine learning:

Resource Allocation: The compute-optimal scaling laws provide concrete guidance for balancing model size against dataset size. For a fixed compute budget, the optimal allocation typically favors larger models trained on more data than current practice suggests.

Simulation Strategy: The quantified benefits of data repetition justify training longer on existing simulations rather than immediately generating new samples. A 10x increase in epochs provides roughly 4x effective data, which may be more efficient than generating 4x more simulation.

Feature Engineering vs. Raw Data: The superior asymptotic performance of low-level features challenges the HEP tradition of hand-crafted high-level variables. While engineered features may perform better in data-scarce regimes, scaling compute favors learning directly from particle 4-vectors.

Compute Investment: Current HEP models use 10¹⁸-10¹⁹ FLOPs, while foundation models use 10²³-10²⁴ FLOPs. The demonstrated scaling laws suggest that HEP could achieve substantial performance gains by increasing compute budgets, potentially approaching the asymptotic limits identified in this work.

Takeaways

1. Neural scaling laws from LLM research apply to high energy physics tasks, with power-law relationships between compute, model size, dataset size, and performance holding across four orders of magnitude
2. Compute-optimal training requires careful balancing of model parameters and training tokens, with the optimal allocation differing from current HEP practice
3. Data repetition in simulation-constrained environments provides quantifiable effective dataset size gains following $D_{\text{eff}} \propto R^{0.6}$, making extended training on existing data more efficient than previously assumed
4. Low-level particle representations (4-vectors) achieve better asymptotic performance limits than engineered high-level features, with advantages increasing at scale
5. Particle multiplicity directly affects both scaling efficiency and performance ceilings, suggesting that preserving fine-grained detector information improves model capacity
6. The HEP community has significant headroom for performance improvements through increased compute investment, with current models operating far below the identified asymptotic limits