The Geometry of Alignment Collapse: When Fine-Tuning Breaks Safety

Paper: 2602.15799 Authors: Max Springer, Chung Peng Lee, Blossom Metevier, Jane Castleman, Bohdan Turbal, Hayoung Jung, Zeyu Shen, Aleksandra Korolova Categories: cs.LG, cs.AI

Abstract

This paper investigates a critical vulnerability in aligned language models: fine-tuning on benign tasks can unpredictably degrade safety guardrails, even without harmful training data or adversarial intent. The authors challenge the prevailing assumption that fine-tuning updates orthogonal to safety-critical directions preserve alignment, demonstrating that this orthogonality is structurally unstable under gradient descent dynamics. Through novel geometric analysis, they prove that alignment concentrates in low-dimensional subspaces with sharp curvature, creating a brittle structure invisible to first-order methods. The paper establishes the Alignment Instability Condition and derives a quartic scaling law showing alignment loss grows with the fourth power of training time, fundamentally reframing alignment fragility as an intrinsic geometric property rather than a patchable bug.

Key Contributions

• Geometric characterization of alignment collapse: Proves that safety alignment concentrates in low-dimensional subspaces with sharp curvature, making it structurally fragile to fine-tuning
• Alignment Instability Condition: Formalizes three geometric properties that jointly predict safety degradation during fine-tuning
• Quartic scaling law: Establishes that alignment loss scales as $\mathcal{O}(t^4)$ with training time $t$, governed by alignment sharpness and curvature coupling
• Structural critique of current paradigm: Demonstrates that dominant safe fine-tuning approaches address only initial conditions of a fundamentally dynamic problem
• Second-order mechanism: Shows how curvature in the fine-tuning loss generates acceleration that systematically steers trajectories into alignment-sensitive regions

The Geometry of Alignment

The paper’s central insight is that alignment safety is not uniformly distributed across parameter space but concentrates in low-dimensional subspaces. This geometric structure has profound implications:

Sharp curvature manifolds: The authors prove that safety-critical parameters lie on manifolds with high curvature. While first-order gradient methods (standard in deep learning) can initially avoid these regions, the curvature itself creates second-order effects that pull optimization trajectories back toward alignment-sensitive areas.

Orthogonality illusion: The prevailing wisdom suggests that if fine-tuning gradients are orthogonal to safety directions, alignment should be preserved. This paper shows this orthogonality is structurally unstable—small perturbations in parameter space can rapidly collapse this independence due to the curved geometry of the loss landscape.

Low-dimensional concentration: Rather than safety being a high-dimensional property robust to perturbation, it concentrates in surprisingly low-dimensional subspaces. This concentration makes alignment brittle: even benign fine-tuning can inadvertently intersect these critical regions.

The Alignment Instability Condition

The paper formalizes when fine-tuning will degrade safety through three geometric properties:

1. Alignment concentration: Safety-critical parameters occupy a low-dimensional subspace $\mathcal{S}$ with dimension $d_{\mathcal{S}} \ll d$ where $d$ is the full parameter dimension
2. Sharp curvature: The alignment loss exhibits high curvature (large eigenvalues of the Hessian) within $\mathcal{S}$
3. Curvature coupling: The fine-tuning task loss has non-negligible curvature components that couple with the alignment subspace

When all three conditions hold, the paper proves that alignment degradation is inevitable, regardless of the initial orthogonality of gradients.

The Quartic Scaling Law

The main theoretical result establishes:

$\Delta L_{\text{align}}(t) \sim \kappa_{\text{align}} \cdot \kappa_{\text{couple}} \cdot t^4$

where:

• $\Delta L_{\text{align}}(t)$ is the alignment loss at training time $t$
• $\kappa_{\text{align}}$ measures the sharpness of alignment geometry
• $\kappa_{\text{couple}}$ quantifies curvature coupling between fine-tuning and safety parameters

This quartic relationship reveals that alignment degradation accelerates dramatically over time—not linearly or quadratically, but with the fourth power. This explains why safety can appear stable initially but collapse suddenly during extended fine-tuning.

Mechanistic explanation: The quartic scaling emerges from second-order dynamics. Initial fine-tuning updates may avoid alignment subspaces (first-order effect), but the curvature of the loss landscape generates acceleration (second-order effect) that systematically steers trajectories into these regions. The fourth power arises from the compounding of curvature effects over time.

Implications for AI Safety

This work exposes fundamental limitations in current safety practices:

Reactive vs. predictive: Current approaches rely on red-teaming and post-hoc evaluation. This paper argues for predictive diagnostics based on geometric properties that can be measured before deployment.

First-order blindness: Standard gradient-based safety interventions operate in a first-order regime, unable to detect or defend against the second-order curvature effects that drive alignment collapse.

Open-weight deployment risks: For open-weight models, users can fine-tune without oversight. Understanding the geometric conditions for alignment collapse enables better risk assessment and potentially curvature-aware fine-tuning methods.

Beyond orthogonality: The paper demonstrates that ensuring gradient orthogonality is insufficient. Safe fine-tuning requires curvature-aware methods that account for the dynamic, second-order nature of alignment degradation.

Takeaways

1. Alignment safety is geometrically fragile, concentrating in low-dimensional subspaces with sharp curvature that first-order methods cannot protect
2. The quartic scaling law ($\mathcal{O}(t^4)$) explains why safety can appear stable initially but collapse suddenly during extended fine-tuning
3. Current safe fine-tuning approaches address only initial conditions of a fundamentally dynamic problem, missing second-order curvature effects
4. The Alignment Instability Condition provides a predictive framework for assessing when fine-tuning will degrade safety
5. Future safety methods must be curvature-aware, shifting from reactive red-teaming to predictive geometric diagnostics