Why Neural Networks Need Linear, Orthogonal Representations to Generalize Compositionally

Paper: 2602.24264 Authors: Arnas Uselis, Andrea Dittadi, Seong Joon Oh Categories: cs.CV, cs.LG

The Gap

Vision models like CLIP and DINO have shown remarkable empirical success, and researchers have observed that their learned representations exhibit surprisingly linear structure — concepts seem to align with directions in embedding space. But this observation has been purely descriptive. We know that linearity appears, but not why it must appear. The Linear Representation Hypothesis remains an empirical pattern without theoretical necessity.

More critically, we lack a principled understanding of what representational structure is required for compositional generalization — the ability to recognize “red car” after seeing “red ball” and “blue car” separately. Models are trained on massive datasets, yet these cover only a vanishingly small fraction of all possible concept combinations. What geometric constraints must representations satisfy to generalize beyond the training distribution?

The Increment

Before: Linear structure in neural representations was an observed regularity without theoretical justification. After: Linear, orthogonal decomposition is a mathematical necessity for compositional generalization.

The authors formalize three desiderata that any representation must satisfy to support compositional generalization: divisibility (you can isolate individual concept contributions), transferability (concept representations work across different contexts), and stability (small input changes don’t catastrophically alter the representation). From these intuitive requirements, they derive strict geometric constraints through mathematical proof.

Think of it like a modular synthesizer. Each concept (color, shape, texture) is a separate oscillator module. For the system to work compositionally, each module must output a signal (representation component) that can be cleanly mixed with others. This requires two things: the signals must combine linearly (simple addition, not complex interference), and they must be orthogonal (each module occupies its own frequency band, so they don’t bleed into each other). If “redness” and “car-ness” interfere destructively, you can’t compose them reliably. The paper proves this isn’t just good engineering — it’s mathematically necessary given the desiderata.

The theoretical framework also yields dimension bounds: if you want to compose $k$ concepts, your embedding space needs at least $k$ dimensions, and the per-concept factors must be low-rank and near-orthogonal. The authors then validate these predictions empirically across CLIP, SigLIP, and DINO, finding that models exhibiting stronger linear factorization indeed generalize better to unseen combinations.

Key Concepts

Compositional Generalization

Start simple: you’ve seen dogs and you’ve seen snow. Now you see a dog in snow for the first time. Can you recognize both? That’s compositional generalization — recombining known parts in novel configurations.

For neural networks, this is hard because training data is sparse in the combinatorial space. If you have 100 colors and 100 objects, that’s 10,000 combinations. Even massive datasets might show you only 1,000 of them. The model must infer the other 9,000 from structure, not memorization.

Concretely: CLIP might see “red apple” and “green car” during training. At test time, it encounters “green apple.” Does the representation for “green apple” correctly combine the “green-ness” learned from cars with the “apple-ness” learned from red apples? This requires that “green” and “apple” are represented as separable, reusable components.

Linear Decomposition

Imagine you’re encoding the sentence “the red car is fast” as a single vector. Linear decomposition means you can write:

$\text{embedding} = \alpha \cdot \vec{v}_{\text{red}} + \beta \cdot \vec{v}_{\text{car}} + \gamma \cdot \vec{v}_{\text{fast}} + \ldots$

Each concept contributes an additive component. This is opposed to entangled representations where concepts interact nonlinearly — like $\vec{v}_{\text{red}} \odot \vec{v}_{\text{car}}$ (element-wise product) or some deep nonlinear function.

Why does this matter? If representations are linear, you can algebraically manipulate them. Want to remove “red” and add “blue”? Just subtract $\vec{v}_{\text{red}}$ and add $\vec{v}_{\text{blue}}$. This is exactly what’s needed for compositional generalization: swap out one concept for another without retraining.

The paper proves this isn’t optional. If you want divisibility (isolate individual concepts) and transferability (reuse concepts across contexts), linear decomposition is mathematically forced upon you.

Orthogonality

Two vectors are orthogonal if their dot product is zero: $\vec{v}_{\text{red}} \cdot \vec{v}_{\text{car}} = 0$. Geometrically, they point in perpendicular directions.

Why does this matter for compositionality? Suppose “red” and “car” are not orthogonal — they share some overlap in representation space. When you try to compose them, the overlapping parts interfere. Extracting “car-ness” from “red car” becomes ambiguous because part of the “car” signal is contaminated by “red.”

Orthogonality ensures clean separation: each concept occupies its own subspace. When you project the embedding onto the “red” subspace, you get only the color information. Project onto “car,” you get only the object information. No crosstalk.

The paper shows this is necessary for stability: if concepts aren’t orthogonal, small perturbations in one concept can cascade into large changes in another, breaking compositional generalization.

Expert Assessment

Problem significance: This is a foundational question. Compositional generalization is arguably *the central challenge in AI — it’s what separates systems that memorize from systems that understand structure. The affected community is broad: anyone working on vision-language models, few-shot learning, systematic generalization, or interpretability. The theoretical grounding provided here could reshape how we design and evaluate models.

Method maturity: This is primarily a theoretical contribution with empirical validation, not a new training method. The proofs are elegant, but they rest on three desiderata (divisibility, transferability, stability) that are somewhat idealized. Real-world compositional generalization might require additional properties not captured here. The empirical analysis is observational — the authors measure existing models but don’t train new ones to maximize the derived constraints. A stronger validation would involve training models explicitly to satisfy these geometric conditions and demonstrating improved compositional generalization.

Experimental rigor: The experiments are solid but limited in scope. The authors test on CLIP, SigLIP, and DINO — all contrastive or self-supervised vision models. What about supervised models? Generative models? The compositional generalization benchmarks used (e.g., unseen attribute-object combinations) are reasonable but somewhat narrow. The paper would benefit from testing on more diverse compositional tasks (e.g., visual reasoning, relational understanding). The correlation between linear factorization and generalization is shown, but causality is not established — do models generalize *because they’re linear, or are both effects of some other factor?

One notable limitation: the paper assumes standard training (e.g., contrastive learning). The constraints might not hold for models trained with explicit compositional objectives or structured priors. The authors acknowledge this but don’t explore it deeply.

Verdict: weak accept — The theoretical contribution is valuable and the empirical validation is competent, but the work feels incomplete without demonstrating that enforcing these constraints during training actually improves compositional generalization.

Takeaways

Geometric constraints as design principles: The idea that functional requirements (like compositional generalization) impose geometric constraints on representations is broadly applicable. If you’re designing embeddings for any compositional domain (molecules, code, music), ask: what structure must the representation have to support the operations you care about? Then derive the geometry from first principles.

Dimension bounds for compositionality: The paper derives that composing $k$ concepts requires at least $k$ dimensions. This is a practical design constraint: if you’re building a model to handle $n$ compositional factors, your embedding dimension should scale accordingly. Underparameterized embeddings will bottleneck compositional capacity.

Orthogonality as a regularizer: If you want compositional generalization, you could add an orthogonality penalty during training: encourage per-concept factors to be orthogonal. This is actionable — you could implement it as a loss term that penalizes $|\vec{v}_i \cdot \vec{v}_j|$ for distinct concepts $i, j$.

Probing for compositionality: The paper’s analysis method — decomposing embeddings into per-concept factors and measuring their orthogonality — is a useful diagnostic tool. If you’re evaluating a model’s compositional capacity, don’t just test accuracy on held-out combinations. Probe the geometry: are concepts linearly separable? Are they orthogonal? This gives you a mechanistic understanding of *why the model succeeds or fails.

Limits of scaling: The paper suggests that as models scale, they may converge to representations satisfying these constraints. But this is only true if the training objective incentivizes compositional generalization. If your data distribution doesn’t require compositionality (e.g., all combinations are seen during training), scaling won’t magically produce linear, orthogonal structure. The geometry follows from the task, not the scale.