Pseudo-Invertible Neural Networks: Generalizing Moore-Penrose to Deep Learning

Paper: 2602.06042 Authors: Yamit Ehrlich, Nimrod Berman, Assaf Shocher Categories: cs.LG, cs.CV

Abstract

The Moore-Penrose Pseudo-inverse (PInv) serves as the fundamental solution for linear systems. This paper proposes a natural generalization of PInv to the nonlinear regime, introducing Surjective Pseudo-invertible Neural Networks (SPNN) - architectures explicitly designed to admit a tractable non-linear pseudo-inverse.

Key Contributions

• Non-Linear Pseudo-Inverse: Generalization of Moore-Penrose to neural networks
• SPNN Architecture: Networks with tractable pseudo-inverse computation
• Non-Linear Back-Projection (NLBP): Guarantees consistency constraints for non-linear mappings
• Zero-Shot Inverse Problems: Extends diffusion-based methods to non-linear degradations

Background: Linear Back-Projection

For linear systems $Ax = y$, the back-projection moves a sample to its closest consistent state:

$x' = x + A^\dagger(y - Ax)$

where $A^\dagger$ is the Moore-Penrose pseudo-inverse. This ensures $Ax' = y$.

Non-Linear Back-Projection (NLBP)

For non-linear mappings $f(x) = y$, SPNN enables:

$x' = x + f^\dagger(y - f(x))$

where $f^\dagger$ is the learned pseudo-inverse satisfying:

• Consistency: $f(x') = y$
• Minimum norm: $x'$ is closest to $x$ among all solutions

SPNN Architecture

Surjective Pseudo-invertible Neural Networks satisfy:

1. Surjectivity: Every output $y$ has at least one pre-image
2. Tractable Inverse: The pseudo-inverse can be computed efficiently
3. Geometric Properties: Preserves null-space structure

Key Design Principles

• Decompose network into invertible and projection components
• Maintain explicit null-space representation
• Enable efficient back-projection computation

Applications

Zero-Shot Inverse Problems

Diffusion-based null-space projection revolutionized linear inverse problems. SPNN extends this to:

• Optical distortions: Non-linear camera effects
• Semantic abstractions: Classification as degradation
• Complex degradations: Any non-linear information loss

Semantic Control

Enable precise control over generative outputs:

• Invert classification to generate specific classes
• Control semantic attributes without retraining
• Zero-shot adaptation to new degradation types

Results

Application	Capability
Non-linear deblurring	Zero-shot inversion
Semantic generation	Class-conditional without retraining
Optical correction	Complex distortion removal

Takeaways

1. Fundamental generalization: Moore-Penrose extends naturally to neural networks
2. Geometric guarantees: NLBP preserves consistency and minimum-norm properties
3. Practical impact: Enables zero-shot solving for non-linear inverse problems
4. No retraining needed: Works with existing diffusion priors