Exact Posterior Score Estimation for Solving Linear Inverse Problems

Paper: 2606.17048 Authors: Abbas Mammadov, Ozgur Kara, Kaan Oktay, Iskander Azangulov, Adil Kaan Akan, Hyungjin Chung, James Matthew Rehg, Yee Whye Teh Categories: cs.LG, cs.CV, stat.ML

The Gap

Diffusion and flow-based models provide a powerful prior by learning to denoise Gaussian-corrupted data.
But solving a linear inverse problem (e.g., super-resolution, inpainting, deblurring) requires the posterior score, not the unconditional score.

Existing approaches fall into two camps, each with clear shortcomings:

1. Training‑free steering (DPS, GDP, DMPS): keep a fixed pretrained denoiser,
   then at each sampling step add an approximate measurement‑matching correction.
   These corrections rely on likelihood gradients or projections, are computationally expensive (often 10–20× more denoiser evaluations), and can be unstable.

2. Conditional restoration models (e.g., SR3, Palette): train a task‑specific network from scratch,
   discarding the denoising architecture of the original prior.
   This loses the plug‑and‑play flexibility and requires separate models for each inverse problem.

The gap is clear: no method obtains the exact posterior score while keeping the denoising structure of the prior.
This paper closes that gap.

     Problem                     Assumption                  Method                Evidence              Conclusion
+--------------------+     +---------------------+     +-------------------+    +--------------+      +------------------+
| Linear inverse     |     | Posterior score     |     | Closed-form       |    | EPS beats    |      | Posterior        |
| problem needs      | --> | is intractable /    | --> | posterior score   | -> | baselines on | -->  | sampling =       |
| posterior score,   |     | approximated with   |     | via Gaussian      |    | FFHQ/ImageNet|      | denoising under  |
| prior only gives   |     | gradient corrections|     | interpolants      |    | with 10x     |      | anisotropic      |
| unconditional score|     | or separate models  |     |                   |    | fewer evals  |      | covariance       |
+--------------------+     +---------------------+     +-------------------+    +--------------+      +------------------+

The Increment

One sentence: Before this paper, exact posterior score for linear inverse problems under diffusion/flow priors was unavailable; after this paper, we have a closed‑form expression that turns posterior sampling into a denoising problem with an anisotropic noise covariance, eliminating the need for likelihood gradients or extra training tasks.

Core Mechanism

The key insight is a reparameterization of the joint distribution of the clean data $x$ and the measurement $y = A x + \eta$.
The authors work with general Gaussian interpolants — a family of stochastic processes that connect the data distribution to a Gaussian noise source.
They derive the exact posterior score $\nabla_{x_t} \log p(x_t \mid y)$ in closed form:

More concretely, the posterior score can be written as the score of an operator‑dependent shifted pivot under an anisotropic noise covariance.
This means that instead of denoising the standard Gaussian corruption, the denoiser must learn to denoise a specially constructed noisy image whose noise covariance is shaped by the forward operator $A$.

The training objective Exact Posterior Score (EPS) directly matches this analytic score.
It preserves the input/output structure of standard denoising (noisy image → clean image) but with a modified corruption process that incorporates the measurement.
As a result, EPS can be trained from scratch or fine‑tuned from a pretrained denoiser with minimal changes.

At inference, EPS uses the same sampler as the underlying backbone (e.g., DDIM, DPM‑Solver) — no likelihood gradients, no projections, no extra steps.

            TRAINING PATH                              INFERENCE PATH
+-----------------------------+               +-------------------------------+
|  Clean image x0             |               |  Measurement y               |
|         |                   |               |         |                     |
|         v                   |               |         v                     |
|  y = A x0 + eta             |               |  Construct shifted pivot     |
|         |                   |               |  using y and operator A      |
|         v                   |               |         |                     |
|  Construct noisy pair       |               |         v                     |
|  (z, x0) with anisotropic   |               |  Run standard sampler        |
|  covariance derived from A  |               |  (e.g., DDIM) with EPS       |
|         |                   |               |  denoiser                     |
|         v                   |               |         |                     |
|  Train denoiser D(z) -> x0  |               |         v                     |
|  (EPS loss on posterior     |               |  Reconstructed x0            |
|   score)                    |               |                               |
+-----------------------------+               +-------------------------------+

Structural Metaphor: The Conductor’s Score

Imagine an orchestra playing a symphony from a standard sheet (the unconditional score).
If you want the orchestra to adapt to a specific recording (the measurement), you can’t just give them the original sheet — you need a personalized score that tells each musician how to adjust for the room acoustics and the microphone positions.

• The prior (unconditional score) is the generic sheet music.
• The measurement $y$ is the recorded audio from the live performance.
• The posterior score is the adjusted score that accounts for the specific recording setup.
• The shifted pivot is the new reference point from which each musician reads their part.
• The anisotropic noise covariance is the different “tuning offsets” for different instrument groups (high strings vs low brass), determined by the microphone placement (the operator $A$).

EPS essentially teaches each denoiser (musician) to read that personalized score directly, so during inference they simply play from that score without needing a separate conductor to shout corrective instructions (likelihood gradients).

Key Concepts

• Posterior Score:
  The gradient of the log‑probability of the noisy data given the observation.
  Think of it as a “direction arrow” pointing toward more likely clean images, but constrained by what you already observed (the measurement).
  In diffusion models, the unconditional score points toward high‑quality natural images; the posterior score does the same but ensures consistency with $y$.

• Gaussian Interpolants:
  A family of stochastic processes that smoothly interpolate between the data distribution and a Gaussian distribution.
  Standard diffusion is a special case (VP‑SDE).
  This paper generalizes the interpolant to any linear Gaussian corruption, including the one induced by the inverse problem.
  This generalization is what allows the closed‑form posterior score.

• Shifted Pivot:
  Instead of denoising from pure Gaussian noise, the denoiser now operates on a “pivot” point that is a linear combination of the measurement $y$ and Gaussian noise.
  The pivot is shifted because the measurement provides partial information about the clean image.
  This pivot becomes the input to the denoiser, and the output is the clean image.

Framework Shift

The old paradigm: unconditional score + approximate correction at each step.
The new paradigm: exact posterior score → standard denoising sampler.

Before (mainstream approach):        After (this paper):
+----------------------------+      +----------------------------+
| Unconditional denoiser D   |      | EPS denoiser D_EPS         |
|         |                   |      |         |                  |
|         v                   |      |         v                  |
| Sample with score updates  |      | Sample with standard       |
| (DDPM / DDIM)              |      | sampler (no extra steps)   |
|         |                   |      |         |                  |
|         v                   |      |         v                  |
| For each step t:           |      | Output: reconstructed x0   |
|   s = unconditional score  |      |                            |
|   correction = grad loss   |      |                            |
|   (e.g., DPS: ||y - A x||) |      |                            |
|   x_{t-1} = update(s + cor)|      |                            |
+----------------------------+      +----------------------------+

One sentence: From *approximate correction of unconditional score to exact closed‑form posterior score, the core shift is that the measurement information is baked into the denoiser during training, not injected during inference.

Expert Assessment

Problem choice: Real gap.
The field has been wrestling with approximate posterior sampling for years.
This paper gives an exact solution — assuming linear Gaussian measurement models — which covers a large class of practically relevant inverse problems.

Method maturity: Clever insight, not brute force.
The derivation is mathematically elegant and leverages existing theory of Gaussian interpolants.
There’s no hidden simplicity that was overlooked — the hard part was realizing the closed form exists.
One could argue the approach is somewhat restricted to linear forward models, but that’s inherent to the problem setting.

Experimental integrity: Fair baselines, solid numbers.
They compare against DPS, GDP, and other training‑free methods, as well as task‑specific restoration models.
EPS consistently beats or matches on FID, LPIPS, and distributional metrics.
The 10× reduction in denoiser evaluations is a clear win.
No obvious red flags; code is likely to be released.

Writing quality: Good but dense.
The method section uses notation that could be more clearly linked to the intuition.
The diagram in the paper could be more helpful; the “shifted pivot” concept deserves a dedicated figure.
If the authors rewrote Section 3 with a running example (e.g., a simple 1D deblurring), it would elevate the paper.

Verdict: Strong accept — exact posterior score for linear inverse problems under diffusion priors is a significant theoretical contribution with immediate practical impact.

Takeaways

• Direct fine‑tuning recipe: Practitioners can take any pretrained diffusion denoiser, fine‑tune it with the EPS loss for their specific operator $A$, and get a sampler that uses the same inference loop with no extra overhead.
• Modular design: The EPS trainable component is independent of the sampler — it works with DDIM, DPM‑Solver, or any ODE/SDE solver that expects a denoiser.
• Unified view of inverse problems: The framework shows that all linear Gaussian inverse problems can be reduced to the same core denoising task, differing only in the noise covariance structure. This insight can guide future work on adaptive sampling or meta‑learning across operators.
• Avoiding adversarial corrections: By removing likelihood gradients, EPS circumvents the gradient‑based instabilities that plague training‑free methods (especially in high‑resolution image domains).