State Credit, Not Gradient Decay, Is What Stops Recurrent Models Extrapolating

Paper: 2609.09157 Author: Hanwen Jiang Categories: cs.CL, cs.LG

The Gap

Recurrent models are the natural architecture for long-context modelling, and models trained with backpropagation through time often fail beyond their training horizon. The classical explanation is about gradients: they vanish or explode along temporal paths, so the learning signal does not survive the distance.

The paper opens with a counterexample that the classical account cannot absorb: dense per-token losses can still train a shared recurrent rule despite severe decay. If decay were the deciding factor, severe decay should prevent that. It does not — so decay alone does not determine whether learning fails. Something else has to explain why extrapolation breaks.

   RECURRENT MODELS AND THE EXTRAPOLATION FAILURE

   recurrent models: the NATURAL architecture for long-context
     models trained with BPTT OFTEN FAIL BEYOND THEIR
     TRAINING HORIZON
        |
        v
   THE CLASSICAL EXPLANATION: GRADIENTS
     they VANISH or EXPLODE along temporal paths
     -> the learning signal does not survive the distance
        |
        v
   [THE COUNTEREXAMPLE THE CLASSICAL ACCOUNT CANNOT ABSORB]
     DENSE per-token losses can STILL TRAIN a shared recurrent
     rule DESPITE SEVERE DECAY
        |
        v
   if decay were the deciding factor, severe decay should
   PREVENT it -- it does NOT
        |
        v
   -> DECAY ALONE DOES NOT DETERMINE WHETHER LEARNING FAILS
   -> something else must explain why EXTRAPOLATION breaks

The Increment

One sentence: Before this paper, temporal failure was attributed to gradient decay; after it, a distinction between decay and state credit identifies the actual culprit, and a norm-preserving rescaling of that credit extends performance to 128× the training horizon.

Core Mechanism

The contribution begins with a distinction the classical account collapses.

State credit is the signal through which future losses reach earlier recurrent states, before contributing to parameter updates. That “before” is the whole point. Gradient decay is a property of the path from loss to parameters. State credit is a property of the path from loss to earlier states — and a recurrent rule can be trainable in the second sense while being poor in the first. Separating them explains the counterexample: the parameters were being learned adequately, while the signal connecting later outcomes to earlier states was not stable enough for the representation to extrapolate.

The intervention acts on state credit directly. Credit Stabilization through Time (CST) locally rescales the state-credit signal during backward propagation to stabilise its norm, and it does so without rotating the component being corrected, while leaving the forward computation unchanged.

Two properties of that design deserve isolating. “Without rotating” means the rescaling is norm-only: it changes the magnitude of the credit signal and not its direction. A blanket normalisation that also rotated the vector would perturb the very signal being corrected — fixing the scale problem by corrupting the content, which is a recognisable way for a fix to fail. And “leaving the forward computation unchanged” makes this a training-time intervention only: inference is unaffected, so the extrapolation gain is not bought by changing what the model computes at test time.

Because controlled synthetic tasks and real data exhibit different credit dynamics, CST is specialised to each regime. That is a modelling decision rather than an admission: if the dynamics differ across regimes, a single setting cannot be right for both.

The results are stated in the unit that matters for the claim: gains are observed at up to 128× the training length. Extrapolation measured as a multiple of the horizon the model was trained on is exactly the right quantity — an improvement that raised in-horizon accuracy would say nothing about the failure being addressed.

   THE DISTINCTION THE CLASSICAL ACCOUNT COLLAPSES

   STATE CREDIT = the signal through which FUTURE LOSSES reach
   EARLIER RECURRENT STATES, BEFORE contributing to parameter
   updates
        <- "before" is the WHOLE POINT
        |
        +-- GRADIENT DECAY: a property of the path from
        |   loss -> PARAMETERS
        +-- STATE CREDIT:  a property of the path from
            loss -> EARLIER STATES
        |
        v
   a recurrent rule can be TRAINABLE in the second sense while
   being POOR in the first
     -> explains the counterexample: the PARAMETERS were being
        learned adequately, while the signal connecting later
        outcomes to earlier states was not stable enough for the
        REPRESENTATION to extrapolate

   THE INTERVENTION: CST (Credit Stabilization through Time)
     during BACKWARD propagation, LOCALLY RESCALES the
     state-credit signal to STABILISE ITS NORM
       - WITHOUT ROTATING the component being corrected
       - while LEAVING THE FORWARD COMPUTATION UNCHANGED

     WHY THOSE TWO PROPERTIES MATTER
       "without rotating" -> NORM-ONLY: changes the MAGNITUDE of
         the credit signal, not its DIRECTION
         <- a blanket normalisation that also ROTATED would
            perturb the very signal being corrected: fixing the
            scale problem by corrupting the content, a
            recognisable way for a fix to FAIL
       "forward unchanged" -> TRAINING-TIME ONLY: inference is
         unaffected
         -> the extrapolation gain is NOT bought by changing what
            the model computes at TEST time

   SPECIALISATION: controlled synthetic tasks and real data
   exhibit DIFFERENT CREDIT DYNAMICS
     -> CST is specialised to each regime
     <- a modelling decision, not an admission: if the dynamics
        differ across regimes, a single setting cannot be right
        for both

   RESULT in the unit that matters for the claim
     gains observed at up to 128x THE TRAINING LENGTH
       <- extrapolation measured as a MULTIPLE OF THE HORIZON the
          model was trained on
       <- an improvement that raised IN-HORIZON accuracy would say
          nothing about the failure being addressed

Think of it as a relay race where the baton is being passed, not dropped. The classical story is about runners tiring — the signal fading over distance — and its remedy is to keep the handoff strength up. The counterexample shows that a team can have plenty of handoff strength and still lose, because the problem is the legibility of the baton: each runner receives something whose scale has drifted so far that they cannot tell a strong pass from a weak one. Stabilising the scale of what gets handed back — without altering its direction, and without changing how anyone runs forward — is what lets the line extend. The distinction also tells you where to intervene: strengthening handoffs (gradients) is not the same as making the baton readable (state credit).

Key Concepts

• State credit versus gradient decay: the signal from future losses to earlier states, distinguished from the signal from loss to parameters. The distinction explains why severe decay can coexist with successful parameter learning and failed extrapolation.
• Norm-only rescaling: stabilising the magnitude of the credit signal without rotating it. It avoids fixing the scale problem by perturbing the content, which is how such interventions typically go wrong.
• Forward computation untouched: a training-time-only intervention, so inference cost and semantics are unchanged. It rules out the explanation that the model simply got a different function to compute.
• Regime-specific dynamics: synthetic and real credit dynamics differ, so stabilisation is specialised. It follows from measurement rather than convenience.
• Extrapolation in units of the training horizon: 128× the training length. Measuring the gain this way keeps it a claim about extrapolation rather than about in-horizon accuracy.

Framework Shift

Before (temporal failure as gradient decay):
  BPTT-trained recurrent models fail beyond their horizon
  -> attributed to vanishing / exploding gradients
  -> remedy: improve gradient flow
  -> but dense per-token losses train successfully despite
     severe decay, which the account cannot explain

After (decay separated from state credit):
  state credit = future losses reaching earlier states,
    before parameter updates
  CST stabilises the norm of that signal in the backward pass,
    without rotating it and without touching forward computation
  -> specialised per regime, because credit dynamics differ
  -> gains at up to 128x the training length

From treating temporal failure as a gradient-flow problem, to distinguishing parameter learning from state-to-state credit and stabilising the latter, the core shift is that a model can learn its parameters while failing to propagate information through its own states.

Expert Assessment

Problem choice: Excellent, and it starts from a counterexample rather than from a theory. Long-context extrapolation is among the more consequential limitations of recurrent architectures, and proposing a distinction because the classical account leaves an observation unexplained is how a genuinely new mechanism gets identified.

Method maturity: The state-credit definition is precise, and the “before contributing to parameter updates” qualifier is what separates it from gradient decay rather than relabelling it. The intervention’s two constraints — norm-only, and forward-unchanged — are the right guardrails, since relaxing either would leave the result attributable to something other than the claimed mechanism. Specialising per regime follows from measured differences rather than convenience. The scope is a training-time intervention on recurrent models, and 128× is reported as the observed extent rather than as a general law.

Experimental integrity: The claim is supported by the counterexample, a mechanistic distinction, a targeted intervention, and gains measured in the right units. The main limitation is that the intervention modifies the backward pass, which is more invasive than a hyperparameter change — teams will want to know how it interacts with gradient clipping, other stabilisation techniques and any non-recurrent components in a mixed architecture, and how sensitive the outcome is to the specialisation choice between regimes.

Writing quality: The distinction is stated in one sentence and everything follows from it, which is what makes a mechanism paper readable. Because the practical question is whether this composes with an existing training stack, a short section on what CST does to a standard backward pass — and what it deliberately leaves alone — would make adoption much easier to assess.

Verdict: strong accept — it identifies a mechanism the classical account conflates with gradient decay, intervenes on it precisely, and reports extrapolation gains measured in the units the problem is actually about.

Takeaways

• Separate parameter learning from state-to-state information flow. A model can train well and still fail to propagate signal through its own history.
• When a theory leaves an observation unexplained, propose a distinction rather than a better version of the theory. The counterexample here was more informative than the decay story.
• Keep the correction norm-only. Rescaling a signal while rotating it perturbs the content, which fails in a way that resembles the original problem.
• Report extrapolation as a multiple of the training horizon. In-horizon accuracy is silent about the failure being addressed.