SenCache: Accelerating Diffusion Model Inference via Sensitivity-Aware Caching

Paper: 2602.24208 Authors: Yasaman Haghighi, Alexandre Alahi Categories: cs.CV, cs.LG

Abstract

Diffusion models have achieved remarkable success in video generation, delivering state-of-the-art quality. However, their inference process remains computationally expensive due to the requirement of numerous sequential denoising steps. This paper introduces SenCache, a sensitivity-aware caching framework that addresses the computational bottleneck through a principled approach. Unlike existing heuristic-based caching methods that require extensive manual tuning, SenCache formalizes caching error by analyzing model output sensitivity to perturbations in denoising inputs (noisy latent and timestep). The framework demonstrates that sensitivity serves as a reliable predictor of caching error and proposes a dynamic policy that adaptively selects caching timesteps on a per-sample basis. Experimental validation on Wan 2.1, CogVideoX, and LTX-Video demonstrates superior visual quality compared to existing caching methods under equivalent computational budgets.

Key Contributions

• Theoretical Framework: Formalization of caching error through sensitivity analysis of model outputs to input perturbations, providing a principled foundation for understanding when caching is safe
• Sensitivity-Based Predictor: Identification of model output sensitivity as a key predictor of caching error, enabling informed decisions about cache/reuse timesteps
• Dynamic Caching Policy: Introduction of SenCache, an adaptive algorithm that selects caching timesteps dynamically on a per-sample basis rather than using fixed heuristics
• Empirical Validation: Demonstration of improved visual quality over existing caching methods across multiple state-of-the-art video generation models (Wan 2.1, CogVideoX, LTX-Video)
• Theoretical Justification: Explanation of why prior empirical heuristics work and how they can be extended to sample-specific approaches

Methodology and Technical Approach

The core innovation of SenCache lies in its sensitivity-aware formulation of the caching problem. Traditional caching methods in diffusion models reuse previously computed outputs at certain timesteps to avoid redundant computation. However, determining which timesteps are safe for caching has relied on empirical heuristics without theoretical justification.

SenCache addresses this by analyzing the sensitivity of model outputs $\mathbf{f}_\theta(\mathbf{z}_t, t)$ to perturbations in both the noisy latent $\mathbf{z}_t$ and the timestep $t$. The framework establishes that when sensitivity is low, the model output changes minimally between consecutive timesteps, making caching safe. Conversely, high sensitivity indicates significant changes that would result in large caching errors if outputs were reused.

The sensitivity metric is computed efficiently during inference without requiring additional forward passes. Based on this metric, SenCache dynamically decides at each timestep whether to compute a fresh output or reuse a cached one. This adaptive approach allows the method to handle varying content complexity across different samples, unlike fixed caching schedules that apply the same pattern to all inputs.

The mathematical formulation considers both first-order and higher-order sensitivity terms, providing a comprehensive understanding of how errors propagate through the denoising process. This theoretical grounding not only improves performance but also offers interpretability—explaining why certain timesteps are more amenable to caching than others.

Experimental Results and Performance

SenCache was evaluated on three prominent video generation models: Wan 2.1, CogVideoX, and LTX-Video. The experiments compared SenCache against existing caching baselines under controlled computational budgets, measured by the number of function evaluations (NFE).

Results demonstrate consistent improvements in visual quality metrics across all tested models. For instance, on Wan 2.1, SenCache achieved higher perceptual quality scores while using the same or fewer denoising steps compared to uniform caching strategies. The adaptive nature of SenCache proved particularly beneficial for complex scenes with high motion or intricate details, where fixed caching schedules often introduced visible artifacts.

Qualitative analysis revealed that SenCache better preserves temporal coherence and fine-grained details in generated videos. The method’s ability to identify critical timesteps—where fresh computation is essential—prevented the accumulation of errors that plague aggressive caching strategies. Meanwhile, it successfully exploited redundancy in smoother regions of the denoising trajectory, achieving computational savings without quality degradation.

Ablation studies confirmed the importance of the sensitivity metric, showing that removing the adaptive component and reverting to fixed schedules resulted in performance drops. The experiments also validated that the sensitivity computation overhead is negligible compared to the savings from reduced denoising steps.

Implications and Future Directions

SenCache represents a significant step toward making diffusion-based video generation more practical for real-world applications. By reducing inference costs without sacrificing quality, the method enables deployment scenarios that were previously computationally prohibitive, such as real-time video editing or high-resolution generation on resource-constrained devices.

The theoretical framework introduced by SenCache opens avenues for future research in adaptive inference strategies. The sensitivity-based approach could potentially be extended to other acceleration techniques, such as distillation or pruning, creating hybrid methods that combine multiple optimization strategies. Additionally, the framework’s interpretability provides insights into the denoising dynamics of diffusion models, which could inform architecture design and training procedures.

One promising direction is the integration of SenCache with learned acceleration methods. While SenCache is training-free, combining it with models specifically trained for fast inference could yield even greater speedups. Another area of exploration is the application of sensitivity-aware principles to other generative models beyond diffusion, such as autoregressive transformers or flow-based models.

The work also raises questions about the fundamental trade-offs between computation and quality in generative models. Understanding these trade-offs through the lens of sensitivity analysis could lead to more efficient model designs that naturally exhibit favorable caching properties.

Takeaways

1. Principled caching framework: SenCache provides the first theoretically grounded approach to caching in diffusion models, moving beyond empirical heuristics
2. Sensitivity as error predictor: Model output sensitivity to input perturbations reliably predicts caching error, enabling informed cache/reuse decisions
3. Adaptive per-sample optimization: Dynamic caching policies that adapt to individual samples outperform fixed schedules across diverse content
4. Training-free acceleration: SenCache achieves significant speedups without requiring model retraining or fine-tuning, making it immediately applicable to existing models
5. Consistent quality improvements: Experimental validation across multiple state-of-the-art video generation models demonstrates robust performance gains under equivalent computational budgets