Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting

Paper: 2602.17634 Authors: Xinghong Fu, Yanhong Li, Georgios Papaioannou, Yoon Kim Categories: cs.LG, cs.AI

Abstract

This paper introduces Reverso, a family of efficient time series foundation models designed for zero-shot forecasting that challenges the prevailing assumption that larger models are necessary for better performance. While recent time series foundation models have followed the scaling paradigm successful in language and vision, resulting in models with hundreds of millions of parameters, Reverso demonstrates that small hybrid architectures combining long convolutions and linear RNN layers (specifically DeltaNet) can achieve comparable performance while being orders of magnitude smaller. The authors present a comprehensive recipe including architectural choices, data augmentation strategies, and inference techniques that collectively push the performance-efficiency Pareto frontier significantly forward.

Key Contributions

• Introduction of a hybrid architecture that interleaves long convolution layers with DeltaNet (linear RNN) layers, achieving competitive performance with 100x fewer parameters than transformer-based models
• Demonstration that large-scale transformers are not necessary for effective time series foundation models
• Development of data augmentation and inference strategies specifically tailored for efficient zero-shot time series forecasting
• Comprehensive empirical validation showing Reverso models significantly advance the performance-efficiency trade-off in time series forecasting

Architecture Design

The core innovation of Reverso lies in its hybrid architecture that departs from the transformer-dominated paradigm. Instead of relying on attention mechanisms that scale quadratically with sequence length, Reverso employs a combination of:

Long Convolution Layers: These layers capture local temporal patterns efficiently through convolution operations with extended receptive fields. Unlike standard convolutions, long convolutions can model dependencies across extended time horizons without the computational overhead of attention.

DeltaNet Layers: DeltaNet represents a class of linear RNNs that maintain the modeling capacity of traditional RNNs while offering linear-time complexity. By interleaving DeltaNet layers with convolution layers, Reverso achieves a balance between local pattern recognition and long-range dependency modeling.

This architectural choice is motivated by the observation that time series data exhibits different characteristics than natural language or images. The temporal structure and periodicity in time series can be effectively captured by convolutions and linear recurrences without requiring the full expressiveness (and computational cost) of self-attention.

Training and Inference Strategies

Beyond architecture, the paper describes several critical strategies that enhance Reverso’s performance:

Data Augmentation: The authors employ domain-specific augmentation techniques that preserve the statistical properties of time series while increasing training diversity. These augmentations help the model generalize across different time series domains without overfitting to specific patterns.

Inference Optimization: The paper introduces inference-time strategies that leverage the model’s efficiency. The linear complexity of the architecture enables fast inference even on long sequences, making Reverso practical for real-time forecasting applications.

Zero-shot Transfer: The training recipe is specifically designed to maximize zero-shot transfer performance, allowing the model to forecast on unseen time series domains without fine-tuning. This is achieved through careful dataset curation and training objectives that encourage learning of generalizable temporal patterns.

Experimental Results

The empirical evaluation demonstrates that Reverso models achieve performance competitive with much larger transformer-based foundation models across diverse benchmarks. Key findings include:

• Models with only a few million parameters match or exceed the performance of models with hundreds of millions of parameters
• The efficiency gains translate to significant reductions in inference time and memory footprint
• Zero-shot forecasting performance remains strong across different domains, including finance, energy, and web traffic
• The performance-efficiency Pareto frontier is substantially improved, offering practitioners viable options for resource-constrained deployments

Implications for Foundation Model Research

Reverso’s success challenges the assumption that scaling is the primary path to better foundation models in time series. The results suggest that:

Domain-Specific Architectures Matter: While transformers have proven universal in language and vision, time series may benefit from architectures specifically designed for temporal data. The inductive biases provided by convolutions and linear RNNs appear well-suited to time series structure.

Efficiency as a First-Class Objective: Rather than treating efficiency as a post-hoc optimization problem, designing for efficiency from the start can lead to fundamentally different and better solutions. Reverso demonstrates that the performance-efficiency trade-off is not fixed but can be shifted through careful architectural and algorithmic choices.

Practical Deployment Considerations: The orders-of-magnitude reduction in model size makes foundation models accessible for edge deployment, real-time applications, and resource-constrained environments where large transformers are impractical.

Takeaways

1. Large transformers are not necessary for effective time series foundation models—hybrid architectures with convolutions and linear RNNs can achieve comparable performance at a fraction of the size
2. Reverso models are 100x smaller than transformer-based alternatives while maintaining competitive zero-shot forecasting accuracy
3. Domain-specific architectural choices that respect the structure of time series data can outperform generic scaling approaches
4. The combination of architectural efficiency, targeted data augmentation, and optimized inference strategies is key to pushing the performance-efficiency Pareto frontier
5. Efficient foundation models enable new deployment scenarios including edge devices, real-time systems, and cost-sensitive applications