OSI-FL: One-Shot Incremental Federated Learning with Catastrophic Forgetting Resilience

Paper: 2602.17625 Authors: Obaidullah Zaland, Zulfiqar Ahmad Khan, Monowar Bhuyan Categories: cs.LG, cs.DC

Abstract

This paper introduces One-Shot Incremental Federated Learning (OSI-FL), a groundbreaking framework that simultaneously tackles two critical challenges in federated learning: communication efficiency and catastrophic forgetting in incremental learning scenarios. Traditional federated learning requires multiple communication rounds and struggles with dynamic data streams, while OSI-FL achieves effective model training with just a single communication round. The framework leverages frozen vision-language models (VLMs) to extract category-specific embeddings from client data, which are then transmitted to a central server. A pre-trained diffusion model synthesizes training data from these embeddings, enabling server-side model training without accessing raw client data. To address catastrophic forgetting as new tasks arrive incrementally, OSI-FL incorporates Selective Sample Retention (SSR), which intelligently preserves the most informative samples based on loss metrics. Experimental validation across three benchmark datasets demonstrates superior performance over both traditional multi-round federated learning and existing one-shot approaches in class-incremental and domain-incremental settings.

Key Contributions

• First federated learning framework to simultaneously address one-shot communication and catastrophic forgetting in incremental learning scenarios
• Novel architecture combining frozen VLMs for embedding extraction with diffusion models for privacy-preserving data synthesis
• Selective Sample Retention (SSR) mechanism that identifies and retains top-$p$ most informative samples per category-task pair based on loss metrics
• Comprehensive evaluation demonstrating superiority over baselines in both class-incremental and domain-incremental learning across CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets

Technical Methodology

The OSI-FL framework operates through a carefully orchestrated pipeline. Each client maintains a frozen vision-language model that processes local data to generate category-specific embeddings. These compact representations capture semantic information about data distributions without exposing raw samples, preserving privacy. In a single communication round, clients transmit these embeddings to the central server.

The server employs a pre-trained diffusion model conditioned on the received embeddings to synthesize artificial training samples. This generative approach reconstructs data distributions similar to those at clients while maintaining privacy guarantees. The synthesized dataset enables the server to train a global model without direct access to client data.

As new tasks arrive incrementally, the framework faces the dual challenge of incorporating new knowledge while retaining previously learned information. The Selective Sample Retention mechanism addresses this by computing loss values for all samples and retaining the top-$p$ most informative ones per category-task combination. These retained samples are strategically incorporated into subsequent training iterations, creating a memory buffer that bounds forgetting. The selection criterion based on sample loss ensures that the most challenging and representative examples are preserved, maximizing the efficiency of the limited memory budget.

Experimental Results and Analysis

The experimental evaluation spans three benchmark datasets with varying complexity levels. On CIFAR-10 and CIFAR-100, OSI-FL demonstrates consistent improvements over traditional federated learning approaches that require multiple communication rounds. The framework shows particular strength in class-incremental scenarios where new categories are introduced sequentially.

In domain-incremental settings, where the task structure remains constant but data distributions shift, OSI-FL maintains robust performance. The Selective Sample Retention mechanism proves crucial in these scenarios, with ablation studies showing significant performance degradation when SSR is removed. The retention of informative samples creates an effective rehearsal mechanism that stabilizes learning across tasks.

Comparison with existing one-shot federated learning methods reveals OSI-FL’s advantages. While other one-shot approaches achieve communication efficiency, they typically suffer from severe catastrophic forgetting. OSI-FL’s integration of VLM-based embeddings with diffusion-based synthesis and strategic sample retention creates a synergistic effect that outperforms component techniques in isolation.

The scalability analysis demonstrates that OSI-FL maintains efficiency even as the number of clients and tasks increases. The single communication round constraint ensures predictable bandwidth requirements, while the SSR mechanism’s computational overhead remains manageable through efficient loss-based ranking.

Implications and Future Directions

OSI-FL represents a significant advancement for privacy-sensitive applications requiring continuous learning from distributed data sources. The framework’s ability to operate with minimal communication makes it particularly suitable for edge computing scenarios, IoT networks, and cross-organizational collaborations where bandwidth is limited or expensive.

The combination of vision-language models and diffusion-based synthesis opens new avenues for federated learning research. Future work could explore adaptive retention strategies that dynamically adjust the number of preserved samples based on task difficulty or available memory. Integration with other continual learning techniques, such as elastic weight consolidation or progressive neural networks, could further enhance forgetting resistance.

The framework’s reliance on pre-trained models (VLMs and diffusion models) raises questions about generalization to domains where such models are unavailable or perform poorly. Investigating domain-specific adaptations or developing lightweight alternatives could broaden applicability. Additionally, theoretical analysis of the privacy guarantees provided by embedding-based communication compared to raw data transmission would strengthen the framework’s foundations.

Takeaways

1. OSI-FL achieves effective federated learning with just one communication round while preventing catastrophic forgetting through intelligent sample retention
2. Vision-language model embeddings combined with diffusion-based synthesis enable privacy-preserving data reconstruction at the server
3. Selective Sample Retention based on loss metrics provides an efficient memory mechanism that bounds forgetting across incremental tasks
4. The framework outperforms both traditional multi-round federated learning and existing one-shot approaches across multiple benchmark datasets
5. OSI-FL’s architecture is particularly well-suited for bandwidth-constrained environments requiring continuous learning from distributed, privacy-sensitive data sources