Byzantine-Robust and Differentially Private Federated Optimization under Weaker Assumptions

Paper: 2603.23472 Authors: Rustem Islamov, Grigory Malinovsky, Alexander Gaponov, Aurelien Lucchi, Peter Richtárik, Eduard Gorbunov Categories: cs.LG, cs.CR, math.OC

Abstract

This paper tackles federated learning when both privacy and adversarial robustness matter. It proposes Byz-Clip21-SGD2M, which mixes robust aggregation, double momentum, and clipping. The method avoids relying on bounded gradients or extra server data, and the analysis gives high-probability convergence under standard smoothness plus sub-Gaussian gradient noise. It also matches strong rates without attackers and improves utility with Byzantine and DP constraints. Tests on MNIST with CNN and MLP models support the method.

Key Contributions

• Combines Byzantine robustness and differential privacy in one federated optimization method
• Uses robust aggregation, double momentum, and clipping
• Removes the need for bounded gradients or auxiliary server data
• Provides high-probability convergence guarantees under smoothness and sub-Gaussian gradient noise
• Shows practical results on MNIST with CNN and MLP models

Methodology

The core idea is to make federated optimization resilient on two fronts at once: adversarial clients and privacy-preserving noise. The proposed optimizer, Byz-Clip21-SGD2M, uses clipping to control the influence of extreme updates, double momentum to stabilize noisy optimization, and robust aggregation to reduce the impact of Byzantine participants.

A key advantage is that the analysis works under weaker assumptions than many prior methods. Instead of requiring bounded gradients or extra server-side data, the paper assumes smooth objectives and sub-Gaussian gradient noise. That makes the result easier to fit into realistic federated learning settings.

Results

The theoretical guarantees show convergence with high probability, and the rates remain strong even without attackers. When Byzantine and differential privacy constraints are both present, the method still improves utility relative to weaker baselines.

The MNIST experiments with CNN and MLP models provide a concrete sanity check. They show that the method is not only mathematically neat but also usable in a standard learning benchmark.

Takeaways

1. Federated learning can be made robust to both malicious clients and privacy constraints in one method
2. Clipping and robust aggregation are useful complements to momentum-based optimization
3. Strong convergence guarantees are possible under weaker assumptions than bounded-gradient settings
4. Byzantine robustness and differential privacy do not have to be treated as separate problems
5. The method’s empirical results on MNIST support the theoretical analysis