Adaptive Federated Learning for Real-time Anomaly Detection and Response in Large-Scale Distributed Networks

Abstract

The problem is the design of an adaptive, distributed anomaly detection system which protects privacy, and has the ability to evolve in real-time to identify and mitigate new threats while minimizing maximum detection delay and damage, in very large distributed environments. In this work, we propose an A-FCL (Adaptive Federated Collaborative Learning) architecture to enhance the separability of unknown anomaly patterns, adopt meta-learning approach (Meta-UAD) to decrease the false positive rate (FPR) and enhance adaptability, and develop a fully automated response architecture based on SOAR/SDN together with RL-AR (Reinforcement Learning with Adaptive Regulation) frameworks to dynamically and securely adapt low-latency response strategies. The relevance of this work is that it is the first to confront most challenging cybersecurity problems of the digital age and deliver an extremely high throughput autonomous security capability in large-scale environments slashing response times from minutes to milliseconds. The research methodology employed analytical, operational, and simulation methods, in which an advanced computer simulation environment was developed to evaluate the A-FCL framework —comprising contrastive learning as well as trust & reputation schemes to mitigate poisoning attacks—on extensive, heterogeneous (Non-IID) network datasets. The result was that the A-FCL model outperformed all the other models with an accuracy of 98.7% and the FPR was greatly reduced to 0.5% (a reduction of about 81.25%) by applying adaptive learning schemes. The system also exhibited excellent scalability, achieving a processing throughput of 22.1 GB/s for 50 edge nodes and an automated response time to critical attacks (such as DDoS) of no more than 45 milliseconds, representing a 96.6% reduction compared to manual response, while ensuring operational safety by reducing the safety violation rate by RL-AR to less than 0.4%.