Reinforcement Learning based Multi-layer Traffic Engineering for Intelligent Wireless Networking: From System to Algorithms

The AI-digital era is characterized by an unprecedented surge in data usage, spanning from data centers to IoT devices. This growth has driven the evolution of AI-optimized networks, designed to fuse AI capabilities with advanced network solutions seamlessly. However, these networks grapple with challenges such as the complexity of network layer protocols, discrepancies between simulated AI models and their real-world implementations, and the need for decentralized AI training due to network distribution.
To address these challenges, we propose the AI-oriented Network Operating System (AINOS). At the core of AINOS are two foundational sub-platforms: the "Network Gym," tailored for AI-driven network training, and "Federated Computing," designed for decentralized training methodologies. AINOS provides a comprehensive toolkit for rapid prototyping, deployment, and validation of AI-optimized networks, bridging the gap from simulation to real-world deployment.
Harnessing the powerful features of AINOS, we prototyped AI-optimized networking solutions using a safe Reinforcement Learning (RL) strategy for Traffic Engineering (TE) at both the link and network layers. At the link layer, we implemented a scalable RL-based traffic splitting mechanism that learns optimal traffic split ratios across Wi-Fi and LTE through guided exploration. For the network layer, we devised an online Multi-agent Reinforcement Learning (MA-RL) approach with domain-specific refinements to determine optimal paths in real-time for wireless multi-hop networks. In our exploration of Network Assisted AI optimization, we reduced Federated Learning training time with our MA-RL multi-routing approach and proposed a robust Decentralized Federated Learning solution that leverages single-hop connections for enhanced network performance. Our results demonstrate the strengths of AI-enhanced networks in proficiently managing heterogeneity and latency.