UAV-based Reliable Optical Wireless Communication via Cooperative Multi-agent Reinforcement Learning Approach

In marine wireless sensor networks, swarms of unmanned aerial vehicles (UAVs) based optical communication system can be leveraged to transmit underwater real-time monitoring data which enables a variety of potential maritime missions, such as video streaming for underwater surveillance and target tracking. However, due to the high directionality of optical beams and the mechanical instability of UAVs caused by the wind, the optical link via UAVs as relays is very fragile. To deal with the challenge, in this study, we model the problem as a partially observed Markov decision process (POMDP), and propose a novel link maintenance scheme based on the cooperative multi-agent deep deterministic policy gradient (MADDPG) approach to control the swarms of UAVs, which integrates the UAV dynamics model and the optical communication model. In this scheme, multiple UAVs act as agents controlling their own states in real-time under the complex wind field, in order to keep mechanical stability through cooperation, and dynamically maintain the reliability of the optical links which maximize communication performance to achieve reliable end-to-end optical communication and reduce energy consumption. Through numerical simulations, it demonstrates that the proposed optimization scheme is effective and achieves robust performance in terms of communication quality and energy consumption.