Multi-agent reinforcement learning for privacy-aware distributed CNN in heterogeneous IoT surveillance systems

Although Deep Neural Networks (DNN) have become the backbone technology of several Internet of Things (IoT) applications, their execution in resource-constrained devices remains challenging. To cater for these challenges, collaborative deep inference conducted by IoT devices was introduced. However, the prevalence of DNN computation suffers from severe privacy problems, e.g. data-reverse and model leakage. Particularly, malicious participants can accurately recover the received data to access sensitive information. Furthermore, the system is composed of heterogeneous data-sources represented by different DNN models that wish to execute classifications without exposing their data and models. Though, relaying the trained models to a centralized unit managing the collaboration leads to major risks because some features can be revealed through these models, in addition to dependency and scalability problems. In this paper, we present an approach that targets the privacy of collaborative inference via controlling the amount of data assigned to different participants, to prevent them from reversing attempts. Moreover, each independent data-source requesting inference will be responsible to manage the distribution of its DNN locally. In this context, different sources are required to compete over the pervasive resources while cooperating to maintain privacy welfare. We formulate this methodology, as an integer programming problem, where we establish a trade-off between the latency of co-inference and the privacy required by heterogeneous entities. A distributed solution scheme is also developed based on the Lagrangian dual problem. Next, to relax the optimization, we shape our approach as a cooperative and competitive Multi-Agent Reinforcement Learning (MARL) that supports heterogeneous/independent agents. Our comprehensive simulations demonstrated that our method yields results on par with those of a single RL agent in terms of action performance, while maintaining the privacy of individual agents’ information. Additionally, it surpasses the Independent Q-Learning (IQL) approach, where agents operate autonomously, in safeguarding inference privacy.