Building blocks to address variations in federated medical image analysis

Abstract

Federated learning (FL) has shown promising potential in enabling multiple medical institutions/clients to collaboratively train deep learning models while preserving data privacy. However, in practice, differences in data acquisition protocols, patient demographics, and medical expertise among clients, also known as inter-client variations, can hinder the effectiveness of FL. To address these challenges, we propose four modular building blocks. The first two blocks focus on addressing inter-client statistical variations, which arise due to differences in data acquisition protocols and access to patient demographics. We propose two regularization blocks: parametric and non-parametric. The parametric regularization enables each client to learn a personalized sub-model that regularizes the federated model, while the non-parametric regularization (NPR) explicitly models each class's characteristics using feature prototypes to regularize the federated model. The NPR is particularly useful when some clients suffer from severe class imbalance and missing classes. The third building block addresses inter-client annotation variations, which occur when different medical clients have varying data labeling budgets, leading to variations in available annotations across clients. We address this with a label-agnostic framework that utilizes every available training data, regardless of the availability of annotations. The fourth building block explores the impact of vision foundation models, such as the Segment Anything Model (SAM), on federated medical image analysis. To utilize SAM, it needs to be adapted with a sizeable amount of medical data. In principle, FL enables the privacy-preserving adaptation of SAM. However, SAM tends to increase inter-client variations. To address this, we propose an auxiliary FL framework that bridges FL and SAM to learn a robust federated medical segmentation model. Our extensive experiments validate the effectiveness of our modular building blocks in addressing inter-client variations. We believe that these building blocks can better facilitate federated collaboration and pave the way for the widespread adoption of FL in medical image analysis.

Date of Award	2024
Original language	English
Awarding Institution	The Hong Kong University of Science and Technology
Supervisor	Kwang-Ting Tim CHENG (Supervisor)