Asynchronous Decentralized Federated Lifelong Learning (ADFLL)

Guangyao Zheng December 30, 2024 April 16, 2025

This project introduces the Asynchronous Decentralized Federated Lifelong Learning (ADFLL) framework, an innovative approach to federated learning that addresses the limitations of synchronous training schedules and the lack of lifelong learning in conventional machine learning frameworks for medical applications. ADFLL enables asynchronous and continual learning across agents, allowing them to leverage both their own experiences and knowledge shared by others. The framework was evaluated using deep reinforcement learning (DRL) for landmark localization tasks across diverse imaging modalities, orientations, and sequences. Experimental results demonstrated that ADFLL outperforms baseline models in collaborative learning, showing superior performance on both in-distribution and out-of-distribution test sets. This robust, efficient, and flexible framework is well-suited for deployment in real-world applications requiring privacy-preserving and lifelong collaborative learning. Paper published in Medical Imaging with Deep Learning (MIDL) 2024, Towards a Collective Medical Imaging AI: Enabling Continual Learning from Peers. Github repo: https://github.com/guangyaoz/ADFLL.

Illustration of asynchronous decentralized federated lifelong learning ADFLL set up for cross-modality 3D localization of spleen. The blue, orange, and green boxes represent different agents in the setup. Each agent sequentially encounters two different imaging modalities for along with experiences shared by the other nodes, enabling them to learn to localize the spleen across all four modalities as opposed to just the two they encountered . — Illustration of asynchronous decentralized federated lifelong learning (ADFLL) set up for cross-modality 3D localization of spleen. The blue, orange, and green boxes represent different agents in the setup. Each agent sequentially encounters two different imaging modalities for along with experiences shared by the other nodes, enabling them to learn to localize the spleen across all four modalities (as opposed to just the two they encountered).

To further enhance the efficiency of the ADFLL framework, we integrate a reward distribution-preserving coreset compression technique for selective experience replay buffers. This method compresses stored experiences, reducing computational overhead while maintaining the model’s ability to mitigate catastrophic forgetting. Evaluated on tasks such as ventricle localization in the BRATS dataset and landmark localization in whole-body MRI, the compressed lifelong learning models demonstrated competitive performance with minimal impact on accuracy. For instance, the 10x compressed models achieved mean pixel distances close to conventional lifelong learning models, highlighting the viability of coreset compression in resource-constrained settings. This addition reinforces ADFLL’s scalability and adaptability to real-world applications. Paper published in Medical Imaging with Deep Learning (MIDL) 2023, Selective experience replay compression using coresets for lifelong deep reinforcement learning in medical imaging.

To further address the challenges of deploying the ADFLL framework on low-compute edge devices in rapidly evolving imaging environments, we developed three image coreset algorithms for compressing and denoising medical images in selective experience replay. These include neighborhood averaging, neighborhood sensitivity-based sampling, and maximum entropy coresets, which achieve 27x compression while maintaining strong performance in localizing anatomical landmarks on DIXON water and fat MRI images. Notably, the maximum entropy coreset outperformed conventional lifelong learning models with an average distance error of 11.97±12.02 compared to 19.24±50.77, showcasing its potential to enhance efficiency and adaptability in real-world medical imaging applications. Paper on arXiv: A framework for dynamically training and adapting deep reinforcement learning models to different, low-compute, and continuously changing radiology deployment environments.

This project is funded by DARPA, part of the Shared Experience Lifelong Learning (ShELL) program. Collectively, we have published a paper on Nature Machine Intelligence, A collective AI via lifelong learning and sharing at the edge.