MUSCLE_en.md

MUSCLE - MICCAI 2022

This is a repository for paper "MUSCLE: Multi-task Self-supervised Continual Learning to Pre-train Deep Models for X-ray Images of Multiple Body Parts" accepted by MICCAI 2022.

Introduction

The goal of MUSCLE (MUlti-task Self-supervised Continual LEarning) is to pre-train the deep neural network (DNN) models and deliver decent performance on medical image analysis tasks.

All codes are implemented using PaddlePaddle.

Framework

MUSCLE aggregated multiple X-ray image datasets collected from different human body parts, subject to various X-ray analytics tasks. We proposed Multi-Dataset Momentum Contrastive Representation Learning (MD-MoCo) and Multi-task Continual Learning to pre-train the backbone DNNs in a self-supervised continual learning manner. The pre-trained models could be fine-tuned to target tasks using task-specific heads and achieve superb performance.

Datasets

Datasets	Body Part	Task	Train	Valid	Test	Total
Only Used for the first step (MD-MoCo) of MUSCLE
NIHCC	Chest	N/A	112,120	N/A	N/A	112,120
China-Set-CXR	Chest	N/A	661	N/A	N/A	661
Montgomery-Set-CXR	Chest	N/A	138	N/A	N/A	138
Indiana-CXR	Chest	N/A	7,470	N/A	N/A	7,470
RSNA Bone Age	Hand	N/A	10,811	N/A	N/A	10,811
Used for all three steps of MUSCLE
Pneumonia	Chest	Classification	4,686	585	585	5,856
MURA	Various Bone	Classification	32,013	3,997	3,997	40,005
Chest Xray Masks and labels	Chest	Segmentation	718	89	89	896
TBX	Chest	Detection	640	80	80	800
Total	N/A	N/A	169,257	4,751	4,479	178,757

Experiments

Experiment setups

• Backbone
  • ResNet-18 and ResNet-50
• Task
  • Pneumonia classification (Pneumonia),
  • Skeletal abnormality classification (MURA)
  • Lung segmentation (Lung)
  • Tuberculosis detection (TBX)
• Head
  • Fully-Connected (FC) Layer for classification tasks
  • DeepLab-V3 for segmentation tasks
  • FasterRCNN for detection tasks
• Baselines Pre-training Algorithms
  • Scratch: the models are all initialized using Kaiming’s random initialization and fine-tuned on the target datasets
  • ImageNet: the models are initialized using the officially-released weights pre-trained by the ImageNet dataset and fine-tuned on the target datasets
  • MD-MoCo: the models are pre-trained using multi-dataset MoCo and fine-tuned accordingly
  • MUSCLE−−: all models are pre-trained and fine-tuned with MUSCLE but with Cross-Task Memorization and Cyclic and Reshuffled Learning Schedule turned off

Results for various body parts

• Note: chest of Pneumonia and bones of MURA

Datasets	Backbones	Pre-train	Acc.	Sen.	Spe.	AUC(95%CI)
Pneumonia	ResNet-18	Scratch	91.11	93.91	83.54	96.58(95.09-97.81)
		ImageNet	90.09	93.68	80.38	96.05(94.24-97.33)
		MD-MoCo	96.58	97.19	94.94	98.48(97.14-99.30)
		MUSCLE--	96.75	97.66	94.30	99.51(99.16-99.77)
		MUSCLE	97.26	97.42	96.84	99.61(99.32-99.83)
	ResNet-50	Scratch	91.45	92.51	88.61	96.55(95.08-97.82)
		ImageNet	95.38	95.78	94.30	98.72(98.03-99.33)
		MD-MoCo	97.09	98.83	92.41	99.53(99.23-99.75)
		MUSCLE--	96.75	98.36	92.41	99.58(99.30-99.84)
		MUSCLE	98.12	98.36	97.47	99.72(99.46-99.92)
MURA	ResNet-18	Scratch	81.00	68.17	89.91	86.62(85.73-87.55)
		ImageNet	81.88	73.49	87.70	88.11(87.18-89.03)
		MD-MoCo	82.48	72.27	89,57	88.28(87.28-89.26)
		MUSCLE--	82.45	74.16	88.21	88.41(87.54-89.26)
		MUSCLE	82.62	74.28	88.42	88.5o(87.46-89.57)
	RcsNet-50	Scratch	80.50	65.42	90.97	86.22(85.22-87.35)
		ImngeNet	81.73	68.36	91.01	87.87(86.85-88.85)
		MD-MoCo	82.35	73.12	88.76	87.89(87.06-88.88)
		MUSCLE--	81.10	69.03	89.48	87.14(86.10-88.22)
		MUSCLE	82.60	74.53	88.21	88.37(87.38-89.32)

Results for various tasks

• Note: segmentation task for Lung and detection task for TBX

Backbones	Pre-train	Lung		TBX
		Dice	mloU	mAP	AP-Active	AP-Latent
ResNet-18	Scratch	95.24	94.00	30.71	56.71	4.72
	ImageNet	95.26	94.10	29.46	56.27	2.66
	MD-MoCo	95.31	94.14	36.00	67.17	4.84
	MUSCLE--	95.14	93.90	34.70	63.43	5.97
	MUSCLE	95.37	94.22	36.71	64.84	8.59
ResNet-50	Scratch	93.52	92.03	23.93	44.85	3.01
	ImageNet	93.77	92.43	35.61	58.81	12.42
	MD-MoCo	94.33	93.04	36.78	64.37	9.18
	MUSCLE--	95.04	93.82	35.14	57.32	12.97
	MUSCLE	95.27	94.10	37.83	63.46	12.21

Citation

If our work is helpful to you, please kindly cite our paper as:

@inproceedings{liao2022muscle,  
title={MUSCLE: Multi-task Self-supervised Continual Learning to Pre-train Deep Models for X-ray Images of Multiple Body Parts},  
author={Weibin, Liao and Haoyi, Xiong and Qingzhong, Wang and Yan, Mo and Xuhong, Li and Yi, Liu and Zeyu, Chen and Siyu, Huang and Dejing, Dou},  
booktitle={International Conference on Medical Image Computing and Computer-Assisted Intervention},
year={2022},  
organization={Springer}  
}