Introduction

Rheumatoid arthritis (RA) is a chronic inflammatory disease affects approximately 1% of the US population, caused by destruction of articular and periarticular structures [1]. The early diagnosis and accurate monitoring of rheumatoid arthritis (RA) are essential to delay joint destruction and functional disability and also necessary for evaluating successful therapy. Magnetic Resonance images (MRI) is a sensitive tool for the detection and quantification of synovitis [2]. However, the transition of quantitative MRI in clinical settings requires fast accurate fully automatic segmentation. In this work, we have presented a fully automatic segmentation algorithm for synovitis in wrist MR in subjects with RA using deep learning-based conditional generative adversarial networks (cGAN) [3] and U-Net [4].

Method

26 patients with RA were studied at baseline (n=17), 1-month (n=6), 3-month (n=19) and 1-year (n=10), resulting a total of 52 exams. MRI data were collected on a 3-T scanner (GE Healthcare) with a 16Rx wrist coil (InVivo). The imaging protocol included pre- and post-Gd T1-weighted spoiled gradient echo (SPRG) images (dimension 512x512x20, in plane resolution 0.4mm, 2mm slice thickness. The data were split randomly into 42 (21 subjects) : 10 (5 subjects) for training: testing set. Deep learning architecture based on CGAN was used where two networks are trained simultaneously: one focuses on generating realistic segmentation (generator) and the other discriminating between the manual segmentation and the generated one (discriminator). 2D-UNet with 10 convolution layers of encoder and decoder was used as generator and another typical convolutional neural network with 10 layers was used as a discriminator. The generator UNet took 2D slices with two channels (T1-pre and post contrast) as input and provides pixel wise probability map for synovitis. The output probability map along with two channels was provided to the discriminator for segmentation feedback. Along with adversarial loss from discriminator, Dice coefficient loss between manual and auto segmentation, and the feature loss from the discriminator were used to train the network. Adam optimizer was used with an initial learning rate of 10e-4. Batch size was 10. In training, the input MRI volumes were augmented by random flip along X-axis and Y-axis during runtime. The model was implemented in python using Keras 2.2.2 (5) and Tensorflow 1.10.0 [6] framework and trained on Owens Cluster with NVIDIA Tesla P100 GPU of Ohio Supercomputer Center [7]. The segmentation performance was evaluated using the Dice coefficient comparing the automatic segmentation and the manual segmentation.

Results

An average Dice coefficient of 0.78 was obtained with standard deviation of 0.08 across a test set of 10 volumes, compared with manual segmentations. The example of the automated segmentations along with manual segmentation is shown in Fig 1. The correlation between volumes and intensity of manual segmentation and automatic segmentation are R=0.77 and R=0.87.

Discussion

Synovitis is a very common finding in joints of RA patients, which may serve as biomarkers for early diagnosis and for early treatment response evaluation. However, synovitis quantification is challenging because manual segmentation of such irregular lesions are very time consuming and fully automated methods are desirable. Using a small number of training data, the proposed model demonstrated feasibility of fully automatically synovitis segmentation with reasonable accuracy. In Fig 1A, it can be observed that the auto segmentation has learned the general segmentation regions and can segment the cases even if the structure is irregular due the bone erosion (Fig 1B). Though, model sometimes over segment bright regions within the bone as synovitis (pointed by arrow depicted in Fig 1C) and fails to segment lesions outside the regions (pointed by arrow depicted in Fig 1D).

Conclusion

In this study, we presented a deep learning-based approach to automatically segment synovitis lesions in a patient with RA using CGAN-UNet based network. The proposed network can obtain reasonable segmentation performance using very few manual segmentations and can segment small synovitis lesions. In future work, we will improve performance by including more manual segmentation, other image contrasts such as T2-weighted image and optimizing model parameters. Also, the simultaneous segmentation of other types of lesions such as bone marrow edema will be incorporated in the model.

Acknowledgements

The data collection was supported by UCB Pharma.

References

[1] Mikuls T., Saag K. Comorbidity in rheumatoid arthritis, Rheumatic Disease Clinics of North America, Volume 27, Issue 2, 2001(283-303) [2] Yang H1, Rivoire J, Hoppe M, Srikhum W, Imboden J, Link TM, Li X. Computer-aided and manual quantifications of MRI synovitis, bone marrow edema-like lesions, erosion and cartilage loss in rheumatoid arthritis of the wrist. Skeletal Radiol. 2015 Apr;44(4):539-47. [3] Liu F. SUSAN: segment unannotated image structure using adversarial network. Magn Reson Med. 2019 May;81(5):3330-3345. [4] Norman B, Pedoia V, Majumdar S. Use of 2D U-Net Convolutional Neural Networks for Automated Cartilage and Meniscus Segmentation of Knee MR Imaging Data to Determine Relaxometry and Morphometry. Radiology. 2018;288(1):177-185. [5] Chollet, F. and others (2015) keras, https://keras.io. [6] Abadi M, Barham P, Chen J, et al. TensorFlow: a system for large‐scale machine learning. In Proceedings of the 12th USENIX conference on Operating Systems Design and Implementation, Savannah, GA, 2016. p. 265‐283. [7] Ohio Supercomputer Center. http://osc.edu/ark:/19495/f5s1ph73.