2415
Multi-task MR imaging with deep learning1Paul C Lauterbur Research Center, Shenzhen Inst. of Advanced Technology, shenzhen, China, 2Northeastern University, Shenyang, China
Synopsis
Noises, artifacts, and loss of information caused by the MR reconstruction may compromise the final performance of the downstream applications such as image segmentation. In this study, we develop a re-weighted multi-task deep learning method to learn prior knowledge from the existing big dataset and then utilize them to assist simultaneous MR reconstruction and segmentation from under-sampled k-space data. It integrates the reconstruction with segmentation and produces both promising reconstructed images and accurate segmentation results. This work shows a new way for the direct image analysis from k-space data with deep learning.
Introduction
Magnetic resonance (MR) imaging is of great value in clinical applications such as medical diagnosis [1], disease staging [2] and clinical research [3] since it can produce highly detailed images with excellent soft tissue contrast. Most of the existing MR image reconstruction algorithms overlooked the downstream applications such as segmentation [4] since they take optimal visual quality as the first priority, rather than the specific-task quality. The information discarded during the reconstruction process may influence the final segmentation performance [5]. In this study, we propose a task-driven MR imaging method, equipped with teacher forcing to avoid exposure bias, and re-weighted loss training to help the proposed method achieve simultaneous high quality MR reconstruction and segmentation.Method
Fig. 1 presents the overall workflow of the proposed task-driven MR imaging method. It consists of two key components. Namely, we select D5C5 [6] as the reconstruction module and U-Net [7] as the segmentation module respectively. Trained task-driven MR imaging method takes the under-sampled k-space data as input and outputs both the reconstructed images and the segmented masks. In detail, we treat the training process of task-driven MR imaging as an alternating update between the reconstruction and segmentation modules. A teacher forcing method is designed to stabilize the training process and avoid error accumulation by feeding the fully sampled image into the segmentation sub-module. A reweighted loss function is proposed for properly simultaneously prompting the performance of the reconstruction and segmentation during training, with:$$ L=\alpha(t) L_{recon} + \beta(t) L_{seg} \tag{1}$$
wherer $$$ L_{recon} $$$ and $$$ L_{seg} $$$ denote the loss function of the reconstruction and segmentation respectively, $$$ \alpha(t) $$$ and $$$ \beta(t) $$$ are the corresponding weights, and $$$ t $$$ represents the corresponding training epoch. We set $$$ \alpha(t) $$$ and $$$ \beta(t) $$$ as shown in Eq. (2) and (3),
$$ \alpha(t) = \max (\exp (-t) - 0.2, 0.05) \tag{2}$$
$$ \beta(t) = 1 - \alpha(t) \tag{3}$$
which can change quickly at the first few epochs and converged at the last epochs. We also design a subtraction and max operation to avoid too large or small weight values.
Experimental configuration
We evaluated our method on a competitive MR image dataset, namely ATLAS [10]. This dataset consists of 229 intensity normalized subjects on T1 modality in standard space (normalized to the MNI-152 template) collected from 11 cohorts worldwide, with an in-plane resolution of $$$ 1 mm^3 $$$. For k-space data sub-sampling, we performed 1-dimentional masks on the full-sampled acquisition retrospectively. It was designed to simulate physically realizable accelerations by omitting k-space lines in the phase encoding direction. For a volume, all slices were applied to the same under-sampling mask. The overall acceleration factor was set as four. All under-sampling masks were generated in two steps. First step, 8% of all k-space lines from the central region of the fully-sampled k-space were kept. The kept adjacent lowest frequency k-space lines provided a fully-sampled k-space region. The remaining k-space lines were selected with a set probability uniformly at random to achieve the desired acceleration factor. It was chosen to meet the general conditions for compressed sensing [11,12].For segmentation results, we compare our approach with existing methods which segment from under-sampled data, including SynNet [8], LI-Net [8], SERANet [5] and SegNetMRI [9]. We also compare our method with a widely used segmentation algorithm U-Net [7] which is obtained from fully-sampled data. For reconstruction results, we compare the proposed method with SegNetMRI and D5C5 [6].
Result
Table1 and Figure 3 illustrate the segmentation performance of the proposed approach on ATLAS dataset compared with existing segmenting from k-space data methods. Figure 3 shows that our approach performs well on segmenting various size of lesions. The violin plot (Figure 4) demonstrates that the proposed method achieves better segmentation performance not only on average dice score but also on median value and standard-deviation. This means our method is quite robust to lesions. The box plot (Figure 5) among different lesion sizes illustrates that the proposed method performs well on small lesions, while other algorithms cannot segment small lesions correctly.Table 1 and Figure 2 indicate the promising reconstruction result of our method compared with SegNetMRI and D5C5. Our approach not only performs well in quantitative metrics, but also in visual results (with smallest error). Table 1 illustrates that our approach achieves promising evaluation metrics (PSNR and SSIM) compared with D5C5 and SegNetMRI. Figure 2 shows that the error our approach achieves is lower than SegNetMRI, and close to D5C5, which suggests a promising reconstruction performance.
Conclusion and outlook
Our study proposes a deep-learning based approach for multi-task MR imaging. Our method has achieved encouraging segmentation results and promising reconstruction performance on ATLAS dataset, compared to other 6 state-of-the-art methods. The works shows a new way for direct image analysis from k-space data.Acknowledgements
This research was partly supported by Scientific and Technical Innovation 2030 - "New Generation Artificial Intelligence" Project (2020AAA0104100, 2020AAA0104105), the National Natural Science Foundation of China (61871371, 81830056), Key-Area Research and Development Program of GuangDong Province (2018B010109009), the Basic Research Program of Shenzhen (JCYJ20180507182400762), Youth Innovation Promotion Association Program of Chinese Academy of Sciences (2019351).References
[1] F. Bruno, F. Arrigoni, P. Palumbo, R. Natella, N. Maggialetti, A. Reginelli, A. Splendiani, E. Di Cesare, L. Brunese,G. Guglielmi et al., “New advances in mri diagnosis of degener-ative osteoarthropathy of the peripheral joints,” La radiologiamedica, vol. 124, no. 11, pp. 1121–1127, 2019.
[2] S. A. Taylor, S. Mallett, S. Ball, S. Beare, G. Bhatnagar,A. Bhowmik, P. Boavida, J. Bridgewater, C. S. Clarke, M. Dug-gan et al., “Diagnostic accuracy of whole-body mri versusstandard imaging pathways for metastatic disease in newlydiagnosed non-small-cell lung cancer: the prospective streamlinel trial,” The Lancet Respiratory Medicine, vol. 7, no. 6, pp. 523–532, 2019.
[3] S. Debette, S. Schilling, M.-G. Duperron, S. C. Larsson, and H. S. Markus, “Clinical significance of magnetic resonanceimaging markers of vascular brain injury: a systematic reviewand meta-analysis,” JAMA neurology, vol. 76, no. 1, pp. 81–94,2019.
[4] J. Caballero, W. Bai, A. N. Price, D. Rueckert, and J. V. Hajnal,“Application-driven mri: joint reconstruction and segmentationfrom undersampled mri data,” in International Conference onMedical Image Computing and Computer-Assisted Interven-tion. Springer, 2014, pp. 106–113
[5] Q. Huang, X. Chen, D. Metaxas, and M. S. Nadar, “Brainsegmentation from k-space with end-to-end recurrent attentionnetwork,” in International Conference on Medical Image Com-puting and Computer-Assisted Intervention. Springer, 2019,pp. 275–283.
[6] J. Schlemper, J. Caballero, J. V. Hajnal, A. N. Price, andD. Rueckert, “A deep cascade of convolutional neural networksfor dynamic MR image reconstruction,” IEEE transactions onMedical Imaging, vol. 37, no. 2, pp. 491–503, 2017.
[7] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutionalnetworks for biomedical image segmentation,” in InternationalConference on Medical image computing and computer-assistedintervention. Springer, 2015, pp. 234–241.
[8] Schlemper, Jo, et al. "Cardiac MR segmentation from undersampled k-space using deep latent representation learning." International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2018.
[9] Sun, Liyan, et al. "Joint cs-mri reconstruction and segmentation with a unified deep network." International Conference on Information Processing in Medical Imaging. Springer, Cham, 2019.
[10] Liew, Sook-Lei, et al. "A large, open source dataset of stroke anatomical brain images and manual lesion segmentations." Scientific data 5 (2018): 180011.
[11] Candès, Emmanuel J., Justin Romberg, and Terence Tao. "Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information." IEEE Transactions on information theory 52.2 (2006): 489-509.
[12] Lustig, Michael, David Donoho, and John M. Pauly. "Sparse MRI: The application of compressed sensing for rapid MR imaging." Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine 58.6 (2007): 1182-1195.
Figures
