4714
Complex-Valued Convolutional Neural Networks for MRI Reconstruction1Electrical Engineering, Stanford University, Stanford, CA, United States, 2Radiology, Stanford University, Stanford, CA, United States
Synopsis
To improve MRI reconstruction accuracy, we propose various complex-valued frameworks for reconstructions using convolutional neural networks. By introducing complex-valued convolution and activation functions, we improve reconstruction of our subsampled images and achieve competitive results compared to the real-valued counterpart of our model.
Introduction
The reduction of scan times has widespread clinical benefits, especially for pediatric patients. A typical scan requires that patients remain still for a few seconds to a few minutes to produce diagnostic quality images. Scan times can be significantly reduced by subsampling in k-space, and high-quality images can be reconstructed using advanced reconstruction techniques.4,6,7 Most recently, convolutional neural networks (CNNs) have been shown to provide a rapid and robust solution to MRI reconstruction.1,5 Because many of the current deep learning frameworks do not readily support complex-valued networks, most work has split the real and imaginary components as two separate real-valued channels. Recent work with deep learning applied to MR fingerprinting has shown performance can be improved over real-valued networks with complex-valued networks.3,10 Since the raw MR data is complex, complex-valued networks should have a larger impact to CNNs applied to general MRI reconstruction. Our work investigates complex-valued CNNs for image reconstruction in lieu of two-channel real-valued CNNs.Methods
Recent work suggests that complex-valued CNNs could improve accuracy in comparison to real-valued CNNs when dealing with complex-valued data. Research on this topic remains scarce, because traditionally, CNNs have been applied to real-valued data. Initial work with CNN for complex data consists of feeding the data into CNNs by using a 2-channel architecture where the channels contain the real and imaginary components of the data. However, this architecture does not accurately represent the data because it disregards phase information, which is valuable in many MRI applications including blood flow, quantitative susceptibility mapping (QSM), fat-water separation, disease detection, and brain segmentation. For example, in a two-channel CNN, the rectifier activation function (ReLU) is applied separately to the real and imaginary components of the data, which does not preserve the phase component. Recent work in applying complex-valued CNNs to computer vision tasks as well as music and speech spectrum demonstrates that complex models are highly competitive with their real two-channel counterparts.9 Additionally, complex-valued neural networks have been applied to MRI fingerprinting, the task of identifying tissue parameters, with improvements in accuracy in comparison to real models.10
In this work, we apply the concept of complex-valued CNNs to the problem of subsampled image reconstruction by modifying components of our current CNN within our deep unrolled architecture, as described by 2, to be complex-valued. The structure of our network is displayed in Figure 1. Specifically, we perform complex convolution, which relies on the distributive property of convolution. This reduces the number of parameters the network learns. We also explore training the network with complex-valued convolution using various complex-valued activation functions which keep the pre-activated phase intact as well as activation functions which are based on the phase component. These activation functions include modReLU and zReLU, as described by 9, and as well as the cardioid activation function, as described by 10. We evaluate the performance in terms of accuracy of the complex-valued models compared to their real-valued counterpart.
Results
We trained two otherwise identical CNNs with real-valued convolution and complex-valued convolution, and approximately 900K trainable parameters each. As shown in Figure 2, the validation loss was 0.693 and 0.639, respectively. Comparison images between complex and real convolutions are shown in Figures 3 and 4. The network with complex convolution typically better reconstructs the phase. Additionally, we compared the test results of a network trained with complex convolution and various activation functions, including ReLU, modReLU, zReLU, and the cardioid functions. These metrics are shown in Figure 5.Discussion
By changing the convolutional layer from applying separate convolutional kernels on the real and imaginary components on the data, to applying a complex-valued filter to the complex-valued data, we achieve a lower magnitude loss on our images with the same number of trainable parameters. We see competitive results with various complex-valued activation functions. Interestingly, modReLU is the only one which achieves a lower loss between the phase of the output image and the truth image compared to the input image. Additionally, modReLU also has one of the lowest NMRSE and a competitive PSNR and SSIM compared to ReLU. The results also suggest a potential for a better performing activation function for complex-valued networks; thus, we are interested in exploring kernel activation functions, which allows the network to learn a trainable function for reconstructions, as described by 3,8.Conclusion
In this work, we have explored a variety of complex-valued convolutional network architectures with improved results compared to real-valued architectures. Our work shows potential for reducing MRI scan times by more accurately reconstructing images from subsampled data acquisitions using complex-valued CNNs.Acknowledgements
We gratefully acknowledge the support of GE Healthcare and NIH R01-EB009690.References
1. Chen, Feiyu et al. “Variable-Density Single-Shot Fast Spin-Echo MRI with Deep Learning Reconstruction by Using Variational Networks.” Radiology (2018): 180445.
2. Cheng, Joseph. “Highly Scalable Image Reconstruction Using Deep Neural Networks with Bandpass Filtering.” arXiv:1805.03300 [cs.CV], 8 May 2018, arxiv.org/abs/1805.03300.
3. Daval-Frerot, G, et. al., “Exploring Complex-Valued Neural Networks with Trainable Activation Functions for Magnetic Resonance Imaging.” ISMRM. Workshop on Machine Learning, 26 Oct. 2018.
4. Griswold, M, et. al., “Generalized autocalibrating partially parallel acquisitions (GRAPPA),” Magnetic Resonance in Medicine, vol. 47, no. 6, pp. 1202–1210, Jun. 2002. [Online]. Available: http://www.ncbi.nlm.nih.gov/pubmed/12111967
5. Hammernik K, et. al., “Learning a Variational Network for Reconstruction of Accelerated MRI Data,” arXiv:1704.00447 [cs.CV], Apr. 2017. [Online]. Available: http://arxiv.org/abs/1704.00447
6. Lustig, M, et al., “Sparse MRI: The Application of Compressed Sensing for Rapid MR Imaging.” Magnetic Resonance in Medicine, vol. 58, no. 6, pp. 1182–1195, Dec. 2007. [Online]. Available: http://www.ncbi.nlm.nih.gov/pubmed/17969013
7. Pruessmann, K P, et al., “SENSE: sensitivity encoding for fast MRI.” Magnetic Resonance in Medicine, vol. 42, no. 5, pp. 952–62, Nov. 1999. [Online]. Available: http://www.ncbi.nlm.nih.gov/pubmed/10542355
8. Scardapane, Simone, et al., “Kafnets: Kernel-Based Non-Parametric Activation Functions for Neural Networks.” Neural Networks (Elsevier), 23 Nov. 2017.
9. Trabelsi, Chiheb, et. al., “Deep Complex Networks.” ICLR 2018, 25 Feb. 2018.
10. Virtue, Patrick, et. al., “Better than Real: Complex-Valued Neural Nets for MRI Fingerprinting.” 2017 IEEE International Conference on Image Processing (ICIP), 1 July 2017.
Figures
