Synopsis

Keywords: Image Reconstruction, Machine Learning/Artificial Intelligence

Conventional medical image reconstruction methods are less parametric and lack generality due to random error and noise. A novel artificial Fourier transform (AFT) framework is developed which determines the mapping between k-space and i-space like DFT while can be fine-tuned with further training. The flexibility of AFT allows it to be simply incorporated into any existing deep learning network as learnable or static blocks. Reconstruction and denoising tasks are combined into a unified network that simultaneously enhances the image quality. AFT-Net achieves competitive results compared with other methods and proofs to be more robust to additional noise and contrast differences.

Introduction

Medical image reconstruction from the sensor domain to the image domain is critical because it lays the foundation for the following image analysis tasks. How we reconstruct and process the raw data acquired from the imaging system sensors directly influences the overall image quality. To solve the problem mentioned above, a unified complex-valued image reconstruction approach is proposed for magnetic resonance images (MRI). The framework described in this study is the artificial Fourier transformation (AFT) which has the full functionality of the state-of-the-art Fourier transform and can be fine-tuned through further training. AFT can be incorporated into any existing deep learning pipeline with fully adjustable parameters. We utilize AFT combined with deep complex-valued networks to design the AFT-Net for MRI reconstruction plus denoising and reconstruction plus accelerated imaging.

Material and Methods

2.1 Datasets
The models are trained on a real-world dataset that contains mouse brain T2w MRI scans over 243 subjects. Each subject is scanned for 4 repetitions, and the size of each slice is $$$201\times402$$$ with resolution $$$0.075mm\times0.075mm$$$. The trained networks are further evaluated on MRI scans with other modalities and different contrast on 10 subjects randomly selected from the testing set. For the dataset with simulated additional Gaussian noise in k-space, we add random Gaussian noise to the real and the imaginary part of k-space data separately. For the dataset with different contrast caused by GBCA, we use the T2-weighted contrast-enhanced MRI scanned from the same 10 subjects.

2.2 Network Architecture
The general workflow and the network structure are shown in Figure 1. We implement complex-valued neural networks and propose AFT with two successive blocks. Each block consists of two complex linear layers followed by a transpose operation. To combine reconstruction with other tasks, we incorporate our AFT with a complex-valued U-Net¹ which extracts higher features in the k-space or i-space domain and forces the network to represent sparsely in that domain.

Results

3.1 Reconstruction
We train our AFT on the real-world T2-weighted 4-coils k-space raw data and compare it with the comparative method AUTOMAP². The residual map between the ground truth and AFT prediction is presented in Figure 2a. It can be seen that no mouse brain structural information is shown in the residual, so the error is mainly caused by precision loss during floating-point calculation in matrix multiplication. The maximum error exists in the eyes and neck region, where motion artifacts are commonly present due to respiration and eye movement, which is inevitable for in-vivo MRI scans.

3.2 Reconstruction plus denoising
The AFT-Net trained on the real-world dataset aims to combine reconstruction and denoising. The models are compared with the numerical method, real-valued neural networks, DFT-based networks, and other methods. Given k-space data containing noise, AFT-Net can learn the mapping between two domains, remove noise and preserve useful structural information as shown in Figure 3. The proposed AFT outperforms AUTOMAP even without implementing non-linear activation functions and CNNs. Comparing AFT with AFT_CUNet, CNNs extract higher features and introduce non-linearity, enhancing overall performance. Comparing DFT_CUNet with AFT_CUNet, the AFT can remove noise artifacts in k-space before feeding into CNNs while keeping the anatomy information needed. Comparing AFT_UNet with AFT_CUNet, the complex-valued network leverages the correlation between the real and imaginary parts and is superior to the real-valued counterpart. The real-valued network performs worse in terms of PSNR and SSIM due to failed elimination of some abnormal values in the image domain.

3.3 Evaluation on the datasets with artifacts
The trained networks are evaluated on the T1w MRI with additional Gaussian noise and T2w MTI with GBCAs. The networks trained on the real-world dataset are used as pre-trained models and retrained on the T1w MRI. The results are shown in Figure 4. Our AFT-Net can keep the structure of vessels while removing noise. The AFT-Net is also more robust to the contrast difference presented in the image domain and shows an overall improvement compared with AUTOMAP. AFT-Net can be easily applied to medical image reconstruction and denoising with other modalities using transfer learning.

3.4 Reconstruction plus accelerated imaging
We also verify that reconstruction can be combined with the accelerated imaging task which can not be easily achieved in i-space. K-space data from the real-world T2w MRI dataset are undersampled in the phase-encoding direction by 2 times³ and the results are shown in Figure 5. Our AFT-Net achieves significant improvement and increases the input PSNR by 10dB, which indicates the generality and wide potential application of AFT-Net.

Discussion and Conclusion

Figure 2, 3, 4, and 5 demonstrate that AFT-Net achieves competitive results compared with other methods and proofs to be more robust to additional noise and contrast differences. An extensive study on the transfer learning on the dataset with different contrast caused by different scan modalities demonstrates the strength and generality of our approach.

One remaining limitation is that we only implement a complex-valued network with linear layers and CNNs, which are less effective than some advanced architectures. In our future work, we will aim to replace multi-layer perceptron and CNNs with transformer-based and diffusion-based models while extending the concept of AFT to more medical imaging tasks from 2D imaging processing to 1D spectroscopy analysis.

Acknowledgements

This study was performed at the Zuckerman Mind Brain Behavior Institute at Columbia University and Columbia MR Research Center site.