Synopsis

Keywords: Image Reconstruction, Machine Learning/Artificial Intelligence

For accelerated MR image reconstruction, machine learning (ML)-based methods outperform traditional sparsity-based methods by exploiting large datasets to learn effective priors. However, most ML methods output only a single image reconstruction when in fact there may be many plausible reconstructions given the measurement and prior. To extract this diagnostically relevant information, we propose to explore the space of plausible images, i.e., to sample the posterior, using ML. Among ML methods, conditional normalizing flows (CNFs) stand out for rapid sample generation and simple likelihood-based training. In this work, we present the first CNF for posterior sample generation in accelerated multicoil MRI.

Introduction

Accelerated magnetic resonance imaging (MRI) seeks to reduce acquisition time while preserving diagnostically relevant information¹. As an ill-posed inverse problem though, many possible reconstructions are consistent with the measurements and prior. Considering the consequential nature of MRI, generating many posterior samples could be very beneficial. Among methods for posterior sample generation, conditional normalizing flows (CNFs)² stand out for offering rapid inference with simple likelihood-based training and outperforming other methods in a recent super-resolution contest³. Previous work on CNFs for MRI by Denker et al.⁴ demonstrated promising initial results but was limited to single-coil magnitude measurements. We develop a CNF that works with complex multicoil MRI measurements and leverages recent CNF advances to demonstrate excellent performance across multiple performance metrics.

Methods

Data
We use non-fat-suppressed fastMRI⁵ knee data, giving 17 286 training and 3 592 validation images. For preprocessing, each measurement stack is compressed to c=2 complex-valued virtual coils⁶ and cropped to 320 × 320 pixels. Sampling uses a fixed golden ratio offset (GRO)⁷ Cartesian mask with acceleration 4 and a 13 pixel wide autocalibration signal (ACS) region. Coil-sensitivity maps are estimated using ESPIRiT⁸. All inputs are normalized by the 95th percentile of the zero-filled (ZF) magnitude image before being used by the networks.
Architecture
Our CNF consists of a conditional invertible neural network (cINN) G and a conditioning network H. H accepts measurement information and converts it into a form that is usable by G. Given the conditioning information, G maps Gaussian random vectors to posterior samples.
For G, we use multi-scale RealNVP⁹ with l = 3 downsampling layers, m = 15 flow steps per layer, and a transition step at the beginning of each layer to ease the dimensionality change between layers¹⁰. For H, we use the UNet from Zbontar et al.⁵ (with 128 output channels in the first convolution block and 4 pooling layers) and configure it to accept a ZF c-coil image and generate feature maps that are concatenated with the inputs to G’s coupling layers.
We train G to map the non-measured-subspace projections of the c-coil complex ground-truth images to Gaussian random vectors. This allows the cINN to focus on learning the unknown data. During inference, our cINN maps Gaussian random vectors to these “inverse-zero-filled” images, which are then combined with the measured information to yield the final (data-consistent) predictions. Our data-consistency method is reminiscent of “frequency separation” from Song et al.¹¹ but operates in the complex domain. Our architecture is illustrated in Figure 1.
Baseline
We compare to the single-coil CNF from Denker et al.⁴ To convert our multicoil measurements to a format usable by that method, we coil-combine the ZF multicoil images using the estimated coil maps. We use a similar technique to create single-coil ground-truth training data.
Training
We first train the UNet for 50 epochs to recover the multicoil ground-truth. In particular, we train the UNet to minimize squared-error in the non-measured subspace and replace the measured subspace with the measured data. Next, the UNet and cINN are jointly trained for 100 epochs to minimize the standard negative log-likelihood loss² with batch size 16. We train Denker et al.⁴ for 300 epochs.
Metrics
All images are unnormalized and converted to the magnitude domain before metrics are calculated. When computing Fréchet Inception Score (FID)¹² and conditional FID (cFID)¹³, we use VGG-16 embeddings¹⁴ and 8 posterior samples per validation image. When computing normalized mean-squared error (NMSE), peak-signal-to-noise ratio (PSNR), and structural-similarity index (SSIM), we use mean images computed from p = 50 posterior samples.

Results/Discussion

Figure 2 shows average performance over the validation set and Figures 3 and 4 show typical image recoveries. Relative to the Denker’s single-coil CNF⁴, our multicoil CNF shows significant improvements in all metrics and fewer reconstruction artifacts in the generated images. The posterior standard-deviation maps in Figure 4 provide insight into where recovery uncertainty may exist. Figure 5 shows that both CNFs achieve the expected amount of PSNR gain¹⁵ as a function of the number of averaged samples, p, which suggests that both methods achieve the proper tradeoff between accuracy and diversity. Future studies will focus on optimizing the architectures of G and H and validating CNF in different MRI applications.

Conclusion

We propose the first CNF for posterior sampling in multicoil accelerated MRI. By leveraging recent advances in CNF, we demonstrate improvement over the previous single-coil, magnitude image applications. The generated posterior samples and standard-deviation maps, unavailable with point-estimate methods, provide additional diagnostic and trustworthiness resources for clinicians.

Acknowledgements

This work was funded by NIH project R01EB029957.