0659

Neural Implicit Quantitative Imaging

Felix Zimmermann¹, Simone Hufnagel¹, Patrick Schuenke¹, Andreas Kofler¹, and Christoph Kolbitsch¹
¹Physikalisch-Technische Bundesanstalt (PTB), Braunschweig and Berlin, Germany

Synopsis

Keywords: Analysis/Processing, Machine Learning/Artificial Intelligence

Motivation: 3D quantitative MRI presents a challenging inverse problem. The application of learned reconstruction methods is hindered by the need for extensive training data and the large size of high-resolution voxel representations of multi-dimensional data. Implicit neural fields have shown promise in cine imaging and slice-to-volume registration.

Goal(s): Explore the use of neural fields for representing 3D high-resolution quantitative parameters in qMRI.

Approach: We integrate motion correction, sensitivity map estimation, and 3D parameter neural fields into an end-to-end, scan-specific optimization without training data.

Results: Demonstration of feasibility in the context of cardiac qMRI and initial results of whole-heart 3D T1 maps.

Impact: Introduction of implicit neural fields into qMRI, allowing for continuous representation of the quantitative parameters in 3D space. Our novel end-to-end reconstruction with motion correction, sensitivity map estimation provides fast high-resolution, whole-heart T1-maps without relying on training data.

Introduction

Quantitative mapping could play a crucial role in diagnostic imaging, but is limited by the trade-off between scan time and signal-to-noise-ratio, as well as the complexity of the reconstruction problem^1-3. Despite the general success of learned regularization for MR image reconstruction^4,5, the application to high-resolution, 3D quantitative imaging is hindered by the limited availability of training data and the substantial computational and storage demands associated with 3D voxel representations. Consequently, commonly only stacks of 2D maps with reduced through-plane resolution are considered^5,6.

Neural implicit fields offer an innovative means of representing multidimensional data by directly mapping input coordinates to various quantities⁷, leveraging familiar neural network components. Notably, neural fields can be used for subject-specific optimization without the need for additional training data, benefiting from the architecture's efficiency, inherent regularization properties, and stochastic optimization. While neural fields have exhibited effectiveness in diverse MR applications^8-14, they have not been used for quantitative MRI.

Here, we propose an end-to-end approach encompassing motion-corrected reconstruction, coil-sensitivity map estimation, slice-to-volume super-resolution, and quantitative mapping in 3D. We perform initial experiments of this novel technique for whole-heart T₁ mapping.

Methods

We recast the well-known quantitative imaging inverse problem³ of obtaining voxel-wise parameter maps from acquired k-space data $$$\mathbf{k}$$$ to
$$\min_{\phi,\psi,\gamma} \|\mathbf{k} - \mathbf{F} \mathbf{C}_{\gamma(\mathbf{r})} \mathbf{S} q(\psi(\mathbf{r}’))\|, \quad \mathbf{r}'=\phi(\mathbf{r})$$ with signal model $$$q$$$, $$$\mathbf{S}$$$ an integration over the slice profile, sensitivity map dependent coil-expand $$$\mathbf{C}$$$, and Fourier transform $$$\mathbf{F}$$$. Here, we introduce a continuous parameter field $$$\psi$$$ mapping from 3D-coordinates $$$\mathbf{r}$$$ to quantitative parameters, a motion transform $$$\phi$$$ and coil-sensitivity field $$$\gamma$$$. We model $$$\phi, \psi, \gamma$$$ by neural fields.

Hence, our proposed architecture (Fig.1) consists of 1) a neural motion field. 2) a coil sensitivity field. 3) the main quantitative parameter field, mapping coordinates to the parameters of interest (e.g. T₁). 4) a forward model, applying the signal model (here: cont. acquisition inversion recovery) and obtaining for each mini-batch the estimated k-space signal by applying the sensitivity maps and the radial Fourier transform. We employ the Fourier slice theorem to capitalize on the ability to sample our model in arbitrary orientations¹³, thus avoiding a NUFFT. The forward model includes the slice profile via Monte-Carlo integration¹⁰.

Inspired by prior works^13,15, the motion field comprises a slice-conditioned affine transform, random Fourier embeddings¹⁶ in space (256 features) and time (64 features), and a 2 hidden layer MLP (256 features). The coil-sensitivity module as well as the parameter module use Hashgrid¹⁷ embeddings. For the sensitivity-maps, we limit the capacity by using only 3 resolutions in the embedding and a single hidden layer, while the $$$\psi$$$ uses 18 resolutions (scaling factor 1.3, hashlength 2²²) and 3 hidden layers (256 features). We use the output activation of the parameter field to avoid impossible parameter values, i.e., we enforce positivity of T₁ by a Softplus. The trainable parameters of the neural fields are optimized end-to-end, minimizing (1) using AdamW¹⁸ with cycled learning-rate, weight decay=10^-3, beta₁=0.99, and mini-batches of 8 spokes. Total runtime was 30min (Nvidia A6000). We promote smooth sensitivity maps by including Gaussian jitter to the respective input coordinates and volume preservation in the motion module by a Jacobian determinant penalty.

Data was collected from a healthy volunteer with a 3T Siemens Verio with 32-channel cardiac coil (compressed to 8 channels). Three stacks, consisting of five slices each, were acquired in four LAX orientations¹⁹. After a slice-selective inversion pulse, data was continuously acquired using a golden angle radial scheme with spatial resolution 1.3×1.3×4.0mm³. Total scan time 3min in 12 breath-holds. For comparison, a 3(3)3(3)5-MOLLI was acquired in 4ch and SAX orientation.

Results

Initial results demonstrate the ability to reconstruct quantitative 3D T₁ maps from multislice 2D acquisitions (Fig.2). These maps can be visualized in explicitly acquired orientations (4ch-view) and non-acquired orientations (SAX) by evaluation of $\psi$ (Fig.3), illustrating successful 3D representation (Fig.4). Additionally, Figure 5 illustrates the effect of the cardiac motion field on a single slice.

Discussion

Modeling the parameter maps using neural fields shows great potential in qMRI. Our current approach already replaces hand-crafted^19,20 multi-step pipelines with a single model-based end-to-end optimization while not relying on any training data. Even without any regularization on the parameter maps, we obtain in our initial results maps in orientations not explicitly acquired in the challenging problem of full-heart T₁ mapping with 100% scan efficiency. By simple modification of the forward model, the approach is transferable to other qMRI sequences. The possibility of evaluating the fields in arbitrary orientations and resolutions allows for many regularisation approaches, which are known to improve fidelity¹⁹, and thus will be further investigated.

Acknowledgements

Supported by the German Research Foundation (GRK2260, BIOQIC).
Supported by the Metrology for Artificial Intelligence in Medicine (M4AIM) project, which is funded by the German Federal Ministry for Economic Affairs and Climate Action (BMWi) as part of the QIDigital initiative.
The project received funding from the European Partnership on Metrology, co-financed from the European Union’s Horizon Europe Research and Innovation Programme, and by the Participating States.

References

1 Haaf P,et al. "Cardiac T1 Mapping and Extracellular Volume (ECV) in clinical practice: a comprehensive review". J Cardiovasc Magn Reson. 2016

2 Schelbert, EB, et al. "State of the Art: Clinical Applications of Cardiac T1 Mapping". Radiology. 2016

3 Hufnagel, Simone, et al. "3D model-based super-resolution motion-corrected cardiac T1 mapping." Physics in Medicine & Biology 2022

4 Sriram, Anuroop, et al. "End-to-end variational networks for accelerated MRI reconstruction." MICCAI 2020

5 Hammernik, K et al. "Physics-Driven Deep Learning for Computational Magnetic Resonance Imaging: Combining physics and machine learning for improved medical imaging." IEEE Signal Processing Magazine 2023

6 Zimmermann, F., et al. "PINQI: An End-to-End Physics-Informed Approach to Learned Quantitative MRI Reconstruction." arXiv:2306.11023 (2023)

7 Mildenhall B, et al. "Nerf Representing scenes as neural radiance fields for view synthesis". Communications of the ACM 2021

8 Wu Q, et al. "IREM: High-resolution magnetic resonance image reconstruction via implicit neural representation". International Conference on Medical Image Computing and Computer-Assisted Intervention 2021

9 Feng, Jie, et al. "Spatiotemporal implicit neural representation for unsupervised dynamic MRI reconstruction." arXiv:2301.00127 (2022).

10 Xu, Junshen, et al. "NeSVoR: Implicit Neural Representation for Slice-to-Volume Reconstruction in MRI." IEEE Transactions on Medical Imaging (2023)

11 Vogt, N. et al "Implicit CINE: a deep-learning super-resolution model for multi-planar real-time MRI", ISMRM 2023, Abstract 3884

12 Feng, R, et al. "IMJENSE: scan-specific IMplicit representation for Joint coil sENSitivity and image Estimation in parallel MRI", ISMRM 2023, Abstract 0820

13 Catalán, Tabita, et al. "Unsupervised reconstruction of accelerated cardiac cine MRI using Neural Fields." arXiv:2307.14363 (2023)

14 Huang, Wenqi, et al. "Neural Implicit k-Space for Binning-Free Non-Cartesian Cardiac MR Imaging." International Conference on Information Processing in Medical Imaging 2023.

15 Kunz, J et al. "Implicit Neural Networks with Fourier-Feature Inputs for Free-breathing Cardiac MRI Reconstruction." arXiv:2305.06822 (2023).

16 Tancik, Matthew, et al. "Fourier features let networks learn high frequency functions in low dimensional domains". Advances in Neural Information Processing Systems 2020

17 Müller, Thomas, et al. "Instant neural graphics primitives with a multiresolution hash encoding." ACM Transactions on Graphics (2022)

18 Loshchilov, Ilya, and Frank Hutter. "Decoupled weight decay regularization." arXiv:1711.05101 (2017)

19 Hufnagel, S. et al "Towards isotropic 3D whole-heart T1 mapping using model-based motion-corrected super-resolution reconstruction", ISMRM 2023, Abstract 1093

20 Bano, W, et al. "Model‐based super‐resolution reconstruction of T2 maps." Magn Reson Med. 2020

Figures

Schematic overview. We model the 3D parameter fields (red) by a coordinate-based network. We include a field for motion and for the CSMs. Our forward model consists of the signal function $$$q$$$, the integral over the slice profile and a radial Fourier transform (implemented via Radon Transform). All weights are optimized per subject, minimizing the discrepancy between predicted k-data and acquired k-data.

Obtaining 2D views arbitrary orientations. After the scan-specific optimization, the neural parameter field $$$\psi$$$ can be evaluated at 3D positions. Thus parameter maps in all orientations, even those not included explicitly acquired, can be rendered by simple evaluation.

Application Example. We apply the proposed framework to the task of 3D motion-corrected cardiac T₁ mapping. From k-data of thick slices recorded in a few orientations and positions, we aim to reconstruct full 3D T₁ maps. Note also the sparsity in inversion time/cardiac phase, as for each slice only a single inversion recovery is acquired.

Resulting T₁ Maps rendered in 4-chamber view (acquired) and SAX (orientation not explicitly acquired but purely reconstructed out of the different aquired views) and 2D MOLLI reference, explicitly acquired in both orientations (right). Note that the 3D reconstruction is performed without any explicit regularization on the T₁ maps, although different ones could be included to further enhance the results.

Effect of the coordinate deformation/motion correction block. By applying the deformation to the sampling grid before mapping the coordinates to T₁ values (center), views corresponding to different cardiac phases can be created (right). Note, that the neural motion field only corrects motion based on consistency of the predicted signal intensity with the measured data. Thus, it does not affect time points or locations with negligible signal due to the inversion model.

Proc. Intl. Soc. Mag. Reson. Med. 32 (2024)

0659

DOI: https://doi.org/10.58530/2024/0659