0752
Learning non-rigid registration in k-space from highly-accelerated cardiac and respiratory MR data1Medical Image And Data Analysis (MIDAS.lab), Department of Diagnostic and Interventional Radiology, University Hospital of Tuebingen, Tuebingen, Germany, 2School of Computation, Information and Technology, Technical University of Munich, Munich, Germany, 3Klinikum Rechts der Isar, Technical University of Munich, Munich, Germany, 4Department of Computing, Imperial College London, London, United Kingdom, 5Department of Radiology, Stanford University, Stanford, CA, United States
Synopsis
Keywords: Motion Correction, Motion Correction, Image registration, motion estimation, Cardiovascular, Lung, MR-Guided Radiotherapy, motion-compensated reconstruction, Multimodal motion correction
Motivation: Time-resolved motion estimation from accelerated MR data enables high-quality imaging, intra-modality motion correction and real-time tracking during MR-guided radiotherapy. Conventionally, image registration is solved in the image domain and, therefore, remains susceptible to aliasing artifacts for highly-accelerated acquisitions.
Goal(s): We aim to propose a robust non-rigid image registration framework from highly-accelerated data without additional information.
Approach: We introduce a novel Local-All-Pass Attention Network (LAPANet) that performs accurate motion estimation directly from the acquired k-space.
Results: LAPANet provides reliable estimates for fully-sampled and undersampled data, up to 104-fold for cardiac motion and 148-fold for respiratory motion, and outperforms established image-based registrations in different trajectories.
Impact: Our framework can reliably estimate non-rigid motion from highly-accelerated data without a-priori information. This enables faster acquisition through integration into motion-compensated reconstructions, intra-modality motion correction for other imaging methods and real-time motion characterization and tracking for guided radiotherapy and interventions.
Introduction
Real-time non-rigid image registration from MR data is valuable for faster motion-compensated reconstruction1–3, intra-modality motion correction4–6 and MR-guided radiotherapy and interventions7–9. Nevertheless, accommodating a high frame rate motion estimation demands the processing of highly-undersampled data. In this case, image-based registration methods10–15 prove inadequate in providing precise motion characterization and localization due to residual aliasing artifacts, even for well-equipped reconstruction methods. Alternatively, image registration can be solved in k-space. However, existing techniques adopt a patch-based paradigm, which imposes inherent constraints on the accessibility to contextual information16,17, or rely on additional priors7,18. Here, we propose a novel framework, called the Local-All Pass Attention-Network (LAPANet), to perform non-rigid image registration directly from the full-sized acquired k-space. We investigate the application of LAPANet for cardiac and respiratory motion for the fully-sampled and for highly-accelerated data, acquired with Cartesian and radial trajectories. For all cases, we demonstrate the superior performance to image-based approaches for highly-accelerated acquisitions.Methods
Architecture: Based on the Local-All Pass (LAP) algorithm12, non-rigid deformation in k-space can be decomposed into rigid displacements that span local windows represented as all-pass filter operations. To approximate these filters, we used a four-level multi-resolution deep-learning network, illustrated in Fig.1. We stack the full-sized real and imaginary components of the 2D pairs of coil-weighted fixed and moving k-spaces to form the network’s input. The Global Residual Modules inject the encoding levels with a multi-scale k-space pyramid. At each level, the stacked k-spaces are delineated into local windows emulating phase-modulated tapering functions16, by halving the processing window size. This module integrates local features with high-level hierarchical information using self-attention and then recalibrates dynamically the leveraged coil information using the Attention-weighted Squeeze and Excitation Block. Each encoding and decoding level incorporates sequentially the Channel Integration Module and the Dilated Fusion Module. The interplay of strided convolutions and multi-head self-attention blocks in both modules allow for synergistically improved spatial context interpretation. Queries, keys, and values are acquired through depthwise convolutions19 to help preserve spatial context without positional encoding and leverage valuable coil information stored in the channels. The adopted coarse-to-fine decoding is supported by the Motion Attention Modules, which refine the current level motion estimation by integrating features from preceding levels using pixelwise weights for the horizontal (X) and vertical (Y ) motion components.Datasets: We used 134 short-axis 2D cine scans (38 healthy subjects and 98 patients), acquired in-house on a 1.5T MRI with 2D bSSFP (TE/TR=1.06/2.12ms, resolution=1.9×1.9mm2, slice thickness=8mm, Nt=25 temporal phases, 8 breath-holds of 15s duration) and a 3D respiratory dataset5 (24 healthy subjects and 36 patients), acquired on a 3T PET/MR with a 3D T1 weighted spoiled gradient echo sequence (TE/TR=1.23/2.60ms, resolution=1.9×1.9×1.9mm3, Nt=6 temporal phases, field-of-view=500×500×360mm3).
Training and Experiments: The data was divided into distinct groups of training/testing subjects, resulting in 631250/107812 and 48672/7614 image pairs per acceleration factor to study the cardiac and respiratory motion. Training is performed on changing accelerations 2x-104x (cardiac) and 2x-148x (respiratory) and using varying undersampling strategies (VISTA20, 2D golden radial21 and 3D variable-density Poisson-Disc/vdPD). The loss function includes the photometric loss (Lphoto), the spatial smoothness loss22 (Lsm) and the translational photometric loss (LTphoto) to encourage the encoder to learn improved features:
$$\mathcal{L}=\mathcal{L}_{photo}+\alpha \mathcal{L}_{sm}+\beta \mathcal{L}_{Tphoto}$$
LAPANet was trained on an NVIDIA V100 GPU using an AdamW optimizer23 (batch size=32, learning rate=1e−4, weight decay=1e−3, α=0.01, β=0.2) with a cosine annealing schedule24. LAPANet was compared to image-based deep learning registrations: VoxelMorph13 and GMA-RAFT25; and conventional registrations LAP12 and Elastix10. We reported the mean and the standard deviation of the averaged Normalized Root Mean Squared Error and dice scores for the left and right ventricles and the statistical significance of differences between the competing methods and LAPANet determined with a paired t-test.
Results and Discussion
LAPANet demonstrates consistent cardiac motion estimation results for the VISTA undersampling and radial trajectory. Contrarily, image-based registrations prove ineffective for highly-accelerated acquisitions, as shown in Fig.2/3. The superior performance of LAPANet is further substantiated through the quantitative assessment across high accelerations, as indicated in Table 1. Similarly, LAPANet exhibits improved and consistent performance in respiratory motion estimation compared to image-based registrations for high accelerations, as depicted in Fig.4. Overall, image-based methods showcase a concurrent decline in performance with increasing acceleration due to the amplified occurrence of undersampling artifacts, while LAPANet maintains reliable estimations over varying trajectories and motion types.Conclusion
We introduced LAPANet, a generalizable deep-learning framework that performs non-rigid registration in complex-valued k-space for different sampling trajectories. Unlike image-based registrations, our method offers reliable motion estimation at high frame rates from data acquired in a few milliseconds, which can be leveraged for different applications.Acknowledgements
No acknowledgement found.References
1. M. Usman et al., “Motion corrected compressed sensing for free-breathing dynamic cardiac MRI,” MRM, vol. 70, no. 2, pp. 504–516, 2013.
2. A. Bustin et al., “3d whole-heart isotropic sub-millimeter resolution coronary magnetic resonance angiography with non-rigid motion-compensated prost,” JCMR, vol. 22, no. 1, pp. 1–16, 2020.
3. G. Cruz et al., “One-heartbeat cardiac cine imaging via jointly regularized non-rigid motion corrected reconstruction,” in Proc. Int. Soc. Magn. Reson. (ISMRM), p. 0070, 2021.
4. J. Ouyang et al., “Magnetic resonance-based motion correction for positron emission tomography imaging,” in Semin. Nucl. Med., vol. 43, pp. 60–67, Elsevier, 2013.
5. T. Küstner et al., “MR-based respiratory and cardiac motion correction for pet imaging,” Med. Image Anal., vol. 42, pp. 129–144, 2017.
6. P. M. Robson et al., “Correction of respiratory and cardiac motion in cardiac PET/MR using MR-based motion modeling,” Phys. Med. Biol., vol. 63, no. 22, p. 225011, 2018.
7. N. R. Huttinga et al., “MR-MOTUS: model-based non-rigid motion estimation for mr-guided radiotherapy using a reference image and minimal k-space data,” Phys. Med. Biol., vol. 65, no. 1, 2020.
8. M. L. Terpstra et al., “Deep learning-based image reconstruction and motion estimation from undersampled radial k-space for real-time mri-guided radiotherapy,” Phys. Med. Biol., vol. 65, no. 15, 2020.
9. I. Y. Ha et al., “Model-based sparse-to-dense image registration for realtime respiratory motion estimation in image-guided interventions,” IEEE TBME, vol. 66, no. 2, pp. 302–310, 2018.
10. S. Klein et al., “Elastix: a toolbox for intensity-based medical image registration,” IEEE TMI, vol. 29, no. 1, pp. 196–205, 2009.
11. T. Vercauteren et al., “Diffeomorphic demons: Efficient non-parametric image registration,” NeuroImage, vol. 45, no. 1, pp. S61–S72, 2009.
12. C. Gilliam and T. Blu, “Local all-pass filters for optical flow estimation,” ICASSP, pp. 1533–1537, 2015.
13. G. Balakrishnan et al., “Voxelmorph: a learning framework for deformable medical image registration,” IEEE TMI, vol. 38, no. 8, pp. 1788–1800,2019.
14. T. C. Mok and A. Chung, “Fast symmetric diffeomorphic image registration with convolutional neural networks,” in Proc. IEEE/CVF, pp. 4644–4653, 2020.
15. C. Munoz et al., “Self-supervised learning-based diffeomorphic non-rigid motion estimation for fast motion-compensated coronary mr angiography,” Magn. Reson. Imag., vol. 85, pp. 10–18, 2022.
16. T. Küstner et al., “Non-rigid “image” registration in k-space,” in Proc. Int. Soc. Magn. Reson. (ISMRM), 2020.
17. T. Küstner et al., “Lapnet: non-rigid registration derived in k-space for magnetic resonance imaging,” IEEE TMI, vol. 40, no. 12, pp. 3686–3697, 2021.
18. H.-C. Shao et al., “Real-time mri motion estimation through an unsupervised k-space-driven deformable registration network (ks-regnet),” Phys. Med. Biol., vol. 67, no. 13, p. 135012, 2022.
19. H. Wu et al., “Cvt: Introducing convolutions to vision transformers,” in Proc. IEEE/CVF, pp. 22–31,2021.
20. R. Ahmad et al., “Variable density incoherent spatiotemporal acquisition (VISTA) for highly accelerated cardiac MRI,” MRM, vol. 74, no. 5, pp. 1266–1278, 2015.
21. S. Winkelmann et al., “An optimal radial profile order based on the golden ratio for time-resolved MRI,” IEEE TMI, vol. 26, no. 1, pp. 68–76, 2006.
22. T. Brox et al., “High accuracy optical flow estimation based on a theory for warping,” in European conference on computer vision, pp. 25–36, Springer, 2004.
23 I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101, 2017.
24 I. Loshchilov and F. Hutter, “Sgdr: Stochastic gradient descent with warm restarts,” arXiv preprint arXiv:1608.03983, 2016.
25 A. Ghoul et al., “Attention-guided network for image registration of accelerated cardiac cine,” ISMRM, 2023.
26 S. Baker et al., “A database and evaluation methodology for optical flow,” IJCV, vol. 92, pp. 1–31, 2011.8
Figures
