Introduction

Achieving precise radiation dose coverage for abdominal tumors is challenging due to respiratory motion.¹ Recently, four-dimensional magnetic resonance imaging (4D-MRI) has emerged as a potential solution for motion management, offering excellent soft-tissue contrast without ionizing radiation.² However, 4D-MRI currently is heavily undersampled, resulting in significantly compromised image quality. This limitation poses challenges in accurately visualizing complex anatomical structures and performing precise motion estimation. Traditional enhancement methods face difficulties in recovering missing or corrupted detailed information because of the inherent challenges posed by the inverse problem.³ Furthermore, these methods often introduce undesirable structural distortions, exacerbating the issue. Given that 4D-MRI acquisitions encompass multiple respiratory cycles and different frames contain shared anatomical structures and complementary information, we propose a time-compensated approach called Tco-SEN. Furthermore, we employ a two-stage network design to maximize the preservation of structures.

Methods

The abdominal MRI data utilized in this study were collected from 26 patients with liver tumors who were undergoing radiotherapy. The imaging was performed using a TWIST-VIBE sequence with a scanning time of 0.69 seconds per volume. The ground truth data, which served as a reference, was obtained from breath-hold high-quality (HQ) 3D-MRI scans acquired using the same imaging sequence but with an acquisition time of 11 seconds. Figure 1 presents a schematic illustration of the problem definition and an overview of the proposed method Tco-SEN. We first sort all frames into 8 respiratory phases based on the amplitude of the estimated surrogate signal. Unlike traditional methods that select only one representative frame from each bin, the Tco-SEN method utilizes multiple frames within the same bin to fully exploit the spatial-temporal information. The objective function can be described as follows.
$$\hat{\theta} = \underset{F}{\mathrm{argmin}} \ \mathcal{L}\left(F_\theta\left(I^i,I^j, I^k\right), I^{HQ}\right)$$


where $$$F_\theta: \left(I^i, I^j, I^k\right)\rightarrow I^{HQ}$$$ can be regarded as an image restoration model with learnable parameters $$$\hat{\theta}$$$, and $$$\mathcal{L}$$$ denotes the distance between 4D-MRI frames $$$(I^i, I^j, I^k)$$$ of the same bin and the ground truth MRI $$$I^{HQ}$$$.
To enhance the recovery of lost high-frequency information, such as edges, texture details, and blood vessels, while preserving the anatomical structure through deep learning models, we have devised a two-stage restoration process. A detailed demonstration of the Tco-SEN is presented in Figure 2. In the first stage, we restore HQ edge maps by utilizing multiple frames of the same bin to recover high-frequency information. Since slight anatomical differences may exist among frames due to respiratory movements, even though the input 2D images are on the same plane along the z-direction, we incorporate the backbone of the Enhanced Deformable Convolutional Networks (EDVR)⁴ framework, utilizing the deformable convolutional layers for an implicit registration. In the second stage, the generated HQ edge map is used as additional prior information to further improve the image quality of LQ 4D-MRI.

As for the loss function, in addition to L1 loss and MS-SSIM, here we also propose an edge loss to preserve structure. We formulate the edge loss by diminishing the mean absolute error between the edge map extracted from the enhanced 4D-MR image and the corresponding HQ MR image using the Sobel operator. The loss function of the reconstruction network is defined as follows:
$$\mathcal{L}\left(\mathbf{R}\right) = \lambda_1\mathcal{L}_{L1}\left(\mathbf{R}\right) + \lambda_2\mathcal{L}_{SSIM}\left(\mathbf{R}\right)+\lambda_3\mathcal{L}_{L3}\left(\mathbf{R}\right)= \lambda_1\mathbb{E}_{\left(I^{en},I^{HQ}\right)}\left[\Vert I^{HQ}-I^{en}\Vert\right] + \lambda_2\left[1-\text{SSIM}\left(I^{HQ}, I^{en}\right)\right]+\lambda_3\mathbb{E}_\left(I^{en},I^{HQ}\right)\left[\Vert S(I^{HQ})-S(I^{en})\right\Vert])$$.

Results

To evaluate the proposed method, we compared it with three state-of-the-art methods: EDSR⁵, EDVR, and SRCNN⁶. Figure 3 presents a qualitative comparison of an example patient. Furthermore, we employed five evaluation metrics to quantitatively measure the recovery accuracy as illustrated in Figure 4. It is evident that Tco-SEN significantly enhances the image quality of 4D-MRI, exhibiting improved visibility of organ shapes with fewer artifacts and less noise. To evaluate the potential benefits of Tco-SEN in radiotherapy, we conducted motion modeling experiments on various image types. Due to the absence of ground truth deformation vector fields (DVFs), we assessed the registration accuracy by comparing images. DVFs were calculated using the Demons between the LQ 4D-MRI, the Tco-SEN improved image, and the HQ 3D-MRI pairs. These predicted DVFs were then applied to the HQ 3D-MRI as the moving image, and the resulting deformed images were compared to a fixed image. Figure 5 demonstrates that Tco-SEN plays a favorable role in precise motion modeling of 4D-MRI, both qualitatively and quantitatively.

Conclusion

We have presented Tco-SEN, a temporal-compensation method for enhancing 4D-MR images. The Tco-SEN decomposes the 4D-MRI reconstruction problem into two stages and each stage focuses on one mission, which reduces the learning difficulty of the entire network. The results highlight the effectiveness of our approach in improving image quality and subsequent motion modeling.

Acknowledgements

This research was partly supported by research grants from NIH R01 CA226899, NSFC Young Scientists Fund 82202941, General Research Fund (GRF 15104822, GRF 15102219), the University Grants Committee, and the Health and Medical Research Fund (HMRF 10211606), the Health Bureau, The Government of the Hong Kong Special Administrative Regions.