Introduction

MRI acquisition time can be reduced by sampling fewer points in k-space, which is typically achieved via under-sampling or reductions in matrix size. However, both approaches can lead to lower image quality, via reductions in signal-to-noise ratio and/or reduced spatial resolution. This trade-off is amplified in the growing fields of low-field MRI¹ (where low signal-to-noise per unit time causes prolonged scan times) and MRI-guided radiation therapy² (where real-time, low latency, cine-MRI is required for treatment adaptation).
Deep learning has been used in MRI image enhancement, including super-resolution – where low spatial resolution images are up-sampled to a high spatial resolution. Here, we present the transformer residual cross (T-REX) network, a unified transformer-based neural network that applies super-resolution to up-sample a low spatial resolution thick slice MRI to a high spatial resolution thin multi-slice MRI.

Methods

T-REX comprises of a convolutionally-based encoding arm for feature extraction and compression to encode a high dimensionality to a low dimensionality. Once low dimensionality is attained, this representation is passed through a multi-layer transformer encoder. The output of the transformer encoder is then provided to a convolutionally-based decoding arm for further feature extraction and up-sampling to the desired dimensionality. Residual blocks, between encoding and decoding arms, comprising of convolutions and skip connections extract and retain high frequency features. A basic overview of T-REX is provided in Figure 1.
UPENN-GBM³ from the Cancer Imaging Archive⁴ was used to train and evaluate T-REX in this study. 40 unstripped volumetric brain MRIs (corresponding to 40 patients) acquired at 3.0 T were used as training data. Two test datasets (10 patients each) were also derived for 3.0 T and 1.5 T to assess the generalisability of the model to lower field strengths. The normalised (min-max) root mean-square-error (NRMSE) was computed between each reconstructed patient volume and the reference image. This workflow was conducted twice for T1w and T2w datasets (i.e., a T1w and T2w T-REX models were trained and evaluated on 3.0 T and 1.5 T data).
Reference high spatial resolution volumetric brain MRIs were processed to produce synthetic inputs for T-REX. In-plane (axial) spatial resolution was decreased through central cropping of k-space. Additive Gaussian noise was added to these low spatial resolution slices (only during model training) to simulate low-field acquisitions. The slice thickness was increased by taking an average down-sampled slice for a neighbourhood of slices. The in-plane up-sampling factor was 4× while the slick thickness decrease factor was 5×. For example, a 4×4×5 mm³ voxel was up-sampled to an isotropic 1 mm³. During model training, the thick slices were overlapped to increase the training data. This pipeline is shown in Figure 2. For model testing, each thick slice generated 5 thin slices that were stacked to encompass the total field-of-view of the reference MRI for a direct volumetric comparison.

Results

T-REX was successfully trained on T1w and T2w images creating both a T1w and T2w model. Qualitatively, T-REX recovered high frequency features as shown in Figure 3 and 4. Additionally, T-REX generalised well to lower (1.5 T testing data vs 3.0 T training data) field strengths (Figure 4). The NRMSE values across the 10 testing patients for both sequences and field strengths are presented in figure 5.

Discussion

T-REX was successful at the two distinct simultaneous tasks of decreasing slice thickness and increasing in-plane spatial resolution in a unified neural network. Although trained on 3.0 T MRI, it could generalise well to 1.5 T (for a variety of MRI scanners) indicating transferability to low-field MRI and MRI-guided radiation therapy. Further work will be conducted to improve model performance, especially in recovering thin slices from thick slices for 1.5 T as noticeable artifacts were still present using T-REX (Figure 4 coronal view). Additionally, further work to experiment with T-REX to different anatomical sites will be undertaken, especially sites with high motion (e.g., liver) for the downstream use of MRI-guided radiation therapy.

Conclusion

We present T-REX, a transformer residual cross network for volumetric super-resolution. We trained T-REX on 3.0 T and evaluated on 1.5 T and 3.0 T. Error metrics remained low and stable even when evaluating on MRI data acquired at half the field strength the model was trained on. These promising initial results show that T-REX can be applied to low-field MRI and MRI-guided radiation therapy.

Acknowledgements

This work is supported by an Australian Government Research Training Program (RTP) Scholarship and a NHMRC Investigator Grant.