Introduction

Deep learning (DL) algorithms have previously been proposed [ref] that use complementary contrast information from multi-contrast MRI images such as T2 and FLAIR. These models are capable of reconstructing T1Gd (post-contrast) images from pre-contrast images and/or low-dose contrast images. Though the reconstruction is clinically equivalent in regions of pathology, there is a lack of explainability in terms of mapping the source of contrast signals that the network has used in order to produce contrast-enhancement in a certain region. In a real-world clinical scenario, explainable models are desired in order to justify the prognosis. In this work, we explore the feasibility of an explainable multi-contrast deep learning model with anomaly-aware attention mechanism for reducing gadolinium dose in contrast-enhanced brain MRI. The explainability is achieved by means of a branched UNet model where each branch processes different input contrasts separately and the corresponding branches are fused in order to produce the final synthesis.

Methods

Protocol
Fig1 depicts the clinical protocol where T2 and FLAIR images are acquired first followed by a T1-weighted pre-contrast image. Followed by this a low-dose T1-w scan is acquired with 10% of standard dose (0.01 mmol/kg). Finally the remaining 90% dose is injected to acquire a full-dose T1-w scan. The T1pre, T1low, T2 and FLAIR are used as model input to predict the T1-w post-contrast scan.

Anomaly-aware attention mechanism
Fig2 shows the unsupervised anomaly detection (UAD) scheme using FLAIR images. This scheme is similar to the one proposed in [1], but uses FLAIR instead of T2 images. FLAIR images are better in generating the UAD masks as they have better in-plane resolution and well defined hyper-intense signals in the regions of anomalies.

Dataset
With IRB approval and informed consent, an in-house dataset of 185 patient scans were obtained using the above mentioned protocol. 150 cases were used for model training and 35 cases were used for validation and testing.

DL network architecture
Fig3 shows the branched DL network architecture. The network has three branches with T1+T1low, T1+T2 and T1+FLAIR as respective inputs. Each branch is processed by a UNet-like encoder-decoder structure to extract features and produce “synthesized pseudo T1Gd images”. These are pseudo-T1Gd images because they do not have the full-contrast enhancement, but contain the enhancement derived from the respective inputs in the branch. The outputs from the respective branches are concatenated into channels and passed through a convolution layer to produce the “boosted” T1Gd image. This image is again combined with the T1pre contrast and passed through another UNet encoder-decoder to produce the final synthesized image. The individual branches are trained separately with the T1Gd ground truth as the target image. The pre-trained networks’ weights are then transferred to the individual branches before beginning to train the end-to-end network. Moreover, having separate encoding pathways for the individual contrasts, instead of squashing the multiple modalities as channels, performed better, as the separate encoders were able to learn the unique features offered by the different contrasts.

Loss function
The model was trained with a combination of L1, SSIM and perceptual (VGG) losses. The L1-loss was weighted with the UAD mask to make the model pay more attention to the anomalous regions. Hence, the anomaly-aware attention mechanism was achieved through adding the UAD masks as part of the input and also weighting the L1-loss. The loss function between ground-truth full dose $$$I_{T1CE}$$$ and DL synthesized full-dose $$$I*_{T1CE}$$$ is
$$\mathcal{L}(I_{T1CE}, I*_{T1CE}) = \lambda_{L1}L_{L1}(M_{UAD} . I_{FLAIR}) + \lambda_{SSIM}L_{SSIM} + \lambda_{VGG}L_{VGG} $$
where $$$\lambda$$$s are the respective loss weights and $$$M_{UAD}$$$ is the UAD mask computed from the FLAIR image. The lambda values are empirically chosen from the validation set.

Results

Fig4 depicts the explainable nature of the model. For the example shown, the T1+T1low branch does not produce any enhancement, whereas the T1+T2 branch produces a mild enhancement. The T1+FLAIR branch seems to have the maximum contribution to the final output. These enhancements are fused by the center convolution layer which is reflected in the boosted image and in turn in the final output. Fig5 shows the qualitative results for a few cases alongside the T1Gd ground truth.

Discussions

Based on the qualitative results we have shown the feasibility of developing an explainable model that can work on multi-contrast images for reducing gadolinium dosage in brain MRI. The final result of the model can be traced back to the individual input branches and the contrast synthesis can be mapped to the different multi-modal input images. Since different pathologies enhance differently in T2 and FLAIR images, these kinds of models can be used to explain the respective contributions of the input contrasts.

Conclusion

In this work we have explored the feasibility of developing an explainable multi-contrast DL model for reduced gadolinium dosage in contrast-enhanced brain MRI.

Acknowledgements

We would like to acknowledge the grant support of NIH R44EB027560.

References

1. Pasumarthi, Srivathsa Gong, Enhao Zaharchuk, Greg Zhang, Tao. "Anomaly-aware multi-contrast deep learning model for reduced gadolinium dose in contrast-enhanced brain MRI - a feasibility study". Proceedings of ISMRM 2021