2244
Synthesising ultra-strong gradients diffusion MRI with high-resolution convolutional neural networks1Italian National Institute of Health, Rome, Italy, 2Cardiff University Brain Research Imaging Centre, Cardiff University, Cardiff, United Kingdom
Synopsis
Keywords: Other AI/ML, Machine Learning/Artificial Intelligence, Synthesis
Motivation: Ultra-strong gradients scanners allow to explore microstructure, but these systems are not widespread because of their associated challenges.
Goal(s): Our goal is to leverage image synthesis and deep learning to design a neural network able to predict high b-values data from low b-values.
Approach: .We implemented a U-net architecture and tailored a loss function able to learn tissue-based features in a patch-based fashion. We trained it on a large dataset and tested it quantitatively and qualitatively.
Results: .Qualitative and quantitative results showed a remarkable agreement between synthetic high-b values and the ground-truth. A preliminary test with a microstructural model also gave encouraging results.
Impact: Being able to synthesise high b-value data from clinical data could unleash the availability of advanced microstructural models to study the human brain and body, with applications in fundamental research and in the clinical settings.
Introduction
MRI scanners with ultra-strong gradients are invaluable for studying tissue microstructure in vivo1. However, such systems are currently bespoke and have extremely limited distributions. The advantages they confer can nevertheless be shared more widely, especially with dedicated large datasets to build of bridges with lower-performance clinical systems. One such bridge can be built computationally, using image synthesis. While medical image synthesis is rapidly gaining popularity2,3, only a handful of approaches have been applied to diffusion MRI, mainly for apparent diffusion coefficient estimation in the prostate4,5. Here we propose to implement image synthesis and predict the powder average of high b-value signals from lower b-value signals, with the ultimate goal of fitting microstructural models.Network architecture and loss design strategy
The powder-average diffusion signal scales non-linearly with the b-value6. To learn this non-linear relationship, we leveraged a U-net convolutional neural network7, formed by a contracting path and a symmetric expanding path that allow it to learn multi-scaled structures from the data. To encode the need for a tissue-dependent mapping into our model2, we tailored the loss function of the network relying on three components:- the mean absolute error:
- the structural similarity index:
- two tissue-specific terms (using white matter and grey matter masks):
$$\begin{equation*} \mathcal {L}_{GM} = \Vert (f_{\theta}(\mathbf {X})-\mathbf {Y})\odot \mathbf {M_{GM}}\Vert _{1} \end{equation*}$$
Each term was weighted through its own hyper-parameter.
Dataset and pre-processing
Data were collected from 161 healthy participants using ultra-strong gradients (Siemens Connectom 3T). The protocol has been previously described8 and included a multi-shell diffusion-weighted acquisition (13 b=0 volumes; 20 directions w/ b=200s/mm2 and w/ b=500s/mm2; 30 dir. w/ b=1200s/mm2; 61 dir w/ b=2400s/mm2, w/ b=4000s/mm2 and w/ b=6000s/mm2). The diffusion data was pre-processed using the Marchenko-Pastur PCA approach9,10 with MRtrix11, then corrected for gradient non-linearities using custom scripts. Susceptibility-induced and eddy current-related distortions were removed using FSL (topup, eddy)12, and gibbs ringing was removed using Kellner’s approach13. From these pre-processed data, a brain mask as well as white and grey matter segmentations were estimated with MRtrix11,14. We finally computed, for each b-value, the arithmetic mean signals over all the directions (powder average), normalised by averaged non-diffusion-weighted volumes (b=0).Training procedure and experiments
Through preliminary experiments, we identified an optimal configuration that includes 5 downsample steps and a patch-based strategy, feeding it 16 sub-volumes of 16x16x16 voxels, sampled within the brain mask, for each subject in batches of 4. The dataset was split into training, validation and test sets (80-10-10 proportion), with the test set only fed to the network at the end. For each experiment, training lasted for 40 epochs (learning rate=0.0001, loss hyper-parameters=1). To test a refinement step, we trained an additional 40 epochs (learning rate=0.00001) and a greater weight on tissue-specific terms of the loss. As a first experiment, we sought to estimate the powder-average signal at b=6000s/mm2. In a more challenging experiment, we wanted to predict both b=4000s/mm2 and b=6000. Peak signal-to-noise ratio (PSNR) and mean absolute error were used for quantitative assessment. Finally, for a sample test subject, we used the synthetic b=4000s/mm2 and b=6000s/mm2 to fit the SANDI (soma and neurite density imaging) model15 to the data.Results
Fig.1 shows qualitative results for three sample subjects from the test set. To provide a quantitative perspective on the synthesised data, fig.2 shows high PSNR and low MAE values achieved by the model, averaged across different masks and subjects. Both qualitative and quantitative results highlight a strong agreement between synthetic and ground-truth data, with a slightly marginal improvement in the models that have been refined. Once used for downstream analysis, e.g. SANDI parameter estimation, our synthesized data produce equally accurate parametric maps (Fig.3).Discussion and conclusions
These results show a remarkable potential of image synthesis to complement clinical datasets. A drawback as in most of the convolutional architectures is the dependency on the contrast learned from the training set. While the current training set may not generalise to any given contrast or in pathology, it is possible to resort to harmonisation to adapt the initial training set, and to explore how uncertainty estimation could aid synthesis in pathological scenarios. One limitations of advanced microstructure techniques, such as SANDI, is the requirements for demanding acquisitions, often unfeasible on clinical scanners or for reasonable duration. Our results show that with much less demanding measurements (only a few relatively low b-shells) it is possible to satisfy the requirements of advanced techniques and produce high-fidelity quantitative maps.Acknowledgements
The WAND data were acquired at the UK National Facility for In Vivo MR Imaging of Human Tissue Microstructure funded by the EPSRC (grant EP/M029778/1), and The Wolfson Foundation, and supported by a Wellcome Trust Investigator Award (096646/Z/11/Z) and a Wellcome Trust Strategic Award (104943/Z/14/Z). Matteo Mancini is supported by the Italian National Institute of Health with a Starting Grant, and by the Wellcome Trust through a Sir Henry Wellcome Fellowship (213722/Z/18/Z). Marco Palombo is supported by UKRI Future Leaders Fellowship grant no. MR/T020296/2.References
1. Jones D K, Alexander D C, Bowtell R, et all. Microstructural imaging of the human brain with a ‘super-scanner’: 10 key advantages of ultra-strong gradients for diffusion MRI,NeuroImage, 2018; 182: 8-38
2. Chen C ,Raymond C, Speier W, et al. Synthesizing MR Image Contrast Enhancement Using 3D High-Resolution ConvNets, IEEE Transactions on Biomedical Engineering. 2023; 70 (2): 401-412.
3. Wang Y, Wu W, Yang Y, et al. Deep learning-based 3D MRI contrast-enhanced synthesis from a 2D noncontrast T2Flair sequence, Medical Physics, The International Journal of Medical Physics Research and Practice. 2022; 49 (7): 4478-4493
4. Wright, C., Mäkelä, P., Bigot, A. et al. Deep learning prediction of non-perfused volume without contrast agents during prostate ablation therapy. Biomed. Eng. Lett. 2023; 13: 31–40
5. Peng B, Liu B, Bin Y, et all. Multi-Modality MR Image Synthesis via Confidence-Guided Aggregation and Cross-Modality Refinement, IEEE Journal of Biomedical and Health Informatics, 2022; 26 (1): 27-35
6. Afzali M, Aja-Fernández S, Jones DK. Direction-averaged diffusion-weighted MRI signal using different axisymmetric B-tensor encoding schemes, Magnetic Resonance in Medicine. 2020; 84(3):1579-1591
7. Olaf Ronneberger, Philipp Fischer, Thomas Brox. U-Net: Convolutional Networks for Biomedical Image Segmentationx, Lecture Notes in Computer Science. 2015
8. Koller K, Rudrapatna U, Chamberland M, et all. MICRA: Microstructural image compilation with repeated acquisitions, Neuroimage. 2021; 225:117406
9. Cordero-Grande, L., Christiaens, D., Hutter, J., et al. Complex diffusion-weighted image estimation via matrix recovery under general noise models. Neuroimage. 2019; 200, 391-404.
10. Veraart, J., Fieremans, E., Novikov, D. S. Diffusion MRI noise mapping using random matrix theory. Magnetic resonance in medicine. 2016: 76(5), 1582-1593.
11. Tournier, J. D., Smith, R., Raffelt, D., et al. MRtrix3: A fast, flexible and open software framework for medical image processing and visualisation. Neuroimage. 2019: 202, 116137.
12. Smith, S. M., Jenkinson, M., Woolrich, et al. Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage. 2004: 23, S208-S219.
13. Kellner, E., Dhital, B., Kiselev, et al. Gibbs‐ringing artifact removal based on local subvoxel‐shifts. Magnetic resonance in medicine. 2016: 76(5), 1574-1581.
14. Dhollander T, Raffelt D, Connelly A. Unsupervised 3-tissue response function estimation from single-shell or multi-shell diffusion MR data without a co-registered T1 image. ISMRM Workshop on Breaking the Barriers of Diffusion MRI, 2016
15. Palombo M, Ianus A, Guerreri M, et all. SANDI: A compartment-based model for non-invasive apparent soma and neurite imaging by diffusion MRI, NeuroImage. 2020; 215: 116835
Figures
