1965
Medical Image Registration Using Deep Learning Techniques Applied to Pediatric Magnetic Resonance Imaging (MRI) Brain Scans
Andjela Dimitrijevic1,2, Vincent Noblet3, and Benjamin De Leener1,2,4
1NeuroPoly Lab, Institute of Biomedical Engineering, Polytechnique Montréal, Montréal, QC, Canada, 2Research Center, Ste-Justine Hospital University Centre, Montréal, QC, Canada, 3ICube-UMR 7357, Université de Strasbourg, CNRS, Strasbourg, France, 4Computer Engineering and Software Engineering, Polytechnique Montréal, Montréal, QC, Canada
1NeuroPoly Lab, Institute of Biomedical Engineering, Polytechnique Montréal, Montréal, QC, Canada, 2Research Center, Ste-Justine Hospital University Centre, Montréal, QC, Canada, 3ICube-UMR 7357, Université de Strasbourg, CNRS, Strasbourg, France, 4Computer Engineering and Software Engineering, Polytechnique Montréal, Montréal, QC, Canada
Synopsis
Deep learning techniques have a potential in allowing fast deformable registration tasks. Studies around registration often focus on adult populations, even if there is a need for pediatric research where less data and studies are being produced. In this study, we compared three methods for intra-subject registration on publicly available Calgary Preschool dataset. Using the DeepReg framework, pre-registering with a rigid and affine transformation (proposed RigidAffineReg method) showed the least negative JD values and the highest Dice score (0.924±0.045). By achieving faster alignments, this tool for pediatric MRI scans could help proliferate larger scale population research in brain developmental studies.
Introduction
Registration consists of bringing a pair of images into spatial correspondence. It is used for intra or inter-patient analysis in the medical field and is an essential step to ensure a normalized visualization between brain scans. There is only a limited number of registration methods dedicated to the pediatric brain, mainly because of the major changes that occur during neurodevelopment and the need for huge amounts of data. Conventional deformable registration involves estimating a transformation by reducing it to an optimization problem often iterative. This process is commonly quite time-consuming before converging to an optimal solution. Newly used deep learning (DL) techniques utilizing convolutional neural networks (CNN) can allow faster registrations by applying a learning-based approach. Hence, applying DL methods to pediatric MRI brain scans can improve registration and future diagnostics for medical applications. Indeed, these recently developed methods estimate deformation fields directly from an input 3D pair of images. Therefore, the general objective of this study is to evaluate the performance and robustness of DL techniques for registering pediatric brain MRI scans.Methods
The main objective of this project is to develop and validate a deep learning framework for fast image registration on pediatric brain data. The project has three specific objectives :1) Selecting as well as preprocessing an appropriate pediatric dataset that provides a large range of age,
2) Evaluating the fundamental types of registration methods that the framework would train on and,
3) Training an existing DL framework for intra-subject registration and evaluating its performance (accuracy and time) depending on age and age interval.
After carrying out a state-of-the-art study, few DL-based approaches for pediatric intra-subject non-invasive magnetic resonance imaging are extracted. This confirmed the need to implement a DL-based registration framework for pediatric populations. Using DeepReg1 library as well as Calgary Preschool dataset2 of 247 T1-weighted images and 434 pairs, three methods are being explored; NoReg, RigidReg and RigidAffineReg. NoReg method consists of going through a U-Net like CNN architecture without any registration task performed beforehand. While RigidReg and RigidAffineReg methods are respectively rigidly and rigidly together with affine registered prior to the network using ANTs3. The pair-based strategy used consists of previously registering each moving/fixed image pair. Finally, the unsupervised network is evaluated with obtained white matter (WM), gray matter (GM) and cerebrospinal fluid (CSF) segmentations considering the Dice score as a performance metric like it was done in numerous DL applications4–7. The generated deformation fields are also regulated and then evaluated using the Jacobian determinant (JD).
Results & Discussion
Fig.1 presents the general procedure for training each pre-pocessed image pair and obtaining the dense displacement field (DDF). A batch size of 2 pairs was used and trained during 150 epochs. The obtained results of the test set evaluated on the best runs for each method are illustrated in Fig.2. It is possible to observe that NoReg (0.770±0.088), RigidReg (0.916±0.051) and RigidAffineReg (0.924 ± 0.045) seem to be having on average better Dice scores than the initial Dice score of 0.419±0.188. Previously mentioned values are mean and standard deviations calculated over all three regions and subjects. As a visual assessment of the best and worst pairs for RigidReg and RigidAffineReg methods considering their obtained Dice scores, Fig.3 illustrates moving, fixed and predicted images. Table 1 regroups information on important acquired evaluation metrics considering the three methods used. Dice scores as well as the number of negative JD values evaluated on the test set are shown. These negative values indicate unwanted local folding at certain regions in the brain. Furthermore, GPU total training time for each chosen run depending on the method is presented. GPU testing time is the time required to predict a new DDF when using the chosen trained model for a single pair of moving/fixed images. CPU time per pair of the pre-registration tasks done on ANTs for RigidReg and RigidAffineReg methods is also presented. Also, as depicted by Fig.4, for RigidReg and RigidAffineReg methods, the Dice scores in fact decrease as the age interval expands.Conclusion
A DL registration framework has been implemented to test three methods using pediatric MRI T1-weighted images. These methods allowed breaking down the non-linear registration task into its simpler parts RigidReg and RigidAffineReg. These two methods use a pair-based strategy where pairs are pre-registed using ANTs3. For both methods, results demonstrated a significant increase in Dice score when using a neural network compared to without one. NoReg was added to assess the performance of the network without pre-registration. It was dominated by RigidReg and RigidAffineReg whether it be for their higher Dice scores or smaller amount of negative JD values. NoReg method remains pertinent because it requires no pre-registration and still achieves a mean Dice score of 0.770±0.088. However, due to increased variability, it can’t grasp the relationship between age interval and obtained Dice scores. Nonetheless, there remains additional analyses to be made. One could try inspecting the impact of all the evaluated metrics on more labeled anatomical structures of the brain not only on WM, GM and CSF. It would also be interesting to assess the network’s performance using Hausdorff distance between segmentations to better account for changes in brain microstructures.Acknowledgements
This study was supported by Polytechnique Montréal, by the Canada First Research Excellence Fund, and by the TransMedTech Institute.References
1. Fu, Y. et al. DeepReg: a deep learning toolkit for medical image registration. Journal of Open Source Software 5, (2020).
2. Reynolds, J. E., Long, X., Paniukov, D., Bagshawe, M. & Lebel, C. Calgary Preschool magnetic resonance imaging (MRI) dataset. Data Brief 29, 105224 (2020).
3. Avants, B. B., Epstein, C. L., Grossman, M. & Gee, J. C. Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med. Image Anal. 12, 26–41 (2008).
4. Dalca, A. V., Balakrishnan, G., Guttag, J. & Sabuncu, M. R. Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration. Lect. Notes Comput. Sci. 729–738 (2018) doi:10.1007/978-3-030-00928-1_82.
5. Dalca, A. V., Balakrishnan, G., Guttag, J. & Sabuncu, M. R. Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces. Med. Image Anal. 57, 226–236 (2019).
6. Kuang, D. & Schmah, T. FAIM -- A ConvNet Method for Unsupervised 3D Medical Image Registration. arXiv [cs.CV] (2019).
7. He, K., Zhang, X., Ren, S. & Sun, J. Deep Residual Learning for Image Recognition. arXiv [cs.CV] (2015).
8. Balakrishnan, G., Zhao, A., Sabuncu, M. R., Guttag, J. V. & Dalca, A. V. VoxelMorph: A Learning Framework for Deformable Medical Image Registration. CoRR abs/1809.05231, (2018).
Figures
Fig.1. Schema of the training procedure to obtain deformation field (ɸ) with given moving (M) and fixed (F) 3D pair of images. Validation technique using segmentations to calculate Dice scores is also shown. Inspired by Guha Balakrishnan et al.8
Fig.2. Dice scores results on the evaluation set represented as boxplots for pair-based strategy obtained for each method compared to the initial dice scores when no registration is done. The Dice scores are calculated in the white matter (WM), gray matter (GM) and cerebrospinal fluid (CSF)
Fig.3. Visual representation in axial view of worst (left) and best (right) predicted fixed images for most performant RigidReg (R) (up) and RigidAffineReg (RA) (down) methods considering best and worst Dice scores obtained with their respective segmentations. Red arrows indicate regions where the registration went wrong and green regions which seem well represented. Numbers on the left indicate the chosen slices in the third dimension.
Table 1. Average Dice scores calculated over all regions and subjects in the test set for pair-based strategy for all three studied methods. The number of negative Jacobian determinant (JD) values are calculated considering the 1.1 million voxels present per image and summing all negative values per method. Next to it are percentages of negative JD values. All values in parentheses are the calculated standard deviations. For the given times, h is for hours, m for minutes and s for seconds.
Fig.4. Dice scores versus age intervals for paired-based strategy of every method, NoReg (NR) on first row, RigidReg (R) on second row and RigidAffineReg (RA) on third row for each segmentation, WM is represented in red, GM in green and CSF in blue.
DOI: https://doi.org/10.58530/2022/1965