0872

Fully automated registration of cross-sectional liver images using fully convolutional and affine transformation networks
Kyle A Hasenstab1, Guilherme M Cunha1, Kang Wang1, Brian Hurt1, Alexandra Schlein1, Timoteo Delgado1, Ryan L Brunsing2, Armin Schwartzman3, Kathryn Fowler1, Albert Hsiao1, and Claude B Sirlin1

1Radiology, University of California, San Diego, San Diego, CA, United States, 2Radiology, Stanford University, Stanford, CA, United States, 3Family Medicine and Public Health, University of California, San Diego, San Diego, CA, United States

Synopsis

Proper spatial alignment of anatomical landmarks during and between liver imaging exams is challenging due to the dynamic morphology of the liver. Liver-focused registration algorithms have been developed but are typically semiautomatic. We propose a fully-automated pipeline for affine-based registration of inter- and intra-exam liver images and assess performance on clinical liver MRI exams at 1.5T and 3T. The proposed pipeline achieved comparable or superior accuracy and scalability to that reported for previously proposed algorithms. Retrospective image review by an expert abdominal radiologist confirmed subjective improvement in anatomic registration and lesion co-localization. Proof of concept of multimodal scalability was demonstrated.

Introduction

Medical image registration (MIR) increases radiologist confidence and diagnostic accuracy1. However, proper spatial alignment of liver and liver lesions during and between imaging exams is challenged by the dynamic morphology of the liver and its surrounding organs as well as unavoidable variability in patient positioning, movement, and physiological motion2-4.

Both affine and deformable liver-focused MIR algorithms have been developed to address these challenges but are typically semiautomatic and slow due to CPU implementation5-9. Moreover, the majority of liver-focused MIR algorithms were tested on small numbers of patients and may not be generalizable. Convolutional neural networks (CNN) have proven fast and effective for deformable MIR10-11, but deformable methods are unreliable due to their exceedingly flexible parameterizations12-14.

We propose a fast two-step, fully automated pipeline for affine-based registration of inter- and intra-exam liver images comprising a liver segmentation CNN and affine transformer (AT) parallelized on a GPU for speed15. Performance is evaluated on a large cohort of clinical liver MRI exams through a variety of image similarity metrics. Clinical utility is illustrated through case examples. Multimodal scalability is explored.


Methods

Imaging Data

This retrospective, single-center, HIPAA-compliant, IRB-approved study included 1016 image acquisitions from 253 unique patients who underwent a total of 564 gadoxetate-enhanced 1.5T or 3T liver MRI exams from 2011 to 2018. The dataset comprised baseline and follow-up 3D T1-weighted hepatobiliary phase (HBP) acquisitions, including multiple intra-exam acquisitions. All possible within-patient pairs of acquisitions, including those acquired in the same exam, were used for registration validation, producing 2225 acquisition pairs.

Registration Algorithm

Acquisition pairs, comprising a static acquisition (e.g., baseline acquisition) and a moving acquisition (e.g., follow-up acquisition), are first sent to an independently developed liver segmentation CNN with U-net architecture to create intensity-populated liver masks (intensity-masks), which are used to constrain the registration to focus on the liver (Figure 1). Intensity-masks are registered using an AT network with a single 12-neuron dense layer representing affine parameters. Estimated parameters are used to map the native moving acquisition to static acquisition space. Since the registration pipeline is individually applied to each acquisition pair and is fully unsupervised, we do not require a leave-out dataset for testing.

Evaluation

Binary liver masks, intensity-masks, and whole images were used as input into the proposed pipeline to study the impact of liver-focused inputs on registration accuracy. Performance was evaluated by computation time, image similarity metrics, and subjectively by an expert radiologist through retrospective image review. Differences in image similarity metrics were statistically evaluated using Wilcoxon signed-rank tests due to departures from normality.

Results

Median [10th, 90th percentile] intersection-over-union (IoU) percentages were 87.35% [70.71%, 94.08%] for binary-mask, 86.73% [69.37%, 94.05%] for intensity-mask, and 77.90% [38.08%, 92.23%] for whole-image inputs. Intensity correlations (ICs) were 0.91, [0.79, 0.97] for binary-mask, 0.91 [0.80, 0.97] for intensity-mask, and 0.85, [0.52, 0.96] for whole-image inputs. Liver-focused inputs produced significantly higher median IoU and IC than whole-image inputs. Binary-mask inputs provided significantly higher IoU and lower IC than intensity-mask inputs, but differences were small. Binary-mask inputs achieved convergence (49.75s [40.99s, 67.18s]) significantly faster than intensity-mask (84.56s [66.68s, 119.98s]) or whole-image (75.51s [55.05s, 112.86s]) inputs. Subjectively, manual registration by an expert radiologist resulted in ambiguous lesion localization and an inability to register all parts of the liver simultaneously. The affine-registration mitigated these problems (Figures 2-4) and improved radiologist confidence for lesion localization.

Discussion

Liver-focused MIR constrains the algorithm to maximize spatial alignment across the liver region only, producing more accurate registration than whole-image inputs. Minor differences in binary-mask and intensity-mask performance suggest liver-masking drives the registration, with inclusion of intensities providing only marginal improvement. An accurate fully-automated, liver-focused pipeline for CT/open-MR registration has been proposed6. However, performance was validated across only 18 livers and produced an average liver overlap of 78%, with prolonged runtime (540s). Our study included 2225 acquisition pairs from 253 patients and produced a comparable overlap score (87%), with median runtime <60s for binary-mask input. Preliminary application across different imaging phases and modalities shows proof-of-concept of multimodal scalability (Figure 5). Additionally, affine-registered images improved reader confidence, which should be further confirmed in prospective studies.

Conclusion

Our proposed liver registration pipeline achieves comparable or superior accuracy and scalability to previously proposed algorithms. Fully-automated liver registration can automate intra-exam lesion characterization and improve reader confidence of inter-exam comparisons16-18. Prospective research is necessary to assess the large-scale impact of registered liver images on patient outcomes when incorporated into clinical radiology workflow.

Acknowledgements

This study would not be possible without the invaluable contributions from the members of the Liver Imaging Group (LIG) and the Augmented Imaging/Artificial Intelligence Data Analytics (AIDA) Laboratory at University of California, San Diego.

References

  1. Erickson BJ, Patriarche J, Wood C, et al. Image Registration Improves Confidence and Accuracy of Image Interpretation. Cancer Informatice. 2007;4:19-24.
  2. Blackall JM, King AP, Penney GP, et al. A Statistical Model of Respiratory Motion and Deformation of the Liver. International Conference on Medical Image Computing and Computer-Assisted Intervention. 2001;1338-1340.
  3. Blackall JM, Ahmad S, Miquel ME, et al. MRI-based measurements of respiratory motion variability and assessment of imaging strategies for radiotherapy planning. Physics in Medicine and Biology. 2006;51(17):4147-4169.
  4. Preiswerk F, De Luca V, Arnold P, et al. Model-guided respiratory organ motion prediction of the liver from 2D ultrasound. Medical Image Analysis. 2014;18(5):740-751.
  5. Feng Q, Zhou Y, Li X, et al. Liver DCE-MRI Registration in Manifold Space Based on Robust Principal Component Analysis. Scientific Reports. 2016;6:34461
  6. Foruzan AH and Motlagh HR. Multimodality liver registration of Open-MR and CT scans. International Journal of Computer Assisted Radiology and Surgery. 2015;10(8):1253-1267.
  7. Gunay G, Luu MH, Moelker, A, et al. Semiautomated registration of pre- and intraoperative CT for image-guided percutaneous liver tumor ablation interventions. Medical Physics. 2017;44(7):3718-3725.
  8. Luu MH, Klink C, Niessen W, et al. Non-Rigid Registration of Liver CT Images for CT-Guided Ablation of Liver Tumors. PLoS One. 2016;11(9): e0161600.
  9. Song Y, Totz J, Thompson S, et al. Locally rigid, vessel-based registration for laparoscopic liver surgery. International Journal of Computer Assisted Radiology and Surgery. 2015;10(12):1951-1961.
  10. Vos BD, Berendsen FF, Viergever MA, et al. End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision. 2017;204-212.
  11. Shan S, Yan W, Guo X, et al. Unsupervised End-to-end Learning for Deformable Medical Image Registration. arXiv:1711.08608.
  12. Fortin D, Basran PS, Berrang T et al. Deformable versus rigid registration of PET/CT images for radiation treatment planning of head and neck and lung cancer patients: a retrospective dosimetric comparison. Radiation Oncology. 2014;9:50.
  13. Woerner AJ, Choi M, Harkenrider MM, et al. Evaluation of Deformable Image Registration-Based Contour Propagation From Planning CT to Cone-Beam CT. Technology in Cancer Research and Treatment. 2017;16(6):801-810.
  14. Batchelar D, Collins A, Jaswal J. A Comparison of Rigid to Deformable Registration for Dose Escalation to the Dominant Lesion in HDR Prostate Brachytherapy. Brachytherapy. 2015;14(1):S50.
  15. Jaderberg M, Simonyan K, Zisserman A, et al. Spatial Transformer Networks. In NIPS 2015.
  16. Jansen MJA, Kuijf HJ, Pluim JPW, et al. Automatic classification of focal liver lesions based on clinical DCE-MR and T2-weighted images: A feasibility study. 2018 IEEE 15th International Symposium on Biomedical Imaging. 2018.
  17. Pregliasco GA, Collin A, Gueguen A, et al. Improved Detection of New MS Lesions during Follow-Up Using an Automated MR Coregistration-Fusion Method. American Journal of Neuroradiology. 2018;39(7):1226-1232.
  18. Schellingerhour D, Lev MH, Bagga RJ, et al. Coregistration of head CT comparison studies: assessment of clinical utility. Academic Radiology. 2003;10(3):242-248.

Figures

Figure 1: Diagram of the proposed registration pipeline. Image pairs are first sent to an independently developed 2D liver segmentation CNN with U-net architecture to create intensity-populated liver masks (intensity-masks). A follow-up intensity-mask (moving image) is registered to a baseline intensity-mask (static image) using an affine transformer. Optimal affine transformation parameters are determined by maximizing similarity between the baseline intensity-mask and registered follow-up intensity-mask. Affine transformation parameters are then used to map the whole follow-up image to the baseline image.

Figure 2a-b: Baseline and follow-up: Slices are manually registered using the bifurcation of the right portal vein as anatomical reference (yellow arrows). Note difference in morphology of posterior aspect and left lobe of liver (orange circles) due to different liver position.

Figure 2c: Registered follow-up: Posterior aspect and left lobe of liver are more consistent with appearance on baseline image.


Figure 3: Two independent HBP acquisitions within the same exam: Slices are aligned using the right and middle hepatic veins as anatomical landmarks (yellow arrows). Note the nodular lesion in segment IVa on the first acquisition (a) (white arrow) is not visible in the corresponding slice of the subsequently acquired image (b), probably due to differences in breath hold and liver rotation. Following image registration (c), vessels and lesion are registered.

Figure 4a-b: Baseline and follow-up: Images manually registered using anatomical landmarks (yellow arrows). Lesion (white arrow) observed in follow-up image without correspondence on baseline scan.

Figure 4c: Affine registered image shows correspondence of the anatomical landmarks.

Figure 4d-e: Baseline and follow-up images registered using left portal vein as anatomical landmark. Nodule seen on baseline scan is not shown in aligned follow-up slice. Confidence on lesion correspondence is low due to differences in lesion size and liver appearance.

Figure 4f: Following image registration, the nodular image can be confidently confirmed as a preexisting lesion that grew from baseline.


Figure 5a-c: Cross-modality image registration. T1 weighted MR Image (a) is registered to previous CT study (b). Note difference in the morphology of the posterior aspect and left lobe of the liver, which is corrected using the proposed registration (c).

Figure 5d-f: Intra-exam image registration. Hepatobiliary phase image (d) is registered to arterial phase image (e). Note difference in the morphology of the left lobe of the liver and the presence of a third lesion not seen in the corresponding arterial phase slice. After affine registration anatomical landmarks and liver metastasis are colocalized.


Proc. Intl. Soc. Mag. Reson. Med. 27 (2019)
0872