Introduction

High quality scan prescription that optimally covers the area of interest with scan planes aligned to relevant anatomical structures is crucial for error-free radiologic interpretation. Consistency of prescription is especially important for evaluation of disease progression in serial imaging studies. Manual prescription quality varies significantly depending on the operator's skill and training.
Machine learning (ML) approaches have demonstrated the ability to perform accurate slice planning in organs such as brain and knee^1,2. Compared to the previous approaches, ML models did not require long computation times during inference, but still relied on custom logic and large amounts of manually annotated training data specific to acquisition type and anatomical region. Deployment of the models and integration with the scanner software remained a challenge.
An approach that utilized metadata embedded in the image headers for training was proposed previously and applied to prescription of knee MRI exams³. The goal of the current study was to investigate whether this technique could be adapted for lumbar spine acquisitions and performed on the scanner during patient examinations.

Methods

Model and Training: The prescription pipeline was based on three multislice, rotational, region-based convolutional neural network object detection models³ (MS-R²CNN, Fig 1a), based on R²CNN architecture^4,5, that generated inclined bounding boxes in axial, sagittal, and coronal planes.
For training, we have used a dataset of 497 spine MRI exams containing sagittal T1 and T2 weighted images of lumbar spine and corresponding localizers. The dataset was shuffled and split into training and validation sets with a ratio of 70:30%. Geometric prescription parameters were extracted from the headers of the DICOM files of the diagnostic images. The localizer images, immediately preceding the sagittal acquisitions were sorted by orientation and, along with the extracted prescription parameters, stored in TFRecord files.
Three models were trained using a Titan Xp GPU (NVidia, Santa Clara, CA) on axial, sagittal, and coronal localizer slices and corresponding inclined boxes until the models overfit. Random horizontal flipping and rotation by a random angle were implemented for data augmentation. The model weights could optionally be pre-trained on a DOTA satellite imaging dataset⁶. Hyperparameter search was performed to determine the optimal learning rate parameters and pre-training strategy. For validation, metrics, such as intersection over union (IOU), difference in center positions, box sizes, and tilt angles between the model output and ground truth were quantified on the entire validation dataset.
Inference pipeline: During inference, the localizer series was split into three slabs by orientation and fed into MS-R²CNN models to generate three in-plane prescriptions that were combined to generate position, dimensions, and orientation of a single image slab (Fig. 1b). The inference pipeline was built in Python 3.5 with TensorFlow 1.8, packaged in a Docker container, and deployed to a 3T MRI scanner (GE Healthcare, Waukesha, WI). The pipeline interfaced with the scanner software using vendor’s external software API to load selected localizer images and apply generated prescriptions during the examinations.
Volunteer experiments: We have evaluated the technique in experiments in three healthy volunteers. In each experiment, after acquiring the localizer images, a trained MRI technologist prescribed sagittal T1-weighted FSE and T2-weighted frFSE acquisitions. The same acquisitions were then performed with the automated prescription software. To assess reproducibility, the subjects were repositioned to be no longer aligned with MRI axes and both manual and automated acquisitions were repeated.

Results

Figure 2 shows examples of generated (blue) and ground truth (green) boxes overlaid on the localizer images from the validation set. Figure 3 shows plots of generated box parameters vs. the ground truth values. The plots show good agreement between inferred and ground truth values for center positions, dimensions, and tilt angles in the coronal plane. In most manually prescribed exams, the tilt angle was zero in axial and sagittal planes. Despite this limitation of the training data, random rotation augmentation allowed the models to learn to produce accurate tilt angles. The IOU between generated and ground truth boxes (mean±st.dev.) was 0.78±0.09 for axial, 0.82±0.14 for sagittal, and 0.77±0.11 for the coronal model.
Experiments in healthy volunteers demonstrated high accuracy of prescription in all subjects. Figure 4 shows the automated prescription being applied to the scan plan at two positions of the same subject. Figure 5 shows comparison between manual prescriptions by a trained operator and prescriptions generated by the ML pipeline for two positions of the same subject, as well as examples of slices acquired with these prescriptions.

Discussion

Our results validate the ability of ML-based automated prescription pipeline to be trained to generate accurate oblique prescriptions for a new acquisition type – sagittal lumbar spine MRI – without any additional manual annotation or feature engineering. This allows to easily extend this technique to other acquisition types and anatomical regions. We have also demonstrated the feasibility of performing ML-based automated prescription directly on the scanner within the time constraints of a clinical MRI examination.

Conclusion

This study demonstrates the feasibility of using oriented object detection-based pipelines on the scanner for automated prescription of lumbar spine acquisitions. Automated prescription has potential to improve the quality of MRI examinations by reducing prescription errors and enabling more consistent and easier to interpret imaging studies.

Acknowledgements

The authors would like to thank Dan Rettman and Maggie Fung (GE Healthcare) for help with MRI scanner deployment. This work was supported by GE Healthcare and NIH UH3AR076724.

References

1. Shanbhag DD, Bhushan C, de Alm Maximo A, et al. A generalized deep learning framework for multi-landmark intelligent slice placement using standard tri-planar 2D localizers. Proc Intl Soc Mag Reson Med. 2019.
2. Bhushan C, Shanbhag DD, Maximo A, Patil U, Madhavan R, Frick M, Amrami KK, Yeo DTB, Foo T. Consistency in human and machine-learning based scan-planes for clinical knee MRI planning. Proc Intl Soc Mag Reson Med. 2020.
3. Ozhinsky E, Pedoia V, Majumdar S. Oriented Object Detection Convolutional Neural Network for Automated Prescription of Oblique MRI Acquisitions. Proc Intl Soc Mag Reson Med. 2020.
4. Ma J, Shao W, Ye H, et al. Arbitrary-Oriented Scene Text Detection via Rotation Proposals. IEEE Trans Multimedia. 20(11):3111-3122. doi:10.1109/TMM.2018.2818020.
5. https://github.com/DetectionTeamUCAS/R2CNN_Faster-RCNN_Tensorflow
6. Xia G-S, Bai X, Ding J, et al. DOTA: A Large-Scale Dataset for Object Detection in Aerial Images. The IEEE Conference on Computer Vision and Pattern Recognition CVPR. 2018:3974-3983.