0861
Deep Learning-Driven Automatic Scan Plane Alignment for Needle Tracking in MRI-Guided Interventions
Xinzhou Li1,2, Yu-Hsiu Lee3, David S. Lu1, Tsu-Chin Tsao3, and Holden H. Wu1,2
1Radiological Sciences, University of California, Los Angeles, Los Angeles, CA, United States, 2Bioengineering, University of California, Los Angeles, Los Angeles, CA, United States, 3Mechanical and Aerospace Engineering, University of California, Los Angeles, Los Angeles, CA, United States
1Radiological Sciences, University of California, Los Angeles, Los Angeles, CA, United States, 2Bioengineering, University of California, Los Angeles, Los Angeles, CA, United States, 3Mechanical and Aerospace Engineering, University of California, Los Angeles, Los Angeles, CA, United States
Synopsis
Misalignment between the MRI scan plane and needle trajectory degrades visualization and localization of the needle. This may prolong procedure time and increase errors in MRI-guided interventions. By leveraging an accurate deep learning-based needle localization algorithm, this work proposed an automatic workflow to realign the MRI scan plane with the needle. A scan plane control module was implemented for scan parameter updates. In one degree-of-freedom needle insertion experiments, the automatic workflow accurately aligned the scan plane with the needle (orientation difference 1.9°) with processing time <2 sec.
Introduction
Intra-procedural MRI provides crucial information for image-guided percutaneous interventions such as targeted biopsy and focal ablations (1). Deep learning (DL) needle localization algorithms (2) have potential to estimate the relative position between the needle and target lesion for device adjustment. However, misalignment between the MRI scan plane and needle may cause algorithm failure due to incomplete or missing features. Automatic alignment of scan planes with the needle can improve visualization to ensure procedural success and improve efficiency. Existing techniques require additional hardware to monitor the device for scan plane alignment (3,4). Recent studies used DL-based methods to localize anatomical features for automatic cardiac MRI scan plane selection (5). Therefore, this study aimed to develop an automatic method for intra-procedural MRI scan plane alignment with needle using DL-based needle localization.Methods
Deep Learning-Based Needle Localization: We used a mask region-proposal based convolutional neural network (Mask R-CNN) that was trained to accurately localize the needle feature tip and axis orientation with pixel-level accuracy on intra-procedural MRI (2). This method is also capable of using MRI to estimate physical needle tip and axis orientation with sub-millimeter accuracy (6). The processing time was 75 ms and 200 ms for needle feature and physical needle localization, respectively.Scan Plane Control (SPC) Module: We implemented the SPC software module on an external workstation to control a golden-angle-ordered 2D radial spoiled gradient-echo sequence (TR/TE: 5.26/3.0 ms, spatial resolution: 1.2x1.2 mm2, slice thickness: 5 mm, temporal resolution/footprint: 100/370 ms and 100/1050 ms) on a 3T MRI scanner (Prisma, Siemens) via a local network connection. The objective of the SPC module was to use the Mask R-CNN needle localization results to automatically align the scan plane of the sequence with the needle (Fig. 1).
Workflow: After positioning the subject (or object) and performing initial insertion of the needle at the specified entry point, localizer scans were acquired as the reference for initial plane selection (step I, Fig. 2). However, there may be misalignment between the initial scan plane and needle axis over the course of insertion due to limited visualization of initially inserted needle, or needle trajectory changes during insertion (e.g., deflection). To address this, we propose a series of automated steps (II-IV, Fig. 2) to obtain a final plane aligned with the needle. Even though the alignment of the intermediate plane with the needle may not be perfect in step III, the needle feature axis in this plane is adequate to realign the image plane with the needle.
Experiments: To evaluate the proposed workflow, we performed 1 degree-of-freedom (DOF) phantom needle insertion experiments 5 times (different needle and initial plane orientations) using a remote-controlled master-slave needle actuator (Fig. 3). To begin, the operator inserted the needle into the gelatin/tissue phantom to a shallow depth (<10 mm) at the entry point and executed step I to start the SPC module and the scan. During the experiment, the operator inserted the needle to reach a depth ~4 cm. Subsequently, the automated steps II-IV in the workflow obtained the final plane that aligned with the needle after insertion. After each experiment, a high-resolution 3D confirmation scan (spatial resolution: 1.1x1.1x2 mm3) was acquired to measure the reference needle position (7).
Data Analysis: The orientation difference (dθ) and Hausdorff distance (HD) between the initial/final scan plane with respect to the reference needle position were calculated to evaluate the accuracy of the scan plane alignment with the needle. The reference needle position was determined by manually segmenting the 3D needle feature on the confirmation scan. The average dθ and HD from the 1-DOF experiments were calculated.
Results
The proposed automatic scan plane alignment method improved alignment of the imaging plane with the reference needle feature (example in Fig. 4). The average dθ and HD of the initial image plane versus the reference needle feature was 16.4° and 9.4 mm. The average dθ and HD of the realigned final plane was reduced to 1.9° and 1.6 mm (Table 1). The average processing time of each image plane update was about 1-2 sec.Discussion
The accurate scan plane alignment results of the 1-DOF experiments demonstrated the potential of our automated workflow to improve step-and-shoot MRI-guided procedures, such as in-bore targeted biopsy (1). Accurate DL-based needle localization is the key to the success of SPC module. Only single-slice 2D MRI and two additional scans were required. The proposed workflow can be executed within 20 sec (imaging and processing), which has potential to accelerate procedures compared with 3D or multi-slice intra-procedural imaging acquired with manual plane selection. In addition, improved scan plane alignment with the needle can improve the accuracy of the Mask R-CNN needle localization algorithm. It is possible to extend the current workflow to accommodate in-bore needle manipulation for multiple DOF. The automatic workflow can also be combined with interactive real-time sequences to support a range of MRI-guided interventions.Conclusion
In this study, we proposed an automatic MRI scan plane alignment workflow based on deep learning needle localization results. This workflow achieved fast (1-2 sec processing) and accurate scan plane alignment (dθ ~ 2°) for needle insertion in phantoms and has potential to improve needle tracking in MRI-guided interventions.Acknowledgements
This study was supported in part by Siemens Medical Solutions USA and UCLA Radiological Sciences.References
1. Tan N, Lin W-C, Khoshnoodi P, Asvadi NH, Yoshida J, Margolis DJA, Lu DSK, Wu H, Sung KH, Lu DY, Huang J, Raman SS. In-bore 3-T MR-guided transrectal targeted prostate biopsy: prostate imaging reporting and data system version 2–based diagnostic performance for detection of prostate cancer. Radiology 2016;283(1):130-139.
2. Li X, Young AS, Raman SS, Lu DS, Lee Y-H, Tsao T-C, Wu HH. Automatic needle tracking using Mask R-CNN for MRI-guided percutaneous interventions. International Journal of Computer Assisted Radiology and Surgery 2020.
3. Elayaperumal S, Plata JC, Holbrook AB, Park Y, Pauly KB, Daniel BL, Cutkosky MR. Autonomous Real-Time Interventional Scan Plane Control With a 3-D Shape-Sensing Needle. IEEE Transactions on Medical Imaging 2014;33(11):2128-2139.
4. Neumann M. Automatic multimodal real-time tracking for image plane alignment in interventional Magnetic Resonance Imaging. Université de Strasbourg, 2014.
5. Blansit K, Retson T, Masutani E, Bahrami N, Hsiao A. Deep Learning–based Prescription of Cardiac MRI Planes. Radiology: Artificial Intelligence 2019;1(6):e180069.
6. Li X, Lee Y-H, Tsao T-C, Wu HH. Physical Needle Localization using Mask R-CNN for MRI-Guided Percutaneous Interventions. Proceedings of the ISMRM 28th Annual Meeting, Paris, France 2020:4144.
7. Mikaiel S, Simonelli J, Li X, Lee Y-H, Lee YS, Sung K, Lu DS, Tsao T-C, Wu HH. MRI-guided targeted needle placement during motion using hydrostatic actuators. The International Journal of Medical Robotics and Computer Assisted Surgery 2020;16(2):e2041.
Figures
Figure 1: The scan plane control (SPC) module used needle localization results from the Mask R-CNN algorithm to automatically align the MRI scan plane with the needle. Images with high frame rate (i.e. temporal footprint of 370 ms/image) were used for Mask R-CNN needle feature localization and images with moderate frame rate (i.e. 1050 ms/image) were used for Mask R-CNN physical needle localization. If the procedure has not been completed yet, the needle localization results would be used to set the new slice orientation and position for the next scan.
Figure 2: The workflow for automatic MRI scan plane alignment with the needle using Mask R-CNN and the scan plane control (SPC) module. (I) An initial plane was manually selected based on the needle feature at the entry point on 3-plane localizer images. (II-IV) Scans 1-3 automatically started using the plane selection from the previous step. Mask R-CNN needle localization results were used to automatically select a new plane that aligned with the needle.
Figure 3: Experimental setup to evaluate the scan plane alignment workflow using a gel/tissue phantom. (a) A remote-controlled master-slave actuator using a beaded chain mechanism allowed the operator to manually control needle translation (i.e. insertion, retraction) and rotation during the scan. The operator worked from the end of the MRI table and could see intra-procedural images on the in-room monitor to manually adjust the master controller for needle insertion during the MRI scan. (b) A photo of the experimental setup. B0: main magnetic field.
Figure 4: (a) The needle feature at the entry point and after insertion were displayed for initial plane. The incomplete needle feature after insertion was caused by misalignment between the initial plane with the needle trajectory. Orientation difference (dθ) and Hausdorff distance (HD) are 19.2° and 13.9 mm. The reference needle was extracted by segmenting needle feature on a high-resolution 3D confirmation scan. (b) After executing the proposed automated workflow, the final plane was aligned with the needle and the complete needle feature is visible. dθ and HD were 1.8° and 1.8 mm.
Table 1: Quantitative analysis results of MRI scan plane to needle alignment accuracy based on 1 degree-of-freedom needle insertion experiments. Rotation angle: orientation between needle shaft and B0. dθ: orientation difference between image scan plane and reference needle on 3D confirmation scans. HD: Hausdorff distance between image scan plane and reference needle.