Synopsis

Keywords: Image Reconstruction, Software Tools

The open-source software tool BART provides various state-of-the-art image reconstruction algorithms for MRI. We add an interface for streaming image and k-space data to BART, which should facilitate the application of modern image reconstruction to real-time MRI. Our implementation matches the requirements of interventional MRI in terms of frame rate and latency.

Furthermore, we present an intuitive way of interactive slice positioning for MR-guided Interventions using a game controller featuring an inertial measurement unit.

Introduction

MRI-guided clinical interventions may be an alternative to conventional X-Ray guided interventions^1-3. However, the required technical setup is challenging. Fast image reconstruction is required as well as low latency control of scan parameters such as slice orientation. While interventional MRI has yet to become a mainstream method, it is a very active area of research and continues to see technical innovation such as the exploration of mixed-reality⁴.

BART⁵ is an open-source framework which provides numerous methods for reconstruction of MR images. It can be used for online-reconstruction e.g. with the Gadgetron framework⁶ and implements state-of-the art algorithms for real-time reconstruction⁷.

We extended BART to allow streaming of input and output data. Based on this, we implement a reconstruction pipeline to use the framework for the purpose of low latency real-time MRI. Furthermore, we explore a method to interactively control slice orientation during an MRI examination from inside the scanner room. For this purpose we modified a game controller featuring an inertial measurement unit (IMU). The different components of the project are combined into a framework for interactive real-time MRI. We demonstrate its functionality and show first results on a phantom.

Methods

An overview of our setup is shown in Fig. 1. Real-time image reconstruction was implemented in BART⁵ in the form of a new command, bart rtreco. This command reconstructs MR images using coil compression, k-space filtering, a sliding window adjoint nonuniform fast Fourier transform (NUFFT) and optionally RSS coil combination and Median filtering. The program ingests acquired k-space frame-by-frame from an in-memory filesystem. A separate routine accumulates k-space lines transmitted from the scanner and triggers the reconstruction process. Reconstructed images are then streamed to a real-time viewer.

Interactive control of the scanner is realized through a library which can be linked against the MRI sequence. Commands are send to the scanner computer using a plain-text protocol: To rotate a slice normal by e.g. 180 degree about the readout-axis, a single line containing the command rotr(90) would be send.

Upon start of the measurement, the sequence launches a separate control thread which awaits commands send over TCP/IP. Received commands are injected into a ring buffer to be fetched by the sequence. This allows independent testing and ensures that command processing never interferes with real-time sequence processing.

On the sender side, a game controller (Joycon by Nintendo, Kyoto, Japan) featuring an IMU was connected⁸ to the computer used for reconstruction and visualization. Rotation of the controller was sensed using its IMU and translated to sequence control commands. A corresponding rotation of the slice normal is then realized by the sequence. Slice offset and in-plane rotation were controlled separately using buttons on the controller. The controller was furthermore modified to allow its use inside the MR room.

To test the setup, we acquired data of a structural phantom using a Vida Magnetom (Siemens Healthcare GmbH, Erlangen) with a custom radial FLASH sequence. Acquisition parameters are given in Fig. 2. The phantom along with the modified game controller are shown in Fig 3. All computation was done on an Intel Core i5-4690 quad-core processor with 15 GiB of RAM. Timing of the code was measured using microsecond-precision timestamps on the measurement system and the Linux host. Clock offsets were determined using iputils-clockdiff⁹ with about 1 ms precision.

Results

Images were reconstructed and displayed at a frame rate of 22.7 fps. We analyze our reconstruction system by measuring the time needed for the different processing steps. In addition to the results shown in Fig. 4, the time needed for reconstruction of a single frame was below 44 ms for 99 percent of all frames. Delay between start of acquisition and finished reconstruction was approximately 92 ms. Every step takes less time than the acquisition time for a single frame, which is crucial for real-time reconstruction.

Interactive control was tested by navigating to transversal and sagittal views starting from a coronal perspective, as shown in Fig. 5. The images show some artifacts due to the strong radial undersampling.

Discussion

Measurement of the latency in our setup demonstrates that the framework is suitable for interactive real-time MRI. A considerable part of the visualization latency could be amounted to the initialization of the viewing program. The sliding window NUFFT reconstruction has some pronounced artifacts, but it allows tests on standard hardware without GPU. As a next step, we will integrate iterative reconstruction algorithms into the streaming framework.

Slice navigation using a game controller was demonstrated. The motion sensing approach using the build-in IMU makes the task very intuitive. In future work we will focus on improving the control latency and explore IMU-based slice control in more detail.

We plan to release all vendor-independent software components as open-source.

Conclusion

We extended BART with an interface for streaming of k-space and image data. A simple reconstruction pipeline was created based on the new interface. We demonstrate feasibility for interactive real-time MRI. As a next step, we will add iterative reconstruction algorithms available into the reconstruction pipeline.

References

(1): Reddy, S. R. V., Arar, Y., Abou Zahr, R., Gooty, V., Hernandez, J., Potersnak, A., ... & Hussain, T. (2020). Invasive cardiovascular magnetic resonance (iCMR) for diagnostic right and left heart catheterization using an MR-conditional guidewire and passive visualization in congenital heart disease. Journal of Cardiovascular Magnetic Resonance, 22(1), 1-11.

(2): Nageotte, S. J., Lederman, R. J., & Ratnayaka, K. (2020). MRI catheterization: ready for broad adoption. Pediatric cardiology, 41(3), 503-513.

(3): Unterberg-Buchwald, C., ... & Uecker, M. (2017). Targeted endomyocardial biopsy guided by real-time cardiovascular magnetic resonance. Journal of Cardiovascular Magnetic Resonance, 19(1), 1-10.

(4): Franson D, Dupuis A, Gulani V, Griswold M, Seiberlich N. (2021). A System for Real-Time, Online Mixed-Reality Visualization of Cardiac Magnetic Resonance Images. Journal of Imaging. 2021; 7(12):274

(5): BART Toolbox for Computational Magnetic Resonance Imaging, DOI: 10.5281/zenodo.592960

(6): Diakite, M., Campbell-Washburn, A. & Xue, H. (2018) Integration of the BART Toolbox into Gadgetron Streaming Framework for Inline Cloud-Based Reconstruction. Proc. Intl. Soc. Mag. Reson. Med. 26

(7): Uecker, M., Hohage, T., Block, K.T. and Frahm, J. (2008), Image reconstruction by regularized nonlinear inversion - Joint estimation of coil sensitivities and image content. Magn. Reson. Med., 60: 674-682.

(8): https://github.com/dekuNukem/Nintendo_Switch_Reverse_Engineering

(9): https://github.com/iputils/iputils/blob/master/clockdiff.c