2002

Compatibility of a 3D Time-of-Flight Video Camera for Motion Capture with a 0.55T MRI System
Douglas Brantner1, Leanna Pancoast1, Jerzy Walczyk1, Roy Wiggins1, Ryan Brown1, and Christopher Collins1
1Center for Advanced Imaging Innovation and Research, New York University, New York, NY, United States

Synopsis

We evaluated compatibility of a 3D Time-of-Flight (ToF) video camera with a 0.55T MRI system. Initial tests showed severe MR image artifacts when the ToF camera was operating, and USB communication with a computer outside the MR suite was interrupted during MR image acquisition. With adequate conductive shielding of the ToF camera and typical MR imaging bandwidths, MR images could be acquired with no measurable effects from the ToF camera, and with no effects of the MRI system on ToF signal acquisition.

Introduction

Characterization of real-time subject motion during MRI is a significant challenge for a variety of studies. Respiratory belts and off-frequency RF antennas can provide rough measures of the temporal nature of bulk motion(1). Accelerometers can be used to measure motion of specific portions of the body without knowledge of location(2,3). Standard video cameras require advanced methods and calibration to provide explicit 3D spatial information(4). Here we evaluate MR compatibility of a Time-of-Flight (ToF) camera, capable of measuring 3D location of all points within the camera’s field of view at relatively high temporal and spatial resolution.

Methods

A Pico-Flexx ToF camera (PMD; Siegen, Germany) was used to measure 3-dimensional information from all surfaces in its field of view when fixed to the upper bore surface in a commercial Aera MAGNETOM MRI system modified to operate at 0.55T (Siemens Healthcare, Erlangen, Germany). The ToF camera (Fig. 1a) measures distance to each location on the surface in front of it by measuring the round-trip travel time for bursts of 850nm laser light(5). MR images and noise spectrum measurements were performed without the camera in the MR suite (baseline) and with the camera operating with different configurations of RF shielding. The ToF camera was operated by a laptop via a 500cm long USB cable connected to a shielded USB connector installed into the patch panel. When the ToF camera was operated without any additional shielding, interference between the camera and the MR system produced significant zipper-type artifacts on MR images and caused the camera to lose communication with the laptop. Experiments with different levels of shielding led to a configuration with negligible interference between the camera and the MR system.

To accomplish shielding of the camera and USB cable, the camera was fixed inside an 11cm length of ¾-inch copper pipe with a 1.9x3.5cm window cut into it. A 0.7mm-thick piece of glass coated with an electrically conductive material (Indium-doped tin oxide, or ITO)(6) was cut to fit the rectangular hole in the pipe and affixed with copper tape(Fig. 1b). The USB cable was connected to the camera, passed through a copper fitting, a 143cm section of ¾-inch pipe, 371cm of ½-inch flexible braided copper sleeve (TechFlex; Sparta, NJ)(Fig. 1c) and connected to a USB connector in the patch panel. Copper tape was used to ensure shield continuity.

MR system noise spectrum and image SNR were measured without the ToF Camera in the MR suite (baseline) and with it operating in the magnet bore. Measurements of the noise floor were accomplished using a product noise spectrum sequence. Image SNR maps were acquired using GRE acquisitions (with and without RF). Image SNR measurements were acquired with two different receiver bandwidths.

Integrity of ToF camera performance was assessed by evaluating distance to a flat surface before, during, and after MR image acquisition. Ability of the ToF camera to detect physiological motion was demonstrated within the magnet with a human subject.

Results

Figure 2 shows MR image SNR and noise spectra for baseline (no ToF camera present in suite) and during operation of the shielded ToF camera. Although some perturbations at the upper end of the noise spectrum are apparent when the camera is operating, they have negligible effect on image SNR, even when the signal bandwidth is relatively broad (1500Hz/pixel, 192kHz total). The largest perturbations of the noise spectrum occur at frequencies approximately 150kHz and 250kHz above the center acquisition frequency (23.56MHz). This is well above the frequency span for ordinary acquisitions.

Figure 3 shows average distance to a rectangular region near the middle of the patient bed measured with the ToF camera before, during, and after MR imaging. A piece of cardboard was placed on the bed to provide a flat surface for this measurement. Over the entire 90-second period (including 70 seconds of MR imaging) the measured distance varied by less than 0.5mm.

Figure 4 shows average distance to a rectangular region on the abdomen of a human subject in the magnet during normal breathing as measured with the ToF camera. The respiratory time course is clearly visible as the abdominal region rises and falls by several millimeters with each breath.

Discussion and Conclusion

With adequate conductive shielding of the ToF camera and a reasonable imaging bandwidth, there were no measurable effects of the camera operation on MRI performance, and no effects of the MRI system on ToF signal acquisition. Interference apparent in the noise spectrum indicates that effects may be seen in images acquired over different frequency spans. However, by carefully selecting the bandwidth and center frequency of acquisition we anticipate most MR sequences could be run with no measurable effect from the ToF camera.

Future developments may reduce interference further. While this initial demonstration included bulky support structures (Fig. 1c) and measured only respiratory motion (Fig. 4), we anticipate that future implementations could include a much more streamlined physical profile and a wider variety of patient motions. Work at higher field strengths may require greater care in positioning the ToF camera to avoid damaging it. Use of more than one ToF camera could allow for tracking motion over more of the body(4).

Acknowledgements

The authors thank Jan Paska, Radhika Tibrewala and the rest of the RF team for discussions on the work. This work was was performed under the rubric of the Center for Advanced Imaging Innovation and Research (CAI2R, www.cai2r.net) at the New York University School of Medicine, which is an NIBIB Biomedical Technology Resource Center (NIH P41 EB017183).

References

  1. T Vahle, DS Rigie, R Brown, T Zhao, L Feng, M Fenchel, P Speier, F Boada. PET-MR Motion Correction of Entire Listmode Data Sets Using Pilot Tone Navigation. 2017 ISMRM Annual Meeting, p. 3897
  2. L Pancoast, D Brantner, R Wiggins, J Walczyk, R Brown. Wireless Body Sensor Data Acquisition Platform for Motion Tracking. 2021 ISMRM Annual Meeting, p. 0566.
  3. Chen B, Weber N, Odille F, Large-Dessale C, Delmas A, Bonnemains L, Felblinger J. Design and Validation of a Novel MR-Compatible Sensor for Respiratory Motion Modeling and Correction. IEEE Trans Biomed Eng 2017;64(1):123-133.
  4. L Qin, P van Gelderen, JA Derbyshire, F Jin, JLee, JA de Zwart, Y Tao, JH Duyn. Prospective Head Movement Correction for High Resolution MRI using an In-bore Optical Tracking System. Magn Reson Med 2012;62(4)924-34
  5. Pasinetti, S., Hassan, M.M., Eberhardt, J., Lancini, M., Docchio, F., Sansoni, G., 2019. Performance Analysis of the PMD Camboard Picoflexx Time-of-Flight Camera for Markerless Motion Capture Applications. IEEE Transactions on Instrumentation and Measurement 68, 4456–4471.. doi:10.1109/tim.2018.2889233
  6. Maniyara, R.A., Graham, C., Paulillo, B., Bi, Y., Chen, Y., Herranz, G., Baker, D.E., Mazumder, P., Konstantatos, G., Pruneri, V., 2021. Highly transparent and conductive ITO substrates for near infrared applications. APL Materials 9, 021121.. doi:10.1063/5.0040864
  7. He, Y., Liang, B., Zou, Y., He, J., Yang, J., 2017. Depth Errors Analysis and Correction for Time-of-Flight (ToF) Cameras. Sensors 17, 92.. doi:10.3390/s17010092
  8. K. Tanaka, Y. Mukaigawa, H. Kubo, Y. Matsushita and Y. Yagi, "Recovering Transparent Shape from Time-of-Flight Distortion," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 4387-4395, doi: 10.1109/CVPR.2016.475.

Figures

a) Pico-Flexx Time-of-Flight (ToF) camera, b) ToF camera and USB cable inside RF shield, c) experimental arrangement of the shielded camera in the 0.55T MRI system. The shielding includes copper pipe, copper tape, braided copper sleeve, and glass with conductive ITO coating.

MRI SNR and noise floor measurements with shielded ToF camera in MR system (bottom row) compared to baseline (top row). Although operation of ToF camera causes perturbations of noise spectrum (far right) most notable at about 150kHz and 250kHz above center frequency (23.56MHz), this has negligible effects on SNR of images acquired with bandwidths from 19.2kHz (far left) to 192kHz (center). Parameters used: TE 5ms, TR 500 ms, FA 25 degrees, FOV=256mm, matrix=128, slice thickness = 10mm.

Evaluation of effects of MR system on operation of ToF camera. The mean ToF-measured distance from the camera to the patient table (averaged over a ~120 x 85 mm region of interest) was recorded during a 90-second period including 10 seconds before, 70 seconds during, and 10 seconds after MR imaging using a 3D RAVE sequence. The measured distance varied by less than 0.5mm. ToF camera operated at 5 frames per second.

Distance maps from ToF camera to the subject’s torso at two different timepoints during normal respiration (a and b). Respiratory signal (c) measured over a ~38x41mm region of interest (black box) on the subject’s abdomen. ToF camera operated at 5 frames per second. Distance between inhalation and exhalation is consistently greater than 7mm. Buttons and seams of the subject’s shirt are apparent in the depth maps, but the button material causes greater depth measurements, likely due to refractive index (7,8), requiring greater care in analysis.

Proc. Intl. Soc. Mag. Reson. Med. 30 (2022)
2002
DOI: https://doi.org/10.58530/2022/2002