0186

Electrical Property Mapping using Vision Transformers and Canny Edge Detection
Ilias Giannakopoulos1, Xinling Yu2, Giuseppe Carluccio3, Gregor Koerzdoerfer4, Karthik Lakshmanan1,5, Hector Lise de Moura1, Jose Cruz Serralles1, Jerzy Walczyk,1, Zheng Zhang2, and Riccardo Lattanzi1,5,6
1Bernard and Irene Schwartz Center for Biomedical Imaging, Department of Radiology, NYU Grossman School of Medicine, New York, NY, United States, 2UC Santa Barbara, Santa Barbara, CA, United States, 3Universita di Napoli Federico II, Napoli, Italy, 4Siemens Medical Solutions, New York, NY, United States, 5Center for Advanced Imaging Innovation and Research (CAI2R), Department of Radiology, NYU Grossman School of Medicine, New York, NY, United States, 6Vilcek Institute of Graduate Biomedical Sciences, NYU Grossman School of Medicine, New York, NY, United States

Synopsis

Keywords: Electromagnetic Tissue Properties, Electromagnetic Tissue Properties, Machine Learning

Motivation: To estimate tissue electrical properties (EP) non-invasively for specific absorption rate management and as biomarkers for pathology characterization.

Goal(s): To train neural networks for mapping transmit magnetic fields (B1+) onto EP.

Approach: We developed a 3D vision transformer that takes the B1+ and an edge mask based on Canny filtering of the MR image as the inputs. The targets were the EP of the object. We trained on simulated tissue mimicking objects and fine-tuned on realistic head models.

Results: Our network successfully reconstructed the EP in a phantom experiment, and detected a synthetic cyst in a realistic head model in simulation.

Impact: We propose a supervised learning approach using vision transformers and Canny edge detection to perform electrical property (EP) mapping. The network successfully reconstructs the EP using experimentally measured fields and is a promising first step towards clinically-usable in-vivo EP reconstructions.

INTRODUCTION

Electrical property (EP) maps of tissue, namely permittivity and conductivity, can enable the estimation of local specific absorption rate distributions, serve as biomarkers for diseases1,2 and enhance treatment methods. MR-based EP tomography utilizes the measured transmit magnetic fields (B1+) to reconstruct EP maps based on the Maxwell's equations. Partial differential methods3 exhibit noise artifacts at the boundaries between tissues4, while integral-based approaches5 are computationally expensive. Existing supervised machine learning (ML) methods can lead to overfitting due to the scarcity of available training data6. Previously proposed ML methods7 that use both T2w images and B1+ maps could result in learning direct correlations between T2 and EP maps, and neglect the electrodynamic constraints between EP and B1+. In this work, we generated8 a large simulated dataset and trained a vision transformer9 to perform EP reconstructions. We incorporated an edge mask10 as an additional input to the network to mitigate boundary artifacts.

METHODS

We generated 8160 tissue-mimicking ellipsoid phantoms with random principal semi-axes. These included 500 homogeneous cases, 1000 cases with an additional inner compartment, and 6660 cases with 23 inner smaller compartments. Each compartment had randomly assigned properties, with relative permittivity ranging from 11 to 120 and conductivity from 0.07 to 2.5 S/m. We split the dataset into 6065 training, 1463 validation, and 632 testing cases. Additionally, we generated 1200 variations of six realistic human head models11,12,13 to fine-tune the network. The variations (216 heads) based on one model (Duke) were split into a validation (108) and a testing (108 cases, different than those used for validation) dataset. The rest of heads were used for training (984). All models were discretized with 5 mm voxel resolution. The network's inputs were the simulated8 magnitude of quadrature B1+ and its phase, which was approximated as the half of the transceive phase14, for a 3T birdcage head coil. We also computed edge masks based-on Canny edge detection filters10 from the conductivity maps and included them as a third input of the network. We also collected experimental measurements15 for a cylindrical phantom, using a GRE acquisition and an in-house birdcage coil.

Our network (Figure 1) consisted of three sequential 3D TransUNets9 (cascades) with two pooling layers each, GELU activations16, dropout, residual connections, and 16 attention17 heads per transformer. We trained using the simulated phantoms for 100 epochs, using a cyclical learning rate schedule18, and a dropout rate of 0.05. We included an additional cascade and fine-tuned a new network on the head models for 100 epochs, and increased dropout to 0.2 to avoid overfitting. We minimized the mean squared error (MSE) during training and also monitored structural similarity (SSIM).

RESULTS

Figure 2 presents the MSE, normalized MSE (NMSE), and SSIM during validation and testing for both networks. The average NMSE (and SSIM) in the test cases were 1.5% (0.82) and 0.8% (0.93) for the permittivity and conductivity, respectively, when the network was trained with the tissue-mimicking phantoms. The values were 5.1% (0.91) and 13.6% (0.91) for the fine-tuned network tested on heads. Figure 3 presents a qualitative comparison for two representative reconstructions. Figure 4 presents the experimental reconstruction, where the NMSE (and SSIM) were 13.5% (0.7) and 15.6% (0.7), for the permittivity and conductivity, respectively. Figure 5 presents the network performance in detecting a pathology unseen during training.

DISCUSSION

Our proposed network could accurately map the EP of tissue-mimicking phantoms. Canny filtering enhances boundary preservation between tissues, overcoming a common shortcoming of other EP reconstruction techniques. Although the edge masks were generated using the conductivity in simulation, they could as effectively be generated from GRE images in experiments. The model demonstrated robust performance also in reconstructing EP for out-of-distribution models and in identifying pathologies that were absent from the training data. The network's performance was satisfactory also with experimental data. The high errors in a few regions of the phantom were expected due to discrepancies between simulated and experimental conditions (like slice interpolation and coil differences). In fact, our results present higher homogeneity and accuracy comparing to other partial differential methods19,20, for which the experimental reconstructions were severely corrupted by noise amplifications associated with numerical derivatives. In our case, we anticipate an error reduction by aligning the simulated training data with the experimental MRI setup and retraining using higher resolution.

CONCLUSION

The proposed network architecture with the edge mask as an additional input demonstrated exceptional EP reconstructions in numerical phantoms and for out-of-distribution cases, such as pathology detection and experimental data. Our approach seems a promising direction towards clinically-usable in-vivo EP reconstructions.

Acknowledgements

This work was supported in part by NIH R01 EB024536 and was performed under the rubric of the Center for Advanced Imaging Innovation and Research (CAI2R, www.cai2r.net), an NIBIB National Center for Biomedical Imaging and Bioengineering (NIH P41 EB017183).

References

  1. Tha, Khin Khin, et al. "Noninvasive electrical conductivity measurement by MRI: a test of its validity and the electrical conductivity characteristics of glioma." European radiology 28 (2018): 348-355.
  2. Holder, D. S. "Detection of cerebral ischaemia in the anaesthetised rat by impedance measurement with scalp electrodes: implications for non-invasive imaging of stroke by electrical impedance tomography." Clinical Physics and Physiological Measurement 13.1 (1992): 63.
  3. Zhang, Xiaotong, Jiaen Liu, and Bin He. "Magnetic-resonance-based electrical properties tomography: A review." IEEE reviews in biomedical engineering 7 (2014): 87-96.
  4. Mandija, Stefano, et al. "Error analysis of helmholtz‐based MR‐electrical properties tomography." Magnetic resonance in medicine 80.1 (2018): 90-100.
  5. Serrallés, José EC, et al. "Noninvasive estimation of electrical properties from magnetic resonance measurements via global Maxwell tomography and match regularization." IEEE Transactions on Biomedical Engineering 67.1 (2019): 3-15.
  6. Mandija, Stefano, et al. "Opening a new window on MR-based electrical properties tomography with deep learning." Scientific reports 9.1 (2019): 8895.
  7. Jung, Kyu‐Jin, et al. "Data‐driven electrical conductivity brain imaging using 3 T MRI." Human Brain Mapping 44.15 (2023): 4986-5001.
  8. Giannakopoulos, Ilias I., et al. "A hybrid volume-surface integral equation method for rapid electromagnetic simulations in MRI." IEEE Transactions on Biomedical Engineering 70.1 (2022): 105-114.
  9. J. Chen et al., ''Transunet: Transformers make strong en-coders for medical image segmentation,'' arXiv preprint arXiv:2102.04306, 2021.
  10. Bao, Paul, Lei Zhang, and Xiaolin Wu. "Canny edge detection enhancement by scale multiplication." IEEE transactions on pattern analysis and machine intelligence 27.9 (2005): 1485-1490.
  11. Christ, Andreas, et al. "The Virtual Family—development of surface-based anatomical models of two adults and two children for dosimetric simulations." Physics in Medicine & Biology 55.2 (2009): N23.
  12. Dimbylow, Peter. "Development of the female voxel phantom, NAOMI, and its application to calculations of induced current densities and electric fields from applied low frequency magnetic and electric fields." Physics in Medicine & Biology 50.6 (2005): 1047.
  13. Makarov, Sergey N., et al. "Virtual human models for electromagnetic studies and their applications." IEEE reviews in biomedical engineering 10 (2017): 95-121.
  14. van Lier, Astrid LHMW, et al. "Electrical properties tomography in the human brain at 1.5, 3, and 7T: a comparison study." Magnetic resonance in medicine 71.1 (2014): 354-363.
  15. Akoka, S., et al. "Radiofrequency map of an NMR coil by imaging." Magnetic resonance imaging 11.3 (1993): 437-441.
  16. Hendrycks, Dan, and Kevin Gimpel. "Gaussian error linear units (gelus)." arXiv preprint arXiv:1606.08415 (2016).
  17. Vaswani, Ashish, et al. "Attention is all you need." Advances in neural information processing systems 30 (2017).
  18. Smith, Leslie N. "Cyclical learning rates for training neural networks." 2017 IEEE winter conference on applications of computer vision (WACV). IEEE, 2017.
  19. Balidemaj, Edmond, et al. "Feasibility of electric property tomography of pelvic tumors at 3T." Magnetic resonance in medicine 73.4 (2015): 1505-1513.
  20. Eda, Naohiro, et al. "A method for electrical property tomography based on a three-dimensional integral representation of the electric field." IEEE Transactions on Medical Imaging 41.6 (2021): 1400-1409.

Figures

Figure 1: (Top) Simulation setups used for the data generation. The inputs (|B1+|, transmit phase approximated as half of the transceive phase, and edge mask) and targets (Permittivity and Conductivity) are shown for an axial slice for a phantom and a head model. (Bottom) Network architecture and extended representation for one cascade. Yellow and green layers represent convolutions with kernel size = 3 or 1. Red and orange layers represent up-convolutions and average pooling. Cyan, magenta, and blue layers represent instance normalization, dropout, and GELU activation functions.


Figure 2: (Top) Average validation error for the MSE (left) and SSIM (right) for the network trained on phantoms and the network that was fine-tuned on the head models. (Middle and bottom) Histograms of differences in NMSE (left) and SSIM (right) for the network trained and tested on phantoms and the network fine-tuned on the head models and tested only on Duke's variations. The results account for all volumes in the test datasets.

Figure 3: (Top) Relative permittivity and (Bottom) conductivity reconstructions for 2 representative examples from the phantoms and heads datasets. The absolute error with respect to the ground-truth is also presented. For the phantom, we also present a reconstruction without using edge masks as part of the inputs in training. The latter reconstruction has larger boundary errors, artifacts, and misses fine details (cyan arrows), which are instead preserved in the reconstruction with the network that used the edge mask. In the conductivity map of the head, the CSF is underestimated.

Figure 4: (Left) Relative permittivity and (right) conductivity reconstructions for the phantom experiment. (Bottom) Absolute error with respect to the ground-truth. The reconstruction preserve truthfully the homogeneity of the inner compartment and underestimate the values in the top left region of the outer compartment. Although the error is relatively high, this is expected due to miss-match of the simulations used for training and the experimental setup. In future work, we plan to include cylinders in the training and re-process the experiments with the resulting network.

Figure 5: Conductivity map for a head model (variation of the Duke's head) in which we inserted a cyst (conductivity = 0.8). (Top row) The reconstructed map (left) preserved the conductivity values of the pathology. In particular, the NMSE in the interest surrounding the cyst was less than 6% on average (Bottom row).


Proc. Intl. Soc. Mag. Reson. Med. 32 (2024)
0186
DOI: https://doi.org/10.58530/2024/0186