Introduction

Hyperpolarized ¹²⁹Xe MRI is increasingly used in 3D imaging of ventilation and gas exchange¹. However, quantification of these images relies on accurate delineation of the subject’s thoracic cavity. This typically uses a separate breath-hold ¹H image that is segmented and registered to the functional scan². Nevertheless, these additional images take additional time to acquire, may not be well-matched to the lung inflation volume of the ¹²⁹Xe scan, or may simply be unavailable. Moreover, performing segmentations takes time and training, which is a primary cause of inter-analyst variability. Therefore, it is highly desirable to derive a high-quality thoracic cavity mask from the ¹²⁹Xe MRI scan alone. This task is particularly challenging because the lung boundary we are seeking to delineate from ¹²⁹Xe MRI is often adjacent to the most prominent ventilation defects. Here, we demonstrate a 3D convolutional neural network (CNN), trained on expert segmentations of both ¹H and ¹²⁹Xe images, that reliably delineates the thoracic cavity from only the ¹²⁹Xe MRI scan.

Methods

Dataset generation:
Our study used 232 image datasets that included 3D radial acquisitions of both ¹²⁹Xe ventilation and ¹H thoracic cavity anatomy. The ¹H images were segmented by expert readers to generate thoracic cavity masks. These data were divided into 185 sets for training and 47 for validation. Final testing of the 3D-CNN was done using an additional 33 “pristine” datasets where masks were generated by expert readers who segmented the ventilation images while using the ¹H images as a reference. Because the training data predominantly contained subjects with minor or no defects, and because severe defects provide the largest segmentation challenge, this pool was increased by template augmentation³ to create a more balanced dataset. Representative expert reader segmentations for the training data are shown in Fig 1, which provides both the ¹²⁹Xe ventilation and registered ¹H thoracic cavity images as well as the superposed segmentation outline used to visually confirm accuracy.

Model architecture:
Model training consisted of two key modules. The first was a basic segmentation model (SM) underlying all models described here and adapted from the V-Net for medical image segmentation⁴; however, each model differed in the specifics of their training. The second was a perceptual loss model which consists of an encoder-decoder model (EDM) and a predictor model (PM) similar to Anatomically Constrained Neural Networks (ACNNs)⁵. The PM structure was similar to that of the EDM encoder. Each model was trained separately, and upon the convergence of loss, were combined and trained simultaneously as shown in Fig 2.

Loss:
The segmentation model was first trained using Dice-Focal loss ( L_Dice & L_focal ), followed by a perceptual loss function L_perc. The total loss function is given by:

$$$L_{tot}=L_{Dice}+0.1*L_{focal}+0.001*L_{perc}$$$

The focal loss function⁶ encourages the model to include ventilation defect areas in the segmentation by lowering the relative penalty of false positives to false negative and makes the model focus on poorly ventilated areas by increasing the overall penalty in that area. This is illustrated in Fig 3 showing that binary cross entropy causes defects to shift the lung boundary, but that this is overcome by the addition of focal loss. The perceptual loss function is added to encourage the segmentation to following the lung shape and is given by the Euclidean distance (ED) between the output from the encoder to the ground-truth mask and the SM prediction. For training the perceptual loss model, loss is given by the binary cross entropy for EDM and ED for PM.

Training:
The SM, EDM, and PM were trained individually over 126,000 steps with batch size = 1. Subsequently, the frozen-weight encoder in the EDM was used to train the SM for an additional 126,000 steps. The SM was trained using ventilation images as inputs and the ground-truth segmentation as the outputs. EDM was trained using the ground-truth segmentation for both inputs and outputs. The PM used either the ¹H structural or ¹²⁹Xe ventilation images as the inputs; the outputs were from the encoder in the EDM using the associated segmentation.

Results

We found that the model performance improves significantly when training data undergoes an additional registration step. The registered dataset is created by registering the ventilation images to the ground-truth segmentations to improve alignment and SNR. However, while this facilitates training of the network, it causes it to struggle with segmenting low-SNR images. Therefore, the perceptual loss component is used to constrain the network to predict segmentation that follows the known anatomical structure. This is illustrated in Fig 4, showing incorrect segmentation of low-SNR images when using only the traditional 3D-CNN with binary cross entropy. Fig 5 shows quantitative comparison between models, further demonstrating that the proposed model performs better than the traditional model and comparable to expert readers (Dice score: 0.955). We did not observe significant differences between training PM with ventilation or ¹H images.

Discussion

We have demonstrated that a 3D-CNN using only ¹²⁹Xe MRI inputs generates thoracic cavity segmentations that are similar to those of expert readers using both ¹²⁹Xe MRI and volume-matched ¹H scans. These methods may help reduce inter-reader subjectivity and allow quantification of pulmonary ¹²⁹Xe MRI when corresponding proton images are not volume-matched, poorly registered, or unavailable.

Acknowledgements

R01HL105643, R01HL12677, NSF GRFP DGE-1644868