Introduction

Quantification of hyperpolarized ¹²⁹Xe MRI requires accurate delineation of the subject’s thoracic cavity. This typically involves time-consuming and labor-intensive segmentation of a breath-hold ¹H MR image, which is then registered to the functional scan for quantitative analysis¹. Alternatively, it is also possible to train a convolutional neural network (CNN) to directly segment the ¹²⁹Xe ventilation scan. We have previously demonstrated that this approach can yield relatively high Dice scores². However, careful inspection of such segmentation reveals poor performance around peripheral ventilation defects. Overcoming this problem requires training the CNN with much larger datasets with well-registered masks and ¹²⁹Xe scans containing a wide range of patterns of ventilation heterogeneity from minimal to extensive. Unfortunately, these datasets are not only costly to acquire but also require extensive and precise manual segmentation. Here, we propose using the Pix2Pix generative adversarial network³ to synthesize ¹²⁹Xe images that can be tuned to a desired ventilation defect percentage (VDP) range and texture/histogram input, together with corresponding aligned thoracic cavity masks. We then use this synthetic data to augment the training data for a standard (U-net)⁴ segmentation model. We compare segmentation performance when the model is trained with hand-segmented data alone, synthetic data alone, and both datasets combined.

Methods

Dataset selection and processing:
Our study employed 73 acquired imaging datasets consisting of 14-slice 2D GRE coronal ¹²⁹Xe ventilation and registered SSFSE ¹H thoracic cavity scans. These were registered and segmented by expert readers to delineate the thoracic cavity, enabling generation of histogram-based defect maps⁵. The 73 datasets were split 5-fold and then split into individual slices for cross validation to train both the Pix2Pix data synthesis model and the U-net segmentation model. To expand the range of histograms on which Pix2Pix trained, each ¹²⁹Xe image was augmented in 5 different ways using gamma correction with [0.2,0.6,1.4,1.6] values and adaptive histogram equalization. Both the manually generated defect maps and their original and augmented histograms were used to train the Pix2Pix model.

Training:
The Pix2Pix model was trained for 100 epochs with batch size of 1, with other hyperparameters as outlined in the original work³. The Pix2Pix input is a concatenation of a histogram tiled to all spatial locations of image's size and the defect map. The U-net segmentation model was trained over 3000 epochs with batch size of 16 using binary cross entropy. For each training session, the segmentation model with the highest validation Dice score was retained.

Synthetic image generation:
Synthetic ¹²⁹Xe images were generated using 704 thoracic cavity masks publicly available from 2D X-ray scans associated with Kaggle challenge⁶. Each mask was used to create a separate defect map with a target level of defect percentage based on the distribution of VDP values in the set of 73 2D GRE images. This process, as outlined in Figure 1, begins by randomly placing circle-shaped “defects” within the mask. This included 1) randomly choosing left or right lung 2) calculating the radius of a circle that satisfies the area of the VDP target, 3) multiplying this by a random scaling value drawn from an exponential distribution with mean = 0.5, 4) randomly placing the circle 5) calculating the remaining area and 6) repeating these steps in the opposite lung. The effect of the exponential distribution on ventilation defect pattern are shown in Figure 2 along with a montage of resultant ventilation defect maps. These, along with the 704 randomly picked histograms from the original training dataset are then fed into Pix2Pix to synthesize ¹²⁹Xe ventilation images.

Results

Figure 3 illustrates representative synthetic ¹²⁹Xe images generated by Pix2Pix for the given input histograms and defect maps. Figure 4 shows a montage of synthetic ¹²⁹Xe ventilation images and their perfectly aligned masks. A total of 704 generated 2D slices were used in training the U-net segmentation model. Figure 5 shows the Dice scores achieved when training the U-net segmentation model with the original data, synthetic data, and the combination of both. The model trained with only synthesized data exhibited the lowest Dice score (0.777). The one trained only with original data was somewhat better (0.861) and the one trained with all the data performed best (0.869).

Discussion

We have demonstrated a method to rapidly generate synthetic ¹²⁹Xe ventilation images containing arbitrary defects and perfectly aligned with corresponding thoracic cavity masks. Although the addition of this data resulted in only marginal improvement in the performance of the U-net segmentation model, there are several opportunities for improvement. First, the masks used to synthesize the data were derived from X-ray projections and most closely reflect only the middle slice of the thoracic cavity. Thus, data synthesis can be improved by using masks from a truly 3D dataset. Second, we limited the number of synthesized data to 704 for the purpose of comparison. The performance may be improved as we generate more training data since there is no limit to how many we choose to synthesize. Third, the relative sizes and numbers of defects (generated here based on an exponential distribution) may not reflect those of the real ventilation defects and need to be further optimized.

Acknowledgements

R01HL105643, R01HL12677, NSF GRFP DGE-1644868