Synopsis

Keywords: Machine Learning/Artificial Intelligence, Machine Learning/Artificial Intelligence

Synthetic CT (sCT) image generated from MRI by unsupervised deep learning models tends to have large errors around bone area. To generate better sCT image quality in bone area, we propose to add bony structure constrains in the loss function of the unsupervised CycleGAN model, and modify the single-channel CycleGAN to a multi-channel CycleGAN that takes Dixon constructed MR images as inputs. The proposed model has lowest mean absolute error compared with single-channel CycleGAN with different MRI images as input. We found that it can generate more accurate Hounsfield Unit and anatomy of bone in sCT.

Introduction

MR-only radiotherapy planning (MROP) is beneficial to patients by reducing MR and CT registration errors ¹. MRI is the primary information carrier and planning CT (i.e. CT simulation scan) is redundant if synthetic CT (sCT) can be generated from MRI for patient positioning and dose calculation in MROP workflow.
CycleGAN ² is an unsupervised deep learning (DL) model used in MR-to-sCT conversion when paired CT and MR images are not always available in the clinics. However, its disadvantage compared to supervised DL methods is that content consistency is not guaranteed between MR and sCT, because there are no direct constrains between input and synthetic images. This problem can be observed around bony area by a large error between the sCT and reference CT ^3,4.
T2-weighted imaging (T2WI) is widely used in MROP for prostate radiotherapy, and also used as input to generate sCT. However, bony anatomy on T2WI-generated sCT is often missing. In this work, we propose to use Dixon sequence to facilitate sCT generation, as it separates water and fat signals, and has been used for sCT generation ^5-9.

Methods

1. Multi-channel CycleGAN with Dixon constructed images
Different from original CycleGAN model ² which only has one-to-one image conversion shown in Figure 1(a), we modified the architecture to multi-channel CycleGAN shown in Figure 1(b). The multi-channel CycleGAN has 3 generators and 3 discriminators. One generator takes MR images of fat and in-phase (IP) images as inputs and outputs CT modality images, and the other two generators both take CT images as inputs and output fat and IP MR images separately. Three discriminators are used to distinguish real and sCT, real and synthetic fat images, and real and synthetic IP images. The modified CycleGAN framework takes advantage of Dixon constructed MR images that highlight structures containing fat (i.e., bones) and water (i.e., soft tissues), to generate sCT.
The loss function for multi-channel CycleGAN is listed from equation (1) to (14) shown in Figure 2.
2. Bony structure constrains in loss function
We further added constrains in bone area in to improve bony structure conversion. The assumption is that there is no anatomy change on bony area between CT and MR of the same patients. Bone masks (M_Bone) are generated from pCT images with a global threshold of Hounsfield Unit (HU) larger than 200. The HU values on the generated sCT is forced to be the same as that on pCT in bony area only, listed in equation (15) shown in Figure 2. With this supervision added in the loss function, the objective can be written as equation (16) instead of equation (1), shown in Figure 2.

Results

This study was conducted on a total of 31 patients with prostate cancer: 20 for training and 11 for testing. Each patient has pCT, T2-weighted, and Dixon MR images. Rigid registration between MRIs and pCT of the same patients was performed to preprocess the data for training. For testing, MRIs were deformed to pCT, and the trained models were applied to the deformed MRIs for sCTs generation. The pCT serves as ground truth to evaluate sCT quality.
We compared single-channel CycleGAN models with and without bony structure constrains (Table 1). Comparing single-channel CycleGAN with the same MR input, adding bony structure constrains can reduce mean absolute error (MAE) of HU calculated in the whole body against pCT. Figure 3 shows bone area improvement on sCTs with bony structure constrains added in the single-channel CycleGAN models.
We also compared single-channel and multi-channel CycleGAN models taking fat and IP images as inputs. The MAE in the entire body contour by multi-channel CycleGAN is smallest compared with other single-channel CycleGAN models, shown in Table 1. After separating the body into bone and soft tissue area, MAE calculated separately on these two regions shows that multichannel CycleGAN has best bone quality that is comparable to single-channel CycleGAN taking fat images as input with bony structure constrains (Table 2). It also shows that multichannel CycleGAN has the best soft tissue quality that is comparable to single-channel CycleGAN taking IP images as input with bony structure constrains. The sCT images generated by multichannel CycleGAN with bony structure constrains are shown in Figure 4, compared to pCT.

Discussion and conclusion

To reduce the large uncertainties of bony structure generation on sCT images by the original CycleGAN model, we leverage Dixon reconstructed MR images as inputs of a multi-channel CycleGAN, with additional supervised loss function to constrain the conversion in bony area to recover missing bones on sCT. The modified multi-channel CycleGAN that takes Dixon constructed fat and IP images as inputs can generate good quality sCT images in both bone and soft tissue areas. Dosimetric evaluation will be studied in the future. The generated sCT image has the potential to be used for accurate dose calculation and patient positioning in MROP planning.

Acknowledgements

No acknowledgement found.

References

1. Jonsson J, Nyholm T, Söderkvist K. The rationale for MR-only treatment planning for external radiotherapy. Clinical and translational radiation oncology. 2019;18:60-65.

2. Zhu J-Y, Park T, Isola P, Efros AA. Unpaired image-to-image translation using cycle-consistent adversarial networks. Paper presented at: Proceedings of the IEEE international conference on computer vision2017.

3. Liu Y, Lei Y, Wang T, et al. MRI-based treatment planning for liver stereotactic body radiotherapy: validation of a deep learning-based synthetic CT generation method. The British Journal of Radiology. 2019;92(1100):20190067.

4. Peng Y, Chen S, Qin A, et al. Magnetic resonance-based synthetic computed tomography images generated using generative adversarial networks for nasopharyngeal carcinoma radiotherapy treatment planning. Radiotherapy and Oncology. 2020;150:217-224.

5. Florkow MC, Zijlstra F, Willemsen K, et al. Deep learning–based MR-to-CT synthesis: The influence of varying gradient echo–based MR images as input channels. Magnetic Resonance in Medicine. 2020;83(4):1429-1441.

6. Maspero M, Savenije MHF, Dinkla AM, et al. Dose evaluation of fast synthetic-CT generation using a generative adversarial network for general pelvis MR-only radiotherapy. Physics in Medicine & Biology. 2018;63(18):185001.

7. Maspero M, Seevinck PR, Schubert G, et al. Quantification of confounding factors in MRI-based dose calculations as applied to prostate IMRT. Physics in Medicine and Biology. 2017;62(3):948-965.

8. Su K-H, Hu L, Stehning C, et al. Generation of brain pseudo-CTs using an undersampled, single-acquisition UTE-mDixon pulse sequence and unsupervised clustering. Medical Physics. 2015;42(8):4974-4986.

9. Tyagi N, Fontenla S, Zhang J, et al. Dosimetric and workflow evaluation of first commercial synthetic CT software for clinical use in pelvis. Physics in Medicine and Biology. 2017;62(8):2961-2975.