Introduction

Assessment of scapular bone morphology is essential in the surgical planning for multiple shoulder pathologies, such as instability and osteoarthritis. Three-dimensional (3D) models are considered the most useful planning tools for surgeons as they provide improved conceptualization and accurate determination of glenoid orientation and bone loss. Recently, 3D magnetic resonance imaging (MRI) models of the shoulder have emerged as an equally effective tool as 3D computed tomography (CT) in assessing bone morphology¹. This study, however, utilized high-resolution MRI sequences, which are not commonly used in clinical practice. The purpose of this work is to create a task-based super resolution (TBSR) deep learning approach capable of generating patient specific 3D model of the scapula bone, from a 2D clinical MRI sequence.

Data

MRI scans from 45 subjects (sex: 16 female/29 male, age: 51.93±14.94 years, body mass index (BMI) 26.34±4.60 kg/m2), who had a 3D high-resolution fat-saturated (FS), T2-weighted axial shoulder MRI sequence (1 mm slice thickness, 0.375x0.375 mm in-plane resolution) as well a T2 FS 2D clinical proton density (PD) axial shoulder MRI sequence (3 to 4 mm slice thickness, and in-plane resolution ranging 0.2344-0.293x0.2344-0.293 mm), acquired between March 2018 and February 2021, were identified through our Picture Archiving and Communication System. Table 1 and Table 2 summarize adopted MRI protocols and population demographics. Patient diagnoses included varying shoulder conditions such as partial or complete rotator cuff tear, instability, arthritis, and tendinosis. Scans were excluded if metal implants, suspected infection, or tumor were present. The scapula of each high-resolution MRI was manually segmented with custom, in-house developed software by an expert trained researcher. Two MRIs were manually segmented by an additional orthopedic surgeon, with a 0.86 inter-rater reliability score. Data pre-processing included resizing of the 2D clinical to the 3D high-resolution scan size, as well as landmark-based affine and rigid registration, followed by a cropping step around the scapula bone, using the manual bone annotations, to ensure coverage of the same field-of-view between scan pairs.

Methods

To devise an automatic 3D super-resolution scapular segmentation framework from 2D clinical scans, the cohort was distributed in training, validation, and test set splits, comprising 35, 6, and 4 patients respectively. Inspired by previous work in MRI reconstruction², we cast the current problem to a multi-task learning problem. Figure 1 depicts the proposed architecture. The network is trained to output 3D high-resolution axial shoulder MRI and bone segmentation map, from input 2D clinical proton density axial sequences. The network has the structure of a 2D fully convolutional encoder-decoder, where both tasks share an encoding path. The decoder includes two task-related branches which exchange features through inter-task skip connections, which provide the segmentation branch with fine-detail features relevant to the task. Conversely, the super-resolution branch receives features directly through the encoder via skip connections. The model was trained using a multi-task loss function comprising a multiscale structural similarity index metric (SSIM), mean absolute error and DICE loss. Super resolution (SR) and segmentation terms were equally weighted. Adam optimizer with learning rate 1x10-5, 16 samples per batch, 0.05 dropout rate and early-stopping with 15 validation iterations without segmentation improvements were also used. Data are processed per slice and at inference time the outputs are re-assembled to form the 3D scapula model. Segmentation post-processing involves volume-wise extraction of the largest connected component. For comparison, we trained two additional models which performed segmentation directly from the low resolution 2D (LR-to-Seg) and high-resolution 3D (HR-to-Seg) scans respectively, which served as desired segmentation lower and upper bound respectively.

Results

Mean point to surface distance (MPTS) was computed to evaluate the model’s segmentation performance on the test set. The metric was calculated by comparing the predicted bone segmentation mask and the manual annotation. MPTS was 1.83±0.46 mm. Note that the slice thickness of the clinical MRI is 4 mm. For comparison, LR-to-Seg performance was 2.48±0.94 mm MPTS, whereas HR-to-Seg performance was 0.81±0.08 mm MPTS. Statistically significant difference (p<0.05) was observed when comparing the performance of our super resolution approach against the LR-to-Seg as well as HR-to-Seg against LR-to-Seg. Figure 2 depicts a point-to-point surface distance between the scapula bone model produced by the proposed approach and the manually annotated bone model. Example of the automatic high-resolution segmentation is shown in Figure 3.

Discussion

Task-based super resolution allows to produce accurate segmentation from clinical 2D MRI scans and achieves errors below the slice thickness parameter. Looking at Figure 3, the super-resolution approach leads to smoother identification of the scapula bone, which is especially visible when comparing manual to automatic segmentation in the sagittal and coronal views. While quantitative results are encouraging, they are somewhat affected by minor image registration misalignments.

Conclusions

We have generated highly accurate 3D scapular bone models from standard clinical T2 sequences using a fully-automated deep learning approach. This technique will be invaluable for surgeons in the pre-operative planning of patients with shoulder instability and osteoarthritis by being able to accurately determine glenoid bone loss, wear pattern, inclination, and retroversion. Furthermore, this method avoids the need to undertake the additional cost, time, and radiation associated with obtaining a CT scan in a patient who already has an adequate MRI scan.

Acknowledgements

This work was supported by the NIH R01AR078762 grant.