Synopsis

Keywords: Image Reconstruction, MSK

Image reconstruction and downstream tasks have typically been treated independently by the image processing community, but we hypothesized performing them end-to-end could facilitate further optimization. To these ends, UCSF organized the K2S challenge, where challenge participants were tasked with segmenting bone and cartilage from 8X undersampled knee MRI acquisitions. Top challenge submissions produced high-quality segmentations maintaining fidelity to ground truth, but strong reconstruction performance proved not to be required for accurate tissue segmentation, and there was no correlation between reconstruction and segmentation performance. This challenge showed reconstruction algorithms can be optimized for downstream tasks in an end-to-end fashion.

INTRODUCTION

In knee joint degenerative imaging, MRI has become the modality of choice, with acquisition time reduction and development of postprocessing techniques to extract information from scans emerging as research directions. Thus far, image reconstruction and postprocessing algorithms have been developed independently with few exceptions¹. For reconstruction, algorithms predict full-length acquisition image appearance from undersampled k-space and are generally optimized for standard metrics: normalized root mean square error (nRMSE)², peak signal-to-noise ratio (PSNR)³, and structural similarity index (SSIM)⁴. Although these metrics have limited correspondence to radiologist annotations⁵, perceptual correspondence with fully sampled images has made them metrics of choice when evaluating reconstruction algorithms—namely, they are often optimized for because reconstructed images are intended for human interpretation. If reconstruction and postprocessing were instead viewed as an end-to-end pipeline, reconstructed images would be features for subsequent algorithm input rather than human viewing, possibly leaving additional room for downstream task optimization. This was the rationale behind the K2S challenge, hosted by UCSF researchers at the 25th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), in which challenge participants were tasked with segmenting bone and cartilage from 8X undersampled MRIs.

METHODS

Image Acquisition
3D-Fast-Spin-Echo fat suppressed CUBE images were acquired at a UCSF GE Signa 3T MRI scanner, and an in-house pipeline was developed that leveraged GE Orchestra 1.10 to reconstruct images from raw scanner data and store multicoil k-space. Acquisition parameters were as follows: FOV=15cm²; acquisition matrix=256×256×200; ±62.5kHz readout bandwidth; TR=1002ms; TE=29ms; ARC acceleration by a factor of 4⁶.
Ground Truth Segmentations
A 5-class 3D V-Net pipeline was trained to segment patellar, femoral and tibial cartilage⁷, while a 2D UNet pipeline was trained to segment the patella, femur and tibia⁸. Predicted segmentations were postprocessed and graded for quality by in-house radiologists using a 5-point LIKERT scale. Volumes with the top 350 segmentation scores across bone and cartilage were selected for the K2S dataset, with the corresponding radiologist-approved segmentations being designated as ground truth (Fig. 1).
K2S Challenge
On April 15, the K2S training dataset of 300 ARC-reconstructed fully sampled k-space, segmentations, and 8X center-weighted Poisson undersampling mask with a fully sampled 5% central square in k_y-k_z was released. Challenge participant recruitment ended April 30, with 87 teams registering. The test set of 8X undersampled k-spaces (n=50) was released July 6, with 12 submissions received (Fig. 1). Submissions were evaluated using a weighted sum of the dice similarity coefficient (DSC)⁹ for each of the 6 tissue compartments, where compartment weightings were inversely proportional to tissue size. Submissions from the top 4 teams were analyzed further, and each was asked to submit reconstructed image intermediates from their pipelines for additional analysis.
Top Submissions
K-nirsh pretrained separate 3D nn-UNet architectures¹⁰ that reconstructed coil-combined images from zero-filled coil-combined image space inputs, and segmented bone and cartilage. These pretrained architectures were then fine-tuned end-to-end, using a loss function scheduler that gradually increased the cartilage segmentation loss penalty. UglyBarnacle implemented a compressed sensing approach with combined total variation and L₁-wavelet regularization to reconstruct undersampled images, and a UNet-style architecture to segment reconstructed images. FastMRI-AI used zero-filled root sum-of-squares coil-combined initializations as input for a 3D Single-Attention UNet¹¹ which was trained for segmentation in a patch-based approach, affording cartilage double the weight of bone in the loss function. Finally, NYU-Knee AI trained a variational network to reconstruct coil-combined images from multicoil undersampled inputs and trained separate UNets to segment bone and cartilage from reconstructed images.

RESULTS

Segmentation performances and reconstruction metrics are in Fig. 2: K-nirsh had the best segmentation each tissue compartment, but interestingly, the poorest reconstruction metrics. Fig. 3 shows reconstructions and overlaid segmentations for an example patient, confirming the discordance between reconstruction and segmentation performance. Furthermore, plots of weighted DSC against SSIM and PSNR on a per-volume basis in Fig. 4 show essentially no correlation between reconstruction and segmentation performance the top submissions.
Cartilage thickness was calculated on a per-patient basis for ground truth and submissions¹² to assess the viability of submitted segmentations for biomarker analysis. 3 of the top submissions showed strong correlations between predicted and ground truth cartilage thickness while Bland-Altman plots showed 3 of the top submissions to exhibit essentially no bias in predicted cartilage thicknesses (Fig. 5).

DISCUSSION AND CONCLUSIONS

From the top 4 submissions, cartilage and bone segmentation quality was strong, yielding high DSCs for the 6 tested tissue compartments. Interestingly, strong intermediate pipeline reconstructions were not prerequisites for high-quality segmentation—the top-performing segmentation pipeline had the poorest reconstruction metrics, demonstrating the features desired for human interpretation can differ substantially from those optimal for subsequent algorithm processing. Biomarker analysis in assessing correlations of cartilage thickness between submissions and ground truth showed, however, UglyBarnacle had slightly higher correlations than K-nirsh in femoral and tibial cartilage despite slightly lower segmentation DSCs, illuminating further room for task-specific optimization.
Reconstruction and postprocessing pipelines have largely been viewed independently, but this challenge shows room for optimization when they are viewed as an end-to-end task. Future work can include similar end-to-end optimization for tasks such as anomaly detection, prognosis prediction, and others directly from undersampled k-space.

Acknowledgements

We would like to acknowledge Bruno Astuto Arouche Nunes for developing the multiclass knee cartilage and menisci segmentation pipeline. We acknowledge Andrew Leynes for developing a script that was adapted to automatically transfer raw scanner data from the 3T GE Sigma Scanners to local file systems. We also acknowledge Misung Han and Emma Bahroos for optimizing the FS CUBE sequence acquisition parameters used in this story. Furthermore, we acknowledge Leynes, Jon Tamir, Nikhil Deveshwar, and Peder Larson for assistance in selecting undersampling patterns to use for training algorithms. Additionally, we acknowledge Peter Storey, Jed Chan, Warren Steele, Rhett Hillary, Adam Stowe, and Scott Matsubayashi for their assistance in developing an infrastructure we could to host the considerable amount of data we made available for this challenge, and for developing a system challenge participants could use to gain access to the data. Lastly, we acknowledge Amy Becker for assistance in creating and maintaining the website for the K2S challenge. We thank MICCAI for giving us the opportunity to host this challenge and workshop at the 2022 conference in Singapore. Finally, we also would like to acknowledge our funding source NIH R01AR078762.