INTRODUCTION

Three-dimensional MRI-based statistical shape models (SSM) can predict radiographic knee osteoarthritis progression¹ and distinguish anterior cruciate ligament injury cases and controls². However, current SSMs require dense corresponding points (matching points continuously over all bones), and may introduce bias based on the reference shape. SSMs learn linearly orthogonal features using principal components analysis, potentially limiting features.

Neural implicit representations are a state-of-the-art method for 3D modeling that learn a continuous signed distance function (SDF) corresponding to a shape’s surface (Figure 1). Generative neural implicit representations (neural shape model) extend this methodology to learn the SDF of a large distribution of shapes allowing a single model to encode the variety of shapes found in the training dataset (distribution). While these models have been shown to reconstruct bones from undersampled data³, it is unclear whether they encode clinically important information, and how they compare to SSMs.

The purpose of this work was to determine whether a neural shape model of the femur can predict knee pain, and to compare its predictive ability to conventional SSMs.

METHODS

Data from the 24 and 48-month visit of the right knee of 562 participants enrolled in the Osteoarthritis Initiative were included⁴. Participants were randomly split evenly into training and testing sets. Sagittal Dual Echo in the Steady State knee MRIs (TE/TR= 5/16ms, flip angle=25^o, field-of-view=140×140mm², in-plane resolution=0.36×0.36mm², slice thickness=0.7mm) of the right knee of each participant were segmented using a validated convolutional neural network⁵.

The 24-month visit of the training data was used to learn a neural shape model of the femur using the autodecoder framework⁶.

A multilayer perceptron (MLP) takes as input the xyz position of an arbitrary point and a shape specific latent vector and predicts the SDF of that point and shape. The framework jointly optimizes the weights of the MLP and the latent representation (length 256) using Equation 1:


$$ \min_{\theta, Z}\sum_{k}^{K}\left(\sum_{i}^{X_k}\mathcal{L}\left(f_{\theta}\left(x_i, z_k\right), s_i\right) + \frac{1}{\sigma^2}MSE\left(z_k\right)\right) \tag{1}$$
where K are the 281 training bones, X_k is a set of randomly sampled points for bone k, f is the MLP and θ are the weights optimized to predict SDF values (s_i) from point (x_i) and latent (z_k) inputs. $$$\mathcal{L}$$$ is the L1 loss and the second term is the mean squared error (MSE) used to learn an independent covariance structure.

To obtain latent representations for training and testing pain prediction models, the learned neural shape model was fit to the 48-month training and testing data using Equation 2:

$$z_{recon} = \min_{z}\sum_{i}^{X}\mathcal{L}\left(f_{\theta}\left(x_i, z\right), s_i\right) + \frac{1}{\sigma^2} \|z\|_{2}^2\tag{2}$$
where the MLP parameters (θ) are unchanged, and the latent vector is optimized to predict the SDF values of the bone’s vertices.

To test the model’s ability to learn clinically meaningful information, its ability to predict current self-reported pain was tested. The pain group had > 6-months of pain in the past 12-months and the no-pain group had no pain in the past 12-months⁷. After extracting pain outcomes, there were 111 participants in the pain training dataset and 92 in the pain testing dataset. We trained 4 machine learning classifiers (logistic regression, gradient boosting, random forests, and naïve bayes) to predict pain using the learned 256-dimensional latent representation as input. To compare to prior work, we also used the 90-features of a previously described femur SSM trained with and without cartilage thickness⁸. To determine the effect of training dataset size, we fit each model using 100%, 75%, 50%, and 25% of the training data. Confidence intervals on predictive ability were generated by bootstrapping each model and dataset size 100 times without replacement. Model performance was assessed by calculating area under the receiver operating characteristic curve (AUROC), sensitivity, and specificity on the testing data.

RESULTS

Demographics for participants divided by split are outlined in Table 1. Examples of the learned representation are included in Figure 1. For each dataset size, neural shape models performed best with AUROC of 0.62, 0.66, 0.68, and 0.70 for the 25, 50, 75, and 100% dataset sizes, respectively (Table 2). The best fitting model used 100% of the labelled data, logistic regression and had an AUROC of 0.70 and sensitivity of 0.89.

DISCUSSION

Generative neural shape models of bone learned latent spaces that predict knee pain better than a traditional bone SSM as well as a bone + cartilage SSM. This model required only one step, simultaneously generating correspondence, and encoding clinically relevant features. As expected, increasing training data improved pain predictions. According to AUROC, the neural shape model had comparable performance to the best performing SSMs using 25-50% less training data. The addition of cartilage to the bone only SSM worsened its predictions; this is in-line with a deep learning study where a bone+cartilage model reduced AUROC values by 3% compared to a bone-only model⁷. It is unclear why cartilage worsens predictions. Nonetheless, the neural shape model achieves comparable predictions to these deep learning models, using 1-2 orders of magnitude less data. Visualization of the pain predictions enabled by the neural shape model (Figure 3) shows that osteophyte formation is correlated with pain.

CONCLUSION

Neural shape models learn clinically meaningful bone information. These shape models outperform traditional SSMs for predicting pain, a challenging and clinically important task.

Acknowledgements

This work was supported by the National Institutes of Health (R01 AR077604, R01 EB002524, AR079431, P41 EB027060, and K24 AR062068), the Wu Tsai Human Performance Alliance, and a CIHR Postdoctoral Fellowship.