Introduction

Quantitative compositional MRI plays a central role in Osteoarthritis(OA) research, offering imaging biomarkers that probe the biochemical composition of the articular cartilage¹. While several studies are still limited to analyzing average relaxation time values of manually defined cartilage compartments², there is growing interest in exploring techniques for the automatic analysis of spatial distribution and extraction of relaxometry patterns able to characterize subjects and predict disease progression. Extraction of second order statistical information, or texture analysis^3-5, has been widely used to overcome the limitation of the average ROI-based approaches. However, texture analysis does not address the problem of studying local differences between two groups, and does not allow for the extraction of salient patterns that could characterize cartilage degeneration in a data-driven way. Automation of post-processing pipelines and more effective feature extraction techniques are needed to exploit the real potential of compositional MRI techniques which is, to date, still hampered by tedious and time-consuming manual image post-processing pipelines; and deeply underused due to the handcrafting of overly-simplistic image representations.

Deep Learning applied to MSK Quantitative MRI: Data-Driven Feature Extraction

In the last few years, Computer Vision has been drastically accelerated by the usage of machine learning techniques^6-7. With the large availability of annotated data and processing power, supervised learning can today accomplish challenges never demonstrated before using the concepts of transforming data to knowledge by the observation and computational interpretation of said examples^8-11. Additionally, a completely new concept has changed the machine-learning field; supervised feature extraction. Conventional machine learning techniques were limited in their ability to process natural data in their raw form. For decades, constructing a pattern-recognition or machine learning system required careful engineering and considerable domain expertise to design feature extractors able to transform raw data, such as the pixel values of an image, into a suitable internal representation or feature vector from which the classifier could detect patterns in the input. In contrast, representation learning allows a machine to be fed with raw data and to automatically discover the best representations of the information hidden in the data needed to accomplish a specified task¹². Deep learning neural networks are a representation learning method characterized by the usage of multiple simple, non-linear units to build several interconnected layers. Each layer aggregates the information at increasing levels of abstraction starting with simple image elements, such as edges or contrast, to more complex and semantic aggregations, such as uncovering latent patterns able to accomplish pattern recognition tasks¹³. The key aspect of deep learning is that these models are not designed to solve a specific task, but are adaptive to the specific problems, learning directly from the data and using a general-purpose learning procedure. Deep learning models were recently applied to solve joint tissues segmentation tasks^14-15 and relaxation time feature extraction^16-17 showing promising results in overcoming traditional post processing techniques.

Integration and Analysis of Multidimensional Data: Topological Data Analysis to Study OA

While OA is known as a whole joint disease, the data analytics more commonly used are often limited to either univariate analysis or a subset of the variables. However, innovations in the big data analytics field have brought several multidimensional visualization methods with which we can compare individual patients as a ‘point-cloud’ in multidimensional space, overcoming the inherent limitations of single endpoints. One possible approach considers the usage of data shape or topology¹⁸. Topological data analysis (TDA) explores relationships between data and data shapes. The fundamental idea of TDA is that this method acts as a geometric approach to pattern recognition within data. By extracting fundamental shapes (patterns) in high-dimensional data TDA provides visualization that can provide novel insights about the data and can identify meaningful sub-groups. Extracting patterns via shape data analysis is possible due to three key ideas: 1) coordinate free description, 2) invariance to small deformations and 3) compactness. The first permits the generalizability of the insights extracted from the shape of complex data. The second guarantees robustness to noise while the last allows complex data to be represented in a simple way. These characteristics make TDA a valuable tool for a combined analysis of all the OA features simultaneously, ultimately leading to a more accurate study of risk factors, pathogenesis and natural history of OA and disease phenotyping¹⁹.

Conclusions

In this lecture, recent innovations in the fields of image processing, machine learning and multidimensional/multimodal data analysis to study musculoskeletal disorders will be presented with the aim of discussing the potentials of coupling cutting edge technologies in quantitative compositional MRI and machine learning fields to discover new tools able to automate post-processing, better characterize disease status, and predict progression.

Acknowledgements

I received funding from NIH-NIAMS K99AR070902 (VP) and GE Healthcare A128218 (SM/VP)

References

1. Li X, Majumdar S. Quantitative MRI of articular cartilage and its clinical applications. Journal of magnetic resonance imaging 2013;38:991-1008. 2. Joseph GB, McCulloch CE, Nevitt MC, et al. A reference database of cartilage 3T MRI T2 values in knees without diagnostic evidence of cartilage degeneration: data from the osteoarthritis initiative. Osteoarthritis and Cartilage 2015;23:897-905. 3 Carballido-Gamio J, Stahl R, Blumenkrantz G, Romero A, Majumdar S, Link TM. Spatial analysis of magnetic resonance T1rho and T2 relaxation times improves classification between subjects with and without osteoarthritis. Med Phys 2009;36(9):4059-4067. 4 Schooler J, Kumar D, Nardo L, McCulloch C, Li X, Link TM, Majumdar S. Longitudinal evaluation of T1rho and T2 spatial distribution in osteoarthritic and healthy medial knee cartilage. Osteoarthritis Cartilage 2014;22(1):51-62. 5 Williams A, Winalski CS, Chu CR. Early articular cartilage MRI T2 changes after anterior cruciate ligament reconstruction correlate with later changes in T2 and cartilage thickness. J Orthop Res 2016. 6 Kallenberg M, Petersen K, Nielsen M, Ng AY, Diao PF, Igel C, et al. Unsupervised Deep Learning Applied to Breast Density Segmentation and Mammographic Risk Scoring. Ieee Transactions on Medical Imaging 2016; 35: 1322-1331. 7 Lee H, Grosse R, Ranganath R, Ng AY. Unsupervised Learning of Hierarchical Representations with Convolutional Deep Belief Networks. Communications of the Acm 2011; 54: 95-103. 8 Li XG, Hong CF, Yang YN, Wu XH. Deep Neural Networks for Syllable based Acoustic Modeling in Chinese Speech Recognition. 2013 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (Apsipa) 2013. 9 Hinton G, Deng L, Yu D, Dahl GE, Mohamed AR, Jaitly N, et al. Deep Neural Networks for Acoustic Modeling in Speech Recognition. Ieee Signal Processing Magazine 2012; 29: 82-97. 10 Pan J, Liu C, Wang ZG, Hu Y, Jiang H. Investigation of Deep Neural Networks (Dnn) for Large Vocabulary Continuous Speech Recognition: Why Dnn Surpasses Gmms in Acoustic Modeling. 2012 8th International Symposium on Chinese Spoken Language Processing 2012: 301-305. 11 Leung MKK, Delong A, Alipanahi B, Frey BJ. Machine Learning in Genomic Medicine: A Review of Computational Problems and Data Sets. Proceedings of the Ieee 2016; 104: 176-197. 12 LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521: 436-444.13 Bengio Y, Lee H. Editorial introduction to the Neural Networks special issue on Deep Learning of Representations. Neural Networks 2015; 64: 1-3. 14 Liu F, Zhou Z, Jang H, Samsonov A, Zhao G, Kijowski R. Deep convolutional neural network and 3D deformable approach for tissue segmentation in musculoskeletal magnetic resonance imaging Magn Reson Med. 2017 Jul 21. 15 Norman B, Pedoia V, Majumdar S, Deep Learning Convolutional Neural Networks for Knee Multi-Tissue Automatic Morphometry and Relaxometry, 2nd Conference on Machine Intelligence in Medical Imaging (C-MIMI), Society of Informatic Imaging In Medicine (SIIM), Baltimore, September 2017 16 Ashinsky BG, Bouhrara M, Coletta CE, Lehallier B, Urish KL, Lin PC, Goldberg IG, Spencer RG1.Predicting early symptomatic osteoarthritis in the human knee using machine learning classification of magnetic resonance images from the osteoarthritis initiative. J Orthop Res. 2017 Oct;35(10):2243-2250. 17 Pedoia V, Neumann J, Heilmeier U, Haefeli J, Ferguson A, Link T, Majumdar S, Predicting Osteoarthritis Radiographic Incidence by Coupling Quantitative Compositional MRI and Deep Learning. 25th Annual Conference of International Society of Magnetic Resonance in Medicine (ISMRM), Honolulu, April 2017. 18 Lum PY, Singh G, Lehman A, Ishkanov T, Vejdemo-Johansson M, Alagappan M, et al. Extracting insights from the shape of complex data using topology. Sci Rep 2013; 3: 1236. 19 Pedoia V, Haefeli J, Morioka K, Teng HL, Nardo L, Souza RB, Ferguson AR, Majumdar S. MRI and biomechanics multidimensional data analysis reveals R2 -R1ρ as an early predictor of cartilage lesion progression in knee osteoarthritis. J Magn Reson Imaging. 2017 May 4.