Synopsis

Keywords: Machine Learning/Artificial Intelligence, Head & Neck/ENT, Super Resolution

Low resolution (LR) balanced steady-state free precession (bSSFP) acquisitions confer decreased acquisition times, reduced patient motion and heating, and increased artifact tolerance due to a decrease in TR. The application of a pre-trained super resolution network to LR bSSFP images of the temporal bone allows for these advantages to be realized, without significantly degrading image quality. In the absence of a matched high resolution image, quality was judged by two radiologists. However, radiologist’s ratings were not in agreement, highlighting the fact that there is no single definition of task-specific image quality, which must be considered when super resolution is performed.

Introduction

In many studies that implement super resolution (SR) models, a low resolution (LR) image is synthesized from a high resolution (HR) acquisition (e.g. ^{1, 2}) to generate a paired set of images that is appropriate for use in training and testing a model. However, synthesis of a LR image does not allow for the advantages that a true LR acquisition may provide such as reduced patient heating, less patient motion, and reduced scan times. Balanced steady-state free precession (bSSFP) is a sequence which has the additional benefit of reduced off-resonance induced banding artifacts when performed at LR, due to a reduction in TR. Application of a SR pipeline to LR bSSFP images of the temporal bone could, therefore, enable fast, artifact-tolerant acquisitions of this complex space, without forfeiting the high resolution required to visualize the fine structures of the inner ear.

The consequence of this approach, however, is that there is no HR “reference” image to compare the model output to. But, while paired HR and synthesized LR datasets make it easy to quantify the performance of a model via calculation of full-reference image quality metrics (IQMs), studies have shown that IQMs fail to show strong correlations to the gold standard of radiologist opinion of image quality or diagnostic utility³. Instead, we aim to examine radiologists’ opinion of images resulting from a super resolution pipeline in comparison to standard interpolation techniques. To do so, radiologists will rate their ability to visualize specific structures of the inner ear in images generated from an “originally” protocoled resolution as well as those from a LR acquisition, both brought to HR with the super resolution pipeline and with standard interpolation techniques.

Methods

Informed written consent was obtained from 10 healthy volunteers (6M, 4F, average age 36 ± 11 years) who were scanned under an NSH REB-approved protocol.

Imaging was performed on the head-only 0.5T point-of-care system from Synaptive Medical⁴. The “original” resolution (OR) acquisitions (FOV 18cm, isotropic resolution 0.6mm, 164 slices, RBW 70kHz, TR/TE 7.0/3.4ms, flip angle 60°, scan time 4min 19s) and LR acquisitions (FOV 18cm, isotropic resolution 0.7mm, 140 slices, RBW 70kHz, TR/TE 6.3/3.1ms, flip angle 60°, scan time 3min 11s) were brought to an isotropic HR of 0.3mm using standard interpolation techniques as well as with a SR pipeline, resulting in OR_Int., OR_Sr, LR_Int., and LR_SrInt. image sets (Figure 1). The SR images were generated with a publicly available model that was pre-trained on natural images⁵, and the weights used were the result of a weighted interpolation between two of the sets provided.

Each participant’s dataset was anonymized and shown to two board-certified neuroradiologists in a randomized manner. Raters were asked to independently rank their ability to visualize the superior semicircular canal (SSC), facial nerve, cochlear nerve, vestibule, and cochlea, as well as their impressions of overall image quality on a Likert scale ranging from 1-5. Cohen’s Kappa was calculated to quantify inter-rater reliability, and Wilcoxon signed-rank tests were used to compare ratings between the OR_Int. and LR_SrInt., OR_Int. and OR_Sr, and LR_Int. and LR_SrInt. image sets for each structure examined. A Bonferroni-corrected P value of < 0.02 was considered significant.

Results and Discussion

A representative dataset from one participant is shown in Figure 2. Inter-rater reliability tests revealed poor to fair agreement for visualizing all structures in all image types (Figure 3). The ratings from each rater are therefore considered as two unique case studies. The results of analysis on ratings from both raters are shown in Figure 4.

Comparisons of rater 1’s ratings yielded no significant differences. In contrast, comparison of rater 2’s ratings yielded significant preference for LR_SrInt. images over OR_Int. images when visualizing the cochlea, vestibule, and for overall image quality. When comparing methods used to bring acquisitions to HR, rater 2 preferred OR_Sr images over OR_Int. images for visualizing the cochlea, vestibule, and for overall image quality. Similarly, LR_SrInt. images were preferred over LR_Int. images for visualizing the SSC, facial nerve, cochlea, and vestibule.

The result that raters did not agree is, itself, an interesting one that often goes overlooked (e.g. ⁶). Post-evaluation interviews with raters revealed that inter-rater variation is conferred by differing noise characteristics between pipelines. This indicates that inter-rater variation, as much as IQM calculations, can be a significant challenge when evaluating image quality.

Neither rater rated the LR_SrInt. images significantly lower than the OR_Int. images for any structure examined, nor for overall image quality, permitting a reduction in scan time of 1min 8s. Notably, the LR protocol maintained the RBW of the original protocol so as to maintain comparable SNR. A further reduction in TR, and therefore scan time, would likely be feasible if this was not a concern.

Conclusions

The use of a SR pipeline permitted the advantages of acquiring a bSSFP image at LR to be exploited, without reducing overall image quality, as rated by two board-certified neuroradiologists. Importantly, however, radiologists’ ratings were not in agreement, demonstrating that while IQMs are not an ideal method for assessing image quality, radiologists’ opinions can show similar disadvantages.