Introduction

There is an urgent need for better imaging biomarkers for progressive multiple sclerosis (MS)¹. The Secondary and Primary Progressive Ibudilast NeuroNEXT Trial in Multiple Sclerosis (SPRINT-MS) was a phase II trial of ibudilast that used changes in Brain Parenchymal Fraction (BPF), a measure of whole-brain atrophy, as a primary outcome and found a significant treatment effect². Secondary outcomes included transverse diffusivity (TD), a candidate biomarker for MS. The objective of this work is to assess the impact of equipment changes, a common confound in imaging-based clinical trials. We examine the degree to which hardware changes influenced BPF, TD and the overall outcomes of the trial.

Methods

The trial adhered to ethical guidelines in the Declaration of Helsinki³. All patients provided written informed consent. The trial was multicenter, including 27 scanning sites, and was longitudinal, with imaging occurring at baseline, 24, 48, 72 and 96 weeks⁴. Of the 255 patients who underwent randomization, 244 received at least one dose of study medication and completed more than one scan session. Major changes in scanner hardware affected the data of 39 subjects. Table 1 summarizes the prevalence of each change. BPF and TD were measured as described previously⁴.
The original statistical analysis plan for the SPRINT-MS trial did not account for hardware changes in the analysis. A linear mixed effects model was used to estimate the rate of change in BPF or TD, adjusted for randomization strata. The analysis was modified to assess the impact of hardware changes in four ways: 1) “Exclude:” data acquired after a major hardware change were excluded from the models and the treatment effect estimated. 2) “Binary:” the original model was modified to include hardware change as a time-dependent binary yes/no covariate. 3) “Type:” the original model was altered to include type of hardware change as a time-varying covariate. To assess whether the model goodness-of-fit was improved by including an effect of hardware change, the Akaike information criterions (AICs) were compared to the original model. Statistical analyses were performed in SAS 9.4 Software (SAS Institute, Cary, NC). As this is an exploratory study, no corrections for multiple comparisons were performed.

Results

Table 2 summarizes the impact of accounting for hardware changes on overall outcomes. For BPF, accounting for hardware changes had only a slight impact on the treatment effect. For example, the original treatment effect of 0.00089 BPF units per year changed by 5.6% after accounting for the specific type of hardware change. For TD, accounting for hardware changes had a larger impact. The treatment effect changed by 16% after accounting for the specific type of hardware change. In comparing the model goodness-of-fit to that of the original unadjusted models, values of AIC (table 2) suggest that adjusting for hardware changes in general led to an improvement. The model adjusting for scanner upgrade type, which allowed for each scanner upgrade type to have a different effect, was optimal. Table 3 shows the impact of type of hardware change on imaging measures. Values shown are systematic differences between imaging measures from a particular type of hardware change versus imaging measures not affected by a hardware change. Shifts in values with hardware change are large when compared with the annual changes seen in table 2. However, scanner changes but did not affect overall outcomes. Analysis including a time by treatment by type of change term found no significant interaction between type of scanner change and treatment effect for BPF or TD (table 4, p > 0.5).

Discussion

In this analysis, we found that hardware changes had little impact on the overall outcomes of the longitudinal trial, but that there were changes in BPF and TD associated with hardware change. This study has implications for the design of imaging-based longitudinal clinical trials, where scanner upgrades are expected as a matter of course. Inclusion of a covariate to account for hardware changes is a standard approach14. The results of this study are also important because they were acquired within the context of a clinical trial. The estimates of hardware-related effects therefore have realistic magnitudes for planning future trials.

Conclusion

Because imaging hardware changes can be expected in imaging-based clinical trials, anticipating the impact of such changes on the trial outcomes is prudent. Accounting for the hardware change can be important, but the specific type of imaging metric and hardware change should be considered.

Acknowledgements

his work was supported by grants from the National Institute of Neurological Disorders and Stroke (U01NS082329) and the National Multiple Sclerosis Society (RG 4778-A-6) and by MediciNova through a contract with the National Institutes of Health.