Synopsis

Keywords: Image acquisition: Quantification, Transferable skills: Metrology of MRI, Transferable skills: Reproducible research

For Standards documents to be accepted and widely implemented, the following concepts need to be adopted. (1) There needs to be a clear rationale as the basis for each standards document. (2) The developing committees or groups need to define their operating procedures. (3) Open process is important for consensus to be accepted. (4) Definitions and metrology concepts must be standardized. (5) Gaps in published knowledge must be addressed, often by performing the necessary experiments or data collection activities.

1. There needs to be a clear rationale as the basis for each standards document. What aspect of clinical or research activity will be improved by adherence to the standard? What is the use case for each standard, and how much bias and precision is needed, desired or acceptable to meet the use case? Furthermore, do not confuse the intent of a standards document with the distinct objectives of a “white paper”, “best practices” or “educational” document. Avoid the tendency to lapse into text that tries to educate the broad community, or summarize the history, or cover all possibilities and propose the best approach. The aim of a standards document should be “(a) here is the intent, and (b) do the following to meet the intent”.
2. Operating procedures need to be developed. The organization, network and/or committees need to establish their mission, strategies, policies, and procedures. These need to be based on an understanding of the current problem but also be adaptable to address the problem as it evolves over time. Perspectives on the current and future iterations of the task need to be solicited from various stakeholders in many countries, including academic and industry representatives.
3. Open process is important for consensus to be accepted. Open-membership committees and an open process are important for developing consensus and to encourage or facilitate adoption and implementation by various stakeholders. Some Specifications in a standards document will be based on published data or data obtained from groundwork projects carried out by the committee, but many specifications will have to be based on consensus opinion from experts. If committees have closed membership, it is inevitable that some stakeholders will feel their perspective was not represented, and they are less likely to embrace the consensus-based specifications.
4. Definitions and metrology concepts must be standardized. Ambiguities and inconsistencies in terminology and/or the use of metrological and statistical concepts must be resolved.
5. Gaps in Published knowledge must be addressed. “Groundwork projects” may need to be done to understand the sources of variance in imaging interpretations and/or measurements. Examples of groundwork projects include image acquisition of reference objects (known truth), replicated dozens or hundreds of times with different parameters controlled or varied to assess the impact of each. Other projects can involve measurement challenges using a common set of clinical images. Because ground truth is not known within living bodies, evaluation of clinical images cannot assess bias well, but does give a good sense of current levels of variability. In non-ionizing modalities, test-retest studies of patients can be used to characterize variability since bias would cancel out. As with any performance improvement initiative, it is important (1) to select appropriate performance metrics, (2) to have methods and tools to capture those metrics, and (3) to establish baseline performance against which one can measure progress. Groundwork projects can also help with all three of those. However, performing “groundwork projects” to understand the sources of variance requires money – which is hard to get for these kinds of projects. Such repetitive data collection is generally thought of as “testing”, or QA, and not innovation or research, and therefore is generally not funded by typical research grants.

Acknowledgements

No acknowledgement found.

References

No reference found.