Strengths and Limitations
In addition to assisting trialists with selecting a holistic set of outcomes while designing clinical trials, the generated heatmap serves as an easy-to-interpret visual illustration that can be appended to any systematic review, giving readers quick insight into what outcomes are not properly represented by the studies of interest. This will assist systematic reviewers and decision-makers with assessing the quality of outcome reporting, in tandem with the assessment of the conduct of clinical trials, thereby enabling them to make more informed assessments regarding new interventions. The main limitation to the tool is that it involves an element of subjectivity to determine what constitutes comprehensiveness. Since it is the responsibility of the user to define what the standard is, through a guided process on the ‘Master Sheet’ of the tool, the heatmap output must be interpreted in the context of the user’s defined benchmarks. To ensure the identification of outcomes is truly relevant, it is recommended that the tool is used by individuals with expertise in the conditions they are investigating and have extensive insight into what outcomes would be important in making tangible changes. This level of subjectivity is not exclusive to our tool, however, and is a fairly consistent attribute of many assessment schemes, such as Cochrane’s ROB tool34 and the Newcastle-Ottawa Scale for assessing the quality of observational studies18. This limitation notwithstanding, the COR tool fills a unique void that is not addressed by recommended ROB tools, and unlikely to be addressed by the development of COS.