4 Discussion
No relevant research was found in an extended search on April 23th, this was the first study to systematically evaluate the quality of COVID-19 guidelines with the AGREE II instrument. The overall quality of the 20 guidelines for COVID-19 was highly variable, and significant variability can be seen across domains within guidelines.
Domain scores of scope and purpose in most guidelines were below 60%, indicating that the inclusion and exclusion criteria of target populations and health question remained unclear for early published guidelines.
Stakeholder involvement domains were poorly described, making those guidelines less professional. The involvement of patients in making decision might improve clinical outcomes and increase patients’ guideline adherence2. Therefore, collecting the views and preferences of target populations is a necessary part of standard guideline, and needs a period of time to carry out. However, considering over 60000 people died, and COVID-19 continues to spread worldwide, no guideline described related information in this item. More attention are ought to paid on the composition of the guideline development team and target population preferences by local and regional guideline developers20.
Systematic reviews are expected to form the basis of high quality of CPGs21. The importance had been demonstrated by the guideline manuals published by WHO22. No guideline involved review of outside methodological or health economic experts, which explained all guidelines scored below 60% in the rigor of development domain and suggested a fundamental methodology problem with these CPGs.
Only six guidelines had moderate to high scores in the clarity of the recommendations. This might result from the unclear audience and low scores in stakeholder and rigour of development domain in documents issued by government organizations. CPGs developers should provide a concrete and precise description of different options in different situations, as informed by the body of evidence, and make recommendations easily identifiable to the first-line clinicians audience.
In the applicability domain, most guidelines scored 0%, and did not describe the facilitators and barriers to their application and definite audit criteria. Some CPGs even lacked potential resource and educational tools of applying the recommendations, which might have a great influence on the speed and spread of adoption of guidelines. This finding is inversely proportional to the need of developing guidelines for user-friendliness and clarity suggested in some studies4, 23 and should be taken into more consideration when developing new guidelines or updating new versions.
Information on the domain of editorial independence was also neglected in most guidelines. This is particularly important given that influences of the founding body and conflicts of interest are the most common source of bias in guideline development23, 24. Perhaps some guideline developers did not realize the significance of editorial independence disclosures and management. Studies showed that financial conflicts of interest were prevalent among CPGs in a variety of clinical areas24, and some evidence suggested that such financial conflicts of interest might have an influence on guideline recommendations25. Therefore, the guideline developers can’t emphasize editorial independence domain enough.
Guideline 27,38 and 1914 can be recommended to guide clinical practice, while 116, 49, 1813, 2015 were recommendable with modification. In view of the urgency of the COVID-19 outbreak, the lower quality and limitations of the early published guidelines are understandable and acceptable. But they are expected to update more evidence-based medical recommendations and modify nonstandard methodology of guidelines to increase their credibility and applicability promptly. And guidelines developers are supposed to focus on not only high speed but also rigorous quality, especially in stakeholder involvement, rigour of development, applicability, and editorial independence domain.
This study had limitation as follows, first, the AGREE II instrument established an appraisal system for methodological quality of guidelines, but the evaluation of guideline recommendations was not stated. Second, the excluded guidelines concerning mainly about traditional Chinese medicine, the next research to work on in our team, might cause this study not to be representative of all CPGs.
In conclusion, the overall quality of CPGs for COVID-19 was uneven. Further research is needed for the appraisal of guideline recommendations. The results of our study could contribute to improve development of future guidelines, and affect the reasonable selection and use of guidelines in clinical practice.