Quality of Evidence
For the purposes of recommendations made by the ASRT Practice Standards Council, the definition of “quality of evidence” is confidence in the quantity and quality of evidentiary documentation used to make a recommendation.
The GRADE working group provided the following guidance in assigning a measure of quality in evidence for decisions and/or opinions on provision of healthcare. The following guidance, as adapted from GRADE Working Group, should be used by the Practice Standards Council in assigning a measure of quality in evidence for all resources used in formulating advisory opinion statements:
High = further research is very unlikely to change our confidence in the estimate of effect;
Moderate = further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate;
Low = further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate;
Very low = any estimate of effect is very uncertain.1
Strength of Recommendation
For the purposes of recommendations made by the ASRT Practice Standards Council, the definition of “strength in recommendation” is recommendations to perform, or not perform, a task, should be based on the tradeoffs between benefits on the one hand, and risks, burden and, potentially, costs on the other as supported by evidentiary documentation such as current literature, curriculum, position statements, scopes of practice, laws and federal and state regulations. If benefits outweigh risks and burden as supported by evidentiary documentation, the Practice Standards Council will recommend that medical imaging and radiation therapy professionals are qualified to perform a task. The uncertainty associated with the tradeoff between the benefits and risks and burdens, as supported by evidentiary documentation, will determine the “strength of recommendations”.
The GRADE working group provided the following guidance in assigning a strength of recommendation for decisions and/or opinions on provision of healthcare. The following guidance, as adapted from GRADE Working Group, should be used by the Practice Standards Council in assigning a strength of recommendation for all advisory opinions:
Strong = Based on the available evidence, if the Practice Standards Council is very certain that benefits do, or do not, outweigh risks and burdens they will make a strong recommendation.
Weak = Based on the available evidence, if the Practice Standards Council believes that benefits and risks and burdens are finely balanced, or appreciable uncertainty exists about the magnitude of benefits and risks, they must offer a weak recommendation. When, across the range of potential scenarios in which the task will be performed, fully informed medical imaging or radiation therapy professionals or the institutions these professional work for are liable to make different choices, the Practice Standards Council should offer weak recommendations. 1
Grading under any system is complex, and the Practice Standards Program manager has taken measures to learn the GRADE process and to help with this process. Although we do not expect you to grade recommendations yourselves, we expect that you will need to interact with the Practice Standards Program manager to work through questions that arise about the appropriate grades for the recommendations made by the Practice Standards Council.
As you work with the Practice Standards Program Manager on grading, please keep the following in mind:
- The Practice Standards Council should always make a recommendation about the issue brought forward. Just as a consultant’s note would never conclude “given the lack of data from trials I can make no recommendation about how to treat this patient” similarly the Practice Standards Council should never fail to make a recommendation just because there is no high quality evidence. As a knowledgeable author of the topic, you have more expertise on which to base such a decision than the reader.
- A weak recommendation is still a recommendation. If there is concern whether a recommendation should be strong or weak, it is better to err on the side of making it a weak recommendation.
- Grade High quality in evidence will be uncommon for most things the Practice Standards Council recommends. Observational evidence is nearly always grade very low unless the estimates of effect are very large. For instance, if a recommendation is made based on the practice at one facility in one state, it should be given a grade very low for quality in evidence.
References:
- GRADE working group FAQs. 2009. Available at gradeworkinggroup.org
- GRADE: an emerging consensus on rating quality of evidence and strength of recommendations BMJ 2008;336:924-926 (26 April). Available at http://www.bmj.com/cgi/content/full/336/7650/924