KEY TAKEAWAYS
- ‘Overviews of systematic reviews’ are a feature of evidence-based decision making, but are only as strong as the individual reviews they include. Evaluating potential biases and the methodological quality of systematic reviews is therefore crucial.
- A recent article examines 2 recommended systematic review assessment tools, AMSTAR-2 and ROBIS. While both have value, their use requires proper training, time, and know-how.
Synthesising evidence from multiple systematic reviews (also known as conducting an umbrella review or ‘overview of reviews’) can form a key part of evidence-based decision making and treatment guidelines. However, conducting effective ‘overviews of reviews’ requires careful planning to minimise bias, which can be present at either a primary study or individual review level. In a recent BMJ Medicine methods primer, Carole Lunny and colleagues address the challenges of assessing and reporting bias in systematic reviews. The group offer a detailed examination of AMSTAR-2 and ROBIS, two recommended appraisal tools, and provide practical guidance for authors of ‘overviews of reviews’.
AMSTAR-2 versus ROBIS
The group compared key features of each tool.
AMSTAR-2:
- 16-item checklist
- focuses on the methodological quality of systematic reviews of healthcare interventions, including risk of bias
- reportedly favoured for its quick and easy-to-use format
- may be preferred for broad assessment of systematic review quality.
ROBIS:
- domain-based tool
- 19 items, aimed at identifying biases in systematic reviews
- useful for pinpointing concerns in review conduct and assessing relevance
- requires “more thoughtful assessment and time”
- may be preferred for more nuanced assessments, or comparisons of risk of bias across multiple types of systematic reviews.
Standardising ‘overviews of reviews’
The authors call for a standardised approach to ‘overviews of reviews’ to enhance their credibility and value.
Regardless of the appraisal tool used, the authors call for a standardised approach to ‘overviews of reviews’ to enhance their credibility and value. They outline several key recommendations:
- Report methodological quality or bias by item, domain, and overall judgement, focusing on outcomes.
- Discuss risk of bias for each outcome.
- Highlight any individual review methodological quality issues or potential biases as limitations of the ‘overview of reviews’.
- Use ROBIS to subgroup reviews by risk of bias, identifying overemphasised findings and excluding high-risk reviews.
An expanding toolkit
Previously, the launch of PRISMA-S provided much-needed guidance on reporting literature searches within systematic reviews, and Cochrane’s Hilda Bastian proposed solutions to ensure that systematic review protocols were robust. Now, Lunny and colleagues’ primer, and the tools therein, sit alongside initiatives from the LATITUDES Network to form part of a drive to reduce bias in evidence synthesis.
————————————————–
Categories