Assessing quality of randomized trials supporting guidelines for laparoscopic and endoscopic surgery

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Background Recent studies have highlighted the risk of bias and the fragility of results in randomized controlled trials (RCTs). The aim of our study was to evaluate the clinical practice guidelines created by the Society for Gastrointestinal and Endoscopic Surgeons (SAGES) for fragility, statistical power, and risk of bias. Materials and methods We screened the SAGES clinical practice guideline references for qualifying RCTs. RCTs were assessed for risk of bias using the Cochrane Collaboration Risk of Bias tool 2.0. We used the fragility index and fragility quotient to evaluate the robustness of trial results and conducted a power analysis using G*Power to determine if trials were adequately powered. Results Twenty-two (40.7%) of the 54 trials that we assessed were rated as having a high risk of bias, 17 (31.5%) were rated as having a low risk of bias, and 15 (27.8%) were rated as having some concerns. The median fragility index was 2.5 (interquartile range 1-7). The median fragility quotient was 0.021 (interquartile range 0.003-0.045). Mean sample size was 108, and the mean loss to follow-up was eight patients. Eight of 33 trials (24.2%) were found to be underpowered according to the sample size used in the primary outcome. Conclusions Guidelines created by SAGES are supported by RCTs that are frequently fragile or underpowered or have a high risk of bias. Future RCTs should utilize the Consolidated Standards of Reporting Trials statement, implement strategies to minimize loss to follow-up, and use properly powered sample sizes.

Original languageEnglish
Pages (from-to)233-239
Number of pages7
JournalJournal of Surgical Research
Volume224
DOIs
StatePublished - 1 Apr 2018

Fingerprint

Laparoscopy
Guidelines
Randomized Controlled Trials
Sample Size
Practice Guidelines

Keywords

  • Fragility
  • Guidelines
  • Randomized controlled trial
  • Risk of bias
  • Surgery

Cite this

@article{df7c8aae70dc44dbb97218ae83e3003a,
title = "Assessing quality of randomized trials supporting guidelines for laparoscopic and endoscopic surgery",
abstract = "Background Recent studies have highlighted the risk of bias and the fragility of results in randomized controlled trials (RCTs). The aim of our study was to evaluate the clinical practice guidelines created by the Society for Gastrointestinal and Endoscopic Surgeons (SAGES) for fragility, statistical power, and risk of bias. Materials and methods We screened the SAGES clinical practice guideline references for qualifying RCTs. RCTs were assessed for risk of bias using the Cochrane Collaboration Risk of Bias tool 2.0. We used the fragility index and fragility quotient to evaluate the robustness of trial results and conducted a power analysis using G*Power to determine if trials were adequately powered. Results Twenty-two (40.7{\%}) of the 54 trials that we assessed were rated as having a high risk of bias, 17 (31.5{\%}) were rated as having a low risk of bias, and 15 (27.8{\%}) were rated as having some concerns. The median fragility index was 2.5 (interquartile range 1-7). The median fragility quotient was 0.021 (interquartile range 0.003-0.045). Mean sample size was 108, and the mean loss to follow-up was eight patients. Eight of 33 trials (24.2{\%}) were found to be underpowered according to the sample size used in the primary outcome. Conclusions Guidelines created by SAGES are supported by RCTs that are frequently fragile or underpowered or have a high risk of bias. Future RCTs should utilize the Consolidated Standards of Reporting Trials statement, implement strategies to minimize loss to follow-up, and use properly powered sample sizes.",
keywords = "Fragility, Guidelines, Randomized controlled trial, Risk of bias, Surgery",
author = "Aaron Bowers and Chase Meyer and Daniel Tritz and Courtney Cook and Kaleb Fuller and Caleb Smith and Brian Diener and Matt Vassar",
year = "2018",
month = "4",
day = "1",
doi = "10.1016/j.jss.2017.11.061",
language = "English",
volume = "224",
pages = "233--239",
journal = "Journal of Surgical Research",
issn = "0022-4804",
publisher = "Academic Press Inc.",

}

TY - JOUR

T1 - Assessing quality of randomized trials supporting guidelines for laparoscopic and endoscopic surgery

AU - Bowers, Aaron

AU - Meyer, Chase

AU - Tritz, Daniel

AU - Cook, Courtney

AU - Fuller, Kaleb

AU - Smith, Caleb

AU - Diener, Brian

AU - Vassar, Matt

PY - 2018/4/1

Y1 - 2018/4/1

N2 - Background Recent studies have highlighted the risk of bias and the fragility of results in randomized controlled trials (RCTs). The aim of our study was to evaluate the clinical practice guidelines created by the Society for Gastrointestinal and Endoscopic Surgeons (SAGES) for fragility, statistical power, and risk of bias. Materials and methods We screened the SAGES clinical practice guideline references for qualifying RCTs. RCTs were assessed for risk of bias using the Cochrane Collaboration Risk of Bias tool 2.0. We used the fragility index and fragility quotient to evaluate the robustness of trial results and conducted a power analysis using G*Power to determine if trials were adequately powered. Results Twenty-two (40.7%) of the 54 trials that we assessed were rated as having a high risk of bias, 17 (31.5%) were rated as having a low risk of bias, and 15 (27.8%) were rated as having some concerns. The median fragility index was 2.5 (interquartile range 1-7). The median fragility quotient was 0.021 (interquartile range 0.003-0.045). Mean sample size was 108, and the mean loss to follow-up was eight patients. Eight of 33 trials (24.2%) were found to be underpowered according to the sample size used in the primary outcome. Conclusions Guidelines created by SAGES are supported by RCTs that are frequently fragile or underpowered or have a high risk of bias. Future RCTs should utilize the Consolidated Standards of Reporting Trials statement, implement strategies to minimize loss to follow-up, and use properly powered sample sizes.

AB - Background Recent studies have highlighted the risk of bias and the fragility of results in randomized controlled trials (RCTs). The aim of our study was to evaluate the clinical practice guidelines created by the Society for Gastrointestinal and Endoscopic Surgeons (SAGES) for fragility, statistical power, and risk of bias. Materials and methods We screened the SAGES clinical practice guideline references for qualifying RCTs. RCTs were assessed for risk of bias using the Cochrane Collaboration Risk of Bias tool 2.0. We used the fragility index and fragility quotient to evaluate the robustness of trial results and conducted a power analysis using G*Power to determine if trials were adequately powered. Results Twenty-two (40.7%) of the 54 trials that we assessed were rated as having a high risk of bias, 17 (31.5%) were rated as having a low risk of bias, and 15 (27.8%) were rated as having some concerns. The median fragility index was 2.5 (interquartile range 1-7). The median fragility quotient was 0.021 (interquartile range 0.003-0.045). Mean sample size was 108, and the mean loss to follow-up was eight patients. Eight of 33 trials (24.2%) were found to be underpowered according to the sample size used in the primary outcome. Conclusions Guidelines created by SAGES are supported by RCTs that are frequently fragile or underpowered or have a high risk of bias. Future RCTs should utilize the Consolidated Standards of Reporting Trials statement, implement strategies to minimize loss to follow-up, and use properly powered sample sizes.

KW - Fragility

KW - Guidelines

KW - Randomized controlled trial

KW - Risk of bias

KW - Surgery

UR - http://www.scopus.com/inward/record.url?scp=85040016808&partnerID=8YFLogxK

U2 - 10.1016/j.jss.2017.11.061

DO - 10.1016/j.jss.2017.11.061

M3 - Article

C2 - 29506846

AN - SCOPUS:85040016808

VL - 224

SP - 233

EP - 239

JO - Journal of Surgical Research

JF - Journal of Surgical Research

SN - 0022-4804

ER -