The robustness of trials that guide evidence-based orthopaedic surgery

Research output: Contribution to journalReview article

4 Citations (Scopus)

Abstract

Background: The fragility index (FI) may prove to be a powerful metric of trial robustness. The FI is the minimum number of patient events that would need to become nonevents in order to nullify a significant result. The fragility quotient (FQ) is the FI divided by the total sample size. This study evaluates the robustness of the 20% of orthopaedic clinical trials that were cited as having strong evidence in the American Academy of Orthopaedic Surgeons (AAOS) Clinical Practice Guidelines and that could be analyzed with these indices. Methods: From the AAOS recommendations with strong evidence, we extracted the randomized controlled trials that were cited as having supporting evidence that could be analyzed with the FI. Each trial's FI was calculated using the fragility calculator (www.fragilityindex.com). With use of the Cochrane Risk of Bias Tool 2.0, we evaluated the likelihood of bias. We also performed a post hoc power analysis of eligible studies. Results: The median FI for the 72 trials was 2 events, and the median FQ was 0.022. Of the 72 trials, only 3 (4.2%) were at a low risk of bias, and 35 (48.6%) were at a high risk of bias. Thirty-eight (53%) of the trials were underpowered. We identified a strong correlation between a trial's FI or FQ and the trial's power. Conclusions: Our study found that trials that provided strong evidence for orthopaedic surgery guidelines were largely fragile, underpowered, and at risk of bias.

Original languageEnglish
Pages (from-to)E85
JournalJournal of Bone and Joint Surgery - American Volume
Volume100
Issue number12
DOIs
StatePublished - 1 Jun 2018

Fingerprint

Orthopedics
Practice Guidelines
Sample Size
Randomized Controlled Trials
Clinical Trials
Guidelines

Cite this

@article{e61ac377bc5e408c952c0701ba1eb4cd,
title = "The robustness of trials that guide evidence-based orthopaedic surgery",
abstract = "Background: The fragility index (FI) may prove to be a powerful metric of trial robustness. The FI is the minimum number of patient events that would need to become nonevents in order to nullify a significant result. The fragility quotient (FQ) is the FI divided by the total sample size. This study evaluates the robustness of the 20{\%} of orthopaedic clinical trials that were cited as having strong evidence in the American Academy of Orthopaedic Surgeons (AAOS) Clinical Practice Guidelines and that could be analyzed with these indices. Methods: From the AAOS recommendations with strong evidence, we extracted the randomized controlled trials that were cited as having supporting evidence that could be analyzed with the FI. Each trial's FI was calculated using the fragility calculator (www.fragilityindex.com). With use of the Cochrane Risk of Bias Tool 2.0, we evaluated the likelihood of bias. We also performed a post hoc power analysis of eligible studies. Results: The median FI for the 72 trials was 2 events, and the median FQ was 0.022. Of the 72 trials, only 3 (4.2{\%}) were at a low risk of bias, and 35 (48.6{\%}) were at a high risk of bias. Thirty-eight (53{\%}) of the trials were underpowered. We identified a strong correlation between a trial's FI or FQ and the trial's power. Conclusions: Our study found that trials that provided strong evidence for orthopaedic surgery guidelines were largely fragile, underpowered, and at risk of bias.",
author = "Checketts, {Jake Xavier} and Scott, {Jared T.} and Chase Meyer and Jarryd Horn and Jaclyn Jones and Matt Vassar",
year = "2018",
month = "6",
day = "1",
doi = "10.2106/JBJS.17.01039",
language = "English",
volume = "100",
pages = "E85",
journal = "Journal of Bone and Joint Surgery - American Volume",
issn = "0021-9355",
publisher = "Lippincott Williams and Wilkins Ltd.",
number = "12",

}

TY - JOUR

T1 - The robustness of trials that guide evidence-based orthopaedic surgery

AU - Checketts, Jake Xavier

AU - Scott, Jared T.

AU - Meyer, Chase

AU - Horn, Jarryd

AU - Jones, Jaclyn

AU - Vassar, Matt

PY - 2018/6/1

Y1 - 2018/6/1

N2 - Background: The fragility index (FI) may prove to be a powerful metric of trial robustness. The FI is the minimum number of patient events that would need to become nonevents in order to nullify a significant result. The fragility quotient (FQ) is the FI divided by the total sample size. This study evaluates the robustness of the 20% of orthopaedic clinical trials that were cited as having strong evidence in the American Academy of Orthopaedic Surgeons (AAOS) Clinical Practice Guidelines and that could be analyzed with these indices. Methods: From the AAOS recommendations with strong evidence, we extracted the randomized controlled trials that were cited as having supporting evidence that could be analyzed with the FI. Each trial's FI was calculated using the fragility calculator (www.fragilityindex.com). With use of the Cochrane Risk of Bias Tool 2.0, we evaluated the likelihood of bias. We also performed a post hoc power analysis of eligible studies. Results: The median FI for the 72 trials was 2 events, and the median FQ was 0.022. Of the 72 trials, only 3 (4.2%) were at a low risk of bias, and 35 (48.6%) were at a high risk of bias. Thirty-eight (53%) of the trials were underpowered. We identified a strong correlation between a trial's FI or FQ and the trial's power. Conclusions: Our study found that trials that provided strong evidence for orthopaedic surgery guidelines were largely fragile, underpowered, and at risk of bias.

AB - Background: The fragility index (FI) may prove to be a powerful metric of trial robustness. The FI is the minimum number of patient events that would need to become nonevents in order to nullify a significant result. The fragility quotient (FQ) is the FI divided by the total sample size. This study evaluates the robustness of the 20% of orthopaedic clinical trials that were cited as having strong evidence in the American Academy of Orthopaedic Surgeons (AAOS) Clinical Practice Guidelines and that could be analyzed with these indices. Methods: From the AAOS recommendations with strong evidence, we extracted the randomized controlled trials that were cited as having supporting evidence that could be analyzed with the FI. Each trial's FI was calculated using the fragility calculator (www.fragilityindex.com). With use of the Cochrane Risk of Bias Tool 2.0, we evaluated the likelihood of bias. We also performed a post hoc power analysis of eligible studies. Results: The median FI for the 72 trials was 2 events, and the median FQ was 0.022. Of the 72 trials, only 3 (4.2%) were at a low risk of bias, and 35 (48.6%) were at a high risk of bias. Thirty-eight (53%) of the trials were underpowered. We identified a strong correlation between a trial's FI or FQ and the trial's power. Conclusions: Our study found that trials that provided strong evidence for orthopaedic surgery guidelines were largely fragile, underpowered, and at risk of bias.

UR - http://www.scopus.com/inward/record.url?scp=85056239036&partnerID=8YFLogxK

U2 - 10.2106/JBJS.17.01039

DO - 10.2106/JBJS.17.01039

M3 - Review article

C2 - 29916938

AN - SCOPUS:85056239036

VL - 100

SP - E85

JO - Journal of Bone and Joint Surgery - American Volume

JF - Journal of Bone and Joint Surgery - American Volume

SN - 0021-9355

IS - 12

ER -