TY - JOUR
T1 - The Results of Randomized Controlled Trials in Emergency Medicine Are Frequently Fragile
AU - Brown, Jamin
AU - Lane, Aaron
AU - Cooper, Craig
AU - Vassar, Matt
N1 - Publisher Copyright:
© 2018 American College of Emergency Physicians
PY - 2019/6
Y1 - 2019/6
N2 - Study objective: Randomized controlled trials govern evidence-based clinical practice, and it is therefore critical that their results be robust. We aim to investigate the fragility of randomized controlled trials in emergency medicine by determining how often significance would be nullified with small changes in outcomes using the fragility index. Methods: We conducted a methodological systematic review of randomized controlled trials in emergency medicine published in the top 10 general medicine journals and the top 10 emergency medicine journals. Inclusion criteria required that trials be emergency medicine studies structured with a 2-arm or 2-by-2 factorial design and report at least 1 statistically significant dichotomous outcome. Results: A total of 180 trials met inclusion criteria. The median fragility index across all trials in emergency medicine was 4 (interquartile range [IQR]2 to 10)and the median sample size was 140 (IQR 69.5 to 286). For trials from general medicine journals (n=32), the median fragility index was 9 (IQR 4 to 16.5)and the median sample size was 415.5 (IQR 219.5 to 901); for trials from emergency medicine journals (n=148), the median fragility index was 4 (IQR 1 to 9)and the median sample size was 119 (IQR 60 to 227.25). One third of all trials (62/180)had a loss to follow-up that was greater than or equal to the fragility index. There was a modest correlation between fragility index and total number of events (r=0.36; 95% confidence interval [CI]0.23 to 0.48)and a weak correlation between fragility index and total sample size (r=0.26; 95% CI 0.12 to 0.39). There was no correlation between fragility index and either P value (r=–0.14; 95% CI –0.28 to –0.006)or Science Citation Index (r=0.07; 95% CI –0.08 to 0.22). Conclusion: The statistical significance of the results of randomized controlled trials in emergency medicine was often contingent on a small number of events. Until frequentist interpretation of clinical trials is replaced with Bayesian analysis, the fragility index may have utility as a tool to aid clinicians in assessing the robustness of randomized controlled trials in emergency medicine when considered in conjunction with the fragility quotient and other reported metrics.
AB - Study objective: Randomized controlled trials govern evidence-based clinical practice, and it is therefore critical that their results be robust. We aim to investigate the fragility of randomized controlled trials in emergency medicine by determining how often significance would be nullified with small changes in outcomes using the fragility index. Methods: We conducted a methodological systematic review of randomized controlled trials in emergency medicine published in the top 10 general medicine journals and the top 10 emergency medicine journals. Inclusion criteria required that trials be emergency medicine studies structured with a 2-arm or 2-by-2 factorial design and report at least 1 statistically significant dichotomous outcome. Results: A total of 180 trials met inclusion criteria. The median fragility index across all trials in emergency medicine was 4 (interquartile range [IQR]2 to 10)and the median sample size was 140 (IQR 69.5 to 286). For trials from general medicine journals (n=32), the median fragility index was 9 (IQR 4 to 16.5)and the median sample size was 415.5 (IQR 219.5 to 901); for trials from emergency medicine journals (n=148), the median fragility index was 4 (IQR 1 to 9)and the median sample size was 119 (IQR 60 to 227.25). One third of all trials (62/180)had a loss to follow-up that was greater than or equal to the fragility index. There was a modest correlation between fragility index and total number of events (r=0.36; 95% confidence interval [CI]0.23 to 0.48)and a weak correlation between fragility index and total sample size (r=0.26; 95% CI 0.12 to 0.39). There was no correlation between fragility index and either P value (r=–0.14; 95% CI –0.28 to –0.006)or Science Citation Index (r=0.07; 95% CI –0.08 to 0.22). Conclusion: The statistical significance of the results of randomized controlled trials in emergency medicine was often contingent on a small number of events. Until frequentist interpretation of clinical trials is replaced with Bayesian analysis, the fragility index may have utility as a tool to aid clinicians in assessing the robustness of randomized controlled trials in emergency medicine when considered in conjunction with the fragility quotient and other reported metrics.
UR - http://www.scopus.com/inward/record.url?scp=85058189836&partnerID=8YFLogxK
U2 - 10.1016/j.annemergmed.2018.10.037
DO - 10.1016/j.annemergmed.2018.10.037
M3 - Article
C2 - 30551894
AN - SCOPUS:85058189836
SN - 0196-0644
VL - 73
SP - 565
EP - 576
JO - Annals of Emergency Medicine
JF - Annals of Emergency Medicine
IS - 6
ER -