The fragility of statistically significant results in otolaryngology randomized trials

Research output: Contribution to journalArticle

4 Scopus citations

Abstract

Objectives: The American Academy of Otolaryngology–Head and Neck Surgery regards randomized controlled trials as class A evidence. A novel method to determine the robustness of outcomes in trials is the fragility index. This index represents the number of patients whose status would have to change from a non-event to an event to make a statistically significant result non-significant. Methods: Investigators included otolaryngology journals listed in the top 10 of one or both of Google Scholar Metrics and Clarivate Analytics’ Journal rankings. For inclusion, a randomized controlled trial needed to report a one-to-one random assignment of participants to condition, contain two parallel arms or have used a two-by-two factorial design, and report at least one statistically significant dichotomous outcome. Results: Sixty-nine trials met inclusion criteria. The median fragility index was three events (interquartile range 1–7.5). Median sample size was 72 (interquartile range 50–102.5). Modest correlations were observed between fragility index and total sample size (r = 0.27) and fragility index and event rate (r = 0.46). Investigators found no correlation between fragility index and impact factor or Science Citation Index. In 39% (27/69) of trials, the number lost to follow-up was equal to or greater than the fragility index. Conclusion: A median fragility index of 3 indicates that three people, on average, are needed to alter the outcomes in otolaryngology trials. This indicates that the results of two-group randomized controlled trials reporting binary endpoints published in otolaryngology journals may frequently be fragile.

Original languageEnglish
Pages (from-to)61-66
Number of pages6
JournalAmerican Journal of Otolaryngology - Head and Neck Medicine and Surgery
Volume40
Issue number1
DOIs
StatePublished - 1 Jan 2019

Fingerprint Dive into the research topics of 'The fragility of statistically significant results in otolaryngology randomized trials'. Together they form a unique fingerprint.

  • Cite this