Effects of a proposal to alter the statistical significance threshold on previously published orthopaedic trauma randomized controlled trials

Research output: Contribution to journalArticle

Abstract

Introduction: A recent proposal suggests changing the threshold for statistical significance from a P value of.05 to.005 to minimize bias and increase reproducibility of future studies. P values less than.05 but greater than.005 would be reclassified as “suggestive”, whereas P values less than.005 would be considered significant. The present study explores how lowering the P value threshold would affect the interpretation of previously published orthopaedic trauma randomized controlled trials (RCTs) and whether outcomes from these trials would maintain statistical significance under the proposed P value threshold. Methods: All RCTs published between January 01, 2016 and January 31, 2018 in the Journal of Orthopaedic Trauma, Injury, and Archives of Orthopaedic and Trauma Surgery were screened by at least 2 authors. Data from included trials were extracted in blinded and duplicate fashion. All P values for primary endpoints were included from each study. Results: We identified 124 primary endpoints from 48 trials: 39.5% (49/124) of endpoints had a P value less than.05 and 60.5% (75/124) had a P value greater than.05. Overall, 51.0% (25/49) of statistically significant primary endpoints were less than.005, while 49.0% (24/49) would be reclassified as suggestive. Conclusion: Based on our results, adopting a lower threshold of significance would heavily alter the significance of orthopaedic trauma RCTs and should be further evaluated and cautiously considered when viewing the effect such a proposal on orthopaedic practice.

Original languageEnglish
Pages (from-to)1934-1937
Number of pages4
JournalInjury
Volume50
Issue number11
DOIs
StateAccepted/In press - 1 Jan 2019

Fingerprint

Orthopedics
Randomized Controlled Trials
Wounds and Injuries

Keywords

  • Analysis
  • Clinical trials
  • Evidence
  • Orthopaedic
  • Orthopaedic trials
  • P-value
  • Randomized controlled trials
  • Statistical significance
  • Statistics
  • Trauma
  • Traumatology

Cite this

@article{b00b979c0ce8485ea890f9f88e63a013,
title = "Effects of a proposal to alter the statistical significance threshold on previously published orthopaedic trauma randomized controlled trials",
abstract = "Introduction: A recent proposal suggests changing the threshold for statistical significance from a P value of.05 to.005 to minimize bias and increase reproducibility of future studies. P values less than.05 but greater than.005 would be reclassified as “suggestive”, whereas P values less than.005 would be considered significant. The present study explores how lowering the P value threshold would affect the interpretation of previously published orthopaedic trauma randomized controlled trials (RCTs) and whether outcomes from these trials would maintain statistical significance under the proposed P value threshold. Methods: All RCTs published between January 01, 2016 and January 31, 2018 in the Journal of Orthopaedic Trauma, Injury, and Archives of Orthopaedic and Trauma Surgery were screened by at least 2 authors. Data from included trials were extracted in blinded and duplicate fashion. All P values for primary endpoints were included from each study. Results: We identified 124 primary endpoints from 48 trials: 39.5{\%} (49/124) of endpoints had a P value less than.05 and 60.5{\%} (75/124) had a P value greater than.05. Overall, 51.0{\%} (25/49) of statistically significant primary endpoints were less than.005, while 49.0{\%} (24/49) would be reclassified as suggestive. Conclusion: Based on our results, adopting a lower threshold of significance would heavily alter the significance of orthopaedic trauma RCTs and should be further evaluated and cautiously considered when viewing the effect such a proposal on orthopaedic practice.",
keywords = "Analysis, Clinical trials, Evidence, Orthopaedic, Orthopaedic trials, P-value, Randomized controlled trials, Statistical significance, Statistics, Trauma, Traumatology",
author = "Johnson, {Austin L.} and Sheridan Evans and Checketts, {Jake X.} and Scott, {Jared T.} and Cole Wayant and Mark Johnson and Brent Norris and Matt Vassar",
year = "2019",
month = "1",
day = "1",
doi = "10.1016/j.injury.2019.08.012",
language = "English",
volume = "50",
pages = "1934--1937",
journal = "Injury",
issn = "0020-1383",
publisher = "Elsevier Ltd",
number = "11",

}

TY - JOUR

T1 - Effects of a proposal to alter the statistical significance threshold on previously published orthopaedic trauma randomized controlled trials

AU - Johnson, Austin L.

AU - Evans, Sheridan

AU - Checketts, Jake X.

AU - Scott, Jared T.

AU - Wayant, Cole

AU - Johnson, Mark

AU - Norris, Brent

AU - Vassar, Matt

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Introduction: A recent proposal suggests changing the threshold for statistical significance from a P value of.05 to.005 to minimize bias and increase reproducibility of future studies. P values less than.05 but greater than.005 would be reclassified as “suggestive”, whereas P values less than.005 would be considered significant. The present study explores how lowering the P value threshold would affect the interpretation of previously published orthopaedic trauma randomized controlled trials (RCTs) and whether outcomes from these trials would maintain statistical significance under the proposed P value threshold. Methods: All RCTs published between January 01, 2016 and January 31, 2018 in the Journal of Orthopaedic Trauma, Injury, and Archives of Orthopaedic and Trauma Surgery were screened by at least 2 authors. Data from included trials were extracted in blinded and duplicate fashion. All P values for primary endpoints were included from each study. Results: We identified 124 primary endpoints from 48 trials: 39.5% (49/124) of endpoints had a P value less than.05 and 60.5% (75/124) had a P value greater than.05. Overall, 51.0% (25/49) of statistically significant primary endpoints were less than.005, while 49.0% (24/49) would be reclassified as suggestive. Conclusion: Based on our results, adopting a lower threshold of significance would heavily alter the significance of orthopaedic trauma RCTs and should be further evaluated and cautiously considered when viewing the effect such a proposal on orthopaedic practice.

AB - Introduction: A recent proposal suggests changing the threshold for statistical significance from a P value of.05 to.005 to minimize bias and increase reproducibility of future studies. P values less than.05 but greater than.005 would be reclassified as “suggestive”, whereas P values less than.005 would be considered significant. The present study explores how lowering the P value threshold would affect the interpretation of previously published orthopaedic trauma randomized controlled trials (RCTs) and whether outcomes from these trials would maintain statistical significance under the proposed P value threshold. Methods: All RCTs published between January 01, 2016 and January 31, 2018 in the Journal of Orthopaedic Trauma, Injury, and Archives of Orthopaedic and Trauma Surgery were screened by at least 2 authors. Data from included trials were extracted in blinded and duplicate fashion. All P values for primary endpoints were included from each study. Results: We identified 124 primary endpoints from 48 trials: 39.5% (49/124) of endpoints had a P value less than.05 and 60.5% (75/124) had a P value greater than.05. Overall, 51.0% (25/49) of statistically significant primary endpoints were less than.005, while 49.0% (24/49) would be reclassified as suggestive. Conclusion: Based on our results, adopting a lower threshold of significance would heavily alter the significance of orthopaedic trauma RCTs and should be further evaluated and cautiously considered when viewing the effect such a proposal on orthopaedic practice.

KW - Analysis

KW - Clinical trials

KW - Evidence

KW - Orthopaedic

KW - Orthopaedic trials

KW - P-value

KW - Randomized controlled trials

KW - Statistical significance

KW - Statistics

KW - Trauma

KW - Traumatology

UR - http://www.scopus.com/inward/record.url?scp=85070551331&partnerID=8YFLogxK

U2 - 10.1016/j.injury.2019.08.012

DO - 10.1016/j.injury.2019.08.012

M3 - Article

AN - SCOPUS:85070551331

VL - 50

SP - 1934

EP - 1937

JO - Injury

JF - Injury

SN - 0020-1383

IS - 11

ER -