Abstract
Background: Misinterpretation of p-values in RCTs is extremely problematic since they are the core basis for high levels of recommendation in clinical practice guidelines, especially Orthopaedics. Benjamin et al. proposed a universal protocol change, moving statistical significance from a p-value of .05 to .005 to combat the misinterpretation that is happening in medical literature. In this study, we are looking to evaluate the effect of the protocol suggested by Benjamin et al. on Foot and Ankle-related RCTs in the top 3 Foot and Ankle-related journals.
Methods: We conducted a Pubmed search looking at studies published from January 1st, 2016 to November 10, 2021, in the following three journals; Foot and Ankle International, Journal of Foot and Ankle Surgery, and Foot & Ankle International. The inclusion criteria for the study were RCTs published in the above journals with specifically stated primary endpoints. If a study has multiple primary endpoints, all were included. Exclusion criteria were any study that was not prospective and randomized by design, also any study that did not state primary endpoints was excluded. Two authors extracted the data using a pilot-tested Google form, any disagreements or questions were resolved by published methodologic orthopaedic authors.
Results: Of the 222 endpoints, 101 endpoints (45.5%; 101/222) were at or below the .05 threshold while 121 endpoints (54.5%; 121/222) were above the .05 threshold. We also found that 59 endpoints (26.6%; 59/222) were below .005.
Conclusion: Our results suggest that changing the threshold for statistical significance from .05 to .005 in foot and ankle RCTs would heavily alter literature published in the field. By implementing this methodology, it is a promising measure to be able to increase RCT quality until a more substantial solution can be found. With that being said, caution must be taken when interpreting our results, also requiring further evaluation.
Methods: We conducted a Pubmed search looking at studies published from January 1st, 2016 to November 10, 2021, in the following three journals; Foot and Ankle International, Journal of Foot and Ankle Surgery, and Foot & Ankle International. The inclusion criteria for the study were RCTs published in the above journals with specifically stated primary endpoints. If a study has multiple primary endpoints, all were included. Exclusion criteria were any study that was not prospective and randomized by design, also any study that did not state primary endpoints was excluded. Two authors extracted the data using a pilot-tested Google form, any disagreements or questions were resolved by published methodologic orthopaedic authors.
Results: Of the 222 endpoints, 101 endpoints (45.5%; 101/222) were at or below the .05 threshold while 121 endpoints (54.5%; 121/222) were above the .05 threshold. We also found that 59 endpoints (26.6%; 59/222) were below .005.
Conclusion: Our results suggest that changing the threshold for statistical significance from .05 to .005 in foot and ankle RCTs would heavily alter literature published in the field. By implementing this methodology, it is a promising measure to be able to increase RCT quality until a more substantial solution can be found. With that being said, caution must be taken when interpreting our results, also requiring further evaluation.
| Original language | American English |
|---|---|
| Pages | 93 |
| State | Published - 18 Feb 2022 |
| Event | Oklahoma State University Center for Health Sciences Research Week 2022 : Poster Presentation - Oklahoma State University Center for Health Sciences, Tulsa, United States Duration: 14 Feb 2022 → 18 Feb 2022 |
Conference
| Conference | Oklahoma State University Center for Health Sciences Research Week 2022 |
|---|---|
| Country/Territory | United States |
| City | Tulsa |
| Period | 14/02/22 → 18/02/22 |
Keywords
- Data-dredging
- Foot and Ankle
- RCTs