An assessment of transparency and reproducibility-related research practices in otolaryngology

Objectives/Hypothesis: Clinical research serves as the foundation for evidence-based patient care, and reproducibility of results is consequently critical. We sought to assess the transparency and reproducibility of research studies in otolaryngology by evaluating a random sample of publications in otolaryngology journals between 2014 and 2018. 

Study Design: Review of published literature for reproducible and transparent research practices. 

Methods: We used the National Library of Medicine catalog to identify otolaryngology journals that met the inclusion criteria (available in the English language and indexed in MEDLINE). From these journals, we extracted a random sample of 300 publications using a PubMed search for records published between January 1, 2014 and December 31, 2018. Specific indicators of reproducible and transparent research practices were evaluated in a blinded, independent, and duplicate manner using a pilot-tested Google form. 

Results: Our initial search returned 26,498 records, from which 300 were randomly selected for analysis. Of these 300 records, 286 met inclusion criteria and 14 did not. Among the empirical studies, 2% (95% confidence interval [CI]: 0.4%-3.5%) of publications indicated that raw data were available, 0.6% (95% CI: 0.3%-1.6%) reported an analysis script, 5.3% (95% CI: 2.7%-7.8%) were linked to an accessible research protocol, and 3.9% (95% CI: 1.7%-6.1%) were preregistered. None of the publications had a clear statement claiming to replicate, or to be a replication of, another study. 

Conclusions: Inadequate reproducibility practices exist in otolaryngology. Nearly all studies in our analysis lacked a data or material availability statement, did not link to an accessible protocol, and were not preregistered. Taking steps to improve reproducibility would likely improve patient care. 

Level of Evidence: NA. Laryngoscope, 2019.