An assessment of transparency and reproducibility-related research practices in otolaryngology

Research output: Contribution to journalArticle

Abstract

Objectives/Hypothesis: Clinical research serves as the foundation for evidence-based patient care, and reproducibility of results is consequently critical. We sought to assess the transparency and reproducibility of research studies in otolaryngology by evaluating a random sample of publications in otolaryngology journals between 2014 and 2018. Study Design: Review of published literature for reproducible and transparent research practices. Methods: We used the National Library of Medicine catalog to identify otolaryngology journals that met the inclusion criteria (available in the English language and indexed in MEDLINE). From these journals, we extracted a random sample of 300 publications using a PubMed search for records published between January 1, 2014 and December 31, 2018. Specific indicators of reproducible and transparent research practices were evaluated in a blinded, independent, and duplicate manner using a pilot-tested Google form. Results: Our initial search returned 26,498 records, from which 300 were randomly selected for analysis. Of these 300 records, 286 met inclusion criteria and 14 did not. Among the empirical studies, 2% (95% confidence interval [CI]: 0.4%-3.5%) of publications indicated that raw data were available, 0.6% (95% CI: 0.3%-1.6%) reported an analysis script, 5.3% (95% CI: 2.7%-7.8%) were linked to an accessible research protocol, and 3.9% (95% CI: 1.7%-6.1%) were preregistered. None of the publications had a clear statement claiming to replicate, or to be a replication of, another study. Conclusions: Inadequate reproducibility practices exist in otolaryngology. Nearly all studies in our analysis lacked a data or material availability statement, did not link to an accessible protocol, and were not preregistered. Taking steps to improve reproducibility would likely improve patient care. Level of Evidence: NA. Laryngoscope, 2019.

Original languageEnglish
JournalLaryngoscope
DOIs
StateAccepted/In press - 1 Jan 2019

Fingerprint

Otolaryngology
Publications
Confidence Intervals
Research
Library Catalogs
Patient Care
National Library of Medicine (U.S.)
Laryngoscopes
PubMed
Reproducibility of Results
MEDLINE
Language

Keywords

  • data sharing
  • open access
  • open science
  • otolaryngology
  • protocol
  • replication
  • Reproducibility

Cite this

@article{a5fe7c015ab5469dbbe32d86a0b9c43e,
title = "An assessment of transparency and reproducibility-related research practices in otolaryngology",
abstract = "Objectives/Hypothesis: Clinical research serves as the foundation for evidence-based patient care, and reproducibility of results is consequently critical. We sought to assess the transparency and reproducibility of research studies in otolaryngology by evaluating a random sample of publications in otolaryngology journals between 2014 and 2018. Study Design: Review of published literature for reproducible and transparent research practices. Methods: We used the National Library of Medicine catalog to identify otolaryngology journals that met the inclusion criteria (available in the English language and indexed in MEDLINE). From these journals, we extracted a random sample of 300 publications using a PubMed search for records published between January 1, 2014 and December 31, 2018. Specific indicators of reproducible and transparent research practices were evaluated in a blinded, independent, and duplicate manner using a pilot-tested Google form. Results: Our initial search returned 26,498 records, from which 300 were randomly selected for analysis. Of these 300 records, 286 met inclusion criteria and 14 did not. Among the empirical studies, 2{\%} (95{\%} confidence interval [CI]: 0.4{\%}-3.5{\%}) of publications indicated that raw data were available, 0.6{\%} (95{\%} CI: 0.3{\%}-1.6{\%}) reported an analysis script, 5.3{\%} (95{\%} CI: 2.7{\%}-7.8{\%}) were linked to an accessible research protocol, and 3.9{\%} (95{\%} CI: 1.7{\%}-6.1{\%}) were preregistered. None of the publications had a clear statement claiming to replicate, or to be a replication of, another study. Conclusions: Inadequate reproducibility practices exist in otolaryngology. Nearly all studies in our analysis lacked a data or material availability statement, did not link to an accessible protocol, and were not preregistered. Taking steps to improve reproducibility would likely improve patient care. Level of Evidence: NA. Laryngoscope, 2019.",
keywords = "data sharing, open access, open science, otolaryngology, protocol, replication, Reproducibility",
author = "Johnson, {Austin L.} and Trevor Torgerson and Mason Skinner and Tom Hamilton and Daniel Tritz and Matt Vassar and Mason Skinner",
year = "2019",
month = "1",
day = "1",
doi = "10.1002/lary.28322",
language = "English",
journal = "Laryngoscope",
issn = "0023-852X",
publisher = "Wiley-Blackwell",

}

TY - JOUR

T1 - An assessment of transparency and reproducibility-related research practices in otolaryngology

AU - Johnson, Austin L.

AU - Torgerson, Trevor

AU - Skinner, Mason

AU - Hamilton, Tom

AU - Tritz, Daniel

AU - Vassar, Matt

AU - Skinner, Mason

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Objectives/Hypothesis: Clinical research serves as the foundation for evidence-based patient care, and reproducibility of results is consequently critical. We sought to assess the transparency and reproducibility of research studies in otolaryngology by evaluating a random sample of publications in otolaryngology journals between 2014 and 2018. Study Design: Review of published literature for reproducible and transparent research practices. Methods: We used the National Library of Medicine catalog to identify otolaryngology journals that met the inclusion criteria (available in the English language and indexed in MEDLINE). From these journals, we extracted a random sample of 300 publications using a PubMed search for records published between January 1, 2014 and December 31, 2018. Specific indicators of reproducible and transparent research practices were evaluated in a blinded, independent, and duplicate manner using a pilot-tested Google form. Results: Our initial search returned 26,498 records, from which 300 were randomly selected for analysis. Of these 300 records, 286 met inclusion criteria and 14 did not. Among the empirical studies, 2% (95% confidence interval [CI]: 0.4%-3.5%) of publications indicated that raw data were available, 0.6% (95% CI: 0.3%-1.6%) reported an analysis script, 5.3% (95% CI: 2.7%-7.8%) were linked to an accessible research protocol, and 3.9% (95% CI: 1.7%-6.1%) were preregistered. None of the publications had a clear statement claiming to replicate, or to be a replication of, another study. Conclusions: Inadequate reproducibility practices exist in otolaryngology. Nearly all studies in our analysis lacked a data or material availability statement, did not link to an accessible protocol, and were not preregistered. Taking steps to improve reproducibility would likely improve patient care. Level of Evidence: NA. Laryngoscope, 2019.

AB - Objectives/Hypothesis: Clinical research serves as the foundation for evidence-based patient care, and reproducibility of results is consequently critical. We sought to assess the transparency and reproducibility of research studies in otolaryngology by evaluating a random sample of publications in otolaryngology journals between 2014 and 2018. Study Design: Review of published literature for reproducible and transparent research practices. Methods: We used the National Library of Medicine catalog to identify otolaryngology journals that met the inclusion criteria (available in the English language and indexed in MEDLINE). From these journals, we extracted a random sample of 300 publications using a PubMed search for records published between January 1, 2014 and December 31, 2018. Specific indicators of reproducible and transparent research practices were evaluated in a blinded, independent, and duplicate manner using a pilot-tested Google form. Results: Our initial search returned 26,498 records, from which 300 were randomly selected for analysis. Of these 300 records, 286 met inclusion criteria and 14 did not. Among the empirical studies, 2% (95% confidence interval [CI]: 0.4%-3.5%) of publications indicated that raw data were available, 0.6% (95% CI: 0.3%-1.6%) reported an analysis script, 5.3% (95% CI: 2.7%-7.8%) were linked to an accessible research protocol, and 3.9% (95% CI: 1.7%-6.1%) were preregistered. None of the publications had a clear statement claiming to replicate, or to be a replication of, another study. Conclusions: Inadequate reproducibility practices exist in otolaryngology. Nearly all studies in our analysis lacked a data or material availability statement, did not link to an accessible protocol, and were not preregistered. Taking steps to improve reproducibility would likely improve patient care. Level of Evidence: NA. Laryngoscope, 2019.

KW - data sharing

KW - open access

KW - open science

KW - otolaryngology

KW - protocol

KW - replication

KW - Reproducibility

UR - http://www.scopus.com/inward/record.url?scp=85074616677&partnerID=8YFLogxK

U2 - 10.1002/lary.28322

DO - 10.1002/lary.28322

M3 - Article

C2 - 31593300

AN - SCOPUS:85074616677

JO - Laryngoscope

JF - Laryngoscope

SN - 0023-852X

ER -