Systematic review: Outcome reporting bias is a problem in high impact factor neurology journals

Benjamin Howard, Jared T. Scott, Mark Blubaugh, Brie Roepke, Caleb Scheckel, Matt Vassar

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

Background: Selective outcome reporting is a significant methodological concern. Comparisons between the outcomes reported in clinical trial registrations and those later published allow investigators to understand the extent of selection bias among trialists. We examined the possibility of selective outcome reporting in randomized controlled trials (RCTs) published in neurology journals. Methods: We searched PubMed for randomized controlled trials from Jan 1, 2010–Dec 31, 2015 published in the top 3 impact factor neurology journals. These articles were screened according to specific inclusion criteria. Each author individually extracted data from trials following a standardized protocol. A second author verified each extracted element and discrepancies were resolved. Consistency between registered and published outcomes was evaluated and correlations between discrepancies and funding, journal, and temporal trends were examined. Results: 180 trials were included for analysis. 10 (6%) primary outcomes were demoted, 38 (21%) primary outcomes were omitted from the publication, and 61 (34%) unregistered primary outcomes were added to the published report. There were 18 (10%) cases of secondary outcomes being upgraded to primary outcomes in the publication, and there were 53 (29%) changes in timing of assessment. Of 82 (46%) major discrepancies with reported p-values, 54 (66.0%) favored publication of statistically significant results. Conclusion: Across trials, we found 180 major discrepancies. 66% of major discrepancies with a reported p-value (n = 82) favored statistically significant results. These results suggest a need within neurology to provide more consistent and timely registration of outcomes.

Original languageEnglish
Article numbere0180986
JournalPLoS ONE
Volume12
Issue number7
DOIs
StatePublished - 1 Jul 2017

Fingerprint

Journal Impact Factor
systematic review
Neurology
Publications
Randomized Controlled Trials
Selection Bias
PubMed
funding
clinical trials
Research Personnel
Clinical Trials
methodology

Cite this

@article{54be73f9bcfb432589b391617afe8e1c,
title = "Systematic review: Outcome reporting bias is a problem in high impact factor neurology journals",
abstract = "Background: Selective outcome reporting is a significant methodological concern. Comparisons between the outcomes reported in clinical trial registrations and those later published allow investigators to understand the extent of selection bias among trialists. We examined the possibility of selective outcome reporting in randomized controlled trials (RCTs) published in neurology journals. Methods: We searched PubMed for randomized controlled trials from Jan 1, 2010–Dec 31, 2015 published in the top 3 impact factor neurology journals. These articles were screened according to specific inclusion criteria. Each author individually extracted data from trials following a standardized protocol. A second author verified each extracted element and discrepancies were resolved. Consistency between registered and published outcomes was evaluated and correlations between discrepancies and funding, journal, and temporal trends were examined. Results: 180 trials were included for analysis. 10 (6{\%}) primary outcomes were demoted, 38 (21{\%}) primary outcomes were omitted from the publication, and 61 (34{\%}) unregistered primary outcomes were added to the published report. There were 18 (10{\%}) cases of secondary outcomes being upgraded to primary outcomes in the publication, and there were 53 (29{\%}) changes in timing of assessment. Of 82 (46{\%}) major discrepancies with reported p-values, 54 (66.0{\%}) favored publication of statistically significant results. Conclusion: Across trials, we found 180 major discrepancies. 66{\%} of major discrepancies with a reported p-value (n = 82) favored statistically significant results. These results suggest a need within neurology to provide more consistent and timely registration of outcomes.",
author = "Benjamin Howard and Scott, {Jared T.} and Mark Blubaugh and Brie Roepke and Caleb Scheckel and Matt Vassar",
year = "2017",
month = "7",
day = "1",
doi = "10.1371/journal.pone.0180986",
language = "English",
volume = "12",
journal = "PLoS ONE",
issn = "1932-6203",
publisher = "Public Library of Science",
number = "7",

}

Systematic review : Outcome reporting bias is a problem in high impact factor neurology journals. / Howard, Benjamin; Scott, Jared T.; Blubaugh, Mark; Roepke, Brie; Scheckel, Caleb; Vassar, Matt.

In: PLoS ONE, Vol. 12, No. 7, e0180986, 01.07.2017.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Systematic review

T2 - Outcome reporting bias is a problem in high impact factor neurology journals

AU - Howard, Benjamin

AU - Scott, Jared T.

AU - Blubaugh, Mark

AU - Roepke, Brie

AU - Scheckel, Caleb

AU - Vassar, Matt

PY - 2017/7/1

Y1 - 2017/7/1

N2 - Background: Selective outcome reporting is a significant methodological concern. Comparisons between the outcomes reported in clinical trial registrations and those later published allow investigators to understand the extent of selection bias among trialists. We examined the possibility of selective outcome reporting in randomized controlled trials (RCTs) published in neurology journals. Methods: We searched PubMed for randomized controlled trials from Jan 1, 2010–Dec 31, 2015 published in the top 3 impact factor neurology journals. These articles were screened according to specific inclusion criteria. Each author individually extracted data from trials following a standardized protocol. A second author verified each extracted element and discrepancies were resolved. Consistency between registered and published outcomes was evaluated and correlations between discrepancies and funding, journal, and temporal trends were examined. Results: 180 trials were included for analysis. 10 (6%) primary outcomes were demoted, 38 (21%) primary outcomes were omitted from the publication, and 61 (34%) unregistered primary outcomes were added to the published report. There were 18 (10%) cases of secondary outcomes being upgraded to primary outcomes in the publication, and there were 53 (29%) changes in timing of assessment. Of 82 (46%) major discrepancies with reported p-values, 54 (66.0%) favored publication of statistically significant results. Conclusion: Across trials, we found 180 major discrepancies. 66% of major discrepancies with a reported p-value (n = 82) favored statistically significant results. These results suggest a need within neurology to provide more consistent and timely registration of outcomes.

AB - Background: Selective outcome reporting is a significant methodological concern. Comparisons between the outcomes reported in clinical trial registrations and those later published allow investigators to understand the extent of selection bias among trialists. We examined the possibility of selective outcome reporting in randomized controlled trials (RCTs) published in neurology journals. Methods: We searched PubMed for randomized controlled trials from Jan 1, 2010–Dec 31, 2015 published in the top 3 impact factor neurology journals. These articles were screened according to specific inclusion criteria. Each author individually extracted data from trials following a standardized protocol. A second author verified each extracted element and discrepancies were resolved. Consistency between registered and published outcomes was evaluated and correlations between discrepancies and funding, journal, and temporal trends were examined. Results: 180 trials were included for analysis. 10 (6%) primary outcomes were demoted, 38 (21%) primary outcomes were omitted from the publication, and 61 (34%) unregistered primary outcomes were added to the published report. There were 18 (10%) cases of secondary outcomes being upgraded to primary outcomes in the publication, and there were 53 (29%) changes in timing of assessment. Of 82 (46%) major discrepancies with reported p-values, 54 (66.0%) favored publication of statistically significant results. Conclusion: Across trials, we found 180 major discrepancies. 66% of major discrepancies with a reported p-value (n = 82) favored statistically significant results. These results suggest a need within neurology to provide more consistent and timely registration of outcomes.

UR - http://www.scopus.com/inward/record.url?scp=85024923220&partnerID=8YFLogxK

U2 - 10.1371/journal.pone.0180986

DO - 10.1371/journal.pone.0180986

M3 - Article

C2 - 28727834

AN - SCOPUS:85024923220

VL - 12

JO - PLoS ONE

JF - PLoS ONE

SN - 1932-6203

IS - 7

M1 - e0180986

ER -