Evidence of selective reporting bias in hematology journals: A systematic review

Research output: Contribution to journalArticle

12 Citations (Scopus)

Abstract

Introduction: Selective reporting bias occurs when chance or selective outcome reporting rather than the intervention contributes to group differences. The prevailing concern about selective reporting bias is the possibility of results being modified towards specific conclusions. In this study, we evaluate randomized controlled trials (RCTs) published in hematology journals, a group in which selective outcome reporting has not yet been explored. Methods: Our primary goal was to examine discrepancies between the reported primary and secondary outcomes in registered and published RCTs concerning hematological malignancies reported in hematology journals with a high impact factor. The secondary goals were to address whether outcome reporting discrepancies favored statistically significant outcomes, whether a pattern existed between the funding source and likelihood of outcome reporting bias, and whether temporal trends were present in outcome reporting bias. For trials with major outcome discrepancies, we contacted trialists to determine reasons for these discrepancies. Trials published between January 1, 2010 and December 31, 2015 in Blood; British Journal of Haematology; American Journal of Hematology; Leukemia; and Haematologica were included. Results: Of 499 RCTs screened, 109 RCTs were included. Our analysis revealed 118 major discrepancies and 629 total discrepancies. Among the 118 discrepancies, 30 (25.4%) primary outcomes were demoted, 47 (39.8%) primary outcomes were omitted, and 30 (25.4%) primary outcomes were added. Three (2.5%) secondary outcomes were upgraded to a primary outcome. The timing of assessment for a primary outcome changed eight (6.8%) times. Thirtyone major discrepancies were published with a P-value and twenty-five (80.6%) favored statistical significance. A majority of authors whom we contacted cited a pre-planned subgroup analysis as a reason for outcome changes. Conclusion: Our results suggest that outcome changes occur frequently in hematology trials. Because RCTs ultimately underpin clinical judgment and guide policy implementation, selective reporting could pose a threat to medical decision making.

Original languageEnglish
Article numbere0178379
JournalPLoS ONE
Volume12
Issue number6
DOIs
StatePublished - 1 Jun 2017

Fingerprint

systematic review
Hematology
hematology
Randomized Controlled Trials
Hematologic Neoplasms
leukemia
funding
decision making
Leukemia
Blood
Decision making
blood

Cite this

@article{261ea6e753604da088cbafd672a62834,
title = "Evidence of selective reporting bias in hematology journals: A systematic review",
abstract = "Introduction: Selective reporting bias occurs when chance or selective outcome reporting rather than the intervention contributes to group differences. The prevailing concern about selective reporting bias is the possibility of results being modified towards specific conclusions. In this study, we evaluate randomized controlled trials (RCTs) published in hematology journals, a group in which selective outcome reporting has not yet been explored. Methods: Our primary goal was to examine discrepancies between the reported primary and secondary outcomes in registered and published RCTs concerning hematological malignancies reported in hematology journals with a high impact factor. The secondary goals were to address whether outcome reporting discrepancies favored statistically significant outcomes, whether a pattern existed between the funding source and likelihood of outcome reporting bias, and whether temporal trends were present in outcome reporting bias. For trials with major outcome discrepancies, we contacted trialists to determine reasons for these discrepancies. Trials published between January 1, 2010 and December 31, 2015 in Blood; British Journal of Haematology; American Journal of Hematology; Leukemia; and Haematologica were included. Results: Of 499 RCTs screened, 109 RCTs were included. Our analysis revealed 118 major discrepancies and 629 total discrepancies. Among the 118 discrepancies, 30 (25.4{\%}) primary outcomes were demoted, 47 (39.8{\%}) primary outcomes were omitted, and 30 (25.4{\%}) primary outcomes were added. Three (2.5{\%}) secondary outcomes were upgraded to a primary outcome. The timing of assessment for a primary outcome changed eight (6.8{\%}) times. Thirtyone major discrepancies were published with a P-value and twenty-five (80.6{\%}) favored statistical significance. A majority of authors whom we contacted cited a pre-planned subgroup analysis as a reason for outcome changes. Conclusion: Our results suggest that outcome changes occur frequently in hematology trials. Because RCTs ultimately underpin clinical judgment and guide policy implementation, selective reporting could pose a threat to medical decision making.",
author = "Cole Wayant and Caleb Scheckel and Chandler Hicks and Timothy Nissen and Linda Leduc and Mousumi Som and Matt Vassar",
year = "2017",
month = "6",
day = "1",
doi = "10.1371/journal.pone.0178379",
language = "English",
volume = "12",
journal = "PLoS ONE",
issn = "1932-6203",
publisher = "Public Library of Science",
number = "6",

}

Evidence of selective reporting bias in hematology journals : A systematic review. / Wayant, Cole; Scheckel, Caleb; Hicks, Chandler; Nissen, Timothy; Leduc, Linda; Som, Mousumi; Vassar, Matt.

In: PLoS ONE, Vol. 12, No. 6, e0178379, 01.06.2017.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Evidence of selective reporting bias in hematology journals

T2 - A systematic review

AU - Wayant, Cole

AU - Scheckel, Caleb

AU - Hicks, Chandler

AU - Nissen, Timothy

AU - Leduc, Linda

AU - Som, Mousumi

AU - Vassar, Matt

PY - 2017/6/1

Y1 - 2017/6/1

N2 - Introduction: Selective reporting bias occurs when chance or selective outcome reporting rather than the intervention contributes to group differences. The prevailing concern about selective reporting bias is the possibility of results being modified towards specific conclusions. In this study, we evaluate randomized controlled trials (RCTs) published in hematology journals, a group in which selective outcome reporting has not yet been explored. Methods: Our primary goal was to examine discrepancies between the reported primary and secondary outcomes in registered and published RCTs concerning hematological malignancies reported in hematology journals with a high impact factor. The secondary goals were to address whether outcome reporting discrepancies favored statistically significant outcomes, whether a pattern existed between the funding source and likelihood of outcome reporting bias, and whether temporal trends were present in outcome reporting bias. For trials with major outcome discrepancies, we contacted trialists to determine reasons for these discrepancies. Trials published between January 1, 2010 and December 31, 2015 in Blood; British Journal of Haematology; American Journal of Hematology; Leukemia; and Haematologica were included. Results: Of 499 RCTs screened, 109 RCTs were included. Our analysis revealed 118 major discrepancies and 629 total discrepancies. Among the 118 discrepancies, 30 (25.4%) primary outcomes were demoted, 47 (39.8%) primary outcomes were omitted, and 30 (25.4%) primary outcomes were added. Three (2.5%) secondary outcomes were upgraded to a primary outcome. The timing of assessment for a primary outcome changed eight (6.8%) times. Thirtyone major discrepancies were published with a P-value and twenty-five (80.6%) favored statistical significance. A majority of authors whom we contacted cited a pre-planned subgroup analysis as a reason for outcome changes. Conclusion: Our results suggest that outcome changes occur frequently in hematology trials. Because RCTs ultimately underpin clinical judgment and guide policy implementation, selective reporting could pose a threat to medical decision making.

AB - Introduction: Selective reporting bias occurs when chance or selective outcome reporting rather than the intervention contributes to group differences. The prevailing concern about selective reporting bias is the possibility of results being modified towards specific conclusions. In this study, we evaluate randomized controlled trials (RCTs) published in hematology journals, a group in which selective outcome reporting has not yet been explored. Methods: Our primary goal was to examine discrepancies between the reported primary and secondary outcomes in registered and published RCTs concerning hematological malignancies reported in hematology journals with a high impact factor. The secondary goals were to address whether outcome reporting discrepancies favored statistically significant outcomes, whether a pattern existed between the funding source and likelihood of outcome reporting bias, and whether temporal trends were present in outcome reporting bias. For trials with major outcome discrepancies, we contacted trialists to determine reasons for these discrepancies. Trials published between January 1, 2010 and December 31, 2015 in Blood; British Journal of Haematology; American Journal of Hematology; Leukemia; and Haematologica were included. Results: Of 499 RCTs screened, 109 RCTs were included. Our analysis revealed 118 major discrepancies and 629 total discrepancies. Among the 118 discrepancies, 30 (25.4%) primary outcomes were demoted, 47 (39.8%) primary outcomes were omitted, and 30 (25.4%) primary outcomes were added. Three (2.5%) secondary outcomes were upgraded to a primary outcome. The timing of assessment for a primary outcome changed eight (6.8%) times. Thirtyone major discrepancies were published with a P-value and twenty-five (80.6%) favored statistical significance. A majority of authors whom we contacted cited a pre-planned subgroup analysis as a reason for outcome changes. Conclusion: Our results suggest that outcome changes occur frequently in hematology trials. Because RCTs ultimately underpin clinical judgment and guide policy implementation, selective reporting could pose a threat to medical decision making.

UR - http://www.scopus.com/inward/record.url?scp=85019998458&partnerID=8YFLogxK

U2 - 10.1371/journal.pone.0178379

DO - 10.1371/journal.pone.0178379

M3 - Article

C2 - 28570573

AN - SCOPUS:85019998458

VL - 12

JO - PLoS ONE

JF - PLoS ONE

SN - 1932-6203

IS - 6

M1 - e0178379

ER -