Publication bias in dermatology systematic reviews and meta-analyses

Paul Atakpo, Matt Vassar

Research output: Contribution to journalArticle

14 Citations (Scopus)

Abstract

Systematic reviews and meta-analyses in dermatology provide high-level evidence for clinicians and policy makers that influence clinical decision making and treatment guidelines. One methodological problem with systematic reviews is the under representation of unpublished studies. This problem is due in part to publication bias. Omission of statistically non-significant data from meta-analyses may result in overestimation of treatment effect sizes which may lead to clinical consequences. Our goal was to assess whether systematic reviewers in dermatology evaluate and report publication bias. Further, we wanted to conduct our own evaluation of publication bias on meta-analyses that failed to do so. Our study considered systematic reviews and meta-analyses from ten dermatology journals from 2006 to 2016. A PubMed search was conducted, and all full-text articles that met our inclusion criteria were retrieved and coded by the primary author. 293 articles were included in our analysis. Additionally, we formally evaluated publication bias in meta-analyses that failed to do so using trim and fill and cumulative meta-analysis by precision methods. Publication bias was mentioned in 107 articles (36.5%) and was formally evaluated in 64 articles (21.8%). Visual inspection of a funnel plot was the most common method of evaluating publication bias. Publication bias was present in 45 articles (15.3%), not present in 57 articles (19.5%) and not determined in 191 articles (65.2%). Using the trim and fill method, 7 meta-analyses (33.33%) showed evidence of publication bias. Although the trim and fill method only found evidence of publication bias in 7 meta-analyses, the cumulative meta-analysis by precision method found evidence of publication bias in 15 meta-analyses (71.4%). Many of the reviews in our study did not mention or evaluate publication bias. Further, of the 42 articles that stated following PRISMA reporting guidelines, 19 (45.2%) evaluated for publication bias. In comparison to other studies, we found that systematic reviews in dermatology were less likely to evaluate for publication bias. Evaluating and reporting the likelihood of publication bias should be standard practice in systematic reviews when appropriate.

Original languageEnglish
Pages (from-to)69-74
Number of pages6
JournalJournal of Dermatological Science
Volume82
Issue number2
DOIs
StatePublished - 1 May 2016

Fingerprint

Dermatology
Publication Bias
Meta-Analysis
Inspection
Decision making
Guidelines

Keywords

  • Dermatology
  • Meta-analysis
  • Publication bias
  • Systematic review

Cite this

@article{7cfa418a0bcd48e0bbe5d790a6c8d701,
title = "Publication bias in dermatology systematic reviews and meta-analyses",
abstract = "Systematic reviews and meta-analyses in dermatology provide high-level evidence for clinicians and policy makers that influence clinical decision making and treatment guidelines. One methodological problem with systematic reviews is the under representation of unpublished studies. This problem is due in part to publication bias. Omission of statistically non-significant data from meta-analyses may result in overestimation of treatment effect sizes which may lead to clinical consequences. Our goal was to assess whether systematic reviewers in dermatology evaluate and report publication bias. Further, we wanted to conduct our own evaluation of publication bias on meta-analyses that failed to do so. Our study considered systematic reviews and meta-analyses from ten dermatology journals from 2006 to 2016. A PubMed search was conducted, and all full-text articles that met our inclusion criteria were retrieved and coded by the primary author. 293 articles were included in our analysis. Additionally, we formally evaluated publication bias in meta-analyses that failed to do so using trim and fill and cumulative meta-analysis by precision methods. Publication bias was mentioned in 107 articles (36.5{\%}) and was formally evaluated in 64 articles (21.8{\%}). Visual inspection of a funnel plot was the most common method of evaluating publication bias. Publication bias was present in 45 articles (15.3{\%}), not present in 57 articles (19.5{\%}) and not determined in 191 articles (65.2{\%}). Using the trim and fill method, 7 meta-analyses (33.33{\%}) showed evidence of publication bias. Although the trim and fill method only found evidence of publication bias in 7 meta-analyses, the cumulative meta-analysis by precision method found evidence of publication bias in 15 meta-analyses (71.4{\%}). Many of the reviews in our study did not mention or evaluate publication bias. Further, of the 42 articles that stated following PRISMA reporting guidelines, 19 (45.2{\%}) evaluated for publication bias. In comparison to other studies, we found that systematic reviews in dermatology were less likely to evaluate for publication bias. Evaluating and reporting the likelihood of publication bias should be standard practice in systematic reviews when appropriate.",
keywords = "Dermatology, Meta-analysis, Publication bias, Systematic review",
author = "Paul Atakpo and Matt Vassar",
year = "2016",
month = "5",
day = "1",
doi = "10.1016/j.jdermsci.2016.02.005",
language = "English",
volume = "82",
pages = "69--74",
journal = "Journal of Dermatological Science",
issn = "0923-1811",
publisher = "Elsevier Ireland Ltd",
number = "2",

}

Publication bias in dermatology systematic reviews and meta-analyses. / Atakpo, Paul; Vassar, Matt.

In: Journal of Dermatological Science, Vol. 82, No. 2, 01.05.2016, p. 69-74.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Publication bias in dermatology systematic reviews and meta-analyses

AU - Atakpo, Paul

AU - Vassar, Matt

PY - 2016/5/1

Y1 - 2016/5/1

N2 - Systematic reviews and meta-analyses in dermatology provide high-level evidence for clinicians and policy makers that influence clinical decision making and treatment guidelines. One methodological problem with systematic reviews is the under representation of unpublished studies. This problem is due in part to publication bias. Omission of statistically non-significant data from meta-analyses may result in overestimation of treatment effect sizes which may lead to clinical consequences. Our goal was to assess whether systematic reviewers in dermatology evaluate and report publication bias. Further, we wanted to conduct our own evaluation of publication bias on meta-analyses that failed to do so. Our study considered systematic reviews and meta-analyses from ten dermatology journals from 2006 to 2016. A PubMed search was conducted, and all full-text articles that met our inclusion criteria were retrieved and coded by the primary author. 293 articles were included in our analysis. Additionally, we formally evaluated publication bias in meta-analyses that failed to do so using trim and fill and cumulative meta-analysis by precision methods. Publication bias was mentioned in 107 articles (36.5%) and was formally evaluated in 64 articles (21.8%). Visual inspection of a funnel plot was the most common method of evaluating publication bias. Publication bias was present in 45 articles (15.3%), not present in 57 articles (19.5%) and not determined in 191 articles (65.2%). Using the trim and fill method, 7 meta-analyses (33.33%) showed evidence of publication bias. Although the trim and fill method only found evidence of publication bias in 7 meta-analyses, the cumulative meta-analysis by precision method found evidence of publication bias in 15 meta-analyses (71.4%). Many of the reviews in our study did not mention or evaluate publication bias. Further, of the 42 articles that stated following PRISMA reporting guidelines, 19 (45.2%) evaluated for publication bias. In comparison to other studies, we found that systematic reviews in dermatology were less likely to evaluate for publication bias. Evaluating and reporting the likelihood of publication bias should be standard practice in systematic reviews when appropriate.

AB - Systematic reviews and meta-analyses in dermatology provide high-level evidence for clinicians and policy makers that influence clinical decision making and treatment guidelines. One methodological problem with systematic reviews is the under representation of unpublished studies. This problem is due in part to publication bias. Omission of statistically non-significant data from meta-analyses may result in overestimation of treatment effect sizes which may lead to clinical consequences. Our goal was to assess whether systematic reviewers in dermatology evaluate and report publication bias. Further, we wanted to conduct our own evaluation of publication bias on meta-analyses that failed to do so. Our study considered systematic reviews and meta-analyses from ten dermatology journals from 2006 to 2016. A PubMed search was conducted, and all full-text articles that met our inclusion criteria were retrieved and coded by the primary author. 293 articles were included in our analysis. Additionally, we formally evaluated publication bias in meta-analyses that failed to do so using trim and fill and cumulative meta-analysis by precision methods. Publication bias was mentioned in 107 articles (36.5%) and was formally evaluated in 64 articles (21.8%). Visual inspection of a funnel plot was the most common method of evaluating publication bias. Publication bias was present in 45 articles (15.3%), not present in 57 articles (19.5%) and not determined in 191 articles (65.2%). Using the trim and fill method, 7 meta-analyses (33.33%) showed evidence of publication bias. Although the trim and fill method only found evidence of publication bias in 7 meta-analyses, the cumulative meta-analysis by precision method found evidence of publication bias in 15 meta-analyses (71.4%). Many of the reviews in our study did not mention or evaluate publication bias. Further, of the 42 articles that stated following PRISMA reporting guidelines, 19 (45.2%) evaluated for publication bias. In comparison to other studies, we found that systematic reviews in dermatology were less likely to evaluate for publication bias. Evaluating and reporting the likelihood of publication bias should be standard practice in systematic reviews when appropriate.

KW - Dermatology

KW - Meta-analysis

KW - Publication bias

KW - Systematic review

UR - http://www.scopus.com/inward/record.url?scp=84959253048&partnerID=8YFLogxK

U2 - 10.1016/j.jdermsci.2016.02.005

DO - 10.1016/j.jdermsci.2016.02.005

M3 - Article

C2 - 26925817

AN - SCOPUS:84959253048

VL - 82

SP - 69

EP - 74

JO - Journal of Dermatological Science

JF - Journal of Dermatological Science

SN - 0923-1811

IS - 2

ER -