Evaluation of Predatory Journal Publication in Systematic Reviews in the Top Five Orthopedic Journals: A Cross-Sectional Analysis

Background: What started out as a noble goal to make the world of research more accessible to the public through open access, journals were quickly adulterated by predatory journals who looked to use researchers for personal financial gain. The real problem with predatory journals goes far beyond their unethical entrepreneurial business practices. Predatory journals lack many of the foundational elements of scholarly writing, like peer review and editing, which is a crucial component of authentic scientific literature. As the medical profession continues to move towards evidence based medicine in clinical decision making, it is imperative to maintain quality research. The purpose of this study is to determine the prevalence of predatory journals within systematic reviews published in the top five orthopedic journals.

Methods: In a cross-sectional design, we systematically identified five orthopedic journals using Google Metrics based on their H-5 index. Using PubMed, we designed a search string to ensure we obtained the proper list of systematic reviews from each journal. The following search query was used for the top five orthopedic journals: (("systematic review"[Publication Type]) OR ("meta analysis"[Publication Type])) AND ("journal name"[Journal]). The first ten qualifying systematic reviews were then analyzed, and the journals cited were obtained. An algorithm created by author A.C. was used to determine predatory journals categorized by Beall’s list, an online public collection of journal titles which have demonstrated predatory submission tactics. Any matches were verified to be predatory by comparing it to the publication website listed on Beallslist.net. Systematic reviews that were deemed to include a predatory journal citation were further analyzed in a masked, duplicate fashion by D.T. and B.D. After the extraction process, result differences were adjudicated by D.T. and B.D.. with authors R.O. available for consult.

Results: Of the 50 systematic reviews examined, three systematic reviews included primary studies that were published in predatory journals (3/50, 6%). The systematic reviews that cited primary studies published in predatory journals were identified in The Journal of Bone & Joint Surgery, one systematic review (1/10, 10%), and Clinical Orthopaedics and Related Research, two systematic reviews (2/10, 20%) (Table 1). None of the identified systematic reviews that included studies published in these predatory journals comprised clinical practice guidelines. In total, the primary studies that were published in a predatory journal were referenced 53 times (Table 2).

Conclusion: Our study shows that systematic reviews from the top orthopedic journals contain primary studies published in predatory journals. Since systematic reviews are an important factor in determining clinical practice guidelines, and ultimately patient care, it is important to ensure the information supporting these guidelines comes from reputable, peer-reviewed sources. The purpose of this study was not to determine the effect predatory journals have on systematic reviews and further research is warranted.