Determinants of selective reporting: A taxonomy based on content analysis of a random selection of the literature

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Background

Selective reporting is wasteful, leads to bias in the published record and harms the credibility of science. Studies on potential determinants of selective reporting currently lack a shared taxonomy and a causal framework.

Objective

To develop a taxonomy of determinants of selective reporting in science.

Design

Inductive qualitative content analysis of a random selection of the pertinent literature including empirical research and theoretical reflections.

Methods

Using search terms for bias and selection combined with terms for reporting and publication, we systematically searched the PubMed, Embase, PsycINFO and Web of Science databases up to January 8, 2015. Of the 918 articles identified, we screened a 25 percent random selection. From eligible articles, we extracted phrases that mentioned putative or possible determinants of selective reporting, which we used to create meaningful categories. We stopped when no new categories emerged in the most recently analyzed articles (saturation).

Results

Saturation was reached after analyzing 64 articles. We identified 497 putative determinants, of which 145 (29%) were supported by empirical findings. The determinants represented 12 categories (leaving 3% unspecified): focus on preferred findings (36%), poor or overly flexible research design (22%), high-risk area and its development (8%), dependence upon sponsors (8%), prejudice (7%), lack of resources including time (3%), doubts about reporting being worth the effort (3%), limitations in reporting and editorial practices (3%), academic publication system hurdles (3%), unfavorable geographical and regulatory environment (2%), relationship and collaboration issues (2%), and potential harm (0.4%).

Conclusions

We designed a taxonomy of putative determinants of selective reporting consisting of 12 categories. The taxonomy may help develop theory about causes of selection bias and guide policies to prevent selective reporting.

Related collections

Most cited references 21

Record: found
Abstract: found
Article: found

Is Open Access

Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data

Daniele Fanelli (2010)

The growing competition and “publish or perish” culture in academia might conflict with the objectivity and integrity of research, because it forces scientists to produce “publishable” results at all costs. Papers are less likely to be published and to be cited if they report “negative” results (results that fail to support the tested hypothesis). Therefore, if publication pressures increase scientific bias, the frequency of “positive” results in the literature should be higher in the more competitive and “productive” academic environments. This study verified this hypothesis by measuring the frequency of positive results in a large random sample of papers with a corresponding author based in the US. Across all disciplines, papers were more likely to support a tested hypothesis if their corresponding authors were working in states that, according to NSF data, produced more academic papers per capita. The size of this effect increased when controlling for state's per capita R&D expenditure and for study characteristics that previous research showed to correlate with the frequency of positive results, including discipline and methodology. Although the confounding effect of institutions' prestige could not be excluded (researchers in the more productive universities could be the most clever and successful in their experiments), these results support the hypothesis that competitive academic environments increase not only scientists' productivity but also their bias. The same phenomenon might be observed in other countries where academic competition and pressures to publish are high.

0 comments Cited 222 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors.

An-Wen Chan, Douglas G. Altman (2005)

To examine the extent and nature of outcome reporting bias in a broad cohort of published randomised trials. Retrospective review of publications and follow up survey of authors. Cohort All journal articles of randomised trials indexed in PubMed whose primary publication appeared in December 2000. Prevalence of incompletely reported outcomes per trial; reasons for not reporting outcomes; association between completeness of reporting and statistical significance. 519 trials with 553 publications and 10,557 outcomes were identified. Survey responders (response rate 69%) provided information on unreported outcomes but were often unreliable--for 32% of those who denied the existence of such outcomes there was evidence to the contrary in their publications. On average, over 20% of the outcomes measured in a parallel group trial were incompletely reported. Within a trial, such outcomes had a higher odds of being statistically non-significant compared with fully reported outcomes (odds ratio 2.0 (95% confidence interval 1.6 to 2.7) for efficacy outcomes; 1.9 (1.1 to 3.5) for harm outcomes). The most commonly reported reasons for omitting efficacy outcomes included space constraints, lack of clinical importance, and lack of statistical significance. Incomplete reporting of outcomes within published articles of randomised trials is common and is associated with statistical non-significance. The medical literature therefore represents a selective and biased subset of study outcomes, and trial protocols should be made publicly available.

0 comments Cited 181 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: found

Is Open Access

Reporting bias in medical research - a narrative review

Natalie McGauran, Beate Wieseler, Julia Kreis … (2010)

Reporting bias represents a major problem in the assessment of health care interventions. Several prominent cases have been described in the literature, for example, in the reporting of trials of antidepressants, Class I anti-arrhythmic drugs, and selective COX-2 inhibitors. The aim of this narrative review is to gain an overview of reporting bias in the medical literature, focussing on publication bias and selective outcome reporting. We explore whether these types of bias have been shown in areas beyond the well-known cases noted above, in order to gain an impression of how widespread the problem is. For this purpose, we screened relevant articles on reporting bias that had previously been obtained by the German Institute for Quality and Efficiency in Health Care in the context of its health technology assessment reports and other research work, together with the reference lists of these articles. We identified reporting bias in 40 indications comprising around 50 different pharmacological, surgical (e.g. vacuum-assisted closure therapy), diagnostic (e.g. ultrasound), and preventive (e.g. cancer vaccines) interventions. Regarding pharmacological interventions, cases of reporting bias were, for example, identified in the treatment of the following conditions: depression, bipolar disorder, schizophrenia, anxiety disorder, attention-deficit hyperactivity disorder, Alzheimer's disease, pain, migraine, cardiovascular disease, gastric ulcers, irritable bowel syndrome, urinary incontinence, atopic dermatitis, diabetes mellitus type 2, hypercholesterolaemia, thyroid disorders, menopausal symptoms, various types of cancer (e.g. ovarian cancer and melanoma), various types of infections (e.g. HIV, influenza and Hepatitis B), and acute trauma. Many cases involved the withholding of study data by manufacturers and regulatory agencies or the active attempt by manufacturers to suppress publication. The ascertained effects of reporting bias included the overestimation of efficacy and the underestimation of safety risks of interventions. In conclusion, reporting bias is a widespread phenomenon in the medical literature. Mandatory prospective registration of trials and public access to study data via results databases need to be introduced on a worldwide scale. This will allow for an independent review of research data, help fulfil ethical obligations towards patients, and ensure a basis for fully-informed decision making in the health care system.

0 comments Cited 145 times – based on 0 reviews      Review now

Bookmark

All references

Author and article information

Contributors

Jenny T. van der Steen:

ORCID: http://orcid.org/0000-0002-9063-7501

Role: ConceptualizationRole: Formal analysisRole: Funding acquisitionRole: InvestigationRole: MethodologyRole: ValidationRole: VisualizationRole: Writing – original draft

Cornelis A. van den Bogert: Role: Formal analysisRole: InvestigationRole: MethodologyRole: Project administrationRole: ValidationRole: VisualizationRole: Writing – review & editing

Mirjam C. van Soest-Poortvliet: Role: InvestigationRole: Project administrationRole: Writing – review & editing

Soulmaz Fazeli Farsani: Role: InvestigationRole: Writing – review & editing

René H. J. Otten: Role: MethodologyRole: Project administrationRole: Writing – review & editing

Gerben ter Riet: Role: Formal analysisRole: MethodologyRole: SupervisionRole: ValidationRole: Writing – review & editing

Lex M. Bouter: Role: ConceptualizationRole: MethodologyRole: SupervisionRole: Writing – review & editing

Martin Chalumeau: Role: Editor

Journal

Journal ID (nlm-ta): PLoS One

Journal ID (iso-abbrev): PLoS ONE

Journal ID (publisher-id): plos

Journal ID (pmc): plosone

Title: PLoS ONE

Publisher: Public Library of Science (San Francisco, CA USA )

ISSN (Electronic): 1932-6203

Publication date (Electronic): 5 February 2018

Publication date Collection: 2018

Volume: 13

Issue: 2

Electronic Location Identifier: e0188247

Affiliations

[1 ] Department of Public Health and Primary Care (PHEG), Leiden University Medical Center (LUMC), Leiden, the Netherlands

[2 ] Department of Primary and Community Care, Radboud university medical center, Nijmegen, the Netherlands

[3 ] Division of Pharmacoepidemiology & Clinical Pharmacology, Utrecht University, Utrecht, the Netherlands

[4 ] Program on Aging, Netherlands Institute of Mental Health and Addiction (Trimbos-instituut), Utrecht, the Netherlands

[5 ] Division of Pharmacoepidemiology & Clinical Pharmacology, Utrecht University, Utrecht, the Netherlands

[6 ] Medical Library, Vrije Universiteit, Amsterdam, the Netherlands

[7 ] Department of General Practice, University of Amsterdam, Academic Medical Center, Amsterdam, the Netherlands

[8 ] Department of Epidemiology & Biostatistics, VU University Medical Center, Amsterdam, the Netherlands

[9 ] Vrije Universiteit, Faculty of Humanities, Department of Philosophy, Amsterdam, the Netherlands

Universite Paris Descartes, FRANCE

Author notes

Competing Interests: The authors have declared that no competing interests exist.

[¤]

Current address: Boehringer Ingelheim GmbH, Ingelheim am Rhein, Germany.

* E-mail: jtvandersteen@ 123456lumc.nl

Author information

Jenny T. van der Steen http://orcid.org/0000-0002-9063-7501

Article

Publisher ID: PONE-D-17-15619

DOI: 10.1371/journal.pone.0188247

PMC ID: 5798766

PubMed ID: 29401492

SO-VID: 780cb177-b340-490c-af78-4a8f796597b3

License:

This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

History

Date received : 24 April 2017

Date accepted : 5 November 2017

Page count

Figures: 2, Tables: 3, Pages: 15

Funding

Funded by: funder-id http://dx.doi.org/10.13039/501100003246, Nederlandse Organisatie voor Wetenschappelijk Onderzoek;

Award ID: Vidi 917.11.339

Award Recipient :

ORCID: http://orcid.org/0000-0002-9063-7501

Jenny T. van der Steen

Funded by: funder-id http://dx.doi.org/10.13039/501100002999, Ministerie van Volksgezondheid, Welzijn en Sport;

Award ID: PhD grant

Award Recipient : Cornelis A. van den Bogert

The design of the study and writing of the article was funded by a career award for JTS from the Netherlands Organisation for Scientific Research (NWO; Innovational Research Incentives Scheme: Vidi grant number 917.11.339), and Leiden University Medical Center, Leiden, The Netherlands. CAvdB was supported by a PhD grant from the Dutch Ministry of Health, Welfare and Sports. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Custom metadata

Data Availability All relevant data are within the paper and its Supporting Information files.

Determinants of selective reporting: A taxonomy based on content analysis of a random selection of the literature

Read this article at

Abstract

Background

Objective

Design

Methods

Results

Conclusions

Related collections

PLOS Climate

Most cited references 21

Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data

Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors.

Reporting bias in medical research - a narrative review

Author and article information

Contributors

Journal

Affiliations

Author notes

Author information

Article

History

Page count

Funding

Categories

Custom metadata

Comments

Comment on this article

Similar content 107

Cited by 8

Most referenced authors 1,844