For Publishers
DiscoveryMetadataPeer reviewHostingPublishing
For Researchers
JoinPublishReviewCollect
Register
Blog
About
Search
Advanced search
My ScienceOpen
Search
For Publishers
For Researchers
Blog
About
89
views
54
references
 Top references
cited by
71
    
0 reviews
 Review
0
comments
 Comment
0
recommends
+1 Recommend
0 collections
    0
    shares
      244
      similar
       All similar
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Instruments for Assessing Risk of Bias and Other Methodological Criteria of Published Animal Studies: A Systematic Review

      review-article

      Read this article at

      ScienceOpenPublisherPMC
      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Background: Results from animal toxicology studies are critical to evaluating the potential harm from exposure to environmental chemicals or the safety of drugs prior to human testing. However, there is significant debate about how to evaluate the methodology and potential biases of the animal studies. There is no agreed-upon approach, and a systematic evaluation of current best practices is lacking.

          Objective: We performed a systematic review to identify and evaluate instruments for assessing the risk of bias and/or other methodological criteria of animal studies.

          Method: We searched Medline (January 1966–November 2011) to identify all relevant articles. We extracted data on risk of bias criteria (e.g., randomization, blinding, allocation concealment) and other study design features included in each assessment instrument.

          Discussion: Thirty distinct instruments were identified, with the total number of assessed risk of bias, methodological, and/or reporting criteria ranging from 2 to 25. The most common criteria assessed were randomization (25/30, 83%), investigator blinding (23/30, 77%), and sample size calculation (18/30, 60%). In general, authors failed to empirically justify why these or other criteria were included. Nearly all (28/30, 93%) of the instruments have not been rigorously tested for validity or reliability.

          Conclusion: Our review highlights a number of risk of bias assessment criteria that have been empirically tested for animal research, including randomization, concealment of allocation, blinding, and accounting for all animals. In addition, there is a need for empirically testing additional methodological criteria and assessing the validity and reliability of a standard risk of bias assessment instrument.

          Citation: Krauth D, Woodruff TJ, Bero L. 2013. Instruments for assessing risk of bias and other methodological criteria of published animal studies: a systematic review. Environ Health Perspect 121:985–992 (2013);  http://dx.doi.org/10.1289/ehp.1206389

          Related collections

          Most cited references54

          • Record: found
          • Abstract: found
          • Article: not found

          Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials.

          K Schulz, I. Chalmers, R Hayes … (1995)
          To determine if inadequate approaches to randomized controlled trial design and execution are associated with evidence of bias in estimating treatment effects. An observational study in which we assessed the methodological quality of 250 controlled trials from 33 meta-analyses and then analyzed, using multiple logistic regression models, the associations between those assessments and estimated treatment effects. Meta-analyses from the Cochrane Pregnancy and Childbirth Database. The associations between estimates of treatment effects and inadequate allocation concealment, exclusions after randomization, and lack of double-blinding. Compared with trials in which authors reported adequately concealed treatment allocation, trials in which concealment was either inadequate or unclear (did not report or incompletely reported a concealment approach) yielded larger estimates of treatment effects (P < .001). Odds ratios were exaggerated by 41% for inadequately concealed trials and by 30% for unclearly concealed trials (adjusted for other aspects of quality). Trials in which participants had been excluded after randomization did not yield larger estimates of effects, but that lack of association may be due to incomplete reporting. Trials that were not double-blind also yielded larger estimates of effects (P = .01), with odds ratios being exaggerated by 17%. This study provides empirical evidence that inadequate methodological approaches in controlled trials, particularly those representing poor allocation concealment, are associated with bias. Readers of trial reports should be wary of these pitfalls, and investigators must improve their design, execution, and reporting of trials.
          Article 0 comments Cited 558 times – based on 0 reviews     Review now
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            A call for transparent reporting to optimize the predictive value of preclinical research.

            Story C Landis, Susan Amara, Khusru Asadullah … (2012)
            The US National Institute of Neurological Disorders and Stroke convened major stakeholders in June 2012 to discuss how to improve the methodological reporting of animal studies in grant applications and publications. The main workshop recommendation is that at a minimum studies should report on sample-size estimation, whether and how animals were randomized, whether investigators were blind to the treatment, and the handling of data. We recognize that achieving a meaningful improvement in the quality of reporting will require a concerted effort by investigators, reviewers, funding agencies and journal editors. Requiring better reporting of animal studies will raise awareness of the importance of rigorous study design to accelerate scientific progress.
            Article 0 comments Cited 438 times – based on 0 reviews     Review now
              Bookmark
              • Record: found
              • Abstract: not found
              • Article: not found

              The hazards of scoring the quality of clinical trials for meta-analysis.

              Peter Jüni (1999)
              Although it is widely recommended that clinical trials undergo some type of quality review, the number and variety of quality assessment scales that exist make it unclear how to achieve the best assessment. To determine whether the type of quality assessment scale used affects the conclusions of meta-analytic studies. Meta-analysis of 17 trials comparing low-molecular-weight heparin (LMWH) with standard heparin for prevention of postoperative thrombosis using 25 different scales to identify high-quality trials. The association between treatment effect and summary scores and the association with 3 key domains (concealment of treatment allocation, blinding of outcome assessment, and handling of withdrawals) were examined in regression models. Pooled relative risks of deep vein thrombosis with LMWH vs standard heparin in high-quality vs low-quality trials as determined by 25 quality scales. Pooled relative risks from high-quality trials ranged from 0.63 (95% confidence interval [CI], 0.44-0.90) to 0.90 (95% CI, 0.67-1.21) vs 0.52 (95% CI, 0.24-1.09) to 1.13 (95% CI, 0.70-1.82) for low-quality trials. For 6 scales, relative risks of high-quality trials were close to unity, indicating that LMWH was not significantly superior to standard heparin, whereas low-quality trials showed better protection with LMWH (P<.05). Seven scales showed the opposite: high quality trials showed an effect whereas low quality trials did not. For the remaining 12 scales, effect estimates were similar in the 2 quality strata. In regression analysis, summary quality scores were not significantly associated with treatment effects. There was no significant association of treatment effects with allocation concealment and handling of withdrawals. Open outcome assessment, however, influenced effect size with the effect of LMWH, on average, being exaggerated by 35% (95% CI, 1%-57%; P= .046). Our data indicate that the use of summary scores to identify trials of high quality is problematic. Relevant methodological aspects should be assessed individually and their influence on effect sizes explored.
              Article 0 comments Cited 383 times – based on 0 reviews     Review now
                Bookmark
                All references

                Author and article information

                Journal
                Journal ID (nlm-ta): Environ Health Perspect
                Journal ID (iso-abbrev): Environ. Health Perspect
                Journal ID (publisher-id): EHP
                Title: Environmental Health Perspectives
                Publisher: National Institute of Environmental Health Sciences
                ISSN (Print): 0091-6765
                ISSN (Electronic): 1552-9924
                Publication date (Electronic): 14 June 2013
                Publication date (Print): September 2013
                Volume: 121
                Issue: 9
                Pages: 985-992
                Affiliations
                [1 ]Department of Clinical Pharmacy, and
                [2 ]Department of Obstetrics, Gynecology, and Reproductive Sciences, University of California, San Francisco, San Francisco, California, USA
                [3 ]Program on Reproductive Health and the Environment, Oakland, California, USA
                [4 ]Institute for Health Policy Studies, University of California, San Francisco, San Francisco, California, USA
                Author notes
                Address correspondence to L. Bero, Department of Clinical Pharmacy, Institute for Health Policy Studies, University of California, San Francisco, 3333 California St., Suite 420, Box 0613, San Francisco, CA 94118 USA. Telephone: (415) 476-1067. E-mail: berol@ 123456pharmacy.ucsf.edu
                Article
                Publisher ID: ehp.1206389
                DOI: 10.1289/ehp.1206389
                PMC ID: 3764080
                PubMed ID: 23771496
                SO-VID: 8ba284a0-655b-45c2-851d-dc7832468a8a
                Copyright statement: Copyright @ 2013
                License:

                This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, properly cited.

                History
                Date received : 10 December 2012
                Date accepted : 10 June 2013
                Date: 14 June 2013
                Date: 01 September 2013
                Categories
                Subject: Review

                ScienceOpen disciplines: Public health
                Data availability:
                ScienceOpen disciplines: Public health

                Comments

                Comment on this article

                scite_

                Similar content244

                See all similar

                Cited by63

                See all cited by

                Most referenced authors997

                See all reference authors