SciELO - Scientific Electronic Library Online

 
vol.40 número3Eficacia del bloqueo facetario en pacientes con síndrome facetario lumbarEnvejecimiento de la población: un reto para la salud pública índice de autoresíndice de materiabúsqueda de artículos
Home Pagelista alfabética de revistas  

Servicios Personalizados

Revista

Articulo

Indicadores

Links relacionados

  • En proceso de indezaciónCitado por Google
  • No hay articulos similaresSimilares en SciELO
  • En proceso de indezaciónSimilares en Google

Compartir


Colombian Journal of Anestesiology

versión impresa ISSN 0120-3347

Rev. colomb. anestesiol. vol.40 no.3 Bogotá jul./oct. 2012

https://doi.org/10.1016/j.rca.2012.05.013 

Scientific and Technological Research

Risk of bias assessment of clinical trials published in the Revista Colombiana de Anestesiología*

Evaluación del «riesgo de sesgo» de los ensayos clínicos publicados en la Revista Colombiana de Anestesiología

José Andrés Calvachea'*, Leticia Barajas-Navab, Claudia Sánchezc, Alberto Giraldod, José Domingo Alarcóne y Mario Delgado-Nogueraf

a Anesthesiology Department, Universidad del Cauca, Colombia. Anesthesiology/Biostatistics Departments, Erasmus University Medical Center, Rotterdam, The Netherlands
b Red Cochrane Iberoamericana, Barcelona, Spain
c Profesor at Universidad Antonio Nariño, Bogotá, Colombia
d Anesthesiologist, Clínica Amiga, Anesthesiology Department, Universidad del Valle, Cali, Colombia
e Red Cochrane Iberoamericana, Professor at Universidad Surcolombiana de Neiva, Neiva, Colombia
f Professor at Pediatrics Department, Universidad del Cauca, Popayán, Colombia

* Please cite this article as: Calvache JA, et al. Evaluación del "riesgo de sesgo" de los ensayos clínicos publicados en la Revista Colombiana de Anestesiología. Rev Colomb Anestesiol. 2012;40:183-91.
* Corresponding author at: Departamento de Anestesiología, Hospital Universitario San José, Cra 610N-142, Tercer piso, Popayán, Colombia. E-mail address jacalvache@gmail.com (J.A. Calvache).
© 2012 Published by Elsevier España, S.L. on behalf of Sociedad Colombiana de Anestesiología y Reanimación.


ARTICLE INFO

Article history: Received 2 April 2012 Accepted 29 May 2012

Abstract

Introduction: Randomized clinical trials (RCT) are one of the most reliable methods of scientific investigation in health sciences. It is a corner stone of evidence based medicine and the backbone of high standard knowledge. Several types of errors can compromise the results and affect its validity.

Objectives: To assess the risk of bias of the clinical trials published in the Revista Colombiana de Anestesiología (RCA) medical journal by applying the «risk of bias detection» tool of the Cochrane Collaboration.

Methods: All the clinical trials in the RCA journal were found by carrying out a systematic research. These trials were randomly distributed among 6 evaluators trained in the use of the «risk of bias detection» tool of the Cochrane Collaboration. Results were presented descriptively, graphically and chronologically to each of the 6 parameters that conform the «risk of bias detection» tool.

Results:The RCA journal has published 40 volumes as of 1973. The searching process identified a total 75 RCT up until 2009. The frequency of RCT publication has risen with time. The cities with most publications were Bogotá DC and Medellín, and most trials were related to the management of acute and chronic pain. The greatest risk of bias (29% of all RCT) was found in the concealing of randomization sequences (parameter 2). 30% of the studies showed four or more parameter values of low risk of bias. A trend of decreasing proportion of high risk values was observed as time passed.

Conclusions:There is a sustained trend of improvement and risk reduction in RCTs' in the RCA journal.

Keywords: Clinical trials, Randomized controlled trial, Selection bias, Anesthesiology.

© 2012 Published by Elsevier España, S.L. on behalf of Sociedad Colombiana de Anestesiología y Reanimación.

Resumen

Introducción: El ensayo clínico aleatorizado (ECA) es una de las mejores formas de adquisición de pruebas científicas en ciencias de la salud. Es catalogado como la piedra angular de la medicina basada en la evidencia y como eje de la formación de conocimiento de alta calidad. Diversos tipos de sesgos pueden comprometer sus resultados y afectar su validez interna. Objetivos:Evaluar el «riesgo de sesgo» de los ensayos clínicos publicados en la Revista Colombiana de Anestesiología (RCA) mediante la aplicación de la herramienta para detección de «riesgo de sesgo» de la Colaboración Cochrane.

Métodos:Mediante una búsqueda sistemática se identificaron todos los ensayos clínicos publicados en la RCA. Estos se distribuyeron de forma aleatoria entre 6 evaluadores entrenados en la utilización de la herramienta para detección de «riesgo de sesgo» de la Colaboración Cochrane. Los resultados se presentaron de forma descriptiva, gráfica y temporal para cada uno de los 6 dominios que constituyen la herramienta.

Resultados: La RCA ha publicado 40 volúmenes desde 1973. El proceso de búsqueda identificó hasta el 2009 un total de 75 ECA. La frecuencia de publicación de ECA ha aumentado con el paso del tiempo, las ciudades con mayor publicación fueron Bogotá DC y Medellín, y en su mayoría están relacionados al manejo del dolor agudo y crónico. El mayor riesgo de sesgo (29% de los ECA) se identificó en el encubrimiento de la secuencia de aleatorización (dominio 2). El 30% de los estudios presentaron 4 dominios o más clasificados como bajo riesgo de sesgo. Se apreció una tendencia a la reducción de la proporción de dominios clasificados como alto riesgo de sesgo con el paso del tiempo.

Conclusiones:Existe una tendencia sostenida al mejoramiento y a la reducción del riesgo de sesgo de los ECA publicados en la RCA, con algunos puntos a fortalecer en el proceso de diseño, conducción, análisis y reporte.

Palabras clave: Ensayos clínicos, Ensayo clínico controlado, aleatorizado, Sesgo de selección, Anestesiología.

© 2012 Publicado por Elsevier España, S.L. en nombre de Sociedad Colombiana de Anestesiología y Reanimación.

Introduction

Clinical trials are quantitative, comparative and controlled experiments in which a group of researchers study two or more randomly assigned interventions in a group of individuals.1 It is one of the finest methods of scientific research in health sciences and the design of it provides a greater input of causality. It is considered to be the corner stone of evidence based medicine and the backbone of high standard knowledge.2 The findings are the basis for high quality scientific evidence, systematic reviews and clinical practice guides.

The place RCTs in the evidence hierarchy has given way to some erroneous conceptions of value, quality and absence of bias that have been related only to RCT in name only. In spite of its advantages, it is not without methodological issues that may compromise its internal and external values.3

Regarding internal value, the possibility of bias that pose a threat to the RCT may be a result of a flawed design, conduction, analysis, interpretation and report. Such possibilities have been classified in 4 groups: selection bias, execution bias, weathering bias and detection bias.4 An inadequate randomization, i.e. failure to conceal subject assigning, no double blind method or differential follow-up loss, has been proven to affect the effects of treatments.5-8 Evidence suggests that improper subject assignment concealment and non-double-blind methods may result in overestimating the effect of the intervention on trial.9

The quality assessment method is a matter of controversy. The absence of a reference standard for the assessment and the uncertainty of it on the effect of the intervention have led to the creation of several scales and checklists.10 However, only 12% of the instruments has been assessed empirically.11 Many of them include assessment of elements related to the report and design of the trial and not directly to the bias.4

In February 2008, the Cochrane Collaboration presented a tool for the assessment of bias risks designed to establish the internal validity of a clinical trial.4 The selection of components included (6 parameters) was based on evidence of its positive or negative associations with the effect.7,8,12 The authors' objective was to discern among the elements used for the design of a clinical trial and those necessary for its report.

Every parameter is assessed in three categories of risk evaluation (high, low, unknown) that finally results in individual assessment of every study and for every parameter with two evaluation graphs. The trials are classiied based on the original report and may also include additional documents such as the trial's protocol.13

The RCA journal is the oficial publication of the Sociedad Colombiana de Anestesiología y Reanimación (SCARE), and its purpose is to inform and update scientiic knowledge of the speciality and related areas of medicine. Since its irst issue in 1973, 40 volumes with 157 issues have been published. It is currently classified as A2 category in Publindex at Colciencias.

The aim of this study was to assess the risk of bias of all RCTs published in the RCA journal by using the Cochrane Collaboration "risk of bias detection" tool.

Methods

It is an observational study that included all randomized clinical trials published in the RCA journal since the publication of its first issue in 1973 up to the contents of volume 37, issue no. 1 in 2009.

A systematic search was designed for the identification of all published clinical trials which included the SCIELO database, a revision of all the RCA journal volumes and the research and location tool of the http://www.revcolanest.com.co. website. Clinical trials and before-after experiments that were not performed on humans were excluded.

Once the studies were identified, they were divided into six equal groups using a computer generated random numbering table (R Development Core Team (2010)14). Each group of trials was further divided into six evaluators with previous experience and training in clinical trial assessment by the use of the Cochrane Collaboration methodology and/or systematic review publication. The original trial numbering and distribution was carried out by the main author of this study.

Each evaluator inputs every study into an electronic survey. This survey collected data corresponding to: 1. Title of the study, 2. Type of clinical trial, 3. Number of studied groups, 4. Number of authors, 5. The studied outcomes and time of study of the primary outcome, 6. Date and location of execution, 7. Number of participants and finally, the six parameters of the Cochrane Collaboration "risk of bias detection" tool.

Data analysis

A database was designed in the R14 statistics package for data analysis. The results are reported descriptively and graphically, and also include the description of the studies, the chronologic and geographic rates and the proportion of bias risk assessment for each parameter among its three categories (high risk, low risk, unknown). In addition, graphs were created for assessment of publication trends, their relation to bias risk and number of recruited subjects.

Results

The RCA journal has published 40 volumes to the present day, four issues per year for a total 157 issues (considering number 1, volume 40 in 2012). Its publication began in July 1973 (Volume 1, Issue 1) and has continued for the past 39 years.

The first clinical trial published in the RCA journal was in issue no. 10, in 1982. The search for articles began at that issue and up to issue no. 1 of volume no. 37 in 2009 and yielded a total 75 published clinical trials. The last eight issues of the RCA journal, which contain 13 new clinical trials were not included (issue no. 3 in 2010, no. 4 in 2011 and no. 1 in 2012), because data collection and analysis were carried out during this time. More details on the searching and selection processes are shown in Fig. 1.

Table 1 shows the general character of the studies and research areas organized by frequency and follow-up time of primary outcome. Fig. 2 shows the execution site and Fig. 3 shows the number of studies published in 3-year intervals in the History of the RCA journal.

Classification of low risk of bias was achieved by 66% of the trials for parameter 4 (follow-up loss), 54% (selective report) and 45% in parameter 1 (randomization). The scores with highest risk of bias were those of parameter 2, related to randomization sequence concealment. 30% of all RCTs achieved

four or more parameter scores for low risk of bias. Tables 2, 3 and Fig. 4 shows the frequencies and the proportions of the tool's parameters for risk of bias assessment achieved by the clinical trials published in the RCA journal.

The relation between year of publication, number of published trials and assessment of the four main parameters of the tool are described in Fig. 5. It shows the increase in the blue area with the passing of time as evidence of sustained reduction in global risk of bias. In addition, Fig. 6 shows the rate of trials that achieve three or more low risk parameter scores in blue and the radius of the circles show the average size of recruited studies.

Finally, Fig. 7 is a box diagram of the total distribution of subjects included in the trials according to year of publication (organized in 3-year intervals).

Discussion

This study shows a view of clinical trial publication in the area of Anesthesiology in Colombia. It was a research initiative aimed at documenting our scientiic work and possibly to planning strategies for improvement in future trials.

We present the discussion of results in two parts: first, the general considerations of execution in the journal and then the risk of bias analysis for the published trials.

General considerations

Most of the trials were carried out in the cities of Bogotá and Medellín. This is probably due to the fact that these cities provide greater organization, collaboration, logistics, inancing possibilities and conditions for clinical trial development, which are demanding because of their complexity. Several international trials have also been published in the RCA journal, although publication of foreign studies has decreased in the past years. The international presence of the RCA journal has grown due to several strategies provided by SCARE and its editorial committee. It is currently indexed in SciELO Colombia, LILACS, EBSCO, Imbiomed, Index Copernicus, Redalyc, LICOCS, Latindex and more recently, Elsevier-Doyma.15 It is in our general interest to become a journal of interest for publishing international clinical trials and indexed in Medline.

The number of clinical trials has increased over time and as of 2003 an estimated ive articles are written every year. The graphic trend appears to be linear and sustained. Many authors, researchers and groups could well be influenced by methodology and analysis assessment teams. In a publication which assessed clinical trials of the Revista Española de Anestesiología, García-Alamino al.16 reported that 30% of trials were studying non-pharmacologic interventions. For the RCA journal, this rate was lower, however present. Because of the characteristics of the speciality, Anesthesiology is an area in which several non-pharmacologic interventions are used and they represent an opportunity for new challenges in the planning and design of experimental research.

Few trials carried out outcome follow-up for more than 24 h. In fact, most of them did so for two hours only. A great number of randomized clinical trials in Anesthesiology are limited to the immediate postoperative period, though many outcomes relevant for the patient (rather than the anesthesiologist or the researcher) may appear during hospital stay follow-up or late recovery phase. This consideration must be accounted for and adapt to the research question and the research scenario.17 Also, in case surrogate outcomes are employed (substitute, or intermediate measurements), which are usually easier to measure, they must be carefully selected to properly represent the clinical event of interest for the patient.18

Clinical trials are studies that demand a great amount of planning and resource availability, even if the design or the interventions are not complex. In the RCA journal, more than 50% of all trials were published by three authors or less. To our consideration, the researchers for the processes of planning, design, execution and publication are few for such an endeavor. Greater numbers of researchers are needed.

Publication bias risk analysis

Clinical trials and their systematic reviews provide the "most valid" evidence on effects of health interventions and their causal associations. However, inferences on causality extracted from these studies may be undermined by flaws in their design, execution, analysis and report. Such errors result in an inaccurate interpretation of the effects of the intervention (biases).5,6,13 Usually, it is not possible to know how much the biases truly affect the results of a clinical trial. As a matter of fact, for readers, it is difficult to judge the adversities that a research team faces in a particular trial. For that reason, the current trend is to judge the "risk of bias", understanding "quality" as the degree to which the design, execution, analysis and report are fit for providing a bias-free randomized clinical trial.

The tool used for bias risk assessment in this study is the same facility used for the development of systematic reviews in the Cochrane Collaboration. Few publications have used it with the intent of establishing the risk of bias of a particular journal (possibly because of its recent design).19 In spite of some of the tool's limitations mentioned in literature,20-22 our aim was to show a global view of the development of randomized clinical trials in the RCA journal.

The analysis of tables and figures presented in this study shows that the amount of parameter scores for low risk of bias increases as time goes by. 30% of studies show four parameters or more with low risk of bias. This trend greatly represents a clear, sustained intent of improvement, formation and development of high quality research. In a future issue of this work team, we expect to include a sub-analysis of the time period between the year 2000 and the present day.

The results of this study are proof of the great interest of the RCA journal, its editorial committee and SCARE to strive forward in the improvement process and the presence of the publication.15,23 However, this effort must also be shared by each of the authors in the journal, who take the first step with the quality of their reports.24

The process of achieving a high quality randomized clinical trial with low risk of bias requires solid institutions

and a health system that guarantees the users' rights. A reform of the educational system in which professors at third degree educational institutions have resources enough to develop their research is also necessary. In Colombia, the current health system follows a poorly controlled search for profitability in medical attention, which is a limitation for clinical trial execution and all other research attempts. The system's narrow margins and the priority of profitability restrain the freedoms of thought and action necessary to achieve better and more cost-effective patient care.

Financing

Own resources.

Conflict of interest

Node declared.

Acknowledgements

To the Sociedad Colombiana de Anestesiología y Reanimación SCARE, the SCARE education department and the Clinical Epidemiology Unit at Universidad del Cauca.

REFERENCES

1. Jadad A, Enkin M. Randomised controlled trials. Blackwell Publishing - BMJ Books; 2007.

2. Altman D. Better reporting of randomised controlled trials: the CONSORT statement. BMJ. 1996;313:570-1.

3. Glasziou P, Vandenbroucke J, Chalmers I. Assessing the quality of research. BMJ. 2004;328:39-41.

4. Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [actualizado March 2011]. The Cochrane Collaboration; 2011. Disponible en: http://www.cochrane-handbook.org/

5. Chalmers T, Celano P, Sacks H, Smith HJ. Bias in treatment assignment in controlled clinical trials. N Engl J Med. 1983;309:1358-61.

6. Moher D, Pham B, Jones A, Cook D, Jadad A, Moher M, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet. 1998;352:609-13.

7. Schulz K. Assessing allocation concealment and blinding in randomised controlled trials: why bother? Evid Based Nurs. 2001;4:4-6.

8. Schulz K, Chalmers I, Hayes R, Altman D. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273:408-12.

9. Jadad A, Moore R, Carroll D, Jenkinson C, Reynolds D, Gavaghan D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials. 1996;17:1-12.

10. Olivo S, Macedo L, Gadotti I, Fuentes J, Stanton T, Magee D. Scales to assess the quality of randomized controlled trials: a systematic review. Phys Ther. 2008;88:156-75.

11. Moher D, Jadad A, Nichol G, Penman M, Tugwell P, Walsh S. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials. 1995;16:62-73.

12. Chan A, Hróbjartsson A, Haahr M, G0tzsche P, Altman D. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291:2457-65.

13. Higgins J, Altman D, Gotzsche P, Jüni P, Moher D, Oxman A, et al., The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011;18:343.

14. Team RDC. R: A language and environment for statistical computing, reference index version 2.14.1. Vienna, Austria: R Foundation for Statistical Computing; 2005 [consultado 3 Feb 2011]. Disponible en: http://www.r-project.org

15. Rincoín-Valenzuela D. Jamais nos hemos resignado a ser simples artesanos de la Anestesia: pasado, presente y futuro de la Revista Colombiana de Anestesiologiia. Rev Colomb Anestesiol. 2012;40:6-7.

16. García-Alamino J, Parera A, Ollé G, Bonfill X. Características y calidad metodologica de los ensayos clínicos publicados en la Reuista Española de Anestesiología y Reanimación. Rev Esp Anestesiol Reanim. 2007;54:333-9.

17. Godlee F. Outcomes that matter to patients. BMJ. 2012;344:e318.

18. Yudkin J, Lipska K, Montori V. The idolatry of the surrogate. BMJ. 2011;343.

19. Liu Y, Yang S, Dai J, Xu Y, Zhang R, Jiang H, et al. Risk of bias tool in systematic reviews/meta-analyses of acupuncture in Chinese journals. PLoS One. 2011; 6:e28130.

20. Hartling L, Ospina M, Liang Y, Dryden D, Hooton N, Krebs Seida J, et al. Risk of bias versus quality assessment of randomised controlled trials: cross sectional study. BMJ.2009;339:b4012.

21. Armijo-Olivo S, Stiles C, Hagen N, Biondo P, Cummings G, Assessment of study quality for systematic reviews: a comparison of the Cochrane Collaboration Risk of Bias Tool and the Effective Public Health Practice Project Quality Assessment Tool: methodological research. J Eval Clin Pract. 2012;18:12-8.

22. Hartling L, Bond K, Vandermeer B, Seida J, Dryden D, Rowe B. Applying the risk of bias tool in a systematic review of combination long-acting beta-agonists and inhaled corticosteroids for persistent asthma. PLoS One. 2011;24:e17242.

23. Reyes G. Revista Colombiana de Anestesiologiia: auín en su volumen 40 sigue en busca de la excelencia como medio de la difusioin de nuestra produccioín cientiífica. Rev Colomb Anestesiol. 2012;40:4-5.

24. Miller D. Hacia una mayor transparencia y exactitud de los reportes cientiíficos en las revistas biomeidicas. Rev Colomb Anestesiol. 2012;40:1-3.

1. Jadad A, Enkin M. Randomised controlled trials. Blackwell Publishing - BMJ Books; 2007.         [ Links ]

2. Altman D. Better reporting of randomised controlled trials: the CONSORT statement. BMJ. 1996;313:570-1.         [ Links ]

3. Glasziou P, Vandenbroucke J, Chalmers I. Assessing the quality of research. BMJ. 2004;328:39-41.         [ Links ]

4. Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [actualizado March 2011]. The Cochrane Collaboration; 2011. Disponible en: http://www.cochrane-handbook.org/        [ Links ]

5. Chalmers T, Celano P, Sacks H, Smith HJ. Bias in treatment assignment in controlled clinical trials. N Engl J Med. 1983;309:1358-61.         [ Links ]

6. Moher D, Pham B, Jones A, Cook D, Jadad A, Moher M, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet. 1998;352:609-13.         [ Links ]

7. Schulz K. Assessing allocation concealment and blinding in randomised controlled trials: why bother? Evid Based Nurs. 2001;4:4-6.         [ Links ]

8. Schulz K, Chalmers I, Hayes R, Altman D. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273:408-12.         [ Links ]

9. Jadad A, Moore R, Carroll D, Jenkinson C, Reynolds D, Gavaghan D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials. 1996;17:1-12.         [ Links ]

10. Olivo S, Macedo L, Gadotti I, Fuentes J, Stanton T, Magee D. Scales to assess the quality of randomized controlled trials: a systematic review. Phys Ther. 2008;88:156-75.         [ Links ]

11. Moher D, Jadad A, Nichol G, Penman M, Tugwell P, Walsh S. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials. 1995;16:62-73.         [ Links ]

12. Chan A, Hróbjartsson A, Haahr M, G0tzsche P, Altman D. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291:2457-65.         [ Links ]

13. Higgins J, Altman D, Gotzsche P, Jüni P, Moher D, Oxman A, et al., The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011;18:343.         [ Links ]

14. Team RDC. R: A language and environment for statistical computing, reference index version 2.14.1. Vienna, Austria: R Foundation for Statistical Computing; 2005 [consultado 3 Feb 2011]. Disponible en: http://www.r-project.org        [ Links ]

15. Rincoín-Valenzuela D. Jamais nos hemos resignado a ser simples artesanos de la Anestesia: pasado, presente y futuro de la Revista Colombiana de Anestesiologiia. Rev Colomb Anestesiol. 2012;40:6-7.         [ Links ]

16. García-Alamino J, Parera A, Ollé G, Bonfill X. Características y calidad metodologica de los ensayos clínicos publicados en la Reuista Española de Anestesiología y Reanimación. Rev Esp Anestesiol Reanim. 2007;54:333-9.         [ Links ]

17. Godlee F. Outcomes that matter to patients. BMJ. 2012;344:e318.         [ Links ]

18. Yudkin J, Lipska K, Montori V. The idolatry of the surrogate. BMJ. 2011;343.         [ Links ]

19. Liu Y, Yang S, Dai J, Xu Y, Zhang R, Jiang H, et al. Risk of bias tool in systematic reviews/meta-analyses of acupuncture in Chinese journals. PLoS One. 2011; 6:e28130.         [ Links ]

20. Hartling L, Ospina M, Liang Y, Dryden D, Hooton N, Krebs Seida J, et al. Risk of bias versus quality assessment of randomised controlled trials: cross sectional study. BMJ.2009;339:b4012.         [ Links ]

21. Armijo-Olivo S, Stiles C, Hagen N, Biondo P, Cummings G, Assessment of study quality for systematic reviews: a comparison of the Cochrane Collaboration Risk of Bias Tool and the Effective Public Health Practice Project Quality Assessment Tool: methodological research. J Eval Clin Pract. 2012;18:12-8.         [ Links ]

22. Hartling L, Bond K, Vandermeer B, Seida J, Dryden D, Rowe B. Applying the risk of bias tool in a systematic review of combination long-acting beta-agonists and inhaled corticosteroids for persistent asthma. PLoS One. 2011;24:e17242.         [ Links ]

23. Reyes G. Revista Colombiana de Anestesiologiia: auín en su volumen 40 sigue en busca de la excelencia como medio de la difusioin de nuestra produccioín cientiífica. Rev Colomb Anestesiol. 2012;40:4-5.         [ Links ]

24. Miller D. Hacia una mayor transparencia y exactitud de los reportes cientiíficos en las revistas biomeidicas. Rev Colomb Anestesiol. 2012;40:1-3.         [ Links ]