English (pdf)
Article in xml format
Article references
Automatic translation
Send this article by e-mail
Cited by SciELO 
Cited by Google 
Similars in
SciELO 
Similars in Google 
Permalink
Introduction
The diagnosis and management of patients with the same medical condition and similar characteristics can vary significantly from one professional to another, making it challenging to ensure the quality of the services provided by health institutions.1 One way to reduce this variation and promote evidence-based management is through the use of clinical practice guidelines (CPGs), which Lohr et al. define as "systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances".2 Currently, CPGs play a key role in strategies to improve decision-making in health systems. 3,4
To ensure the quality of the recommendations, CPGs should be elaborated following specific rigorous methodological standards that include the formulation of PICO questions (population, intervention, comparison, and outcome), the systematic search for studies to answer each question, and the assessment of the methodological quality of those studies by the experts who elaborated the guidelines. 5 However, a substantial number of guidelines do not meet these quality criteria, as has been reported in studies that evaluate the guidelines published in MEDLINE6 and produced in different countries such as Argentina, 7 Canada, USA, 8 and Spain. 9
In Peru, 3 studies have assessed the quality of CPGs using the Advancing guideline development, reporting and evaluation in health care (AGREE II) instrument, which allows for an assessment of the methodological rigor with which the guidelines were prepared and the transparency of the development process. 10 According to this instrument, a CPG has poor quality when it has a score of <60%.11 The first of these studies evaluated the CPGs that address the diagnosis and management of hypertension and diabetes published by the Ministry of Health (MINSA) in 2015. 12 The second evaluated the quality of CPGs regarding various health issues such as obstetric pathologies, infectious diseases and non-communicable diseases -available on the website of MINSA's executive quality directorate for the period 2009-2014. 13 The third evaluated 12 CPGs of gynecological-obstetric diseases from a hospital in Peru. 14 In all 3 studies, the researchers found that the mean score for the AGREE-II instrument domains was <60%.
These studies12-14 suggest that the methods used to elaborate CPGs have shortcomings that could affect the validity of the recommendations. However, all of them assess the guidelines issued before 2015, when the Norma Técnica de Salud para la Elaboración y Uso de Guías de Práctica Clínica del Ministerio de Salud (Technical Standards for the Development and Implementation of Clinical Practice Guidelines of the Ministry of Health) 15 and the technical document Metodología para la Elaboración de Guías de Práctica Clínica del Ministerio de Salud (Methodology for the Development of Clinical Practice Guidelines of the Ministry of Health) were published. 16 These documents establish the methodology for developing CPGs based on the Grading of Recommendations Assessment, Development, and Evaluations system to standardize and homogenize the preparation of the guidelines by providing a clear methodology. It is therefore expected that, by using these tools, the methodological quality of the CPGs developed from that date onwards is better than those developed before their publication.
In this context, it is essential to study the CPGs developed in Peru after the issuance of these regulations, which would allow evaluating the compliance with the regulations and proposing strategies to improve the quality of CPGs. Therefore, the objective of this study is to describe the characteristics of the CPGs approved by public health entities in Peru between July 2015 and September 2017.
Materials and methods
Study design and population
Descriptive cross-sectional study that analyzed the CPGs approved by public health entities in Peru. The guidelines collected by the General Directorate of Insurance and Benefit Exchange (DGAIN) of MINSA between July 2015 and September 2017 that met the following criteria were included: 1) being approved through a resolution issued by the directorate; 2) having consistent information, that is, including the name of the CPG and the number of the approval resolution; 3) being approved as of July 2015, and 4) being submitted to DGAIN in physical or digital media.
Procedures
In July 2017, to evaluate the approved CPGs in Peru, MINSA's DGAIN requested the submission of the CPGs regardless of their date of approval. The request was made to all secondary and tertiary health service provider institutions (IPRESS) -centers that develop and approve local CPGs in Peru15 and belong to the Regional Government (GORE)-, the comprehensive health network directorates (DIRIS), social security (EsSalud), the Peruvian Armed Forces and National Police health departments, and MINSA's General Directorate of Strategic Interventions in Public Health (DIGIESP) (the body responsible for the production of national CPGs). 17 One month later, a reminder phone call was made to the institutions that had not responded to the request.
For this study, the CPGs were selected according to the criteria mentioned above. The variables of interest were extracted by a reviewer trained to evaluate this type of guideline. The information was digitized in an ad hoc database.
Quality criteria
A tool developed by the Analysis and Evidence Generation Unit of the National Health Institute of Peru was used to describe the quality of the CPGs. This instrument took into account three quality criteria suggested by Carrasquilla-Guitiérrez et al. ,18 which were proposed because they pose fundamental differences between evidence-based guidelines and guidelines based on expert opinion or consensus. 19 Each criterion was a variable with three possible categories:
Criterion 1. Persons in the panel of experts making the CPG: 1) only one author is mentioned or none is mentioned, 2) the list with the names of the people that developed the guidelines is available, and 3) the list with the names of the people that elaborated the guidelines is available and specifies who were clinicians and who were in charge of the methodology.
Criterion 2. The protocols implemented in the systems of identification, collection, and evaluation of evidence are presented: 1) CPGs do not include bibliographic references, 2) CPGs have bibliographic references, and 3) CPGs include bibliographic references and assess their level of evidence.
Criterion 3. The level of evidence that supports each recommendation is clear: 1) CPGs do not present the level of evidence or it is not clear, 2) CPGs referrer manual or non-systematic search methods, and 3) CPGs refer systematic search methods.
Other variables
The following information was collected for each CPG: region where it was elaborated, year of approval, institution that elaborated it (MINSA- regional governments or comprehensive health network directorates, EsSalud or Armed Forces), level of the healthcare facility (secondary level: II-1, II-2 or II-E and tertiary level: III-l, III-2 or III-E), and clinical condition addressed in the guideline (using its ICD-10 code).
It should be noted that, according to the organization of the Peruvian health system, the secondary level consists of healthcare facilities that provide intermediate care and that meet 12-22% of the demand for health care. The tertiary level, in turn, consists of healthcare facilities that provide highly complex services, i.e., highly specialized, and meet 5-10% of the care demand. 20
Statistical analysis
Frequencies and percentages were used for the descriptive analysis. Moreover, a chi-square test or an exact Fisher test, as appropriate, was performed to evaluate the association between CPG characteristics and quality criteria. Stata vl4.0 was used for the analyses.
Results
A total of 6 147 CPGs approved in the period 2002-2017 were collected, of which 5 140 were approved using a resolution, and 5 107 had consistent information. Of the latter, 1 376 were approved through a resolution from July 2015, with the most recent CPG being approved in September 2017. However, of these 1 376 guidelines only 558 were received by the DGAIN and were, therefore, included in the study (Figure 1).
Of the 558 CPGs included, 316 were elaborated by institutions in the department of Lima and 254 were approved between July and December 2015. DIRIS produced most of these documents (n=276), followed by regional governments (n=235). Regarding healthcare facilities, most of the CPGs were prepared by level III-1 (n = 359).
Table 1 showsthe characteristics of the CPGs included in the study. It is worth mentioning that only 553 CPGs were considered in the analysis of the variable "institution level", since 5 guidelines of the General Directorate of Strategic Interventions in Public Health did not have a level assigned.
Table 1 Characteristics of the clinical practice guidelines included in the study (n=558).
Source: Own elaboration.
The 558 guidelines were evaluated using the 3 quality criteria. Regarding Criterion 1,65.8% did not specify explicitly an author or described only one author, and none listed the elaboration group separating clinicians from methodologists. As for Criterion 2,81.5% had no citations or bibliographic references. Finally, about Criterion 3, 97.7% did not describe any evidence search method that supported their recommendations. Table 2 shows the evaluation of CPGs.
Table 2 Evaluation of the quality of clinical practice guidelines (n = 558).
| Criterion | n (%) | |
| Level of participation | A. Individual or without explicit author | 367 (65.8) |
| B. List of the members in the elaboration group | 191 (34.2) | |
| C. List of the members in the elaboration group: clinicians and methodologists | 0 (0.0) | |
| Support of the recommendations | A. No citations and/or bibliographic references | 455 (81.5) |
| B. With bibliographic references | 54 (9.7) | |
| C. With bibliographic references and evidence level assessment | 49 (8.8) | |
| Method for finding evidence that supported the recommendations | A. Not described/not clear | 545 (97.7) |
| B. Manual / non-systematic | 2 (0.4) | |
| C. Systematic search | 11 (2.0) | |
Source: Own elaboration.
After conducting the bivariate analysis, it was found that the frequency of "supporting the recommendation with bibliographic references" and "reporting and conducting a systematic search for the evidence" were higher in the Lima CPGs published in 2017 and by MINSA (p<0.05). Table 3 shows the associations between the characteristics of CPGs and their indicators.
Discussion
A total of 558 CPGs were studied nationwide. Concerning the minimum criteria evaluated, more than half of the guidelines had no explicit author or described only one author and had no citations or bibliographic references. Also, few referred to or made clear the method of searching for evidence to support their recommendations.
Quality criteria
Most of the guidelines evaluated, which were approved as of July 2015, did not meet the minimum criteria for level of participation, support of their recommendations, or method of searching for evidence. This is consistent with previous studies on CPGs in Peru,12,13 which have reported scores <60% in all domains of the AGREE-II instrument.
These results reflect the poor methodological quality of the guidelines, which may lead to inadequate recommendations. Thus, health staff must take into account that currently approved CPGs do not meet certain minimum quality criteria and, therefore, must carefully assess the CPGs before applying them to the clinical practice.
The shortcomings of CPGs may be explained by the lack of human or material resources for a rigorous methodological elaboration, as well as insufficient monitoring of their methodological quality.
It is worth mentioning that the Norma Técnica de Salud para la Elaboración y Uso de Guías de Práctica Clínica del Ministerio de Salud15 and the technical document Metodología para la Elaboración de Guías de Práctica Clínica16 detail the methodology for preparing CPGs, including the 3 criteria evaluated. However, the model for the presentation of the CPGs exposed in Annex 01 of the Norma Técnica does not require explicitly reporting these criteria, which seems contradictory since evaluating the methodological quality of the CPG without proper support of how it was elaborated is difficult.
Moreover, "reporting the development of a systematic search for evidence" was found to be more common in CPGs approved in 2017 than in those approved in 2015 and 2016. This may indicate that the methodology or the report of the Peruvian CPGs is improving, and it may be due to better knowledge of the technical standard for its elaboration. However, just over 10% of the CPGs approved in 2017 report conducting a systematic search for evidence.
Number of guidelines
A list of more than 5 000 CPGs was compiled with consistent information and resolution number; 1 376 of them were approved between July 2015 and September 2017. Since only the guidelines submitted voluntarily by the institutions that approve them were collected, this figure may be below the actual records, so the number of guidelines approved in the period evaluated could be higher.
This high number of recently approved CPGs is perhaps explained by the fact that, since 2006, the Health and Medical Support Facilities Regulations21 require IPRESS to have technical policy documents and CPGs to start operations. Also, since 2017, the accreditation of healthcare centers and medical support services establishes as one of its criteria the existence of CPGs for the 10 most frequent pathologies in each service.22 For this reason, an increase in the number of guidelines developed in Peru is expected, especially in institutions whose IPRESS can be developed without the need for approval by an evaluation committee.
In this context, EsSalud established in 2016 that the elaboration of CPGs in this institution must be supervised and approved by its Institute for Health Technology Assessment and Research (IETSI).23 This could be a useful strategy to guarantee the necessary methodological rigor and adequate reporting of the guidelines, although it needs to be complemented by adequately prioritizing the documents to be elaborated, as well as providing appropriate training to clinicians and methodologists for their development and evaluation. 24
Limitations and strengths
One of the limitations of this study is that an instrument developed ad hoc and not one used internationally, such as AGREE-II, was employed to describe the quality of CPGs. This could have happened because it was not feasible to apply AGREE-II in all the guidelines evaluated, and the minimum criteria that all should meet were used instead. Likewise, the CPGs evaluated are likely to be the short versions approved by the resolution, which makes it difficult to conduct proper evaluations given that, many times, they do not have complete methodological information.
Another limitation is that, although all relevant institutions were asked to submit the guidelines, participation was voluntary, which may have generated a participation bias considering that those with lower quality CPGs may have chosen not to submit them. Therefore, the results could overestimate compliance with the quality criteria assessed.
Despite the abovementioned limitations, the present study is the first to evaluate some quality criteria in a broad sample of Peruvian CPGs, including the main health subsystems that produce them in the country.
