texto em 
Inglês (pdf)
Artigo em XML
Referências do artigo
Tradução automática
Enviar este artigo por email
Citado por SciELO 
Citado por Google 
Similares em
SciELO 
Similares em Google 
Permalink
Introduction
The professional training of medical clinicians has an ideal ascending learning curve, in which clinical reasoning and decision making are essential in teaching medical staff, to guarantee meaningful learning 1.
One of the goals of medical teaching is to promote the development of expert reasoning in residents in training. Teaching these cognitive abilities is a complex task, as there is no definitive method for decision making and many of the reasoning mechanisms may not even be conscious and pose a challenge for achieving definitive diagnoses 2.
In fact, although the accumulation of theoretical knowledge is essential for internal medicine practice, experience plays a definitive role in developing internists' professional skills. Throughout the assessment of their first patients, the professionals reorganize their knowledge to carry out optimal diagnosis and treatment through the formation of knowledge networks specifically adapted to clinical practice, which have been termed "illness scripts."
The script concordance test has been described as an innovative tool for evaluating clinical skills, designed to determine residents' ability to make clinical decisions as a reflection of an exceptional skill: clinical reasoning.
The principle of this test is to present residents with a series of clinical situations which they must analyze, and then provide them with additional information and ask them what effect this new information has on the diagnostic, investigative or treatment hypotheses already developed. A special characteristic of this evaluation test is that the "illness scripts" are also consulted with "experienced" physicians, whose opinions are compared with the students', giving rise to the name "script concordance test" 3.
The purpose of this article is to present the general principles for using the script concordance test as a methodology for evaluating clinical decision making in internal medicine residents and discuss its potential usefulness as a clinical evaluation method in the workplace.
Methods
Theoretical basis for the script concordance test
The script theory is derived from cognitive psychology and is based on the existence of structured concept networks in the memory whose complexity grows with clinical experience to understand, transform, categorize and act in an uncertain clinical situation 4,5.
Medical-surgical residents in training are faced with having to take clinical information which is, at times, confusing, unclear and hard to obtain, and use it to formulate a clinical opinion regarding possible diagnoses in order to resolve the case and then propose the patient's general management. One way of evaluating this clinical skill is through the script concordance test.
This test is one of the most used tools for evaluating clinical reasoning. It is a written test whose main objective is to evaluate the organization of knowledge into cognitive networks, rather than evaluate isolated knowledge, and it does this by comparing the clinical decisions of those being tested with those of a group of experts or reference panel 6. The greater the concordance between the examinee's answers and those of the experts, the greater the inferred clinical reasoning development 7.
It is a tool which has begun to be used over the last few years to evaluate "knowing" and "knowing how to" 7. The basic goal of the script concordance test is not to measure theoretical knowledge, but rather the organization of this knowledge into conceptual networks (information scripts). These scripts basically emerge from physicians' clinical practice, during medical acts, when decisions are made in light of a specific clinical situation 8.
Likewise, a progressive process of developing decisionmaking abilities can be seen, from an initial novice phase in which only limited, scattered knowledge is obtained, up to an advanced expert phase in which knowledge is obtained compiled in conceptual networks (scripts). As we mentioned, this method does not aim to measure knowledge, but rather show the organization of this knowledge.
The script concordance test requires the availability of a group of experts to construct and validate it and answer the questions; the structure of this type of questions is as follows 4:
A few clinical vignettes describing authentic, difficult clinical situations which physicians must face in their practice.
Diagnostic, complementary test, or treatment hypotheses in accord with the clinical situation described in the vignette.
New, mutually independent, relevant information for each hypothesis, which is useful for solving the problem presented.
In light of this new information, the examinee is asked: If you, in light of this vignette, think of a given hypothesis and receive new information, how does this affect or impact on your hypothesis?
The answer options are always the same:
It completely rules out the hypothesis.
The hypothesis is less likely.
It has no impact on the hypothesis.
The hypothesis is more likely.
It completely confirms the proposed hypothesis. There is actually no single response, as with multiple choice questions - a single expert consensus answer. Rather, there may be more than one acceptable answer and, therefore, the score is based on a consideration of the concordance between the examinees' answers and the expert responses. The examinee's score is in keeping with the majority opinion of the experts; that is, the experts' knowledge networks are compared with those of the examinees 8.
Foundations for constructing clinical cases and experiences with the script concordance test
To construct the script concordance test questions, certain characteristics must be considered, like uncertainty and creating the ability to think and problem solve using a cognitive foundation established during training. The items which must be taken into account in constructing the test are presented in Table 1.
One of the most relevant aspects in applying the script concordance test is the construction of the clinical cases, which has been one of the biggest difficulties for test validation. To build appropriate clinical cases, there must be three-part clinical scenarios (9, 10), which are presented in Figure 1.
Likewise, it should be noted that the applicability of the concordance test in clinical professors' bedside teaching of medical-surgical residents is very limited. Therefore, the author (TE) designed clinical cases to apply it at Hospital Universitario de La Samaritana in Bogotá (Colombia). The first step was to gather the internal medicine residents' clinical professors and design 10 clinical cases using the described script concordance test format, so that the professors would be aware of this assessment test.
The professors were instructed on the test through a tutorial on the key aspects of the script methodology, and once they understood it, they were asked if the clinical cases were understandable and pertinent to serve as examples for evaluating the residents' clinical reasoning. Once the test conditions were established, it was applied on a sample of five clinical professors specialized in internal medicine, with Likert scale answers given.
When the test results were evaluated, differences were found in the specialists' answers, showing difficulties in the performance of the test and in its applicability for residents.
Discussion
The script concordance test is an innovative tool for evaluating clinical reasoning 13,15, and, according to Charlin et al., it is a useful method in training rotations for medical-surgical residents. Using this tool can help improve the development of these clinical information analysis skills 16.
A change is needed in the evaluation of residents in medical areas, especially internal medicine. Written (especially multiple choice) tests, which are most often used, may have gaps, mainly in the context of uncertainty. For example, in an analysis of complex clinical cases in a medical journal in 2005, Norcini et al. criticized the use of oral clinical cases with real patients ("long cases") as the only means of evaluating the graduate students' ability to reason 20. Therefore, the testing trends in practice sites have changed considerably and the script reasoning test has achieved a central place.
In the same way, Driessen proposes that not observing residents' performance and not providing feedback based on the professors' observations is one of the main problems in evaluating the residents' clinical skills in the workplace 21. According to the experience gathered with the script test, the professor will be alert to the resident's performance, and will supervise and guide the improvement process based on the test's application.
Robertil and Ferraz reported their experience in using the script concordance test in medical schools in Brazil, in which they conducted constructive criticism on the use of the script test and concluded that this tool, as an evaluation method, is hard to construct, apply and correct. This could be a limitation for the use of this test 14.
It is evident that meaningful learning depends on many factors, especially the formal education settings in which testing has a privileged weight, constituting, for most students, the main reason for studying and learning 11. In 1984, Fredericksen stated that, "the most powerful method for changing the way in which students study and the way in which teachers teach is to change the method of testing" 12. This concept determines the importance of a substantial change in the testing method to produce students with a higher academic level and meaningful learning.
The first experiences with the script test in the internal medicine service of Hospital de La Samaritana in Bogotá (Colombia) show a lack of knowledge of the test, the inherent difficulties in its implementation and the need for clinical professors to be trained on how to evaluate it. This paves the way for opening research lines on clinical teaching which would optimize testing in the practice hospitals.
In medical education, all testing tools have limitations. One of the difficulties with this test is the construction of questions, as this requires a condition of uncertainty in order to generate reasoning in the participants. A trial in residents showed that this testing modality does not replace other testing mechanisms, but rather complements the tools available to the trainers 17.
This testing process has a high degree of difficulty in its preparation, implementation and scoring, which constrains its use. In addition, this study concludes that one of the main limitations has to do with the specialists' lack of unified criteria and concepts in the questions, as each professional structures his/her responses according to his/ her own clinical reasoning. This variability in the expert panel responses is a characteristic of script tests 18 which can have a two-way use (advantages and disadvantages) in subsequent studies.
Conclusion
In conclusion, the script concordance test is a standardized tool which can help evaluate the development of clinical reasoning skills in graduate students in medical-surgical specialties 15. New studies are needed to propose its effectiveness, which can be achieved by proposing a comparison between the various methods of testing and unifying criteria to guarantee a comprehensive evaluation of meaningful learning 19.
