English (pdf)
Article in xml format
Article references
Automatic translation
Send this article by e-mail
Cited by SciELO 
Cited by Google 
Similars in
SciELO 
Similars in Google 
Permalink
Introduction
Cognitive functioning assessment includes a variety of domains of functioning, such as general intelligence (e.g., IQ), nonverbal intelligence, attention/executive functioning, memory and learning, visual motor and motor functioning, and language (Campbell, Brown, Cavanagh, Vess, & Segall, 2008). It is well accepted that cognitive functioning and a myriad of psychosocial factors contribute to academic success (Caemmerer, Maddocks, Keith, & Reynolds, 2018; Castillo-Parra, Gómez, & Ostrosky-Solís, 2009; McGrew & Wendling, 2001). For example, it has been proven that different executive function components have an important predictive weight for academic success (Ahmed, Tang, Waters, & Davis-Kean, 2019; Cortés Pascual, Moyano Muñoz, & Quílez Robres, 2019).
In particular, in the school context, the assessment of cognitive functioning assumes an important role in diagnosing learning disabilities and disorders and evaluating cognitive processing strengths and weaknesses of students. This is of particular importance since it makes it possible to better adapt the teaching-learning process to students. The construction of regression equations to estimate academic success from the cognitive test adds a novel perspective to allow this approach in the school’s context.
Although screenings cannot replace a full comprehensive assessment, the traditional instruments for the assessment of cognitive function in children and adolescents take a long time to administer and require special training and expertise in the application process. The Wide Range of Assessment Memory and Learning 2nd edition (WRAML2) is one commonly accepted instrument for the assessment of general cognition and memory (Sheslow & Adams, 1990). The administration of the WRAML2 takes approximately one hour and it requires intensive training for a consistent application. Similar limitations are reported for other measures, such as the Wechsler Individual Achievement Test (Wechsler, 2002) and the Woodcock-Johnson III Tests of Achievement (McGrew & Woodcock, 2001).
The available evidence on this topic shows the existing lack of brief screening tools to assess cognitive functioning in multiple domains, specifically those that can be administered outside of the clinical setting (e.g., in the school context) and can provide objective data about cognition of adolescents. In this regard, the Cognitive Telephone Screening Instrument (COGTEL) is a brief (i.e., test administration takes approximately 10 to 15 minutes only), reliable, and valid instrument for capturing interindividual differences in cognitive functioning that can be administered face-to-face as well as over the phone (Kliegel, Martin, & Jäger, 2007). This instrument covers a broad range of cognitive domains essential for adolescent cognitive assessment, such as prospective, short-term, long-term, and working memory, verbal fluency, and inductive reasoning (using specific tasks also adopted from well-established neuropsychological test batteries, such as the Wechsler scales). Additionally, COGTEL has been shown to be a sensitive indicator of overall cognitive functioning. Most importantly, the COGTEL can be widely applied since it can differentiate individuals on the full performance range of cognitive functioning. Therefore, it cannot only assess cognitive impairments, but it can also indicate interindividual differences within the healthy cognitive performance range (Kliegel et al., 2007).
Previous research evaluating the psychometric properties of the COGTEL (both for the face-to-face assessment as well as for over the phone assessment) has demonstrated its suitability for capturing interindividual differences in cognitive functioning in epidemiological and aging studies (Ihle, Gouveia, Gouveia, & Kliegel, 2017; Kliegel et al., 2007; Tinôco et al., 2019). First, Kliegel et al. (2007), in 164 European community-dwelling adult volunteers, provided evidence for the validity and reliability of the COGTEL to assess cognitive functioning in large-scale epidemiological studies, longitudinal studies, and clinical follow-ups among healthy adults. Second, Ihle et al. (2017) observed good test-retest reliability for the COGTEL total score in a sample of 116 community-dwelling older adults from Brazil. In addition, in the same article, in a separate sample of 868 community-dwelling older adults from the same geographic area, a substantial correlation with MMSE total score was seen, indicating convergent validity of this instrument. Yet, so far, no psychometric evaluation exists on using the COGTEL to assess cognitive functioning in adolescents. Thus, the present study set out to evaluate the psychometric properties of the face-to-face application of the COGTEL in adolescents in the school context and to develop age-specific regression equations to estimate academic achievement using the COGTEL total score as a unique predictor.
Methods
Study design and participants
The participants were 170 adolescents (81 boys and 89 girls; mean age = 15.1 years old, SD = 2.2) from the research project entitled “Physical Education in Schools from Autonomous Region of Madeira” (EFERAM-CIT); https://eferamcit.wixsite.com/eferamcit).The classes included in this study from elementary school (from grade 5 to 9) comprised 82 students (age range between 11.9 to 16.2 years old). From the secondary school (from grade 10 to 12) 88 students were included (age range between 16.9 to 20.6 years old). This project addressed students from five public schools, in the Municipality of Funchal (Portugal), between 2017 and 2018. A convenience sampling method was adopted.
The inclusion criteria for this study were: (a) to attend the classes of the trainee teachers and their promoters that belong to the EFERAM-CIT project and (b) to complete the cognitive assessments (twice 6 months apart). The exclusion criteria were: (a) students a priori identified with cognitive impairment or (b) students with any significant comorbidity (e.g., severe heart diseases).
Participants were informed about the objectives of the study and written informed consent was obtained from their legal guardians. The study received approval from the Scientific Committee of Faculty of Physical Education and Sports at the University of Madeira, the Regional Secretary of Education and the Principals of each school surveyed. The study was conducted in accordance with the ethical standards in sports exercise research (Harriss & Atkinson, 2011).
Measures
Cognitive Telephone Screening Instrument
The COGTEL was used to assess cognitive functioning. The COGTEL consists of six subtests covering prospective memory (0 or 1 point), verbal shortand long-term memory (0-8 points each), working memory (0-12 points), verbal fluency (0 to unlimited; as many words as the participant can name within 1 min), and inductive reasoning (0-8 points; see Kliegel et al. (2007) for a more detailed description). The scores of the 6 subtests can be combined into a weighted total score (7.2 * prospective memory + 1.0 * verbal shortterm memory + 0.9 * verbal long-term memory + 0.8 * working memory + 0.2 * verbal fluency + 1.7 * inductive reasoning score) Kliegel et al. (2007).
COGTEL translation and retranslation procedure The original COGTEL translation team (English to Portuguese) consisted of a committee of five researchers, who published the first Portuguese version (Ihle et al., 2017).
Initially, each subtest of the instrument was discussed with the author of the original English version and the translation from English to Portuguese was performed by a native speaker. Later, the same procedure was carried out from French to Portuguese by a Portuguesedescendent researcher, as COGTEL has also been translated into French. In both cases, each final version was back-translated and a final revision based on both back-translation versions was made. The translation, synthesis, and back-translation procedures were carried out without difficulties and the modifications of the committee were aimed at guaranteeing the semantic, idiomatic, cultural and conceptual equivalence of the translated instrument with the original instrument. Further details about the translation process and pilot study can be found in Ihle et al. (2017) and Tinôco et al. (2019).
Academic achievement
Academic achievement was assessed using students’ school grades, in all topics of the curricula. Students’ grades, in all subjects, were based on the students’ performance, considering the written exams, oral class participation, and paper works. These are common assessment procedures in Portuguese schools. In Portuguese elementary schools, students’ grades range from 1 to 5 (1 = very poor, 2 = poor, 3 = average, 4 = good, and 5 = very good). For the secondary school, students’ grades range from 1 to 20 (1=very poor, 10 = average, and 20 = very good).
General procedures
Data on school marks were collected directly from the head teacher of each class and the administrative services of each school. COGTEL assessments were conducted in face-to-face interviews in private rooms of each school. All participants were individually assessed twice with a lapse of 6 months between each assessment, and under the same conditions, but with different interviewer-participant pairs. The interviews took between 10 to 15 min per student. Test administration was performed by the EFERAM-CIT research team which was exposed to previous training for the application of the COGTEL.
The EFERAM-CIT research team was composed of 12 master students and 5 Ph.D. supervisors from the Social Sciences Faculty of the University of Madeira. Two elements of the original research team, BRG and ERG, who participated in the Portuguese COGTEL translation and retranslation procedure with the author of the original instrument (MK) prepared this EFERAM-CIT research team. Training for the COGTEL application included theoretical discussions, lab demonstrations, and training sessions with the elements of the research team.
Statistical Analyses
The reliability and predictive validity of the COGTEL was evaluated as follows. First, we inspected the testretest reliability of the COGTEL total score using the intraclass correlation coefficient (ICC). The ICC estimates and their 95% confidence intervals were based on a meanrating (k = 2), absolute-agreement, and 2-way mixed-effects model. At this step, we also calculated the COGTEL internal consistency using Cronbach’s alpha coefficient. Second, we evaluated the predictive validity by inspecting the relation between the COGTEL total score with the school grades (the average of all school subjects), using partial correlation, controlled by age. Finally, to estimate the regression equation of academic achievement, we performed two regression equations (one for elementary school and another one for secondary school) with the COGTEL total score as a unique predictor. The level of confidence was set at 95%. Data analysis was performed using IBM SPSS Statistics version 26 (SPSS Inc., an IBM Company, Chicago, Illinois, U.S.A.)
Results
Descriptive Statistics
The characteristics of the participants in terms of gender, age, and parents’ education are presented in Table 1. Means and standard deviations, and range scores for the 6 COGTEL subtests and the COGTEL total score are displayed in Table 2. Histograms showing the Gaussian distribution of COGTEL Total score at Time 1 and 2 are presented in Figure 1 and Figure 2, respectively.
Table 2 Descriptive statistics for COGTEL in test and retest assessments.
Note: CD, cognitive domains; PM, Prospective memory; STM, Short-term memory; WM, working memory; VF, Verbal fluency; IR, Inductive reasoning; LTM; Long-term memory; COGTEL, COGTEL total score; M (SD), means (standard deviations); Max, Maximum; Min, Minimum; Skew, Skewness; Kurt, Kurtosis
Test-Retest Reliability of the COGTEL Instrument
We observed good test-retest reliability for the COGTEL ICC =.77, p <. 001 (CI95% .688-830). The Cronbach’s alpha coefficient was .724, indicating an acceptable internal consistency.
Predictive Validity of the COGTEL Instrument
There was a medium positive correlation of the COGTEL total score with academic marks (an average of all school subjects) (r = .40, p < .001), even when the analyses were controlled by age.
Estimation of Academic Achievement from COGTEL
The academic achievement for Elementary school could be predicted from the COGTEL score as follows: Y = 2.6 + .03*(COGTEL total score). The standard error of the estimate was 0.7%. The COGTEL total score was a significant predictor (β = .45; p < .001) of academic achievement. The total variance explained by the model as a whole was 20%, F (1, 82) =20.86, p < .001.
The academic achievement for Secondary school could be predicted from the COGTEL score as follows: Y = 11.1 + .10*(COGTEL total score). The standard error of the estimate was 2.7 %. The COGTEL total score was a significant predictor (β = .36; p < .001) of academic achievement. The total variance explained by the model as a whole was 13%, F (1, 87) =13.10, p < .001.
Discussion
This study aimed to evaluate the reliability and predictive validity of the COGTEL as a brief, less time-consuming, and practical tool to assess cognitive function in adolescents in the school context. Additionally, the contribution of the COGTEL total score as a unique predictor of academic achievement was examined in detail.
First, in this study, the test-retest reliability observed for the COGTEL total score achieved a good size (r = .77). Ihle et al. (2017), in a previous study in Brazilian older adults, found slightly higher reliabilities (r = .85) with a retest after 7 days. The observed COGTEL reliability in this study is thus comparable to that of the adopted Wechsler scales (ranging from .38 to .87), as reported for young and middle-aged adults (Lo, Humphreys, Byrne, & Pachana, 2012). Similar results were also shown in a comprehensive review for the Mini-Mental State Examination (ranging from. 80 to .95; Tombaugh & McIntyre, 1992). We can conclu de that, even with a large time interval between test and retest, the present study reinforces that the COGTEL can be used as a reliable and stable assessment of cognitive functioning in adolescents.
Second, we observed a medium positive correlation of the COGTEL total score with school grades (the average of all school subjects; r = .40), controlled by age. This suggests predictive validity for the COGTEL instrument in this population of adolescents. The COGTEL consists of six subtests that cover important domains of cognitive function and could predict academic achievement. For instance, there is evidence supporting the association between academic success and higher development in prospective memory (Cejudo, McDaniel, & Bajo, 2019), working memory (Brown, 2018; Schneider, & Niklas, 2017), verbal fluency (Bigozzi, Tarchi, Vagnoli, Valente, & Pinto, 2017), and inductive reasoning (Bhat, 2016; Gómez-Veiga, Chaves, Duque, & García Madruga, 2018). This reinforces our main hypothesis that a better COGTEL score is associated with an overall better academic achievement.
Third, we developed a regression equation to estimate academic achievement using the COGTEL total score, considering it as a unique predictor in the model. Although there is evidence that non-cognitive variables, such as selfconcept, goal orientations, learning strategies, popularity, and parent involvement have a predictive role when it comes to academic achievement (Schneider & Preckel, 2017; Veas, Castejón, Gilar, & Miñano, 2015), cognitive abilities have been shown to be the strongest unique predictor in explaining the variation in academic achievement (Caemmerer et al., 2018; McGrew, & Wendling, 2010). We addressed this issue by developing specific regression equations using the COGTEL total score to estimate academic achievement for elementary and secondary students. This study advocates that the use of regression equations to estimate academic success from cognitive tests has an important practical implication for teachers since they can diagnose individual learning deficits. Since academic achievement is a crucial point to consider for the future success of students, it is important to make available brief, reliable, and valid instruments for use in the school context. This assessment could contribute to a deeper understanding of the individual cognitive abilities and explain the achieved academic success. Additionally, besides the limited evidence on this matter, the implementation of regular assessment routines in regard to cognitive functioning could trigger the monitoring of cognitive performance, allowing the development of student-centred and task-targeted interventions for more effective and integrated approaches in the teachinglearning process.
Some points should be underlined with regard to this study. This is the first study in adolescents that evaluates the psychometric properties of the COGTEL in the assessment of cognitive functioning. Second, this instrument covers important domains of cognitive functions that can predict academic success. Finally, a reliability study was performed with test and retest after six months, showing good results. This confirms good stability in the individual cognitive functioning assessment of adolescents in the school context. On the other hand, we acknowledge some of the limitations of this study. First, the timeframe between test and retest is longer than usually reported in the literature on similar reliability studies which could underestimate the results. Yet, even with the broad timeframe of six months, we show good test-retest reliability of the COGTEL in adolescents in the school context. Moreover, notwithstanding the fact that school grades are considered an important indicator of overall cognitive function, it should be taken into account that there are many other variables that have a predictive role on academic achievement that we did not consider in the regression model. Thus, we acknowledge some limitation of using only the school grades to evaluate the predictive validity of the COGTEL. Finally, we acknowledge that the generalization of our findings may also be limited by factors related to sample size. Importantly, we had a comparable number of students in each subgroup, elementary versus secondary school. In conclusion, the present study suggests the COGTEL as a brief, reliable, and valid instrument for capturing interindividual differences in the cognitive functioning of adolescents. The COGTEL has the advantage of allowing a feasible assessment in multiple populations and contexts, covering a great amount of information related to cognitive abilities. In addition, for adolescents, a regression equation using the COGTEL total score as a unique predictor provides a valid estimate of academic achievement. Therefore, the COGTEL can be used to better understand and monitor individual cognitive abilities in the school context, thus, helping students to achieving future academic success1.
