Introduction

The use of sophisticated symbolic language is, of course, recognised as a crucial feature that separates human from non-human animals (^{Berwick, Friederici, Chomsky, & Bolhuis, 2013}). Indeed, humans are much more intelligent than any other animals (^{Premack, 2007}), and language seems to underlie, or at least greatly influence, many other aspects of human cognition (^{Baldo, Bunge, Wilson, & Dronkers, 2010}; ^{Pinker, 2010}).

That our mental lexicon is an essential part of our human intelligence is highlighted by the fact that vocabulary is almost always found to be the most significant part of general intelligence when compared with other cognitive function tests (^{Johnson, Bouchard Jr, Krueger, McGue, & Gottesman, 2004}; ^{Kan, Wicherts, Dolan, & van der Maas, 2013}). Furthermore, lexical ability is a strong predictor of academic achievement at undergraduate level (^{Pluck, 2019a}), suggesting a wide role in intelligent behaviour. Indeed, the mental lexicon of word knowledge, is one of the fundamental bases of what is frequently referred to in psychology as crystalized intelligence (^{Schipolowski, Wilhelm, & Schroeders, 2014}). Crystalized intelligence is contrasted with fluid intelligence in Raymond Cattel’s influential theory (^{Cattell, 1963}, ¹⁹⁶⁷, ¹⁹⁷³). Supposedly, fluid ability deals with on-line novel problem solving in the present, while crystalized ability is based on learnt verbal information, which is acquired through experience, particularly education (^{Kan et al., 2013}). This classic approach to understanding intelligence, separated into fluid and crystalized components, is supported by modern research (^{Hampshire, Highfield, Parkin, & Owen, 2012}).

In fact, this distinction between fluid and crystalized ability has recently become important in cognitive neuroscience through the identification of two broad, but not overlapping, systems within the human brain. These have been called the ‘multiple-demand system’ and the ‘language system’ (^{Blank & Fedorenko, 2017}; ^{Blank, Kanwisher, & Fedorenko, 2014}; ^{Woolgar, Duncan, Manes, & Fedorenko, 2018}). The distinction is important because it appears that they may be two fundamental parts of the human cognitive system, with language being ‘domain-specific’, i.e. processing only linguistic information, and nothing else, and the ‘multiple-demand system’ being domain general, i.e. involved with multiple, attentional top-down tasks, e.g. executive functions and fluid intelligence, irrespective of the type of material (^{Campbell & Tyler, 2018}). These neuroscientific findings, therefore, closely reflect the original fluid and crystalized ability division of Cattell.

On the one hand, there are many available tests for fluid or multiple-demand system processing. Such fluid intelligence-based and executive top-down systems have been intensely studied in psychology; these include tests on planning, working memory, reasoning etc. (see e.g. ^{Delis, Kaplan, & Kramer, 2001}). This is partly because such abilities have proven to be essential in understanding many themes in contemporary psychology research including child development (e.g. ^{Blakemore & Choudhury, 2006}), educational achievement (e.g. ^{Pluck, Ruales-Chieruzzi, Paucar-Guerra, Andrade-Guimaraes, & Trueba, 2016}), and mental illness (e.g. ^{Snyder, Miyake, & Hankin, 2015}). On the other hand, crystalized ability, e.g. measurement of the extent of the mental lexicon, has been much less aggressively researched in relation to how we understand the wider psychological context. In addition, whereas many of the executive and general intelligence tests are deliberately made language-free (e.g. ^{Pluck, 2019b}), by their very nature, crystalized ability tests are limited by linguistic culture.

This is less of a problem in English speaking countries, where there are multiple, validated crystalized ability tests. However, in Spanish there are fewer options. One very widely used assessment of the language system that can be used in research or for educational or clinical purposes is the Vocabulary subtest, which features in several versions of the Wechsler intelligence tests. This is available in Spanish (^{Wechsler, 2012}), and performance on this test is considered to be a strong measure of crystalized ability (^{Wechsler, 1999}). However, the Wechsler tests have several drawbacks, particularly for Latin American countries; as commercial products they are expensive. Furthermore, administration and scoring are not straight forward; training is required and even then, as participant responses are open-ended, scoring is ultimately subjective, introducing unwanted inter-rater variation in scores. One method which avoids this and has been widely used in Spanish is the Word Accentuation Test, in which word pronunciation is assessed (^{Pluck, Almeida-Meza, Gonzalez-Lorza, Muñoz-Ycaza, & Trueba, 2017}). This is simpler, as the psychologist need only listen for the correct stress pattern. In addition, word pronunciation has been shown to figure very highly on a crystalized verbal ability component with factor analysis (^{Crawford, Stewart, Cochrane, Parker, & Besson, 1989}). However, at its core, word pronunciation is probably controlled by non-declarative procedural ability rather than declarative semantic knowledge, as it is possible to pronounce words without knowing their meaning. Furthermore, there are several problems when using reading aloud as a psychological assessment because performance can be hindered by illiteracy, articulation, or visual difficulties; group testing is not possible; and it penalizes the self-educated who have acquired knowledge through reading but may not know the correct pronunciation in languages with irregular grapheme-phoneme matching (^{Baddeley, Emslie, & Nimmo-Smith, 1992}).

In the English language, a test that avoids all of these problems and has been widely used to measure crystalized-verbal ability is the Spot-the-Word Test (^{Baddeley et al., 1992}; ^{Baddeley, Emslie, & Nimmo-Smith, 1993}). This is a simple lexical decision task in which pairs of words are presented to participants, and the task is to choose which of the two is a real word (one is always a legitimate looking non-word and one is a real English word). This is usually done with visual presentation, but for illiterate participants the words can be read aloud by the psychologist. The test has been shown to have good psychometric properties, and validity was established by its high correlation (r = .861) with an established measure of vocabulary: The Mill Hill Vocabulary Scale (^{Baddeley et al., 1992}).

Despite the widespread clinical and research use of the original English-language Spot-the-Word test, and despite its versatility and simplicity as a measure of crystalized-verbal ability, there is no published version in Spanish available for use in research. There is an experimental version, similar to the Spot-the-Word test, which has been used to measure various aspects of word recognition impairment in Spanish patients with Alzheimer’s disease (^{Cuetos, Arce, Martinez, & Ellis, 2017}). In that version, patients and controls were required to identify real words in sets of four words, in which three were distractor non-words. However, as it was an experimental measure, the psychometric properties of the test were not reported.

In this paper, I present a Spot-the-Word type task in Spanish (called the Spanish Lexical Decision Task), and detail its reliability and validity as a measure of crystalized-verbal ability. The aim is to provide a preliminary development of the test, which could be used with adults from the Spanish-speaking community.

Results

These first analyses are all limited to the Development Sample (general population). The researchers first examined the internal consistency of the full 45-item version Spanish Lexical Decision Task with Cronbach’s alpha. This produced an initial alpha of 0.811, which would be considered ‘good’ internal consistency by standard qualitative interpretations. However, four items had zero variation as they were answered correctly by all participants. Furthermore, several items had low item-total correlations. The analysis followed ^{Clark and Watson (1995)} who suggest that item-total correlations should, in general, be greater than 0.15. These were removed sequentially starting from the lowest values until all the remaining items had item-total correlations of greater than 0.15. This produced a set comprised of 33 items. However, the researchers randomly added back in three of the four items with perfect scores in order to extend the range at the lower end of performance. These 36 items were used to form the final version of the Spanish Lexical Decision Task. The Cronbach’s alpha of this final version was 0.846, which would be considered ‘good’ internal consistency. The final version is shown in Table 1, which includes the mean scores, item:total correlations, and the resulting Cronbach’s alpha values for each item if removed.

Table 1 Items included in the final version of the Spanish Lexical Decision ask, the mean scores in the Development Sample, and their relationship to Cronbach’s alpha variables 

*The correct responses in each trial are shown in bold. 1 indicates items which showed zero variance in the Development Sample (all participants were correct)

The sequence of items in the final 36-item version is the same as in the original 45-item version, except that two of the very easy items were moved to later in the sequence. This is so that people with generally poor performance may be less disheartened by the progressive difficulty of the task as they will encounter items that they will probably know. The positions of the correct responses were altered slightly so that the correct word appears equally often in each of the three columns.

Next, the researchers examined the distribution of the scores on the final version. As there are 36 items, the theoretical range of scores is 0-36; however, as each item is composed of one real word and two foils, the guessing rate is one-third, so effectively the score range would be expected to be from about 12 to 36 (a range of 24 points). In fact, the lowest score in the development sample was 13/36 (achieved by one participant, 1.59% of the sample) and the highest score was 36/36 (achieved by two participants, 3.17% of the sample). This suggests a small ceiling effect; however, the participants in this sample are probably somewhat above average in terms of education (mean 14.27 years). The mean Spanish Lexical Decision Task score was 26.92 (SD = 5.63), which was slightly higher than the median score of 26, suggesting some positive skewing within the distribution. A histogram showing the score distribution is shown in Figure 1 (upper panel). Despite the slight positive skew, the score distribution did not differ significantly from normal (Shapiro-Wilk Test (63) = .969, p = .118).

Twenty of the participants in the development sample were tested twice on the Spanish Lexical Decision Task. For the final 36-item version, the correlation between these two administrations was high, r = .890, p < .001, 95%CI of r = .752 .960, suggesting ‘good’ test-retest reliability. The mean test score was 25.35 (SD = 4.58), and at retest it was 26.00 (SD = 5.05) showing only a small practice effect of 0.65 points, and this change was not significant, t(19) = -1.264, p = .222. To quantify the magnitude of this practice effect, the researchers calculated Cohen’s d (^{Cohen, 1992}), which was d = 0.283. This would be considered a qualitatively ‘small’ practice effect.

Still on the topic of the Development Sample, the researchers next examined the association of Spanish Lexical Decision Task scores with demographic variables. There was no significant correlation between age and task performance, r _s = .103, p = .422, 95%CI of r = -.177 .350. This is consistent with findings showing that crystallised verbal ability of adults is little effected by age and remains stable into at least the eighth decade of life (^{Park, 2000}). This lack of correlation also suggests that the Spanish Lexical Decision Task can, in general, be used in research without the need for age correction of scores. On the other hand, there is a large, positive and significant correlation between test scores and years of formal education, r _s = .630, p < .001, 95%CI of r _s = .458 762. This suggests that the Spanish Lexical Decision Task is measuring crystallised verbal ability, which is defined as “skilled judgment habits (that) have become crystallized . . . as a result of earlier learning” (^{Cattell, 1963}; pp 2-3).

As a further examination of the validity of the Spanish Lexical Decision Task, the researchers examined its correlations with scores on the WAIS-IV in the 50 participants who completed both assessments. There was a large, positive significant correlation between Spanish Lexical Decision Task scores and full IQ on the WAIS-IV, r = .715, p < .001, 95%CI of r = .597-828. The correlations with the subtest standard scores are shown in Table 2, where it can be seen that the Spanish Lexical Decision Task had significant and positive correlations with all of the subtest scores. All of these would be considered qualitatively ‘large’ correlations, the largest being with the Information subtest (r _s = .771, p < .001).

Table 2 Correlations of scores on the Spanish Lexical Decision Task with AIS-IV subtest scores 

Next, the researchers examined the statistical performance of the final 36-item Spanish Lexical Decision Task in the Test Sample, which was composed of 75 undergraduate students. This sample scored a mean score of 29.05 (SD = 2.36) and a median score of 29 (range 24-35). Thus, in this sample there is no evidence of skewing nor of a ceiling effect; however, there is also considerably less variation in the data. This can be seen in Figure 1 (lower panel). The distribution of scores did not differ significantly from the normal distribution (Shapiro-Wilk Test (75) = .974, p = .126).

Figure 1 Score distributions of the Spanish Spot-the-Word test in the Development Sample (upper panel) and the Test Sample (lower panel) 

The correlation between the 36-item Spanish Lexical Decision Task and the full WAIS IQ score in the Test Sample was r = .411, p < .001, 95%CI of r = .197 .588. This r value is somewhat lower than that observed in the Development Sample, but this is likely caused by the restriction of range in the data of the Test Sample. The correlations between the Spanish Lexical Decision Task and the WAIS subtests in the Test Sample are also shown in Table 2. Again, the correlation r values are much attenuated in this restricted-range sample; however, the overall pattern is similar to that observed in the Development Sample, in which the two highest correlations are for the mainly verbal assessments of Similarities and Information. They are particularly strong for the latter.

There was also a significant and positive correlation between Spanish Lexical Decision Task scores and Grade Point Average scores, r _s = .318, p = .006, 95%CI of r = .070 .534. This essentially provides predictive validity of the test as a measure of intelligence. However, there was no significant correlation of the Spanish Lexical Decision Task with the number of years spent in education, r _s = .191, p = .100, 95%CI of r = -.063 .421. However, this, again, may be attenuated by the limited range within the data.

Discussion

In this paper, the researchers present preliminary validity of a simple lexical decision task, similar to the English-language Spot-the-Word Test. The test has good psychometric properties. It was found that both internal consistency and test-retest reliability would be considered qualitatively ‘good’ by the normal interpretations of the values. This newly-developed test can be constructed from the items shown in Table 1 and the instructions in the Method section. Alternately, a ready to use version can be downloaded for free from http://www.gpluck.co.uk. This test could be used whenever a simple and fast assessment of language competence is required for research purposes.

More specifically, this Spanish Lexical Decision Task can be considered as a valid measure of crystallised-verbal ability (as opposed to other forms of non-crystallised linguistic ability such as verbal fluency). The fact it measures crystallised ability is demonstrated by its pattern of correlations with demographic and cognitive variables. Firstly, a measure of crystallised ability would be expected to be unrelated to age, which is exactly what was found. This is because while fluid abilities are known to decline in older adulthood, a hallmark of crystallised abilities is that they remain stable throughout adulthood and into old age (^{Craik & Bialystok, 2006}; ^{Park, 2000}). Secondly, a measure of crystallised ability would be expected to be closely associated with years of education, which, again, is what was found. This is because crystallised ability is defined by it being based on acquired information (^{Cattell, 1963}). This leads us to the third supporting point. Crystallised intelligence is thought to be based on two different forms of acquired information: verbal-lexical information and general knowledge (^{Schipolowski et al., 2014}). Of the seven WAIS subtests that were examined, the only test that is clearly a measure of crystallised ability (in the form of general knowledge) is the Information subtest, and this is the test that has the highest correlations with the Spanish Lexical Decision Task.

The correlations between the WAIS subtest scores and the Spanish Lexical Decision Task were all qualitatively ‘large’ within the Development Sample. However, the correlations in the Test Sample comprised of university students, were much smaller, and the only two that were statistically significant were Similarities and Information: both assessments of verbal skills. This again, is consistent with the suggestion that the new test reported here is a measure of crystalized-verbal ability. The reason for the discrepancy in general between the large correlations observed in the Development Sample and the weaker correlations observed in the Test Sample is likely due to the variation of abilities within the two samples. As observed in Figure 1, there was much more variation in task performance within the Development Sample, compared to the Test Sample. It is known that correlation r values are attenuated when variance within the variables is restricted (^{Howell, 1992}).

Therefore, this Spanish-language test, equivalent to the English-language Spot-the-Word Test, can be considered as a reliable and valid measure of crystallised-verbal ability. It should also be noted that the test had impressive correlations with full-scale IQ scores in both samples studied, and it also correlated significantly with academic achievement as measured by undergraduate GPA scores. It could also, therefore, be used as a simple measure of intelligence for research purposes. As a self-completion scale with only 36 items, it has significant advantages over other intelligence measures, which are slow to administer and usually require one-to-one interviews.

A further potential application could be as a measure of premorbid ability, which is particularly important in neuropsychology (see e.g. ^{García-Torres, Vergara-Moragues, Piñón-Blanco, & Pérez-García, 2015}; ^{Pluck et al., 2012}). Measuring premorbid ability is also important in psychopathology research in which scales that are likely not affected by mental-illness-related cognitive impairments are very useful, for example when matching patients to control participants (see e.g. ^{Pluck, Barajas, Hernandez-Rodriguez, & Martínez, 2020}). Initial reports with the English-language Spot-the-Word Test suggested that it was resistant to age-related cognitive declines that were evident with other neuropsychological tests (^{Yuspeh & Vanderploeg, 2000}). Furthermore, one study found that even demented Alzheimer’s disease patients could perform the Spot-the-Word Test normally (^{Law & O'Carroll, 1998}). This resistance to cognitive impairment is very useful to psychologists as it means that the patient’s normal level of cognitive ability can be estimated. However, other studies have suggested that the Spot-the-Word Test is not as resistant as sometimes claimed, and that dementia does in fact have a substantial effect on performance (^{Beardsall & Huppert, 1997}). Nevertheless, further research and modifications of lexical decision tasks such as the Spanish Lexical Decision Task may be able to improve this feature. One particularly promising aspect may be that studies with lexical decision tasks in Spanish have shown that age of acquisition, imageability, and frequency of words within the language are an important factor in resilience to cognitive impairment. Words that are highly imageable, high frequency, and early acquired are generally quite resistant to dementia and, therefore, may be useful in lexical decision tasks that aim to estimate premorbid ability (^{Cuetos et al., 2017}; ^{Cuetos, Herrera, & Ellis, 2010}).

Despite its potential use in neuropsychology, it must be noted that this test has not been validated for clinical or educational decision making and should not be used for such purposes. However, the test may be useful in clinical and educational research contexts where a rapid assessment of crystallised-verbal knowledge, intelligence, or simply reading ability is required. It is notable that the Spanish Lexical Decision Task seems to be stable over ageing, as shown by the lack of correlation with age. This is as expected for a test primarily measuring crystalized-verbal knowledge, as such skills are known to be stable well into old age (^{Craik & Bialystok, 2006}; ^{Park, 2000}). A useful consequence of this is that there is no need for age adjustment. Raw scores from groups can be used for between-subject comparisons, or for correlational/regression analysis with other variables. As the task is designed for research, and not applied assessments, conversion to normative scores is not necessary.

The test was administered via reading in the current research. However, as with the English Spot-the-Word test, it can potentially be administered via speech with the psychologist reading the words aloud. Such an administration format has not yet been specifically assessed, and so it should be used cautiously. It would be useful for future research to establish the psychometric parameters of that alternative administration format, which would allow the test to be used with illiterate or visually impaired participants or simply to test auditory word recognition.

There are some limitations to the research that should be addressed. The Test Sample was comprised of 75 undergraduate students. While this did allow the predictive validity of the Spanish Lexical Decision Task to be examined via the correlations with GPA, there are some problems with the homogeneity of the sample. Student samples from specific sites in general tend to lack variety because they are drawn from limited demographic backgrounds. In the current research, that meant that there was very limited range of ages and educational experience. In addition, as they were drawn from a private university population, they tended to be from a relatively high socioeconomic background. This was not the case for the Development Sample, however, in which there was a deliberate attempt to recruit a wide range of participants. Although socioeconomic status (SES) was not measured directly, education level is considered a strong proxy measure of SES (^{Galobardes, Shaw, Lawlor, Lynch, & Davey Smith, 2006}), and, in our sample, the range was from 6 -26 years of formal education, indicating heterogeneity within our Development Sample. A further limitation is that although crystalized intelligence is thought to be comprised of both lexical word and general knowledge (^{Schipolowski et al., 2014}), the researchers only correlated scores with the latter aspect, as measured by the Information subtest of the WAIS-IV. This was because the abbreviated form of the intelligence that we used does not include the Vocabulary subtest, which would be the obvious measure of stored language. This omission also meant that the researchers could not calculate a measure of verbal IQ, which should also be seen as a limitation. Similarly, the research did not include any measures of linguistic variables such as reading level, or semantic fluency. Such measures would have allowed a more detailed analysis of the validity of the Spanish Lexical Decision Task, and their absence should be considered a limitation.

In summary, the Spanish Lexical Decision Task is a reliable and valid measure of crystallised-verbal ability, reading, and also intelligence. The test is fast to administer and as a self-report scale does not have any issues associated with interrater reliability. In addition, it can be used either in one-to-one interview sessions or also potentially with appropriate control, in group-testing contexts. The Spanish Lexical Decision Task contributes to the available tools for use in cognitive testing of Spanish speaking research participants.