Artículo de investigación

Multicultural Evidence-Based Assessment Using the Achenbach System Of Empirically Based Assessment (ASEBA) For Ages ½-90+

Evaluación multicultural Basada en evidencia usando el achenbach system of empirically based assessment (ASEBA) para edades ½ - 90+

Thomas M. Achenbach^1*

¹ Department of psychiatry, University of Vermont - United States

^* South Prospect Street, Burlington, VT 05401 USA. E-Mail: Thomas.Achenbach@uvm.edu

Para citar este artículo: Achenbach, T.M. (2015). Multicultural evidence-based assessment using the Achenbach System of Empirically Based Assessment (ASEBA) for ages ½-90+. Psychologia: Avances de la Disciplina, 9(2), 13-23.

Fecha recepción: 10/12/2014. Fecha aceptación: 22/12/2014

Abstract

Evidence-based assessment is a prerequisite for evidence-based mental health services. This article reviews the development, components, and multicultural applications of the Achenbach System of Empirically Based Assessment (ASEBA). The ASEBA includes standardized forms for obtaining self-and collateral-reports of evidence for behavioral, emotional, social, and thought problems and strengths. The problems are scored on empirically based syndromes and DSM-oriented scales. The syndromes have been supported by confirmatory factor analyses of ASEBA data from 54 societies. The strengths are scored on competence, adaptive functioning, and personal strengths scales. Translations of ASEBA forms are available in more than 100 languages. Reports of the ASEBA's use have been published for more than 100 societies and cultural groups. Multicultural norms incorporate data from population samples in 54 societies. ASEBA software enables users to systematically compare data from self- and informant-reports regarding the person being assessed. The Multicultural Family Assessment Module (MFAM) compares scores for children and their parents in relation to norms for age, gender, the type of informant, and user-selected societies. Standardized multicultural evidence-based assessment facilitates communication and collaboration between mental health providers and researchers around the world.

Keywords: Evidence-based assessment, multicultural, international, family assessment, psychopathology.

Introduction

Mental health, medical, and social services are increasingly adopting evidence-based practices. Evidence-based assessment is needed for evidence-based services related to behavioral, emotional, and social problems. Such assessment explicitly documents clients' problems and strengths in order to determine whether help is needed and, if so, what the help should achieve. Evidence-based assessment also provides information needed to tailor interventions to clients' needs and to evaluate clients' progress and outcomes.

Standardized forms for obtaining information about problems and strengths are key components of evidence-based assessment. Such forms enable providers to quickly and economically obtain information on far more problems and strengths than can typically be assessed in clinical interviews. By examining data obtained with the forms, providers can identify issues to address in their interviews with clients. Information obtained from standardized forms is especially valuable when it includes quantitative scores that can be compared with relevant norms, as well as with clients' scores obtained from subsequent re-administrations of forms to evaluate progress and outcomes.

This article reviews the Achenbach System of Empirically Based Assessment (ASEBA; Achenbach, 2009), which provides evidence-based assessment of problems and strengths for ages ½ to 90+ years. The article focuses mainly on ages ½ to 59 years, for which multicultural norms are available for many societies.

Development of ASEBA Forms Early Origins

The ASEBA originated in the 1960s with efforts to remedy the neglect of children's problems in First Edition of the American Psychiatric Association's (1952) Diagnostic and Statistical Manual (DSM-I). DSM-I provided only two categories for childhood disorders, which were designated as Adjustment Reaction of Childhood and Schizophrenic Reaction, Childhood Type. Most children who were seen for mental health services received either no diagnosis or a diagnosis of Adjustment Reaction. The diagnosis of Schizophrenic Reaction was used for severely disturbed children, although their problems seldom resembled the problems of adults who were diagnosed as having schizophrenia.

Neither of the DSM-I child diagnostic categories was derived from research on children's problems. In order to obtain evidence on actual patterns of children's problems, Achenbach (1966) rated behavioral, emotional, and social problems reported in the case records of hundreds of children referred for psychiatric services. The ratings were factor analyzed to identify syndromes of problems that tended to occur together. Many more syndromes were found than were implied by the DSM-I categories of Adjustment Reaction and Schizophrenic Reaction. Other factor-analytic studies also identified more syndromes than were implied by DSM-I (e.g., Borgatta & Fanshel, 1965; Conners, 1970; Dreger et al., 1964; Miller, 1967; Quay, Morse, & Cutler, 1966).

Forms Completed by Parents, Teachers, and Youths for Ages 6-18 Years

The Child Behavior Checklist for Ages 6-18 (CBCL/6-18), Teacher's Report Form (TRF), and Youth Self-Report (YSR) were subsequently developed to obtain assessment information directly from parent figures, teachers, and youths. The CBCL/6-18, TRF, and YSR (Achenbach & Edelbrock, 1983, 1986, 1987) were based on Achenbach's (1966) findings from psychiatric case records and on extensive pilot research in which parents, teachers, and youths completed and provided feedback on draft versions of the forms. Current versions of these forms can be completed on paper and online (Achenbach & Rescorla, 2001). For respondents who cannot complete forms independently, interviewers with no specialized training can read the items aloud and enter the responses.

Problem items. The CBCL/6-18, TRF, and YSR include items describing a broad spectrum of behavioral, emotional, and social problems that parents, teachers, and youths rate on Likert scales as 0 = not true, 1 = somewhat or sometimes true, or 2 = very true or often true. Examples include can't get mind off certain thoughts; can't pay attention for long; can't sit still; cries a lot; gets in many fights; strange behavior; and unhappy, sad, or depressed.

Strengths items. The forms include items for assessing children's strengths. The strengths items of the CBCL/6-18 and YSR assess the amount and quality of participation in sports, non-sports activities, and work; social relations with friends, parents, and siblings; and school-related functioning. The strengths items of the TRF assess academic achievement and adaptive aspects of functioning in school. Each form additionally includes an open-ended item that asks respondents to describe the best things about the child, plus items for describing concerns about the child. The YSR also has items for rating Positive Qualities, such as I like to help others.

Behavior Problem Monitor (BPM). The parent, teacher, and youth versions of the BPM comprise small subsets of CBCL/6-18, TRF, and YSR items (Achenbach, McConaughy, Ivanova, & Rescorla, 2011). Requiring only 1 to 2 minutes to complete, the BPM assesses changes in functioning over brief periods of days, weeks, or months chosen by users to monitor responses to interventions that are designed on the basis of comprehensive initial assessment with the full-length CBCL/6-18, TRF, and YSR.

Forms Completed by Parents, Daycare Providers, and Preschool Teachers for Ages ½-5 Years

Child Behavior Checklist for Ages ½-5 (CBCL/½-5). To extend the ASEBA approach to younger children, the Child Behavior Checklist for Ages ½-5 (CBCL/½-5) is completed by parents and others who see young children from parental perspectives, such as adoptive parents, foster parents, and childcare workers in institutional settings (Achenbach & Rescorla, 2000).

Like the CBCL/6-18, the CBCL/½-5 comprises a broad spectrum of problem items. Each item is rated on the same 0-1-2 Likert scale as the CBCL/6-18. However, to take account of rapid changes that are common among young children, parents are asked to base their ratings on the preceding 2 months, rather than the 6-month rating period specified on the CBCL/6-18.

Language Development Survey (LDS). In addition to problem items, the CBCL/½-5 includes the LDS for assessing language development (Rescorla, 1989). The LDS asks parents questions about several risk factors for language delays. It also asks parents to indicate which words on a vocabulary list the child says. If the child is combining words, parents are asked to report five of the child's longest word combinations. Norms for the number of vocabulary words and the average length of word combinations indicate whether a child's speech is in the normal range or is delayed compared to peers of the same age.

The Caregiver-Teacher Report Form (C-TRF). The C-TRF is a parallel form to the CBCL/½-5. Completed by daycare providers and preschool teachers, the C-TRF has many of the same problem items as the CBCL/½-5 but substitutes items observable in group settings (e.g., disturbs other children) for items more specific to family contexts (e.g., doesn't want to go out of home).

Forms Completed by Adults and Collaterals for Ages 18-90+ Years

Evidence-based assessment is as important for helping adults as for helping children. Evidence-based assessment is also important for assessing the parents of child clients and for assessing adult outcomes for people who needed help when they were children. Furthermore, evidence-based assessment of adults can be greatly enhanced by having forms completed by people ("collaterals") who know the adult clients.

To take account of differences in problems and strengths for adults of different ages, separate ASEBA selfreport and collateral-report forms have been developed for ages 18-59 and 60-90+ (Achenbach, Newhouse, & Rescorla, 2004; Achenbach & Rescorla, 2003). And to take account of age and gender differences within those age ranges, scale scores are normed separately for each gender at ages 18-35, 36-59, 60-75, and 76-90+. The forms for ages 18-59 are the Adult Self-Report (ASR) and Adult Behavior Checklist (ABCL). For ages 60-90+, the forms are the Older Adult Self-Report (OASR) and Older Adult Behavior Checklist (OABCL).

Problem items. The ASEBA adult forms assess diverse behavioral, emotional, social, and thought problems. Some items, such as unhappy, sad, or depressed, are similar to items on ASEBA child forms. Other items, such as feels overwhelmed by responsibilities, are more specific to adults. The problem items for ages 18 to 90+ are rated on the same 0-1-2 Likert scale as is used for children. For ages 18 to 59, the rating period is 6 months. However, to take account of more rapid changes in functioning at older ages, the rating period is 2 months for ages 60 to 90+.

Strengths items. All the forms for ages 18-59 and 60-90+ include ratings for Personal Strengths items such as I make good use of my opportunities. The forms also assess relations with friends and with spouse or partner (if the adult lived with one in the preceding months). The ASR additionally assesses relations with family members, functioning in work (if the adult had paid work in the preceding 6 months), and functioning in educational programs (if the adult was in an educational program in the preceding 6 months).

Observational Assessment Instruments

The foregoing sections described assessment on the basis of self-reports and reports by people who know the individuals who are being assessed. However, experienced observers of children's functioning can also contribute important assessment data. To obtain such data, the ASEBA includes instruments for assessing children on the basis of ratings by experienced observers who see the children within specific settings for specific time periods.

Semistructured Clinical Interview for Children and Adolescents (SCICA). The SCICA (McConaughy & Achenbach, 2001) consists of a protocol for interviewing 6- to 18-year-olds about their interests, problems, and various aspects of functioning. It is administered by an experienced interviewer who then rates the child on problems observed during the interview and problems reported by the child.

Test Observation Form (TOF). The TOF ( McConaughy & Achenbach, 2004) is used to assess 2- to 18-year-olds' behavior during administration of individual ability and achievement tests. Following the test, the test administrator rates the child on problems observed during the test session.

Direct Observation Form (DOF). The DOF (McConaughy & Achenbach, 2009) is used to rate children's behavior observed during 10-minute intervals in classrooms and other group settings. To provide comparisons with other children in the same settings, the observer rates the on-task and problem behavior of two other children, one observed for 10 minutes before observing the identified child and one observed for 10 minutes after observing the identified child. To take account of variations in children's behavior, all three children should be observed on at least three occasions. Ratings of the identified child are then averaged over the observation occasions. Ratings of the two comparison children are also averaged together over the observation occasions. Scores for the identified child are displayed on a profile in relation to scores averaged over the ratings of the two comparison children, as well as in relation to scores obtained by normative samples of children.

Table 1 summarizes the forms that comprise the ASEBA.

Empirically Based Scales for Scoring ASEBA Problem Items
Syndrome Scales

A key goal of the ASEBA is to identify actual patterns of co-occurring problems as a basis for assessment, research, training, and treatment. As ASEBA forms were developed for obtaining ratings by various informants, the forms were used to rate tens of thousands of individuals. The ratings were then factor analyzed to identify syndromes of co-occurring problems. The syndromes were given names that summarize the problems that comprise them. For example, a syndrome that includes items such as gets in many fights and physically attacks people is designated as Aggressive Behavior.

Table 2 lists syndromes that have been derived from factor analyses of ratings of tens of thousands of individuals, plus other scales scored from ASEBA forms.

A scale for scoring individuals in terms of a particular syndrome consists of the problem items that comprise the syndrome. An individual's syndrome score is obtained by summing the ratings of the items comprising the syndrome. For example, on a CBCL/½-5 completed by a child's mother, the child's score for the Sleep Problems syndrome is the sum of the mother's 0-1-2 ratings of the items comprising the syndrome. To enable users to evaluate whether the score is clinically elevated, the score is converted to a standard score (T score) and a percentile based on a normative sample of children.

Internalizing, Externalizing, and Total Problems Scales

The syndromes assess specific patterns of closely related problems, such as the problems of the Aggressive Behavior and Sleep Problems syndromes. Statistical analyses of associations among scores on the various syndromes have revealed groupings of syndromes for which the terms Internalizing and Externalizing were coined (Achenbach, 1966). ASEBA Internalizing scores are computed by summing the scores for syndromes that assess anxiety, depression, social withdrawal, and somatic problems (e.g., headaches) having no apparent physical cause. Externalizing scores are computed by summing the scores for syndromes that assess aggressive behavior and nonaggressive antisocial behavior (e.g., lying, stealing).

To tailor services to clients' characteristics, it is helpful to know whether they have primarily Internalizing or Externalizing problems, both kinds of problems, or neither kind of problems. To provide evidence for whether clients have clinically elevated levels of Internalizing or Externalizing problems, ASEBA forms are scored on Internalizing and Externalizing scales that provide standard scores and percentiles based on appropriate norms.

To summarize an individual's overall problem level in terms of a global score, ASEBA forms provide a normed Total Problems score, which is obtained by summing an individual's ratings on all the problem items of a form.

DSM-Oriented Scales for Scoring ASEBA Problem Items

ASEBA problem items are additionally scored on DSM-oriented scales, which can alert providers to diagnoses that should be considered. The DSM-oriented scales were constructed by inviting experts from many societies to identify ASEBA items that the experts judged to be very consistent with particular DSM diagnostic categories (Achenbach, 2014). This was initially done for DSM-IV categories (American Psychiatric Association, 1994) and was repeated for diagnostic categories that were changed in DSM-5 (American Psychiatric Association, 2013). Items that most experts identified as being very consistent with particular diagnostic categories were used to construct DSM-oriented scales for those categories.

Table 2 lists the names of the ASEBA DSMoriented scales. An individual's score for a DSM-oriented scale is computed by summing the ratings of the scale's items. The sum of ratings for a DSM-oriented scale is then converted to a T score and percentile based on appropriate norms.

Strengths Scales

Table 2 lists scales for scoring the strengths items of the full-length self- and collateral- report forms. For ages ½-5, the CBCL/½-5 includes the LDS for assessing the level of children's speech development.

For ages 6-18, the CBCL/6-18 and YSR are scored on competence scales designated as Activities, which assesses the amount and quality of participation in sports, non-sports activities, and work; Social, which assesses participation in organizations, teams, and clubs, plus friendships and relations with siblings, parents, and other children; School, which assesses academic performance and school problems; and Total Competence, which provides a comprehensive score based on scores for the Activities, Social, and School scales. The YSR is also scored on a Positive Qualities scale, which is the sum of a youth's self-ratings on items that assess favorable characteristics. The TRF has a scale designated as Academic Performance for teachers' ratings of a student's achievement level in academic subjects; and a scale designated as Adaptive Functioning, which is the sum of teachers' ratings of how hard the student is working, how appropriately the student is behaving, how much the student is learning, and how happy the student is.

For ages 18-59 and 60-90+, the self-report and collateral report forms are scored on scales designated as Personal Strengths, Friends, and Spouse/Partner, if the person being assessed lived with one in the preceding 6 months (for the ABCL and ASR) or 2 months (for the OABCL and OASR). The ASR additionally has a scale designated as Family for assessing relations with family members; a scale designated as Job for people who had paid employment in the preceding 6 months; and a scale designated as Education for people who were enrolled in an educational program in the preceding 6 months.

Norms for Scale Scores

When assessment items are scored on scales, the scale scores provide quantitative measures of the degree to which individuals manifest characteristics assessed by the scales, as reported by the person who provides the assessment information. However, in order to judge whether particular scale scores indicate needs for mental health services, it is important to compare an individual's scores with scores for people who are in the normal range. In other words, scores for people who are in the normal range provide a metric with which to determine whether scores obtained by a particular individual indicate more problems or fewer strengths than are reported for typical individuals.

Because problems and strengths may differ for females versus males and also for people of different ages, the ASEBA has separate norms for females and males within particular age ranges. Equally important, because different kinds of informants see people in different contexts and may have different mind sets regarding the people they rate, the ASEBA norms take account of differences between the kinds of informants. For example, because parents and teachers see children in different contexts and may have different mind sets, separate norms are provided for parent and teacher ratings.

Multicultural Applications

ASEBA forms have been translated into more than 100 languages, including Latin American Spanish and Brazilian Portuguese (Achenbach & Rescorla, 2015a). ASEBA forms have been used in more than 100 societies and cultural groups, including several in Latin America (Berube & Achenbach, 2015). Indigenous researchers in many societies arranged to have ASEBA instruments completed for population samples of ½-59-yearolds. Confirmatory factor analyses (CFAs) of problem item ratings from 54 societies have been used to test syndromes that were initially derived from ratings of Anglophone-mostly U.S.-population samples. The CFAs have supported C-TRF, CBCL/6-18, TRF, YSR, ASR, and ABCL syndromes in all the samples tested (Ivanova et al., 2007a, b, c, 2010, 2011, 2015a, b). These findings show that ASEBA syndromes accurately capture patterns of problems that co-occur in ratings of people in 54 societies.

Multicultural Norms

To enable mental health providers and researchers to compare individuals' ASEBA scale scores with appropriate norms for their societies, ratings of representative population samples in 54 societies have been used to construct multicultural norms (Achenbach & Rescorla, 2007, 2010, 2015b). Comparisons of scale scores from many societies have revealed that the mean problem scores for most societies range from 1 standard deviation (SD) below the mean of all societies (the "omnicultural mean") to 1 SD above the omnicultural mean. However, about one-sixth of societies have mean problem scores that are more than 1 SD below the omnicultural mean, while about one-sixth of societies have mean problem scores that are more than 1 SD above the omnicultural mean.

To take account of societal differences in problem scores, three sets of multicultural norms have been constructed for scales scored from each of the self- and collateral-report ASEBA forms for ages ½-59. For each ASEBA form, multicultural norms designated as Group 1 are based on scores from societies whose mean scale scores are more than 1 SD below the omnicultural mean. Multicultural norms designated as Group 2 are based on societies whose mean scale scores are within 1 SD of the omnicultural mean. And multicultural norms designated as Group 3 are based on societies whose mean scale scores are more than 1 SD above the omnicultural mean. ASEBA software enables users to display scale scores in relation to Group 1, 2, or 3 norms, based on data from societies relevant to the people being assessed.

Multi-Informant Assessment

Most mental health providers who work with children now recognize that their observations of children in clinical settings do not provide adequate information about how the children function in everyday settings, such as the home and school. Information should therefore be obtained from parents and teachers as well as from the children themselves. Meta-analyses have yielded a mean correlation of .27 between parent and teacher ratings of children's problems and a mean correlation of .22 between children's self-ratings and ratings of the children by adults, including parents, teachers, and mental health providers (Achenbach, McConaughy, & Howell, 1987). The mean correlation was .60 between pairs of adults who play similar roles with respect to the children they rated, such as mothers and fathers, pairs of teachers, and pairs of mental health providers. Even the correlation of .60 indicates that no one informant is likely to provide the same information as other informants would.

Since the Achenbach et al. (1987) meta-analyses revealed low to moderate correlations between informants' reports of children's problems, so many studies have supported the findings that they are said to be among "the most robust findings in child clinical research" (De Los Reyes & Kazdin, 2005, p. 483). Although many of the studies were from a few rather similar societies, similarly low agreement has been found between parents' CBCL/618 ratings of their children and the children's YSR ratings and also between parents' CBCL/6-18 ratings and teachers' TRF ratings from dozens of very diverse societies (Rescorla et al., 2013, 2014).

In contrast to providers who work with children, providers who work with adults often obtain data only from the adult clients themselves via interviews, tests, and personal histories. However, it has been found that 70% of diagnoses made only on the basis of data from adult clients were wrong when compared with diagnoses made from other sources of data (Meyer et al., 2001). Furthermore, kappa coefficients (Cohen, 1960) yielded chance-corrected agreement averaging 18% between diagnoses based only on data from adult clients and diagnoses based on other sources (Meyer et al., 2001). And meta-analyses have yielded a mean correlation of .45 between adults' self-ratings of psychopathology and ratings by collaterals (Achenbach, Krukowski, Dumenci, & Ivanova, 2005). It is thus clear that comprehensive assessment of adult psychopathology requires information from collaterals as well as from the adult clients.

Cross-Informant Comparisons

The ASEBA provides parallel forms for obtaining assessment data from different kinds of informants who are relevant for particular age ranges. The ASEBA software enables providers to compare data from different informants in the following ways:

1. It displays separate profiles of scale scores based on ratings by each informant who completes an ASEBA form for the person being assessed.
2. It displays side-by-side listings of the 0-1-2 ratings of problem items by multiple informants.
3. For each scale, it displays bar graphs of scores obtained from ratings by multiple informants.
4. It displays correlations between the 0-1-2 ratings of problem items by each pair of informants.

By looking at the various comparisons of data from different informants, providers can

identify important consistencies and differences in the assessed person's functioning, as seen from different perspectives. If certain problems are reported for only one context, such as home or school, this may argue for interventions aimed specifically at that context. On the other hand, if problems are reported by informants who see the assessed person in different contexts, this may argue for interventions to improve the person's functioning in multiple contexts. By having ASEBA forms completed again during and after interventions, providers can document improvements and continuing needs for help, as seen from the perspectives of different informants.

Multicultural Family Assessment Module (MFAM)

The cross-informant comparisons described so far are made by ASEBA software modules for ages ½-5, 6-18, 18-59, and 60-90+. Each module can compare up to 10 forms completed by different informants to assess a person who is within the age range appropriate for the module. For example, the module for ages 6-18 can compare scores from up to 10 CBCL/6-18, TRF, and YSR forms for a 15-year-old girl.

The MFAM enables providers to compare problem scores obtained for 6-18-year-old children with problem scores obtained for their parents (Achenbach, Rescorla, & Ivanova, 2015). Seven syndromes and four DSM-oriented scales have counterparts scored from the ASEBA forms for ages 6-18 and 18-59. If CBCL/6-18, TRF, and/or YSR forms are filled out for a child and ASR and/or ABCL forms are filled out for the child's parents, the MFAM can display bar graphs of the child and parent scale scores side-by-side. Each bar representing a scale score is standardized (i.e., based on appropriate norms) for the age and gender of the person for whom the form was filled out, the type of informant, and the provider's choice of multicultural norms. By looking at the bars, the provider can easily identify problems that may be similar for the child and one or both parents.

As an example, bar graphs of scores from forms completed for a boy and his father may show that both of them have high scores on the Attention Problems syndrome and the DSM-oriented Attention Deficit Hyperactivity Problems scale. This would indicate that both the boy and his father may need help for attention problems. In familyoriented mental health services, providers may elect to show parents the bar graphs that display problem scales scored from forms completed by different informants for their child in comparison with the counterpart scales scored from the ASRs completed by the parents and the ABCLs completed by their spouses. The MFAM output can help parents see the areas in which they and their children may need help, as well as teaching parents about variations in informants' perspectives.

Conclusions

Evidence-based assessment is a prerequisite for evidenced-based services. The assessment forms described in this article have demonstrated their value for research and services in more than a hundred societies and cultural groups. Population samples of hundreds of thousands of people in 54 societies have been willing and able to complete the forms. Analyses of their data have supported the empirically based syndromes scored from assessment forms for ages ½-59. Data from the population samples are incorporated into multicultural norms that enable users to compare scale scores for individuals with norms appropriate for the individuals' age, gender, society, and the type of informant. The widespread use of the assessment forms in diverse societies and contexts facilitates communication and collaboration between mental health providers and researchers across the world.

References

Achenbach, T.M. (1966). The classification of children's psychiatric symptoms: A factor-analytic study. Psychological Monographs, 80(7, Serial No. 615). doi:10.1037/h0093906.         [ Links ]

Achenbach, T.M. (2009). The Achenbach System of Empirically Based Assessment (ASEBA): Development, findings, theory, and applications. Burlington, VT: University of Vermont Research Center for Children, Youth, and Families.         [ Links ]

Achenbach, T.M. (2014). DSM-oriented guide for the Achenbach System of Empirically Based Assessment (ASEBA). Burlington, VT: University of Vermont Research Center for Children, Youth, and Families.         [ Links ]

Achenbach, T.M., & Edelbrock, C. (1983). Manual for the Child Behavior Checklist and Revised Child Behavior Profile. Burlington, VT: University of Vermont, Department of Psychiatry.         [ Links ]

Achenbach, T.M., & Edelbrock, C. (1986). Manual for the Teacher's Report Form and Teacher Version of the Child Behavior Profile. Burlington, VT: University of Vermont, Department of Psychiatry.         [ Links ]

Achenbach, T.M., & Edelbrock, C. (1987). Manual for the Youth Self-Report and Profile. Burlington, VT: University of Vermont, Department of Psychiatry.         [ Links ]

Achenbach, T.M., Krukowski, R.A., Dumenci, L., & Ivanova, M.Y. (2005). Assessment of adult psychopathology: Meta-analyses and implications of cross-informant correlations. Psychological Bulletin, 131, 361-382. doi:10.1037/0033-2909.131.3.361.         [ Links ]

Achenbach, T.M., McConaughy, S.H., & Howell, C.T. (1987). Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101, 213. 232. doi:10.1037/0033-2909.101.2.213         [ Links ]

Achenbach, T.M., McConaughy, S.H., Ivanova, M.Y., & Rescorla, L.A. (2011). Manual for the ASEBA Brief Problem Monitor (BPM). Burlington, VT: University of Vermont Research Center for Children, Youth, and Families.         [ Links ]

Achenbach, T.M., Newhouse, P.A., & Rescorla, L.A. (2004). Manual for the ASEBA older adult forms & profiles. Burlington, VT: University of Vermont Research Center for Children, Youth, and Families.         [ Links ]

Achenbach, T.M., & Rescorla, L.A. (2000). Manual for the ASEBA preschool forms & profiles. Burlington, VT: University of Vermont Research Center for Children, Youth, and Families.         [ Links ]

Achenbach, T.M., & Rescorla, L.A. (2001). Manual for the ASEBA school-age forms & profiles. Burlington, VT: University of Vermont Research Center for Children, Youth, and Families.         [ Links ]

Achenbach, T.M., & Rescorla, L.A. (2003). Manual for the ASEBA adult forms & profiles. Burlington, VT: University of Vermont Research Center for Children, Youth, and Families.         [ Links ]

Achenbach, T.M., & Rescorla, L.A. (2007). Multicultural understanding of child and adolescent psychopathology: Implications for mental health assessment. New York: Guilford Press.         [ Links ]

Achenbach, T.M., & Rescorla, L.A. (2010). Multicultural supplement to the Manual for the ASEBA Preschool Forms & Profiles. Burlington, VT: University of Vermont Research Center for Children, Youth, and Families.         [ Links ]

Achenbach, T.M., & Rescorla, L.A. (2015a). Multicultural guide for the ASEBA forms & profiles for ages ½-59. Burlington, VT: University of Vermont Research Center for Children, Youth, and Families.         [ Links ]

Achenbach, T.M., & Rescorla, L.A. (2015b). Multicultural supplement to the Manual for the ASEBA Adult Forms & Profiles. Burlington, VT: University of Vermont Research Center for Children, Youth, and Families.         [ Links ]

Achenbach, T.M., Rescorla, L.A., & Ivanova, M.Y. (2015). Guide to family assessment using the ASEBA. Burlington, VT: University of Vermont Research Center for Children, Youth, and Families.         [ Links ]

American Psychiatric Association. (1952; 1994; 2013). Diagnostic and statistical manual of mental disorders (1^st ed., 4^th ed., 5^th ed.). Washington, DC: Author.         [ Links ]

Borgatta, E.F., & Fanshel, D. (1965). Behavioral characteristics of children known to psychiatric outpatient clinics. New York: Child Welfare League of America.         [ Links ]

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46. doi:10.1177/001316446002000104         [ Links ]

Conners, C.K. (1970). Symptom patterns in hyperkinetic, neurotic and normal children. Child Development, 4, 667-682.         [ Links ]

De Los Reyes, A., & Kazdin, A.E. (2005). Informant discrepancies in the assessment of childhood psychopathology: A critical review, theoretical framework, and recommendations for further study. Psychological Bulletin, 131, 483-509. doi:10.1037/0033-2909.131.4.483.         [ Links ]

Dreger, R.M., Lewis, P.M., Rich, T.A., Miller, K.S., Reid, M.P., Overlade, D.C., & ... Flemming, E.L. (1964). Behavioral classification project. Journal of Consulting Psychology, 28, 1-13. doi:10.1037/h0046180.         [ Links ]

Ivanova, M.Y., Achenbach, T.M., Dumenci, L., Rescorla, L.A., Almqvist, F., Weintraub, S., ...Verhulst, F.C. (2007a). Testing the 8-syndrome structure of the CBCL in 30 societies. Journal of Clinical Child and Adolescent Psychology, 36, 405-417. doi:10.1080/15374410701444363.         [ Links ]

Ivanova, M.Y., Achenbach, T.M., Rescorla, L.A., Dumenci, L. Almqvist, F., Bathiche, M., ...Verhust, F.C. (2007b). Testing the Teacher's Report Form syndromes in 20 societies. School Psychology Review, 36, 468-483.         [ Links ]

Ivanova, M.Y., Achenbach, T.M., Rescorla, L.A., Dumenci, L., Almqvist, F., Bilenberg, N., ...Verhust, F.C. (2007c). The generalizability of the Youth Self-Report syndrome structure in 23 societies. Journal of Consulting and Clinical Psychology, 75, 729-738. doi:10.1037/0022-006X.75.5.729.         [ Links ]

Ivanova, M.Y., Achenbach, T.M., Rescorla, L.A., Harder, V.S., Ang, R.P., Bilenberg, N., ...Verhulst, F.C. (2010). Preschool psychopathology reported by parents in 23 societies. Testing the seven-syndrome model of the Child Behavior Checklist for Ages 1.5-5. Journal of the American Academy of Child and Adolescent Psychiatry 49, 1215-1224. doi:10.1016/j.jaac.2010.08.019.         [ Links ]

Ivanova, M.Y., Achenbach, T.M., Rescorla, L.A., Bilenberg, N., Bjarnadottir, G., Denner, S., ... Verhulst, F.C. (2011). Syndromes of preschool psychopathology reported by teachers and caregivers in 14 societies using the Caregiver Teacher Report Form (C-TRF). Journal of Early Childhood and Infant Psychology, 7, 87-103.         [ Links ]

Ivanova, M.Y., Achenbach, T.M., Rescorla, L.A., Turner, L.V., Ahmeti-Pronaj, A., Au, A., ... Zasępa, E. (2015a). Syndromes of self-reported psychopathology for ages 18-59 in 29 societies. Journal of Psychopathology and Behavioral Assessment, in press. doi:10.1007/s10862-014-9448-8.         [ Links ]

Ivanova, M.Y., Achenbach, T.M., Rescorla, L.A., Turner, L.V., Árnadóttir, H.A., Au, A., ...Zasępa, E. (2015b). Syndromes of collateral-reported psychopathology for ages 18-59 in 18 societies. International Journal of Clinical and Health Psychology, in press. doi:10.1016/j.ijchp.2014.07.001.         [ Links ]

McConaughy, S.H., & Achenbach, T.M. (2001). Manual for the Semistructured Clinical Interview for Children and Adolescents (2^nd ed.). Burlington, VT: University of Vermont Research Center for Children, Youth, and Families.         [ Links ]

McConaughy, S.H., & Achenbach, T.M. (2004). Manual for the Test Observation Form for Ages 2-18. Burlington, VT: University of Vermont Research Center for Children, Youth, and Families.         [ Links ]

McConaughy, S.H., & Achenbach, T.M. (2009). Manual for the ASEBA Direct Observation Form. Burlington, VT: University of Vermont Research Center for Children, Youth, and Families.         [ Links ]

Meyer, G.J., Finn, S.E., Eyde, L.D., Kay, G.G., Moreland, K.L., Dies, R.R., ... Reed, G.M. (2001). Psychological testing and psychological assessment: A review of evidence and issues. American Psychologist, 56, 128-165.         [ Links ]

Miller, L.C. (1967). Louisville Behavior Checklist for males, 6-12 years of age. Psychological Reports, 21, 885-896.         [ Links ]

Quay, H.C., Morse, W.C., & Cutler, R.L. (1966). Personality patterns of pupils in special classes for the emotionally disturbed. Exceptional Children, 32, 297-301.         [ Links ]

Rescorla, L. (1989). The Language Development Survey: A screening tool for delayed language in toddlers. Journal of Speech and Hearing Disorders, 54, 587-599.         [ Links ]

Rescorla, L., Ginzburg, S., Achenbach, T.M., Ivanova, M.Y., Almqvist, F., Begovac, I., ...Verhulst, F.C. (2013). Cross-informant agreement between parent-reported and adolescent self-reported problems in 25 societies. Journal of Clinical Child and Adolescent Psychology, 42, 262-273. doi:10.1080/15374416.2012.717870.         [ Links ]

Rescorla, L.A., Bochicchio, L., Achenbach, T.M., Ivanova, M.Y., Almqvist, F., Begovac, I., ...Verhulst, F.C. (2014). Parent-teacher agreement on children's problems in 21 societies. Journal of Clinical Child and Adolescent Psychology, 43, 627-642. doi:10.1080/15374416.2014.900719.         [ Links ]