Evidence-Based Assessment of Autism Spectrum Disorder in Schools

Evidence-Based Assessment for Autism Spectrum Disorder

The number of children identified with autism has more than doubled over the last decade. The dramatic increase in prevalence, together with the clear benefits of early intervention, have created a pressing need for schools to identify children who may have an autism spectrum disorder (ASD). As a result, specialized support personnel such as school psychologists are now being asked to participate in the screening, assessment, and educational planning for children and youth on the autism spectrum more than at any other time in the recent past. Moreover, the call for greater use of evidence-based practice has increased demands that professionals be prepared to recognize the presence of risk factors, engage in case finding, and be knowledgeable about evidence-based assessment (EBA) and intervention practices for ASD.

Evidence Based Practice

The challenge to improve the services to children with ASD in our schools is dependent on the adoption of evidence-based practices in diagnosis/identification, assessment, and intervention. The scientific literature identifies two primary elements of evidence-based practice: (a) intervention that includes, but is not limited to, those treatment programs for which randomized controlled trials have shown empirical support for the target population and (b) assessment that guides identification/diagnosis, intervention planning, and outcome evaluation. Evidence-based assessment (EBA) emphasizes the use of research and theory to inform the selection of assessment targets, the methods and measures used in the assessment, and the assessment process itself. Elements of EBA in ASD include the following: (a) the use of psychometrically sound assessments; (b) a developmental perspective that characterizes abilities over the lifespan; (c) assessment of core areas of impairment associated with ASD; and (d) the use of information from multiple sources, including direct and indirect observation from parents and teachers.  

Unfortunately, current research suggests that EBA practices are not implemented in our schools with consistency. For example, a recent nationwide survey of school psychologists’ knowledge of and training and experience with ASD on assessment practices found that less than 25% engaged in EBA (Aiello, Ruble, & Esler, 2017). Most school psychologists reported that they did not engage in comprehensive assessment of ASD, which was defined as assessments that consider all areas of development in addition to the use of psychometrically sound ASD-specific instruments. Even among school psychologists who implemented EBA, the majority relied on ASD checklists that provide limited information and, in the case of the GARS-2, have weak psychometric properties (Aiello et. al., 2017; Norris & Lecavalier, 2010; Wilkinson, 2016). These results indicate a significant gap between best and current practices and the need for guidance regarding which tools demonstrate the strongest psychometric properties for identifying students with ASD.

Psychometric Properties

It is imperative that school psychologists have an understanding of the basic psychometric properties that underlie test use and development when assessing children and youth for ASD. For example, sensitivity and specificity are especially important psychometric characteristics to consider when evaluating the quality and usefulness of tests and rating scales. Sensitivity and specificity are measures of a test's ability to correctly identify someone as having a given disorder or not having the disorder. Sensitivity refers to the percentage of cases with a disorder that screen or test positive. A highly sensitive test means that there are few false negative results (individuals with a disorder who screen negative), and thus fewer cases of the disorder are missed. Specificity is the percentage of cases without a disorder that screens negative. A highly specific test means that there are few false positive results (e.g., individuals without a disorder who screen positive). False negatives decrease sensitivity, whereas false positives decrease specificity. An efficient ASD-specific assessment tool should have high sensitivity and minimize false negatives, as these are individuals with a likely disorder who remain unidentified. Sensitivity and specificity levels of .80 or higher are generally recommended. 

Positive Predictive Value (PPV) and Negative Predictive Value (NPV) are also important validity statistics that describe how well a screening tool or test performs. The probability of having a given disorder, given the results of a test, is called the predictive value. PPV is interpreted as the percentage of all positive cases that truly have the disorder. PPV is a critical measure of the performance of a diagnostic or screening measure, as it reflects the probability that a positive test or screen identifies the disorder for which the individual is being evaluated or screened. NPV is the percentage of all cases screened negative that are truly without the disorder. The higher the PPV and NPV values, the more efficient the instrument at correctly identifying cases. It is important to recognize that PPV is influenced by the sensitivity and specificity of the test as well as the prevalence of the disorder in the sample under study. For example, an ASD-specific measure may be expected to have a higher PPV when utilized with a known group of high-risk children who exhibit signs or symptoms of developmental delay, social skills deficits, or language impairment. In fact, for any diagnostic test, when the prevalence of the disorder is low, the positive PPV will also be low, even using a test with high sensitivity and specificity.

Implications

All school psychologists should be able to conduct psychoeducational assessments of students with ASD to determine learning strengths and challenges, as well as to help determine special education eligibility and develop Individualized Education Plan (IEP) goals and objectives. Given that ASD is no longer considered a low incidence disability, there is an urgent need for practitioners to be well informed, trained, and skilled in the screening and assessment of ASD. Evidence-based assessment (EBA) requires using instruments with strong reliability and validity for the accurate identification of children’s problems and disorders, for ongoing monitoring of children’s response to interventions, and for evaluation of treatment outcomes. We should select and utilize assessments in a manner consistent with available evidence, choose tests that have sound psychometric qualities, and rely on multiple measures to guide high-stakes educational decisions.

Adapted from Wilkinson, L. A. (2017). A best practice guide to assessment and intervention for autism spectrum disorder in schools (Second Edition). London and Philadelphia: Jessica Kingsley Publishers.

Key References and Further Reading

Aiello, R., Ruble, L., & Esler, A. (2017). National Study of School Psychologists’ Use of Evidence-Based Assessment in Autism Spectrum Disorder. Journal of Applied School Psychology, 33(1), 67-88. DOI: 10.1080/15377903.2016.1236307

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

American Psychological Association Statement Policy Statement on Evidence-Based Practice in Psychology (2005). Retrieved on October 26, 2012 from

http://www.apa.org/practice/resources/evidence/evidence-based-statement.pdf

American Psychological Association Task Force on Evidence-Based Practice for Children and Adolescents. (2008). Disseminating evidence-based practice for children and adolescents: A systems approach to enhancing care. Washington, DC: Author.

Campbell, J. M., Ruble, L. A., & Hammond, R. K. (2014). Comprehensive Developmental Approach Assessment Model. In L. A. Wilkinson (Ed.), Autism spectrum disorders in children and adolescents: Evidence-based assessment and intervention (pp. 51-73). Washington, DC: American Psychological Association.

Hixson, M. D., Christ, T. J., & Bruni, T. (2014). Best practices in the analysis of progress monitoring data and decision making. In P. L. Harrison & A. Thomas (Eds.), Best practices in school psychology: Foundations (6th ed., pp. 343–354). Bethesda, MD: National Association of School Psychologists.

Kratochwill, T. R. (2007). Preparing psychologists for evidence based school practice: Lessons learned and challenges ahead. American Psychologist, 62, 826-843.

Kratochwill, T. R., & Hoagwood, K. E. (2006). Evidence-based interventions and system change: Concepts, methods and challenges in implementing evidence-based practices in children’s mental health. Child and Family Policy and Practice Review, 2, 12-17.

Mash, E. J., & Hunsley, J. (2005). Evidence-based assessment of child and adolescent disorders: Issues and challenges. Journal of Clinical Child and Adolescent Psychology, 34, 362-379.

National Association of School Psychologists. (2016). School Psychologists’ Involvement in Assessment. Bethesda, MD: Author.

Ozonoff, S., Goodlin-Jones, B. L., & Solomon, M. (2005). Evidence-based assessment of autism spectrum disorders in children and adolescents. Journal of Clinical Child and Adolescent Psychology, 34, 523–540.

Reynolds, C. R., & Livingston, R. B. (2014). A psychometric primer for school psychologists. In P. L. Harrison & A. Thomas (Eds.), Best practices in school psychology: Foundations (pp. 281–300). Bethesda, MD: National Association of School Psychologists.

Skolnik, Samantha, "School Psychologists’ Integrity of Treatment Integrity" (2016). PCOM Psychology Dissertations. 397. http://digitalcommons.pcom.edu/psychology_dissertations/397

Wilkinson, L. A. (2017). A best practice guide to assessment and intervention for autism spectrum disorder in schools. London and Philadelphia: Jessica Kingsley Publishers.

Lee A. Wilkinson, PhD, is a licensed and nationally certified school psychologist, and certified cognitive-behavioral therapist. He is author of the award-winning books,  A Best Practice Guide to Assessment and Intervention for Autism and Asperger Syndrome in Schools and Overcoming Anxiety and Depression on the Autism Spectrum: A Self-Help Guide Using CBT. He is also editor of a text in the APA School Psychology Book Series, Autism Spectrum Disorder in Children and Adolescents: Evidence-Based Assessment and Intervention in Schools. His latest book is A Best Practice Guide to Assessment and Intervention for Autism Spectrum Disorder in Schools (2nd Edition).

© 2018 Lee A. Wilkinson, PhD