Wednesday, May 30, 2018

Response to Intervention (RtI) and Autism

Response to Intervention (RtI) 
Response to Intervention, commonly referred to as RtI, is defined as “the practice of providing high-quality instruction and interventions matched to student need, monitoring progress frequently to make decisions about changes in instruction or goals, and applying child response data to important educational decisions” (Batsche et al., 2005, p. 3). It is considered a prevention oriented approach to linking assessment and instruction that can inform educators’ decisions about how best to teach their students. RtI employs a multi-level system which includes three levels of intensity or three levels of prevention (primary, secondary, and tertiary), which represent a continuum of supports. Schools use RtI data to identify students at risk for poor learning outcomes, monitor student progress, provide evidence-based interventions and adjust the intensity and nature of those interventions depending on a student’s responsiveness (National Center on Response to Intervention, 2010).
IDEA 2004 allows states to use a process based on a student’s response to scientific, research-based interventions (i.e., RtI) to determine if the child has a specific learning disability (SLD). However, federal law does not require schools to use RtI to determine eligibility for all disabilities. The Office of Special Education Programs (OSEP) has clarified that the IDEA does not address the use of an RtI model for children suspected of having disabilities other than SLD and has emphasized while RtI may be used to determine if a child responds to scientific, research-based intervention as part of the evaluation process, RtI is not, in itself, the equivalent to or replacement for a comprehensive evaluation (United States Department of Education, 2007; Hale, 2008). Schools must use a variety of assessment tools and strategies to gather relevant functional, developmental, and academic information about the child, including information provided by the parent, which may assist in determining eligibility and not use any single measure or assessment as the sole criterion for determining whether a child has a disability, and for determining an appropriate educational program. This requirement applies to all children suspected of having a disability (IDEA, 2004).
RtI and the Autism Spectrum
Although RtI is an important advancement in educational practice, there are serious concerns about identifying a child with autism spectrum disorder (ASD) utilizing the RtI process. The heterogeneity of needs and high level of co-occurring problems demonstrated by children and youth with autism may affect the overall use and generalizability of the RtI model (Gilmartin, 2014). For example, intervention research cannot predict, at the present time, which particular intervention approach works best with which children. Similarly, the needs of children with ASD are complex and often more difficult to identify than those with other disabilities. A lack of understanding of ASD and some of the subtler symptoms of ASD might also result in the use of interventions and teaching methods that are inappropriate for this group of children (Twachtman-Cullen & Twachtman-Bassett, 2011). Moreover, some intervention and assessment procedures for ASD require a specific knowledge base and skills for successful implementation. Teachers may not have the skills to implement scientifically-based instructional practices and assessments. There is research to suggest that school personnel (i.e., general education and special education teachers, school counselors, and paraprofessionals) factual knowledge about the assessment/diagnosis and treatment of autism is low and that few teachers receive training on evidence-based practices for students with ASD (Hendricks, 2011; Williams, Schroeder, Carvalho, & Cervantes, 2011). Even with adequate teacher training, it is difficult to determine if the interventions were implemented with integrity (i.e., accurately and consistently). Although the importance of treatment integrity has been recognized in the literature, this construct has largely been ignored in research and practice. Unfortunately, the measurement of treatment integrity tends to be more the exception than the rule. For example, a recent survey of practicing certified school psychologists’ knowledge and use of treatment integrity in academic and behavioral interventions found that only 18% of the participants consistently measured treatment integrity in a one-on-one consultation, while just 4.6% of the participants consistently measured treatment integrity within a school-based problem-solving team (Skolnik, 2016). 
Concluding Comments
While evidence-based interventions delivered across the levels of RtI might be considered as part of the assessment process, RtI is not a substitute for a comprehensive evaluation in determining a student’s eligibility for special education under the IDEA disability category of autism. The determination of autism should include a variety of information sources and measures, and should not be based on a single measure, process, or information source. At present, the comprehensive development assessment model represents best practice in the evidence-based assessment and identification of ASD in the school context. This approach requires the use of multiple measures including, but not limited to, verbal reports, direct observation, direct interaction and evaluation, and third-party reports. Interviews and observation schedules, together with an assessment of social behavior, language and communication, adaptive behavior, motor skills, sensory issues, atypical behaviors, and cognitive functioning are recommended best practice procedures (Campbell, Ruble, & Hammond, 2014; National Research Council 2001; Ozonoff, Goodlin-Jones, & Solomon, 2007; Wilkinson, 2016). Because ASD affects multiple areas of functioning, an interdisciplinary team approach is essential for establishing a developmental and psychosocial profile of the child in order to guide intervention planning.
Adapted from Wilkinson, L. A. (2016). A best practice guide to assessment and intervention for autism spectrum disorder in schools. London and Philadelphia: Jessica Kingsley Publishers.
 Key References and Further Reading
Batsche, G., Elliott, J., Graden, J. L., Grimes, J., Kovaleski, J. F., Prasse, D…Tilly, W. D. (2005). Response to intervention policy considerations and implementation. Reston, VA: National Association of State Directors of Special Education.
Campbell, J. M., Ruble, L. A., & Hammond, R. K. (2014). Comprehensive Developmental Approach Assessment Model. In L. A. Wilkinson (Ed.), Autism spectrum disorders in children and adolescents: Evidence-based assessment and intervention (pp. 51-73). Washington, DC: American Psychological Association.

Gilmartin, Caitlin E., "Autism Interventions in Educational Settings: Delivery within a Response to Intervention Framework" (2014). PCOM Psychology Dissertations. Paper 306. 

Hale, J. B. (2008). Response to intervention: Guidelines for parents and practitioners. Available from http://www.wrightslaw.com/idea/art/rti.hale.pdf
Hendricks, D. (2011). Special education teachers serving students with autism: A descriptive study of the characteristics and self-reported knowledge and practices employed. Journal of Vocational Rehabilitation, 35, 37–50. doi:10.3233/JVR-2011-0552
Individuals with Disabilities Education Improvement Act of 2004. Pub. L. No. 108-446, 108th Congress, 2nd Session. (2004).
Lane, K. L., Bocian, K. M., MacMillan, D. L. & Gresham, F. M. (2004). Treatment integrity: An essential – but often forgotten – component of school-based interventions, Preventing School Failure, 48, 36–43.
National Center on Response to Intervention (March 2010). Essential components of RTI - A Closer look at response to intervention. Washington, DC: U.S. Department of Education, Office of Special Education Programs, National Center on Response to Intervention.
Ozonoff, S., Goodlin-Jones, B. L., & Solomon, M. (2007). Autism spectrum disorders. In E. J. Mash & R. A. Barkley (Eds.). Assessment of childhood disorders (4th ed., pp. 487-525). New York: Guilford.
Sanetti, L. M., & Kratochwill, T. R. (2014). Introduction: Treatment integrity in psychological research and practice. In L. M. Sanetti & T. R. Kratochwill (Eds.), Treatment integrity: A foundation for evidence-based practice in applied psychology. Washington, DC: American Psychological Association.

Skolnik, Samantha, "School Psychologists’ Integrity of Treatment Integrity" (2016). PCOM Psychology Dissertations. 397. http://digitalcommons.pcom.edu/psychology_dissertations/397
Twachtman-Cullen, D., & Twachtman-Bassett, J. (2011). The IEP from A to Z: How to create meaningful and measurable goals and objectives. San Francisco, CA: Jossey-Bass.
U.S. Department of Education Office of Special Education Programs. Questions and Answers on Response to Intervention (RTI) and Early Intervening Services (EIS), 47 IDELR ¶ 196 (OSERS 2007).
Wilkinson, L. A. (2016). A best practice guide to assessment and intervention for autism spectrum disorder in schools. London and Philadelphia: Jessica Kingsley Publishers.
Williams, K., Schroeder, J. L., Carvalho, C., & Cervantes, A. (2011). School personnel knowledge of autism: A pilot survey. The School Psychologist, 65, 7-9.
Lee A. Wilkinson, PhD, NCSP is a licensed and nationally certified school psychologist, registered psychologist, and certified cognitive-behavioral therapist. He provides consultation services and best practice guidance to school systems, agencies, advocacy groups, and professionals on a wide variety of topics related to children and youth with autism spectrum disorders. Dr. Wilkinson is author of the award-winning books,  A Best Practice Guide to Assessment and Intervention for Autism and Asperger Syndrome in Schools and Overcoming Anxiety and Depression on the Autism Spectrum: A Self-Help Guide Using CBTHe is also editor of a best-selling text in the APA School Psychology Book Series, Autism Spectrum Disorder in Children and Adolescents: Evidence-Based Assessment and Intervention in Schools. His latest book is A Best Practice Guide to Assessment and Intervention for Autism Spectrum Disorder in Schools (2nd Edition).

© 2018 Lee A. Wilkinson, PhD

Thursday, May 3, 2018

Evidence-Based Assessment of Autism Spectrum Disorder in Schools


Evidence-Based Assessment for Autism Spectrum Disorder

The number of children identified with autism has more than doubled over the last decade. The dramatic increase in prevalence, together with the clear benefits of early intervention, have created a pressing need for schools to identify children who may have an autism spectrum disorder (ASD). As a result, specialized support personnel such as school psychologists are now being asked to participate in the screening, assessment, and educational planning for children and youth on the autism spectrum more than at any other time in the recent past. Moreover, the call for greater use of evidence-based practice has increased demands that professionals be prepared to recognize the presence of risk factors, engage in case finding, and be knowledgeable about evidence-based assessment (EBA) and intervention practices for ASD.

Evidence Based Practice

The challenge to improve the services to children with ASD in our schools is dependent on the adoption of evidence-based practices in diagnosis/identification, assessment, and intervention. The scientific literature identifies two primary elements of evidence-based practice: (a) intervention that includes, but is not limited to, those treatment programs for which randomized controlled trials have shown empirical support for the target population and (b) assessment that guides identification/diagnosis, intervention planning, and outcome evaluation. Evidence-based assessment (EBA) emphasizes the use of research and theory to inform the selection of assessment targets, the methods and measures used in the assessment, and the assessment process itself. Elements of EBA in ASD include the following: (a) the use of psychometrically sound assessments; (b) a developmental perspective that characterizes abilities over the lifespan; (c) assessment of core areas of impairment associated with ASD; and (d) the use of information from multiple sources, including direct and indirect observation from parents and teachers.  

Unfortunately, current research suggests that EBA practices are not implemented in our schools with consistency. For example, a recent nationwide survey of school psychologists’ knowledge of and training and experience with ASD on assessment practices found that less than 25% engaged in EBA (Aiello, Ruble, & Esler, 2017). Most school psychologists reported that they did not engage in comprehensive assessment of ASD, which was defined as assessments that consider all areas of development in addition to the use of psychometrically sound ASD-specific instruments. Even among school psychologists who implemented EBA, the majority relied on ASD checklists that provide limited information and, in the case of the GARS-2, have weak psychometric properties (Aiello et. al., 2017; Norris & Lecavalier, 2010; Wilkinson, 2016). These results indicate a significant gap between best and current practices and the need for guidance regarding which tools demonstrate the strongest psychometric properties for identifying students with ASD.

Psychometric Properties

It is imperative that school psychologists have an understanding of the basic psychometric properties that underlie test use and development when assessing children and youth for ASD. For example, sensitivity and specificity are especially important psychometric characteristics to consider when evaluating the quality and usefulness of tests and rating scales. Sensitivity and specificity are measures of a test's ability to correctly identify someone as having a given disorder or not having the disorder. Sensitivity refers to the percentage of cases with a disorder that screen or test positive. A highly sensitive test means that there are few false negative results (individuals with a disorder who screen negative), and thus fewer cases of the disorder are missed. Specificity is the percentage of cases without a disorder that screens negative. A highly specific test means that there are few false positive results (e.g., individuals without a disorder who screen positive). False negatives decrease sensitivity, whereas false positives decrease specificity. An efficient ASD-specific assessment tool should should have high sensitivity and minimize false negatives, as these are individuals with a likely disorder who remain unidentified. Sensitivity and specificity levels of .80 or higher are generally recommended. 

Positive Predictive Value (PPV) and Negative Predictive Value (NPV) are also important validity statistics that describe how well a screening tool or test performs. The probability of having a given disorder, given the results of a test, is called the predictive value. PPV is interpreted as the percentage of all positive cases that truly have the disorder. PPV is a critical measure of the performance of a diagnostic or screening measure, as it reflects the probability that a positive test or screen identifies the disorder for which the individual is being evaluated or screened. NPV is the percentage of all cases screened negative that are truly without the disorder. The higher the PPV and NPV values, the more efficient the instrument at correctly identifying cases. It is important to recognize that PPV is influenced by the sensitivity and specificity of the test as well as the prevalence of the disorder in the sample under study. For example, an ASD-specific measure may be expected to have a higher PPV when utilized with a known group of high-risk children who exhibit signs or symptoms of developmental delay, social skills deficits, or language impairment. In fact, for any diagnostic test, when the prevalence of the disorder is low, the positive PPV will also be low, even using a test with high sensitivity and specificity.

Conclusion

All school psychologists should be able to conduct psychoeducational assessments of students with ASD to determine learning strengths and challenges, as well as to help determine special education eligibility and develop Individualized Education Plan (IEP) goals and objectives. Given that ASD is no longer considered a low incidence disability, there is an urgent need for practitioners to be well informed, trained, and skilled in the screening and assessment of ASD. Evidence-based assessment (EBA) requires using instruments with strong reliability and validity for the accurate identification of children’s problems and disorders, for ongoing monitoring of children’s response to interventions, and for evaluation of treatment outcomes. We should select and utilize assessments in a manner consistent with available evidence, choose tests that have sound psychometric qualities, and rely on multiple measures to guide high-stakes educational decisions.

Adapted from Wilkinson, L. A. (2016). A best practice guide to assessment and intervention for autism spectrum disorder in schools (Second Edition). London and Philadelphia: Jessica Kingsley Publishers.
Key References and Further Reading

Aiello, R., Ruble, L., & Esler, A. (2017). National Study of School Psychologists’ Use of Evidence-Based Assessment in Autism Spectrum Disorder. Journal of Applied School Psychology33(1), 67-88. DOI: 10.1080/15377903.2016.1236307

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

American Psychological Association Statement Policy Statement on Evidence-Based Practice in Psychology (2005). Retrieved on October 26, 2012 from

American Psychological Association Task Force on Evidence-Based Practice for Children and Adolescents. (2008). Disseminating evidence-based practice for children and adolescents: A systems approach to enhancing care. Washington, DC: Author.

Campbell, J. M., Ruble, L. A., & Hammond, R. K. (2014). Comprehensive Developmental Approach Assessment Model. In L. A. Wilkinson (Ed.), Autism spectrum disorders in children and adolescents: Evidence-based assessment and intervention (pp. 51-73). Washington, DC: American Psychological Association.

Hixson, M. D., Christ, T. J., & Bruni, T. (2014). Best practices in the analysis of progress monitoring data and decision making. In P. L. Harrison & A. Thomas (Eds.), Best practices in school psychology: Foundations (6th ed., pp. 343–354). Bethesda, MD: National Association of School Psychologists.

Kratochwill, T. R. (2007). Preparing psychologists for evidence based school practice: Lessons learned and challenges ahead. American Psychologist, 62, 826-843.

Kratochwill, T. R., & Hoagwood, K. E. (2006). Evidence-based interventions and system change: Concepts, methods and challenges in implementing evidence-based practices in children’s mental health. Child and Family Policy and Practice Review, 2, 12-17.

Mash, E. J., & Hunsley, J. (2005). Evidence-based assessment of child and adolescent disorders: Issues and challenges. Journal of Clinical Child and Adolescent Psychology, 34, 362-379.

National Association of School Psychologists. (2016). School Psychologists’ Involvement in Assessment. Bethesda, MD: Author.

Ozonoff, S., Goodlin-Jones, B. L., & Solomon, M. (2005). Evidence-based assessment of autism spectrum disorders in children and adolescents. Journal of Clinical Child and Adolescent Psychology, 34, 523–540.

Reynolds, C. R., & Livingston, R. B. (2014). A psychometric primer for school psychologists. In P. L. Harrison & A. Thomas (Eds.), Best practices in school psychology: Foundations (pp. 281–300). Bethesda, MD: National Association of School Psychologists.

Skolnik, Samantha, "School Psychologists’ Integrity of Treatment Integrity" (2016). PCOM Psychology Dissertations. 397. http://digitalcommons.pcom.edu/psychology_dissertations/397

Wilkinson, L. A. (2016). A best practice guide to assessment and intervention for autism spectrum disorder in schools. London and Philadelphia: Jessica Kingsley Publishers.

Lee A. Wilkinson, PhD, NCSP is a licensed and nationally certified school psychologist, registered psychologist, and certified cognitive-behavioral therapist. He provides consultation services and best practice guidance to school systems, agencies, advocacy groups, and professionals on a wide variety of topics related to children and youth with autism spectrum disorders. Dr. Wilkinson is author of the award-winning books,  A Best Practice Guide to Assessment and Intervention for Autism and Asperger Syndrome in Schools and Overcoming Anxiety and Depression on the Autism Spectrum: A Self-Help Guide Using CBTHe is also editor of a best-selling text in the APA School Psychology Book Series, Autism Spectrum Disorder in Children and Adolescents: Evidence-Based Assessment and Intervention in Schools. His latest book is A Best Practice Guide to Assessment and Intervention for Autism Spectrum Disorder in Schools (2nd Edition).

© 2018 Lee A. Wilkinson, PhD

Follow by Email

Top 10 Most Popular Best Practice Posts

Search BestPracticeAutism.com

Blog Archive

Best Practice Books

Total Pageviews