Sunday, January 21, 2018

Treatment Integrity: An Essential Component of Evidence-Based Practice

Supporting children with special needs requires individualized and effective intervention strategies. It is very important for families, teachers, administrators, and school-based support personnel to be knowledgeable about evidence-based approaches to adequately address these unique needs and help minimize the gap between research and practice. An evidence-based practice can be defined as a strategy, intervention, treatment, or teaching program that has met rigorous peer review and other standards and has a history of producing consistent positive results when experimentally tested and published in peer-reviewed professional journals. The success of an intervention depends on the interaction between the age of the child, his or her developmental level and individual characteristics, strength of the intervention, competency of the professional, and a critical component termed treatment integrity.
  Treatment Integrity
Treatment integrity, also referred to as treatment fidelity, intervention integrity, procedural reliability, and implementation integrity is defined as the accuracy and consistency with which each part or step of a treatment or intervention plan is implemented as intended. It is an important link between the use and effectiveness of interventions in clinical and school settings, and one of the key aspects of scientific investigation. Identifying an evidence-based intervention or treatment is a necessary but insufficient provision for producing behavior change. Knowing that an intervention is effective and understanding how to use it does not guarantee its accurate implementation. For example, absent or weak treatment effects might be the result of the poor integrity of interventions, despite their demonstrated empirical support. As a result, even evidence-based based interventions may fail. This can lead to erroneous conclusions regarding treatment effectiveness. Low levels of treatment integrity also adversely affect external validity (i.e., the degree to which the results from a study can be generalized across people, settings, and behaviors) and the ability of researchers to replicate the procedures and find comparable results. This can have a negative effect on the development of effective interventions and procedures. Assuring treatment integrity is critically important for dissemination of evidence-based practices and the delivery of effective services.
Treatment Integrity in Research and Practice
Although the importance of treatment validity has been recognized in the literature, this construct has largely been ignored in research and practice. Unfortunately, the measurement of treatment integrity tends to be more the exception than the rule.  Relatively few intervention studies have monitored or systematically assessed treatment integrity. For example, a review of published behavioral intervention research studies found that only 18% assessed and reported treatment integrity data. Similarly, a recent survey of practicing certified school psychologists’ knowledge and use of treatment integrity in academic and behavioral interventions found that while the majority of participants believed that it is a critical component to the development, monitoring, and evaluation of school-based interventions, only 18% of the participants consistently measured treatment integrity in a one-on-one consultation, while just 4.6% of the participants consistently measured treatment integrity within a school-based problem-solving team. 

It appears that treatment integrity is more often assumed rather than evaluated and empirically documented. This reliance on a ‘consult and hope’ strategy (consult and hope the intervention is implemented with as planned) is troublesome in that it impedes our ability to establish functional relationships between treatments and outcomes. The measurement of treatment integrity is essential when evaluating school-based interventions not only for children with autism, but for all students with academic, behavior, and developmental concerns. Without information regarding treatment integrity, practitioners have little idea as to whether their intervention plans are effective. Clearly, establishing the integrity of treatments should be regarded as a critical aspect of practice and research.
 Monitoring Treatment Integrity
A consistent finding in the literature is that higher levels of treatment integrity are associated with better outcomes. Thus, it is essential that treatment integrity information be collected when implementing interventions so as to distinguish between ineffective interventions and potentially effective treatments implemented with poor integrity. While implementing intervention procedures with textbook accuracy consistently in “real-world” settings presents a challenge, practitioners should attempt to implement procedures with high levels of integrity as often as possible. Direct and indirect methods  can be used to document the extent to which behavioral interventions agreed upon during consultation are being implemented as intended by the change agent (e.g. teacher or parent). 

Although systematic observation is the most direct means of assessing treatment integrity, this procedure is vulnerable to reactivity effects and tends to be a labor-intensive activity, which is not always possible given the time constraints and logistical problems encountered in most practice settings. There are, however, less direct methods that can be utilized to monitor the integrity of intervention plans. They include: (a) self-reporting; (b) permanent products; (c) behavioral interviews; and (d) performance feedback. While these methods are less intrusive, they tend to be less accurate because they rely primarily on self-reports. 

The social significance of intervention outcomes or social validity is also of critical importance. Consumers must feel assured that the selected intervention strategies are effective and appropriate, and that the social objectives are important to achieve. If the intervention lacks social validity, they are less likely to apply the effort necessary to implement the intervention, thus reducing intervention fidelity. Educators, parents and families expect (and hope) that research will produce interventions and treatments that will improve quality of life of children. Thus, the measurement of treatment integrity and social validity should be a standard feature of intervention practice and research.

Key References and Further Reading
Bruhn, A. L., Hirsch, S. E., & Lloyd, J. W. (2015). Treatment integrity in school-wide programs: A review of the literature (1993-2012). The Journal of Primary Prevention, 36, 335-349.
Brand, D. (2014). Topical articles: Treatment Integrity: Why it is important regardless of discipline. Science in Autism Treatment, 14(2), 6-7, 9-11.
Cochrane, W. S., & Laux, J. M. (2007). Investigating school psychologists’ perceptions of treatment integrity in school- based interventions for children with academic and behavior concerns. Preventing School Failure, 51, 29-34.
DiGennaro Reed, F. D., & Codding, R. S. (2014). Advancements in procedural fidelity assessment and intervention: Introduction to the special issue. Journal of Behavioral Education, 23, 1-18.
Gresham, F. M., Gansle, K. A., Noell, G. H., Cohen, S. & Rosenblum, S. (1993). Treatment integrity of school-based behavioral intervention studies: 1980–1990, School Psychology Review 22, 254–72.
Gresham, F. M., MacMillan, D., Beebe-Frankenberger, M. E. & Bocian, K. M. (2000). Treatment integrity in learning disabilities intervention research: Do we really know how treatments are implemented? Learning Disabilities Research and Practice, 15, 198–205.
Gresham, F. M. (2009). Evolution of the treatment integrity concept: Current status and future directions. School Psychology Review, 38(4), 533-541.
Hagermoser Sanetti, L. M., & Kratochwill, T. R. (2008). Treatment integrity in behavioral consultation: Measurement, promotion, and outcomes. International Journal of Behavioral Consultation and Therapy, 4, 95-114.
Lane, K. L., Bocian, K. M., MacMillan, D. L. & Gresham, F. M. (2004). Treatment integrity: An essential – but often forgotten – component of school-based interventions, Preventing School Failure, 48, 36–43.
Noell, G. H., Witt, J. C., Slider, N. J., Connell, J. E., Gatti, S. L., Williams, K. L., Keonig, J. L., Resetar, J. L. & Duhon, G. J. (2005). Treatment implementation following behavioral consultation in schools: A comparison of three follow-up strategies. School Psychology Review 34, 87–106.
Sanetti, L. M., & Kratochwill, T. R. (2014). Introduction: Treatment integrity in psychological research and practice. In L. M. Sanetti & T. R. Kratochwill (Eds.), Treatment integrity: A foundation for evidence-based practice in applied psychology. Washington, DC: American Psychological Association.
Skolnik, Samantha, "School Psychologists’ Integrity of Treatment Integrity" (2016). PCOM Psychology Dissertations. 397.
Lee A. Wilkinson, PhD, NCSP is a licensed and nationally certified school psychologist, chartered psychologist, and certified cognitive-behavioral therapist. He provides consultation services and best practice guidance to school systems, agencies, advocacy groups, and professionals on a wide variety of topics related to children and youth with autism spectrum disorders. Dr. Wilkinson is author of the award-winning books,  A Best Practice Guide to Assessment and Intervention for Autism and Asperger Syndrome in Schools and Overcoming Anxiety and Depression on the Autism Spectrum: A Self-Help Guide Using CBTHe is also editor of a best-selling text in the APA School Psychology Book Series, Autism Spectrum Disorder in Children and Adolescents: Evidence-Based Assessment and Intervention in Schools. His latest book is A Best Practice Guide to Assessment and Intervention for Autism Spectrum Disorder in Schools (2nd Edition).

© 2018 Lee A. Wilkinson, PhD

Follow by Email

Top 10 Most Popular Best Practice Posts


Blog Archive

Best Practice Books

Total Pageviews