Wednesday, December 7, 2016

Assessment Tools for ASD: Sensitivity Matters


Evidence-based assessment requires using instruments with strong reliability and validity for the accurate identification of children’s problems and disorders, for ongoing monitoring of children’s response to interventions, and for evaluation of the outcomes of intervention. These procedures must also have demonstrated effectiveness in diagnosis, clinical formulation, intervention planning, and outcome assessment. 

Professionals should have an understanding of the basic psychometrics properties that underlie test use and development when assessing children and youth for autism spectrum disorder (ASD). For example, diagnostic validity is an especially important psychometric characteristic to consider when evaluating the quality and usefulness of a screening instrument or more comprehensive measure. It refers to a test’s accuracy in predicting group membership (e.g., ASD versus non-ASD) and can be expressed through metrics such as sensitivity and specificity, and positive predictive value (PPV) and negative predictive value (NPV). 

Sensitivity and specificity are measures of a test's ability to correctly identify someone as having a given disorder or not having the disorder. Sensitivity refers to the percentage of cases with a disorder that screens positive. A highly sensitive test means that there are few false negative results (individuals with a disorder who screen negative), and thus fewer cases of the disorder are missed. Specificity is the percentage of cases without a disorder that screens negative. A highly specific test means that there are few false positive results (e.g., individuals without a disorder who screen positive). False negatives decrease sensitivity, whereas false positives decrease specificity. An efficient screening tool should minimize false negatives, as these are individuals with a likely disorder who remain unidentified. Sensitivity and specificity levels of .80 or higher are generally recommended.  

Positive Predictive Value (PPV) and Negative Predictive Value (NPV) are also important validity statistics that describe how well a screening tool or test performs. The probability of having a given disorder, given the results of a test, is called the predictive value. PPV is interpreted as the percentage of all positive cases that truly have the disorder. PPV is a critical measure of the performance of a diagnostic or screening measure, as it reflects the probability that a positive test or screen identifies the disorder for which the individual is being evaluated or screened. NPV is the percentage of all cases screened negative that are truly without the disorder. The higher the PPV and NPV values, the more efficient the instrument at correctly identifying cases. It is important to recognize that PPV is influenced by the sensitivity and specificity of the test as well as the prevalence of the disorder in the sample under study. For example, an ASD-specific screening measure may be expected to have a higher PPV when utilized with a known group of high-risk children who exhibit signs or symptoms of developmental delay, social skills deficits, or language impairment. In fact, for any diagnostic test, when the prevalence of the disorder is low, the positive PPV will also be low, even using a test with high sensitivity and specificity.
© Lee A. Wilkinson, PhD
Lee A. Wilkinson, PhD, NCSP is a licensed and nationally certified school psychologist, registered psychologist, and certified cognitive-behavioral therapist. He provides consultation services and best practice guidance to school systems, agencies, advocacy groups, and professionals on a wide variety of topics related to children and youth with autism spectrum disorders. Dr. Wilkinson is author of the award-winning books,  A Best Practice Guide to Assessment and Intervention for Autism and Asperger Syndrome in Schools and Overcoming Anxiety and Depression on the Autism Spectrum: A Self-Help Guide Using CBTHe is also editor of a best-selling text in the APA School Psychology Book Series,  Autism Spectrum Disorder in Children and Adolescents: Evidence-Based Assessment and Intervention in Schools. His latest book is A Best Practice Guide to Assessment and Intervention for Autism Spectrum Disorder in Schools (2nd Edition).

Savant Skills and the Autism Spectrum


What are savant skills?
There is a long history of reports of individuals who despite having severe intellectual impairments, demonstrate remarkable skills in a particular area. The term “savant” has been variously defined as those individuals who show (a) normatively superior performance in an area and (b) a discrepancy between their performance in that area and their general level of functioning. Some researchers have differentiated “prodigious” savants (e.g., individuals possessing an exceptional ability in relation to both their overall level of functioning and the general population) from “talented” savants (e.g., individuals showing an outstanding skill in comparison with their overall level of functioning). 

Savant skills have been reported much more frequently in males than in females and have been identified in a wide range of neurological and neurodevelopmental disorders. The most commonly reported savant skills are mathematical skills (calendrical calculations, rapid arithmetic and prime number calculations), music (especially the ability to replay complex sequences after only one exposure), art (complex scenes with accurate perspective either created or replicated following a single brief viewing) and memory for dates, places, routes or facts. Less frequently reported are “pseudo-verbal” skills (hyperlexia or facility with foreign languages), coordination skills and mechanical aptitude.
Research
Research in the past 10 years has generated some controversy about the actual incidence of savant syndrome in autism. Once thought to be rare in people with autism, found in no more than 1 out of 10 individuals, research over the past few years suggests savant skills may be more common than previous estimates. Although there have been many single case or small group studies of individuals with autism who possess savant abilities or exceptional cognitive skills, there have been few systematic, large-scale investigations in this area. Inconsistencies in definition and wide variation in diagnostic criteria, ages and ability levels of the cases reported are problematic, as is a paucity of valid information on rates of savant skills in ASD. The objective of this research study was to investigate the nature and frequency of savant skills in a large sample of individuals with autism who had been initially diagnosed as children.
The total sample was comprised 137 individuals, first diagnosed with autism as children, who were subsequently involved in an ongoing, longitudinal follow-up study. Cognitive assessments (Wechsler Scales) were completed for all participants (100 males and 37 females) between the ages of 11 and 48 years (mean age of 24). Parental report data on savant skills were obtained approximately 10 years later at a subsequent follow-up.  Cognitive ability ranged from severe intellectual impairment to superior functioning. Savant skills were judged from parental reports and specified as “an outstanding skill/knowledge clearly above participant’s general level of ability and above the population norm.”
Results
Of the 93 individuals for whom parental questionnaire and cognitive data were available, 16 (17.2%) met criteria for a parent-rated skill, 15 (16.8%) had an exceptional cognitive skill and 8 (8.6%) met criteria for both. There were 14 calendrical calculators (one also showed exceptional memory and another also showed skill in computation and music). There were four others with computational skills (in one case combined with memory and in another case with music). Visuospatial skills (e.g., directions or highly accurate drawing) were reported in three individuals. One individual had a musical talent, one an exceptional memory skill and one had skills in both memory and art. The subtest on which participants were most likely to meet the specified criteria for an area of unusual cognitive skill was block design followed by digit span, object assembly and arithmetic.
There was a sex difference (albeit statistically non-significant) in the prevalence of savant skills. Almost one-third (32%) of males showed some form of savant or special cognitive skill compared with 19 percent of females. No individual with a non-verbal IQ below 50 met criteria for a savant skill and contrary to some earlier hypotheses; there was no indication that individuals with higher rates of stereotyped behaviors/interests were more likely to demonstrate savant skills.
Discussion
In total, 39 participants (28.5%) met criteria for a savant skill. Cognitively, 23 individuals (17% of total sample) met criteria for one or more exceptional area of skill on the Wechsler Scales. Combining the two, 37 per cent of the sample showed either savant skills or unusual cognitive skills or both, a far higher proportion than previously reported. These results suggest that the rates of savant skills in autism are significant, particularly among males, and although these estimates are higher than reported by other researchers, the findings parallel those of previous studies. Based on these findings, it appears likely that at least a third of individuals with autism show unusual skills or talents that are both above population norms and above their own overall level of cognitive functioning. It should be noted that these data offer no support to claims that savant skills occur most frequently in individuals with autism who are intellectually challenged or that individuals with higher rates of stereotyped behaviors/interests are more likely to demonstrate savant skills.
Apart from the need for further research examining the underlying basis of savant skills and why certain individuals go on to develop any area of exceptional skill and why these skills encompass such different areas, there is a more practical and pressing question; “how can these innate talents be developed to form the basis of truly ‘functional’ skills?” In the present study, only five individuals with exceptional abilities (four related to math and one related to visuospatial ability) had succeeded in using these skills to find permanent employment. For the majority, the isolated skill remained just that, leading neither to employment nor greater social integration. As the authors conclude, “The practical challenge now is to determine how individuals with special skills can be assisted, from childhood onward, to develop their talents in ways that are of direct practical value (in terms of educational and occupational achievements), thereby enhancing their opportunities for social inclusion as adults.”
Key References

Howlin, P., Goode, S., Hutton, J., & Rutter, M. (2009). Savant skills in autism: Psychometric approaches and parental reportsPhilosophical Transactions of the Royal Society B: Biological Sciences, 364, 1359–1367. doi:10.1098/rstb.2008.0328 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2677586/

Marsa, L. (2016). Extraordinary minds: The link between savantism and autism. https://spectrumnews.org/features/deep-dive/extraordinary-minds-the-link-between-savantism-and-autism/

Treffert D. (2000). Extraordinary people: Understanding savant syndrome. Ballantine Books: New York, NY.

Treffert, D. (2009). The Savant Syndrome: An Extraordinary Condition. A Synopsis: Past, Present, Future. Philosophical Transactions of the Royal Society B: Biological Sciences, 364, 1351–1358.

Lee A. Wilkinson, PhD, NCSP is a licensed and nationally certified school psychologist, registered psychologist, and certified cognitive-behavioral therapist. He provides consultation services and best practice guidance to school systems, agencies, advocacy groups, and professionals on a wide variety of topics related to children and youth with autism spectrum disorder. Dr. Wilkinson is author of the award-winning books,  A Best Practice Guide to Assessment and Intervention for Autism and Asperger Syndrome in Schools and Overcoming Anxiety and Depression on the Autism Spectrum: A Self-Help Guide Using CBTHe is also editor of a best-selling text in the APA School Psychology Book Series,  Autism Spectrum Disorder in Children and Adolescents: Evidence-Based Assessment and Intervention in Schools. His latest book is A Best Practice Guide to Assessment and Intervention for Autism Spectrum Disorder in Schools (2nd Edition).




Follow by Email

Top 10 Most Popular Best Practice Posts

Search BestPracticeAutism.com

Blog Archive

Best Practice Books

Total Pageviews