Educators, clinicians and researchers have long recognized the lack of valid and reliable tools for measuring the effectiveness of intervention for children with Autism Spectrum Disorder.42, 43, 44 Developing assessment tools is challenging because children with autism display a vast range of symptoms and severity. In addition, discipline-specific tools may not meet interdisciplinary team needs, and there isn’t a coherent conceptual framework for measuring treatment effectiveness.
Without acceptable tools for measuring change over time, many have resorted to using measures designed for diagnostic purposes including the Childhood Autism Rating Scale (CARS),45 the Gilliam Autism Rating Scale (GARS),46 and the Autism Behavior Checklist (ABC).47 The Autism Treatment Evaluation checklist (ATEC)48 claims to assess the effectiveness of autism treatment. The ATEC is a one page form that includes ratings of subtests for the following categories: speech/communication; sociability; sensory/cognitive awareness; and health/physical/behavior. It is recommended that parents, teachers or caretakers complete the form by indicating "how true" each descriptive phrase (e.g. "Knows own name") is under each category. Assigned Ratings indicate that the ATEC statements are “not true” (N), “somewhat true” (S) or, “very true” (V). The ATEC appears to lack the sensitivity necessary to monitor subtle changes or progress in complex behaviors associated with Autism Spectrum Disorder. In addition, the ATEC is not conceptually relevant and does not identify factors that may hinder improvement.
The Participation Accuracy Independence Scales (PAIRS©) are a cross disciplinary outcomes assessment tool and a computerized data management system. They are used to monitor and refine educational interventions, inform staff meetings and parent staffings, complete quarterly progress reports, and document outcomes for educational programs and for the entire school. They allow school personnel to measure the progress of educational and clinical interventions for children on the autism spectrum. The scales are conceptually consistent with The World Health Organization’s49 perspective on functioning and disability. Specifically, the PAIRS rating scales capture how well and independently students manage in educational and clinical contexts given their level of functioning and known barriers to success.
The PAIRS include two separate rating scales. The first is the Accuracy/Independence (AI) scale, which captures how accurately and independently a student performs an educational activity. Supplemental AI codes are used to identify the types of assistance (e.g., gestural, verbal) that facilitate the highest level of accurate responding. The second is the Participation (P) rating scale, which measures the level of active participation during a teacher-learning exchange. P scale subscripts help to identify probable reasons for a student’s lack of full participation in a lesson. Thus, problems with attention, behavior and other factors that interfere with active learning are monitored so that appropriate adjustments can be made to educational plans to reduce or minimize interfering behaviors. Together the two scales provide a means of evaluating the accuracy of performance, level of independence and the level of active participation in a task. Clinicians and teachers use the scales to assess baseline or entry level of performance on goals and objectives for students’ Individual Educational Plans (IEPs). Because the PAIRS scales have proven to be sensitive to change over time, they are used to monitor progress made on IEPs.
The PAIRS are not discipline specific and they have been adapted by all members of our multidisciplinary team. In addition, the scales have been applied to a wide variety of educational and clinical approaches ranging from structured and didactic to more naturalistic settings of information without requiring an overly complex number system, and they appear to be sensitive enough to capture meaningful change in a student’s performance. Furthermore, the coding system is intuitive and useful for both professionals and the lay public.
All teachers and clinicians at Monarch School use the PAIRS to track outcomes on students’ IEP goals and objectives. Data sheets, which are used to record scale scores for each objective, are scanned into a central database that automatically summarizes individual student data. The computerized system tracks data over time (by academic quarters) according to whether Ohio state standards are met for IEP goals and objectives. Automated reports indicate whether goals and objectives have been Mastered (M), or if Adequate Progress (AP), Minimal (M), or No Progress (NP) was achieved according to pre-specified performance criteria. Reports also summarize the percentage of objectives that have met pre-specified criteria for levels of mastery for each IEP goal.
At the end of the year, student reports are generated that contain both quarter-by-quarter progress and a year-end summary of progress. Reports are organized by functional domain. Therefore, progress can be viewed for goals and objectives within educational categories (e.g., ADLs, Behavioral, Math, Speech-Language, etc.) for each student, group of students, and for the school. The Monarch Outcomes Management system facilitates an evidence-based approach to the educational management of children on the autism spectrum.