CC&E Evaluation & Measurement

We have chosen several measures that fit the conceptual and measurement model for this study. All measures will be administered across intervention and control schools with the exception of the student daily performance report, fidelity measures, and social validity measures which only apply to intervention schools.

The model requires:

  1. Daily data collection by the coach
  2. Weekly summaries of the data by the coach
  3. Periodic behavioral observations in classrooms by research assistants
  4. Twice yearly completion of standardized rating scales and interviews by parents, teachers, and children
  5. Yearly assessment of schoolwide features

Figure 1 below displays the possible pathways of the measurement model that have our hypothesized influences on the outcome variables. The moderating factors of student, school, and teacher variables are those that have been demonstrated to affect a student’s success or failure in school.


Figure 1: Measurement Model Pathways. [enlarge] [D]

For example, a student living in a single parent, low-income family with little supervision and academic home support who attends a classroom with a teacher who has poor classroom management skills in a chaotic, disorganized school is at great risk of failing at that school and not graduating from high school.

On the other hand, if the school organizes itself in such a way that it can provide positive and competent school staff to engage the student in their academic and social success, and to heavily reinforce this success, the factors that influence the success/failure of the student can be mediated, and the student has a greater probability of being successful at school.

By having intervention and comparison schools that are matched on the moderating variables, we can measure our impact on students in the intervention condition, and test the hypothesis that this intervention will lead to improved social and academic outcomes.

Measures

Tables 1-3 below list all the measures we are using to assess the effectiveness of the intervention and the goodness of fit of the structural model in the figure.

Table 1: Moderating Variables.
School Teacher Student At-risk Indicators
School-wide Evaluation Tool (SET) Teacher Knowledge and Skills Survey (TKSS) 2.0 Students Screened into study with School-wide Screen for Behavioral Disorders (SSBD)
School Records:
School-wide diversity, school percentage on free or reduced lunch		   Individual Student and Family risk factors:
Anderson (2004) ethnicity, Sp.Ed., Title 1, free or reduced lunch, grade retention, resides with parents or guardian.
Table 2: Treatment and Mediating Factors.
Relationship Reinforcement Engagement
BASC Student Survey Survey Reinforcing Expectations (SET Protocol):
Interviews with student participants and their teachers 		Engagement in School-Teacher Rating (EIS-TR)
Student Teacher Relationship Survey (STRS) Daily Progress Report (DPR) of reinforcement Behavioral and Emotional Rating Scale (BERS): School Functioning
Classroom behavioral observations-teacher interactions Social Validity:
Student Intervention Rating Profile, Intervention Evaluation Inventory		  
Table 2: Treatment and Mediating Factors (continued).
Functionally-Based Intervention Screening Variable Screening for Participation and Use of Treatment in Comparison Schools
Student enters this level of intervention with below 80% on the DPR for 1 week and/or 3 TFBACs Students screened into study with School-wide Screen for Behavioral Disorders (SSBD) Current & Past Implementation of Programs Screening Tool
Based on TFBAC results the student receives differential reinforcement for teacher attention or escape motivated behavior or a group contingency for peer motivated behavior    
Table 3: Student Outcome Indicators.
Problem Behaviors Social Skills Academic Competence
Behavior Assessment System for Children (BASC): Externalizing Behavior SSRS Social Skills Illinois Test of Basic Skills: total Reading, Math, and Written Language
Social Skills Rating Scale (SSRS): Externalizing Problems BASC Social Skills SSRS Academic Competence
Classroom behavioral observations: disruptive behavior BERS Interpersonal Skills Rate of referral to Special Education
School Records:
SWIS office referral data		    
Suspension data    
Rate of referral and prevalence of EBD or with behavioral IEP objectives