Home Articles Reader Opinion Editorial Book Reviews Discussion Writers Guide About TCRecord
transparent 13
Topics
Discussion
Announcements
 

Methodological Challenges in Estimating Effects of Educational Interventions for Students With Disabilities


by Suzanne E. Graham, Carla M. Evans, Beth S. Fornauf & Joy Dangora Erickson - April 04, 2019

Background/Context: Educational researchers frequently study the impact of treatments or interventions on educational outcomes. A critical aspect of such investigations involves determining whether treatment effects vary by student subgroups, such as race/ethnicity, sex/gender, SES, and disability status. However, estimation of intervention effects for subgroups of students defined by disability status can be potentially misleading when researchers control for prior achievement or other measures of academic ability. Estimating intervention effects for students with disabilities is further complicated by the fact that disability status is often defined and measured by whether a student has an Individualized Education Program (IEP), masking important variation in abilities related to academic achievement and services received.

Purpose/Objective/Research Question/Focus of Study: This paper describes methodological challenges in estimating effects of educational interventions for students with disabilities, provides an applied example using data from an innovative state-level educational intervention, and concludes with implications for policy and practice.

Research Design: The analyses presented here come from a larger secondary analysis evaluating the impact of an innovative state assessment and accountability program, New Hampshire’s Performance Assessment of Competency Education (PACE) pilot program (2014-2016), on eighth-grade student academic achievement.

Findings/Conclusions/Recommendations: The estimated effects of the PACE pilot program on eighth-grade student achievement for students with and without disabilities differ depending on whether prior academic achievement is included as a control variable. Controlling for prior academic achievement, we found that the PACE program narrowed or even reversed the achievement gap between students with and without disabilities. When prior achievement was not included, the achievement gap was attenuated but not reversed. Further investigation revealed limited overlap in the distributions of prior achievement for students with and without disabilities, impacting estimates of program effects when prior achievement is controlled. Consistent with other studies, this study employed a dichotomous measure of disability (IEP vs. no IEP). However, the dichotomization of students by disability status defined by whether they have an IEP conceals important variability in cognitive skills related to achievement and thus in understanding the impact of educational interventions. We recommend that researchers investigating the impact of large educational interventions place more emphasis on understanding the impact of these programs for students with disabilities. Importantly, our work underscores problematic analytic and interpretive issues that can ensue when students from all disability groups are grouped together.



To view the full-text for this article you must be signed-in with the appropriate membership. Please review your options below:

Sign-in
Email:
Password:
Store a cookie on my computer that will allow me to skip this sign-in in the future.
Send me my password -- I can't remember it
 
Purchase this Article
Purchase Methodological Challenges in Estimating Effects of Educational Interventions for Students With Disabilities
Individual-Resource passes allow you to purchase access to resources one resource at a time. There are no recurring fees.
$12
Become a Member
Online Access
With this membership you receive online access to all of TCRecord's content. The introductory rate of $25 is available for a limited time.
$25
Visitor
Choose this to join the mailing list or add an announcement.
$0
Print and Online Access
With this membership you receive the print journal and free online access to all of TCRecord's content.
$210


Cite This Article as: Teachers College Record, Date Published: April 04, 2019
http://www.tcrecord.org ID Number: 22745, Date Accessed: 4/23/2019 6:18:59 PM

Purchase Reprint Rights for this article or review
 
Article Tools
Related Articles

Related Discussion
 
Post a Comment | Read All

About the Author
  • Suzanne Graham
    University of New Hampshire
    E-mail Author
    SUZANNE E. GRAHAM is Associate Professor of Education Statistics and Director of Educational Studies in the Department of Education at the University of New Hampshire, Durham. Her research interests include longitudinal data analysis, multilevel modeling, causal inference, and mathematics achievement.
  • Carla Evans
    National Center for the Improvement of Educational Assessment
    E-mail Author
    CARLA M. EVANS is a Postdoctoral Fellow at the National Center for the Improvement of Educational Assessment (Center for Assessment). Her research focuses on the impacts and implementation of assessment and accountability policies on teaching and learning. She is interested in policy research related to innovative assessment and accountability systems, competency-based education, performance-based assessments, and teacher/teacher preparation program effectiveness initiatives.
  • Beth Fornauf
    University of New Hampshire
    E-mail Author
    BETH S. FORNAUF is a Ph.D. Candidate at the University of New Hampshire. Her research is grounded in Disability Studies and focuses on inclusive education through Universal Design for Learning (UDL) in K-12 and higher education settings.
  • Joy Erickson
    University of New Hampshire
    E-mail Author
    JOY DANGORA ERICKSON is a Ph.D. candidate and Dissertation Year Fellow at the University of New Hampshire. Her research interests include early childhood reading motivation and engagement, early childhood education for citizenship, and culturally responsive literacy instruction.
 
Member Center
In Print
This Month's Issue

Submit
EMAIL

Twitter

RSS