Home Articles Reader Opinion Editorial Book Reviews Discussion Writers Guide About TCRecord
transparent 13
Topics
Discussion
Announcements

State and Local Efforts To Investigate the Validity and Reliability of Scores From Teacher Evaluation Systems


by Corinne Herlihy, Ezra Karger, Cynthia Pollard, Heather C. Hill, Matthew A. Kraft, Megan Williams & Sarah Howard — 2014

Context: In the past two years, states have implemented sweeping reforms to their teacher evaluation systems in response to Race to the Top legislation and, more recently, NCLB waivers. With these new systems, policymakers hope to make teacher evaluation both more rigorous and more grounded in specific job performance domains such as teaching quality and contributions to student outcomes. Attaching high stakes to teacher scores has prompted an increased focus on the reliability and validity of these scores. Teachers unions have expressed strong concerns about the reliability and validity of using student achievement data to evaluate teachers and the potential for subjective ratings by classroom observers to be biased. The legislation enacted by many states also requires scores derived from teacher observations and the overall systems of teacher evaluation to be valid and reliable.

Focus of the study: In this paper, we explore how state education officials and their district and local partners plan to implement and evaluate their teacher evaluation systems, focusing in particular on states’ efforts to investigate the reliability and validity of scores emerging from the observational component of these systems.

Research design: Through document analysis and interviews with state education officials, we explore several issues that arise in observational systems, including the overall generalizability of teacher scores; the training, certification, and reliability of observers; and specifications regarding the sampling and number of lessons observed per teacher.

Findings: Respondents’ reports suggest that states are attending to the reliability and validity of scores, but inconsistently; in only a few states does there appear to be a coherent strategy regarding reliability and validity in place.

Conclusions: There remain a variety of system design and implementation decisions that states can optimize to increase the reliability and validity of their teacher evaluation scores. While a state may engage in auditing scores, for instance, it may miss the gains to reliability and validity that would accrue from periodic rater retraining and recertification, a stiff program of rater monitoring, and the use of multiple raters per teacher. Most troublesome are decisions about which and how many lessons to sample, which are either mandated legislatively, result from practical concerns or negotiations between stakeholders, or, at best case, rest on broad research not directly related to the state context. This suggests that states should more actively investigate the number of lessons and lesson sampling designs required to yield high-quality scores.



To view the full-text for this article you must be signed-in with the appropropriate membership. Please review your options below:

Sign-in
Email:
Password:
Store a cookie on my computer that will allow me to skip this sign-in in the future.
Send me my password -- I can't remember it
 
Purchase this Article
Purchase State and Local Efforts To Investigate the Validity and Reliability of Scores From Teacher Evaluation Systems
Individual-Resource passes allow you to purchase access to resources one resource at a time. There are no recurring fees.
$12
Become a Member
Online Access
With this membership you receive online access to all of TCRecord's content. The introductory rate of $25 is available for a limited time.
$25
Visitor
Choose this to join the mailing list or add an announcement.
$0
Print and Online Access
With this membership you receive the print journal and free online access to all of TCRecord's content.
$210


Cite This Article as: Teachers College Record Volume 116 Number 1, 2014, p. -
http://www.tcrecord.org ID Number: 17292, Date Accessed: 3/26/2017 3:17:58 AM

Purchase Reprint Rights for this article or review
Article Tools
Related Articles

Related Discussion
 
Post a Comment | Read All

About the Author
  • Corinne Herlihy
    Harvard University
    E-mail Author
    CORINNE HERLIHY is the project director of the National Center for Teacher Effectiveness and researcher at the Center for Education Policy Research at Harvard University. Her interests are in measures of teaching effectiveness, teacher evaluation and professional development, and mathematics education.
  • Ezra Karger
    University of Chicago
    E-mail Author
    EZRA KARGER is a third-year undergraduate at the University of Chicago majoring in mathematics, statistics, and economics who is interested in applying quantitative methods to the analysis of education policy as well as labor, education, and behavioral economics.
  • Cynthia Pollard
    Harvard University
    E-mail Author
    CYNTHIA POLLARD works as a research project manager at the National Center for Teacher Effectiveness housed at the Center for Education Policy Research at Harvard University. She is interested in teacher recruitment, and retention and inequality in education achievement.
  • Heather Hill
    Harvard University
    E-mail Author
    HEATHER C. HILL is a professor at the Harvard Graduate School of Education. Her areas of expertise include teacher and teaching evaluation.
  • Matthew Kraft
    Harvard University
    E-mail Author
    MATTHEW A. KRAFT is an advanced doctoral student in the Quantitative Policy Analysis Program at the Harvard Graduate School of Education and a doctoral fellow at the Center for Education Policy Research at Harvard University. His primary areas of research are the economics of education, education policy analysis, and applied quantitative methods. His current projects include analyses of late teacher hiring, individualized teacher coaching, and teacher–-parent communication.
  • Megan Williams
    Harvard University
    E-mail Author
    MEGAN WILLIAMS studies in the Learning and Teaching Program at the Harvard Graduate School of Education. She is a former practitioner who plans to return to working in schools as a teacher leader.
  • Sarah Howard
    Harvard University
    E-mail Author
    SARAH HOWARD is currently a graduate student at the Harvard Graduate School of Education with interests in teacher support and development.
Member Center
In Print
This Month's Issue

Submit
EMAIL

Twitter

RSS