Home Articles Reader Opinion Editorial Book Reviews Discussion Writers Guide About TCRecord
transparent 13
Topics
Discussion
Announcements

Exogenous Variables and Value-Added Assessments: A Fatal Flaw


by David C. Berliner — 2014

Background: There has been rapid growth in value-added assessment of teachers to meet the widely supported policy goal of identifying the most effective and the most ineffective teachers in a school system. The former group is to be rewarded while the latter group is to be helped or fired for their poor performance. But, value-added approaches to teacher evaluation have many problems. Chief among them is the commonly found class-to-class and year-to-year unreliability in the scores obtained. Teacher value-added scores appear to be highly unstable across two classes of the same subject that they teach in the same semester, or from class to class across two adjacent years.

Focus of Study: This literature review first focuses on the confusion in the minds of the public and politicians between teachers’ effects on individual students, which may be great and usually positive, and teachers’ effects on classroom mean achievement scores, which may be limited by the huge number of exogenous variables affecting classroom achievement scores. Exogenous variables are unaccounted for influences on the data, such as peer classroom effects, school compositional effects, and characteristics of the neighborhoods in which some students live. Further, even if some of these variables are measured, the interactions among these many variables often go unexamined. But, two-way and three-way interactions are quite likely to be occurring and influencing classroom achievement. This analysis promotes the idea that the ubiquitous and powerful effects on value-added scores of these myriad exogenous variables is the reason that almost all current research finds instability in teachers’ classroom behavior and instability in teachers’ value-added scores. This may pose a fatal flaw in implementing value-added assessments of teaching competency.

Research Design: This is an analytic essay, including a selective literature review that includes some secondary analyses.

Conclusions: I conclude that because of the effects of countless exogenous variables on student classroom achievement, value-added assessments do not now and may never be stable enough from class to class or year to year to be used in evaluating teachers. The hope is that with three or more years of value-added data, the identification of extremely good and bad teachers might be possible; but, that goal is not assured, and empirical results suggest that it really is quite hard to reliably identify extremely good and extremely bad groups of teachers. In fact, when picking extremes among teachers, both luck and regression to the mean will combine with the interactions of many variables to produce instability in the value-added scores that are obtained. Examination of the apparently simple policy goal of identifying the best and worst teachers in a school system reveals a morally problematic and psychometrically inadequate base for those policies. In fact, the belief that there are thousands of consistently inadequate teachers may be like the search for welfare queens and disability scam artists—more sensationalism than it is reality.



To view the full-text for this article you must be signed-in with the appropropriate membership. Please review your options below:

Sign-in
Email:
Password:
Store a cookie on my computer that will allow me to skip this sign-in in the future.
Send me my password -- I can't remember it
 
Purchase this Article
Purchase Exogenous Variables and Value-Added Assessments: A Fatal Flaw
Individual-Resource passes allow you to purchase access to resources one resource at a time. There are no recurring fees.
$12
Become a Member
Online Access
With this membership you receive online access to all of TCRecord's content. The introductory rate of $20 is available for a limited time.
$20
Visitor
Choose this to join the mailing list or add an announcement.
$0
Print and Online Access
With this membership you receive the print journal and free online access to all of TCRecord's content.
$145


Cite This Article as: Teachers College Record Volume 116 Number 1, 2014, p. -
http://www.tcrecord.org ID Number: 17293, Date Accessed: 7/30/2014 3:02:32 AM

Purchase Reprint Rights for this article or review
Article Tools
Related Articles

Related Discussion
 
Post a Comment | Read All

About the Author
  • David Berliner
    Arizona State University
    DAVID C. BERLINER is Regents’ Professor Emeritus at Arizona State University in Tempe, AZ. He is interested in the study of teaching and educational policy. Recent publications include the following: Berliner, D. C. (2012). Effects of inequality and poverty vs. teachers and schooling on America’s youth. Teachers College Record. Berliner, D. C. (2012). Narrowing curriculum, assessments, and conceptions of what it means to be smart in the US schools: Creaticide by design. In D. Ambrose & R. J. Sternberg (Eds.), How dogmatic beliefs harm creativity and higher-level thinking (79–93). New York, NY: Routledge/Taylor & Francis.
Member Center
In Print
This Month's Issue

Submit
EMAIL

Twitter

RSS