Home Articles Reader Opinion Editorial Book Reviews Discussion Writers Guide About TCRecord
transparent 13
Topics
Discussion
Announcements
 

Everything is Bigger (and Badder) in Texas: Houston’s Teacher Value-Added System


by Audrey Amrein-Beardsley, Clarin Collins, Jessica Holloway-Libell & Noelle Paufler - January 05, 2016

In this commentary, authors discuss the Houston Independent School District's (HISD) highest-stakes use of its contracted value-added system (i.e., the Education Value-Added Assessment System (EVAAS)) to reform and improve student learning and achievement throughout the district's schools. Authors situate their discussion within a related report on the recent release of Houston Superintendent's own evaluation scores. Authors also situate their discussion within the evidence, as per the recent release of the state's large-scale standardized test scores. Authors assert that, perhaps, attaching high-stakes consequences to teachers' value-added output in Houston is not working as intended.

As the saying goes, everything is “bigger and badder” in Texas, and nothing could be truer of the state’s largest public school district, Houston Independent School District (HISD), which serves approximately 215,000 students (75% of whom are socioeconomically disadvantaged). Nationally, HISD ranks seventh in overall size, but it’s definitely the biggest and baddest in terms of punishing and rewarding teachers based on their students’ test scores. With no holds barred, the district attaches more high-stakes consequences to teacher-level value-added estimates derived via value-added models (VAMs) than any other state or district in the nation1 (see also Corcoran, 2010; Harris, 2011; Lowrey, 2012; Sparks, 2011).


Under the command of HISD’s current Superintendent Terry Grier, district leaders are using VAM estimates (as per a $500,000 annual contract the district has with SAS Institute Inc. and its Education Value-Added Assessment Systems (EVAAS)) to determine whether teachers are to be put on professional improvement plans, awarded or denied merit pay, granted annual contract renewals, or terminated. But with Mr. Grier’s recent announcement of his upcoming retirement, we also recently learned that, apparently, those same practices are not even fitting for him as the district’s leader. Following his retirement notice, details surrounding his most recent evaluation were released, along with details describing the school board’s split vote (i.e., 4-3, with two members absent) in favor of not only his “satisfactory” rating, but also his $98,600 merit performance bonus for his value-added last year (this bonus was to be paid atop his $300,000 yearly salary, plus benefits including $19,200 in car and technology allowances and pay for unused leave time).


Illustrated in the Houston Chronicle (Mellon, 2015a, 2015b, 2015c) were the criteria the school board used to determine Mr. Grier’s bonus. Not surprisingly, it was primarily based on annual district-level growth ($53,600) and other job-related criteria2 ($45,000); however, and surprisingly, indicators of Grier’s annual district-level growth were only partially measured using the district’s VAM data (i.e., 2/18 or 11% of the growth measures included in his evaluation were actually derived via the district’s VAM). What remains uncertain is why these data mattered substantively less for Mr. Grier than for his teachers, for whom value-added (or value-detracted) can count for up to 50% of their total teacher evaluation scores, or effectively 100% of their total teacher evaluation scores when growth is used to trump other district measures of teacher effectiveness (e.g., measures of teachers’ instructional practice).3


In addition, during Mr. Grier’s six years as superintendent of HISD, his bonuses have demonstrably grown larger over time, despite his now questionable value-added as the district’s superintendent. Take a look, for example, at the two figures below (see Figures 1 and 2), taken directly from HISD’s recently released reports (2015) highlighting district students’ performance (declining, on average, in blue) as compared to the state of Texas (maintaining, on average, in black), as per the state’s newest (launched in 2012) set of grade 3–8 and high school State of Texas Assessments of Academic Readiness (STAAR) tests.4

 

Figure 1. Houston (blue trend line) v. Texas (black trend line) performance on the STAAR tests, 2012-2015 (HISD, 2015a, p. 19)


[39_18983.htm_g/00002.jpg] 


Figure 2. Houston (blue trend line) v. Texas (black trend line) performance on the state’s STAAR End-of-Course (EOC) tests, 2012-2015 (HISD, 2015b, p. 17)


[39_18983.htm_g/00004.jpg]


When considering these data, keep in mind that Mr. Grier and his district leaders have fired numerous teachers based, in large part, on their students’ value-added scores. For example, in 2011 alone, 221 district teachers were terminated due to demonstrating “a significant lack of student progress attributable to the educator” or “insufficient student academic growth as reflected by [teachers’] value-added scores.”5 These actions resulted in a series of wrongful termination hearings,6 and they ultimately resulted in the filing of an April 2014 lawsuit by the Houston Federation of Teachers (HFT), with the support of the American Federation of Teachers (AFT), on behalf of teacher plaintiffs.7 Plaintiffs charge that district leaders are inappropriately attaching the aforementioned high-stakes consequences to their VAM data, given that their VAM data are allegedly unstable, inaccurate, biased, unfair, incomprehensible, and inactionable (Sawchuck, 2015; Strauss, 2014); hence, plaintiffs also charge, high-stakes consequences cannot and should not be attached to their value-added output. With this impending lawsuit, Mr. Grier and his district leaders recoiled at some of the consequences attached to teachers’ VAM output (e.g., teacher terminations), though it will still be heard in court in June of 2016.


For years, now, Mr. Grier has deemed it appropriate to make such extreme decisions using VAM data for teachers throughout his district. Should we not ask whether he deserves similar treatment, and much less a satisfactory evaluation and an accompanied $98,600 merit performance bonus given the district’s growth, or lack thereof, as compared to the state? Would it not seem fitting to hold superintendents such as Grier, who have championed the use of VAMs to both evaluate and make highly consequential decisions about their teachers, accountable using the same logic, as well as a similar set of accountability measures?


Let us also ponder the specific model used by HISD—the aforementioned EVAAS—and SAS Institute Inc.’s perpetual claims that this model helps teachers become more “proactive [while] making sound instructional choices;” helps teachers use “resources more strategically to ensure that every student has the chance to succeed;” or “provides valuable diagnostic information about [teachers’ instructional] practices” so as to ultimately improve student learning and achievement (SAS Institute Inc., n.d.; see also Kappler Hewitt, 2015).


Perhaps the district’s EVAAS system is not as much of an “educational-improvement and performance-management model that engages all employees in creating a culture of excellence” (HISD, n.d.a) that is yielding “outstanding [student-level] results” (HISD, n.d.b) as is being asserted.


If not (see, again, Figures 1 and 2 above), should anyone really be held accountable using such a system that also fails to meet or evidence its own fundamental claims?


Not that we necessarily want to generalize from this one case beyond Houston’s borders, but when the rooster crows, it often means the sun is a’risin’. Even the simplest evidence presented above should at the very least make us question this particular value-added system, as paid for,8 supported,9 and applied in Houston for some of the biggest and baddest teacher-level consequences in town. Whether all of this occurring in Houston under the leadership of Mr. Grier has also helped to reform or improve Houston’s schools is also certainly suspect given the evidence.10


Notes


1. Attaching high-stakes consequences to teachers’ students’ test scores, prior to the passage of the Every Student Succeeds Act (ESSA; White House, 2015), has been largely incentivized throughout the nation via the federal Race to the Top (RttT) competition (2011; see also Duncan 2009) and the federal waivers excusing states from not meeting No Child Left Behind’s (NCLB, 2001) 100% proficiency targets by 2014 (US Department of Education, n.d.; see also Klein, 2015). However, HISD has been doing this well before RttP incentivized such activities, although it is uncertain whether HISD will continue these practices, now, post-ESSA.

2. Other evaluative criteria included indicators of (a) college preparedness, (b) Mr. Grier’s advance placement initiative, (c) his power up laptop program, (d) his literacy by 3rd grade program, (e) his effective teacher initiative, (f) indicators of school safety and environment, (g) the financial well-being of the district, and (h) his communications with the school board. Of these eight criteria, one—the (5) effective teacher initiative—uses EVAAS, among other criteria, to assess teachers.

3. Many principals throughout HISD have described the district-level pressures placed on them to ensure or guarantee specific evaluation outcomes, whereby value-added scores seem to be trumping other indicators capturing what it means to be an effective teacher (i.e., artificial conflation). In HISD, this is also part of district policy, as they noted in 2012 in the following: “While there is a meaningful, positive relationship between Instructional Practice [IP] ratings and school level EVAAS, many schools are not aligning their assessment of classroom practice with student [EVAAS] growth. Individual teacher ratings and student growth are also correlated, though there is significant room for improvement in aligning teacher IP scores to student [EVAAS] outcomes. Instructional practice [IP] scores show that there is room for improvement for schools to be more consistent in their ratings” (HISD, 2012, p. 3). One year later, in 2013, the district set a district wide goal for what they termed “rating accuracy” for all schools to align their teachers’ IP ratings to student performance, again, as measured via the EVAAS. The explicit goal was to have “at least 85% of [HISD] schools meet the minimum bar for alignment between Instructional Practice [IP] and Student Performance [EVAAS]” (HISD, 2013, p. 3, 11).

4. Note that these figures illustrate growth-by-cohort over time, whereby third graders are compared to different sets of third graders subsequent years. These figures do not demonstrate EVAAS growth, although for this year’s evaluation Mr. Grier demonstrated “added value” in district wide mathematics (+0.2) and “decreased value” in district wide reading (-0.3).

5. Houston Independent School District (Plaintiff) v. Mia Thymes, Hercilia Leon, Mary McWilliams, and Glenda Drew (Defendants); Before Robert Barringer, Franklin Holcomb, Allecia Pottinger, and Audrei Lawton, Certified Hearing Examiners, Texas Education Agency (2011).

6. Ibid.

7. Houston Federation of Teachers, Local 2415, Daniel Santos, Paloma Garner, Ivan Castillo, Andy Dewey, Joyce Helfman, Myla Van Duyn, and Araceli Ramos (Plaintiffs) v. Houston Independent School District (Defendant); United States District Court, Southern District of Texas, Houston Division (Pending, 2016).

8. Again, HISD pays $500,000 annually to SAS Institute Inc. for its EVAAS services.

9. Supported by Battelle for Kids (which is united with SAS Institute Inc.), HISD’s ASPIRE system (launched in 2007−2008) is the district’s “educational-improvement and performance-management model” meant to engage all district employees “in creating a culture of excellence.” It is through this system that all HISD employees are to access information about the district’s value-added reports, as well as related training opportunities, reports, and other resources to inform, in particular, teachers’ practice. For more information browse here: http://portal.battelleforkids.org/Aspire/home.html

10. See Note #4 above.


References


Corcoran, S. P. (2010). Can teachers be evaluated by their students’ test scores? Should they be? The use of value-added measures of teacher effectiveness in policy and practice. Providence, RI: Annenberg Institute for School Reform. Retrieved from http://annenberginstitute.org/publication/can-teachers-be-evaluated-their-students%E2%80%99-test-scores-should-they-be-use-value-added-mea  


Duncan, A. (2009, July 4). The race to the top begins: Remarks by Secretary Arne Duncan. Retrieved from http://www.ed.gov/news/speeches/2009/07/07242009.html


Harris, D. N. (2011). Value-added measures in education: What every educator needs to know. Cambridge, MA: Harvard Education Press.


Houston Federation of Teachers v. Houston Independent School District, (S.D. Tex., Pending, 2016).


Houston Independent School District (HISD). (n.d.a). ASPIRE: Accelerating Student Progress Increasing Results & Expectations: Welcome to the ASPIRE Portal. Retrieved from http://portal.battelleforkids.org/Aspire/home.html


Houston Independent School District (HISD). (n.d.b). ASPIRE: Accelerating Student Progress Increasing Results & Expectations: The next generation of ASPIRE. Retrieved from http://portal.battelleforkids.org/Aspire/aboutaspire


Houston Independent School District (HISD). (2012). HISD Core Initiative 1: An effective teacher in every classroom, teacher appraisal and development system – Year one summary report. Retrieved from http://hisdeffectiveteachers.org/assets/Year_One_Summary_Report_FINAL_PDF.pdf


Houston Independent School District (HISD). (2013). Progress conference briefing.


Houston Independent School District (HISD). (2015a). State of Texas Assessments of Academic Readiness (STAAR) performance, grades 3-8, spring 2015. Retrieved from http://www.houstonisd.org/site/default.aspx?PageType=14&DomainID=8269&PageID=63696&ModuleInstanceID=96404&ViewID=1e008a8a-8e8a-4ca0-9472-a8f4a723a4a7&IsMoreExpandedView=True


Houston Independent School District (HISD). (2015b). State of Texas Assessments of Academic Readiness (STAAR) end-of-course results, spring 2015. Retrieved from http://www.houstonisd.org/Page/69852


Houston Independent School District v. Thymes, Texas Education Agency (2011).


Kappler Hewitt, K. (2015). Educator evaluation policy that incorporates EVAAS value-added measures: Undermined intentions and exacerbated inequities. Education Policy Analysis Archives, 23(76). 1–49. doi:10.14507/epaa.v23.1968 Retrieved from http://epaa.asu.edu/ojs/article/view/1968


Klein, A. (2015). Sec. Duncan in weaker spot on NCLB waiver renewals. Education Week. Retrieved from http://www.edweek.org/ew/articles/2015/02/12/sec-duncan-in-weaker-spot-on-nclb.html


Lowrey, A. (2012). Big study links good teachers to lasting gain. The New York Times. Retrieved from http://www.nytimes.com/2012/01/06/education/big-study-links-good-teachers-to-lasting-gain.html?_r=0


Mellon, E. (2015a). Bonus criteria for HISD chief less specific. The Houston Chronicle. Retrieved from http://www.houstonchronicle.com/news/education/article/Bonus-criteria-for-HISD-chief-less-specific-6644576.php?t=99a0002adb&cmpid=twitter-premium


Mellon, E. (2015b). Split HISD board gives Grier $98,600 in bonuses. The Houston Chronicle. Retrieved from http://www.chron.com/news/education/article/Split-HISD-board-gives-Grier-98-600-in-bonuses-6640782.php?cmpid=twitter-desktop


Mellon, E. (2015c). A Q&A with the HISD board candidates. The Houston Chronicle. Retrieved from http://www.chron.com/news/education/article/A-Q-A-with-the-HISD-board-candidates-6585539.php


No Child Left Behind (NCLB) Act of 2001, Pub. L. No. 107-110, • 115 Stat. 1425. (2002). Retrieved from http://www.ed.gov/legislation/ESEA02/


Race to the Top (RttT) Act of 2011, S. 844--112th Congress. (2011). Retrieved from http://www.govtrack.us/congress/bills/112/s844


SAS Institute Inc. (n.d.). SAS® EVAAS® for K–12: Assess and predict student performance with precision and reliability. Retrieved from www.sas.com/govedu/edu/k12/evaas/index.html


Sawchuck, S. (2015). Teacher evaluation heads to the courts. Education Week. Retrieved from http://www.edweek.org/ew/section/multimedia/teacher-evaluation-heads-to-the-courts.html


Sparks, S. D. (2011). Value-added formulas strain collaboration. Education Week. Retrieved from http://edweek.org/ew/articles/2011/11/16/12collab- changes.h21.html?tkn=OVMFb8PQXxQi4wN6vpelNIr7+NhOFCbi71mI&intc=es


Strauss, V. (2014). Houston teachers sue over controversial teacher evaluation method. The

Washington Post. Retrieved from https://www.washingtonpost.com/news/answer-sheet/wp/2014/04/30/houston-teachers-sue-over-controversial-teacher-evaluation-method/


US Department of Education. (n.d.). ESEA flexibility. Retrieved from

http://www2.ed.gov/policy/elsec/guid/esea-flexibility/index.html


White House. (2015). The Every Student Succeeds Act. Retrieved from

https://www.whitehouse.gov/the-press-office/2015/12/10/white-house-report-every-student-succeeds-act




Cite This Article as: Teachers College Record, Date Published: January 05, 2016
https://www.tcrecord.org ID Number: 18983, Date Accessed: 5/28/2022 10:35:01 AM

Purchase Reprint Rights for this article or review
 
Article Tools
Related Articles

Related Discussion
 
Post a Comment | Read All

About the Author
  • Audrey Amrein-Beardsley
    Arizona State University
    E-mail Author
    AUDREY AMREIN-BEARDSLEY, PhD, is an Associate Professor at Mary Lou Fulton Teachers College at Arizona State University.
  • Clarin Collins
    Arizona State University
    E-mail Author
    CLARIN COLLINS is a research professional at Mary Lou Fulton Teachers College at Arizona State University. Her research interests include national and local policy implementation at the classroom level, teacher influences on policymaking and implementation, and education evaluation and accountability systems. She recently coauthored two related articles, “The SAS Education Value-Added Assessment System (EVAAS) in the Houston Independent School District (HISD): Intended and Unintended Consequences,” with Audrey Amrein-Beardsley (Education Policy Analysis Archives, volume 21, no. 4), and “All Hat and No Cattle: The Value-Added Approach to Educational Reform” with Jessica Hollowey-Libell and Audrey Amrein-Beardsley (Educational Leadership, volume 70, no. 3).
  • Jessica Holloway-Libell
    Kansas State University
    E-mail Author
    JESSICA HOLLOWAY-LIBELL is an assistant professor of Educational Leadership at Kansas State University. Her research interests are in education policies related to teachers and school leaders. Her current research looks at the influence of market-based logics on education policy, specifically as it relates to teacher evaluation policies, practices and instruments, as well as school leadership practice and development.
  • Noelle Paufler
    University of North Texas
    E-mail Author
    NOELLE A. PAUFLER is an assistant professor at the University of North Texas. Her research interests include research methods and educational policy and more specifically, the impact of value-added measures and their related teacher evaluation systems on educators, their professional practices, and the educational system as a whole.
 
Member Center
In Print
This Month's Issue

Submit
EMAIL

Twitter

RSS