Everything is Bigger (and Badder) in Texas: Houston’s Teacher Value-Added System

by Audrey Amrein-Beardsley, Clarin Collins, Jessica Holloway-Libell & Noelle Paufler - January 05, 2016

In this commentary, authors discuss the Houston Independent School District's (HISD) highest-stakes use of its contracted value-added system (i.e., the Education Value-Added Assessment System (EVAAS)) to reform and improve student learning and achievement throughout the district's schools. Authors situate their discussion within a related report on the recent release of Houston Superintendent's own evaluation scores. Authors also situate their discussion within the evidence, as per the recent release of the state's large-scale standardized test scores. Authors assert that, perhaps, attaching high-stakes consequences to teachers' value-added output in Houston is not working as intended.

As the saying goes, everything is bigger and badder in Texas, and nothing could be truer of the states largest public school district, Houston Independent School District (HISD), which serves approximately 215,000 students (75% of whom are socioeconomically disadvantaged). Nationally, HISD ranks seventh in overall size, but its definitely the biggest and baddest in terms of punishing and rewarding teachers based on their students test scores. With no holds barred, the district attaches more high-stakes consequences to teacher-level value-added estimates derived via value-added models (VAMs) than any other state or district in the nation¹ (see also Corcoran, 2010; Harris, 2011; Lowrey, 2012; Sparks, 2011).

Under the command of HISDs current Superintendent Terry Grier, district leaders are using VAM estimates (as per a $500,000 annual contract the district has with SAS Institute Inc. and its Education Value-Added Assessment Systems (EVAAS)) to determine whether teachers are to be put on professional improvement plans, awarded or denied merit pay, granted annual contract renewals, or terminated. But with Mr. Griers recent announcement of his upcoming retirement, we also recently learned that, apparently, those same practices are not even fitting for him as the districts leader. Following his retirement notice, details surrounding his most recent evaluation were released, along with details describing the school boards split vote (i.e., 4-3, with two members absent) in favor of not only his satisfactory rating, but also his $98,600 merit performance bonus for his value-added last year (this bonus was to be paid atop his $300,000 yearly salary, plus benefits including $19,200 in car and technology allowances and pay for unused leave time).

Illustrated in the Houston Chronicle (Mellon, 2015a, 2015b, 2015c) were the criteria the school board used to determine Mr. Griers bonus. Not surprisingly, it was primarily based on annual district-level growth ($53,600) and other job-related criteria² ($45,000); however, and surprisingly, indicators of Griers annual district-level growth were only partially measured using the districts VAM data (i.e., 2/18 or 11% of the growth measures included in his evaluation were actually derived via the districts VAM). What remains uncertain is why these data mattered substantively less for Mr. Grier than for his teachers, for whom value-added (or value-detracted) can count for up to 50% of their total teacher evaluation scores, or effectively 100% of their total teacher evaluation scores when growth is used to trump other district measures of teacher effectiveness (e.g., measures of teachers instructional practice).³

In addition, during Mr. Griers six years as superintendent of HISD, his bonuses have demonstrably grown larger over time, despite his now questionable value-added as the districts superintendent. Take a look, for example, at the two figures below (see Figures 1 and 2), taken directly from HISDs recently released reports (2015) highlighting district students performance (declining, on average, in blue) as compared to the state of Texas (maintaining, on average, in black), as per the states newest (launched in 2012) set of grade 38 and high school State of Texas Assessments of Academic Readiness (STAAR) tests.⁴

 

Figure 1. Houston (blue trend line) v. Texas (black trend line) performance on the STAAR tests, 2012-2015 (HISD, 2015a, p. 19)

 

Figure 2. Houston (blue trend line) v. Texas (black trend line) performance on the states STAAR End-of-Course (EOC) tests, 2012-2015 (HISD, 2015b, p. 17)

When considering these data, keep in mind that Mr. Grier and his district leaders have fired numerous teachers based, in large part, on their students value-added scores. For example, in 2011 alone, 221 district teachers were terminated due to demonstrating a significant lack of student progress attributable to the educator or insufficient student academic growth as reflected by [teachers] value-added scores.⁵ These actions resulted in a series of wrongful termination hearings,⁶ and they ultimately resulted in the filing of an April 2014 lawsuit by the Houston Federation of Teachers (HFT), with the support of the American Federation of Teachers (AFT), on behalf of teacher plaintiffs.⁷ Plaintiffs charge that district leaders are inappropriately attaching the aforementioned high-stakes consequences to their VAM data, given that their VAM data are allegedly unstable, inaccurate, biased, unfair, incomprehensible, and inactionable (Sawchuck, 2015; Strauss, 2014); hence, plaintiffs also charge, high-stakes consequences cannot and should not be attached to their value-added output. With this impending lawsuit, Mr. Grier and his district leaders recoiled at some of the consequences attached to teachers VAM output (e.g., teacher terminations), though it will still be heard in court in June of 2016.

For years, now, Mr. Grier has deemed it appropriate to make such extreme decisions using VAM data for teachers throughout his district. Should we not ask whether he deserves similar treatment, and much less a satisfactory evaluation and an accompanied $98,600 merit performance bonus given the districts growth, or lack thereof, as compared to the state? Would it not seem fitting to hold superintendents such as Grier, who have championed the use of VAMs to both evaluate and make highly consequential decisions about their teachers, accountable using the same logic, as well as a similar set of accountability measures?

Let us also ponder the specific model used by HISDthe aforementioned EVAASand SAS Institute Inc.s perpetual claims that this model helps teachers become more proactive [while] making sound instructional choices; helps teachers use resources more strategically to ensure that every student has the chance to succeed; or provides valuable diagnostic information about [teachers instructional] practices so as to ultimately improve student learning and achievement (SAS Institute Inc., n.d.; see also Kappler Hewitt, 2015).

Perhaps the districts EVAAS system is not as much of an educational-improvement and performance-management model that engages all employees in creating a culture of excellence (HISD, n.d.a) that is yielding outstanding [student-level] results (HISD, n.d.b) as is being asserted.

If not (see, again, Figures 1 and 2 above), should anyone really be held accountable using such a system that also fails to meet or evidence its own fundamental claims?

Not that we necessarily want to generalize from this one case beyond Houstons borders, but when the rooster crows, it often means the sun is arisin. Even the simplest evidence presented above should at the very least make us question this particular value-added system, as paid for,⁸ supported,⁹ and applied in Houston for some of the biggest and baddest teacher-level consequences in town. Whether all of this occurring in Houston under the leadership of Mr. Grier has also helped to reform or improve Houstons schools is also certainly suspect given the evidence.¹⁰

Notes

1. Attaching high-stakes consequences to teachers students test scores, prior to the passage of the Every Student Succeeds Act (ESSA; White House, 2015), has been largely incentivized throughout the nation via the federal Race to the Top (RttT) competition (2011; see also Duncan 2009) and the federal waivers excusing states from not meeting No Child Left Behinds (NCLB, 2001) 100% proficiency targets by 2014 (US Department of Education, n.d.; see also Klein, 2015). However, HISD has been doing this well before RttP incentivized such activities, although it is uncertain whether HISD will continue these practices, now, post-ESSA.

2. Other evaluative criteria included indicators of (a) college preparedness, (b) Mr. Griers advance placement initiative, (c) his power up laptop program, (d) his literacy by 3rd grade program, (e) his effective teacher initiative, (f) indicators of school safety and environment, (g) the financial well-being of the district, and (h) his communications with the school board. Of these eight criteria, onethe (5) effective teacher initiativeuses EVAAS, among other criteria, to assess teachers.

3. Many principals throughout HISD have described the district-level pressures placed on them to ensure or guarantee specific evaluation outcomes, whereby value-added scores seem to be trumping other indicators capturing what it means to be an effective teacher (i.e., artificial conflation). In HISD, this is also part of district policy, as they noted in 2012 in the following: While there is a meaningful, positive relationship between Instructional Practice [IP] ratings and school level EVAAS, many schools are not aligning their assessment of classroom practice with student [EVAAS] growth. Individual teacher ratings and student growth are also correlated, though there is significant room for improvement in aligning teacher IP scores to student [EVAAS] outcomes. Instructional practice [IP] scores show that there is room for improvement for schools to be more consistent in their ratings (HISD, 2012, p. 3). One year later, in 2013, the district set a district wide goal for what they termed rating accuracy for all schools to align their teachers IP ratings to student performance, again, as measured via the EVAAS. The explicit goal was to have at least 85% of [HISD] schools meet the minimum bar for alignment between Instructional Practice [IP] and Student Performance [EVAAS] (HISD, 2013, p. 3, 11).

4. Note that these figures illustrate growth-by-cohort over time, whereby third graders are compared to different sets of third graders subsequent years. These figures do not demonstrate EVAAS growth, although for this years evaluation Mr. Grier demonstrated added value in district wide mathematics (+0.2) and decreased value in district wide reading (-0.3).

5. Houston Independent School District (Plaintiff) v. Mia Thymes, Hercilia Leon, Mary McWilliams, and Glenda Drew (Defendants); Before Robert Barringer, Franklin Holcomb, Allecia Pottinger, and Audrei Lawton, Certified Hearing Examiners, Texas Education Agency (2011).

6. Ibid.

7. Houston Federation of Teachers, Local 2415, Daniel Santos, Paloma Garner, Ivan Castillo, Andy Dewey, Joyce Helfman, Myla Van Duyn, and Araceli Ramos (Plaintiffs) v. Houston Independent School District (Defendant); United States District Court, Southern District of Texas, Houston Division (Pending, 2016).

8. Again, HISD pays $500,000 annually to SAS Institute Inc. for its EVAAS services.

9. Supported by Battelle for Kids (which is united with SAS Institute Inc.), HISDs ASPIRE system (launched in 2007−2008) is the district’s “educational-improvement and performance-management model” meant to engage all district employees “in creating a culture of excellence.” It is through this system that all HISD employees are to access information about the districts value-added reports, as well as related training opportunities, reports, and other resources to inform, in particular, teachers practice. For more information browse here: http://portal.battelleforkids.org/Aspire/home.html

10. See Note #4 above.

References

Corcoran, S. P. (2010). Can teachers be evaluated by their students test scores? Should they be? The use of value-added measures of teacher effectiveness in policy and practice. Providence, RI: Annenberg Institute for School Reform. Retrieved from http://annenberginstitute.org/publication/can-teachers-be-evaluated-their-students%E2%80%99-test-scores-should-they-be-use-value-added-mea  

Duncan, A. (2009, July 4). The race to the top begins: Remarks by Secretary Arne Duncan. Retrieved from http://www.ed.gov/news/speeches/2009/07/07242009.html

Harris, D. N. (2011). Value-added measures in education: What every educator needs to know. Cambridge, MA: Harvard Education Press.

Houston Federation of Teachers v. Houston Independent School District, (S.D. Tex., Pending, 2016).

Houston Independent School District (HISD). (n.d.a). ASPIRE: Accelerating Student Progress Increasing Results & Expectations: Welcome to the ASPIRE Portal. Retrieved from http://portal.battelleforkids.org/Aspire/home.html

Houston Independent School District (HISD). (n.d.b). ASPIRE: Accelerating Student Progress Increasing Results & Expectations: The next generation of ASPIRE. Retrieved from http://portal.battelleforkids.org/Aspire/aboutaspire

Houston Independent School District (HISD). (2012). HISD Core Initiative 1: An effective teacher in every classroom, teacher appraisal and development system Year one summary report. Retrieved from http://hisdeffectiveteachers.org/assets/Year_One_Summary_Report_FINAL_PDF.pdf

Houston Independent School District (HISD). (2013). Progress conference briefing.

Houston Independent School District (HISD). (2015a). State of Texas Assessments of Academic Readiness (STAAR) performance, grades 3-8, spring 2015. Retrieved from http://www.houstonisd.org/site/default.aspx?PageType=14&DomainID=8269&PageID=63696&ModuleInstanceID=96404&ViewID=1e008a8a-8e8a-4ca0-9472-a8f4a723a4a7&IsMoreExpandedView=True

Houston Independent School District (HISD). (2015b). State of Texas Assessments of Academic Readiness (STAAR) end-of-course results, spring 2015. Retrieved from http://www.houstonisd.org/Page/69852

Houston Independent School District v. Thymes, Texas Education Agency (2011).

Kappler Hewitt, K. (2015). Educator evaluation policy that incorporates EVAAS value-added measures: Undermined intentions and exacerbated inequities. Education Policy Analysis Archives, 23(76). 149. doi:10.14507/epaa.v23.1968 Retrieved from http://epaa.asu.edu/ojs/article/view/1968

Klein, A. (2015). Sec. Duncan in weaker spot on NCLB waiver renewals. Education Week. Retrieved from http://www.edweek.org/ew/articles/2015/02/12/sec-duncan-in-weaker-spot-on-nclb.html

Lowrey, A. (2012). Big study links good teachers to lasting gain. The New York Times. Retrieved from http://www.nytimes.com/2012/01/06/education/big-study-links-good-teachers-to-lasting-gain.html?_r=0

Mellon, E. (2015a). Bonus criteria for HISD chief less specific. The Houston Chronicle. Retrieved from http://www.houstonchronicle.com/news/education/article/Bonus-criteria-for-HISD-chief-less-specific-6644576.php?t=99a0002adb&cmpid=twitter-premium

Mellon, E. (2015b). Split HISD board gives Grier $98,600 in bonuses. The Houston Chronicle. Retrieved from http://www.chron.com/news/education/article/Split-HISD-board-gives-Grier-98-600-in-bonuses-6640782.php?cmpid=twitter-desktop

Mellon, E. (2015c). A Q&A with the HISD board candidates. The Houston Chronicle. Retrieved from http://www.chron.com/news/education/article/A-Q-A-with-the-HISD-board-candidates-6585539.php

No Child Left Behind (NCLB) Act of 2001, Pub. L. No. 107-110, " 115 Stat. 1425. (2002). Retrieved from http://www.ed.gov/legislation/ESEA02/

Race to the Top (RttT) Act of 2011, S. 844--112th Congress. (2011). Retrieved from http://www.govtrack.us/congress/bills/112/s844

SAS Institute Inc. (n.d.). SAS® EVAAS® for K12: Assess and predict student performance with precision and reliability. Retrieved from www.sas.com/govedu/edu/k12/evaas/index.html

Sawchuck, S. (2015). Teacher evaluation heads to the courts. Education Week. Retrieved from http://www.edweek.org/ew/section/multimedia/teacher-evaluation-heads-to-the-courts.html

Sparks, S. D. (2011). Value-added formulas strain collaboration. Education Week. Retrieved from http://edweek.org/ew/articles/2011/11/16/12collab- changes.h21.html?tkn=OVMFb8PQXxQi4wN6vpelNIr7+NhOFCbi71mI&intc=es

Strauss, V. (2014). Houston teachers sue over controversial teacher evaluation method. The

Washington Post. Retrieved from https://www.washingtonpost.com/news/answer-sheet/wp/2014/04/30/houston-teachers-sue-over-controversial-teacher-evaluation-method/

US Department of Education. (n.d.). ESEA flexibility. Retrieved from

http://www2.ed.gov/policy/elsec/guid/esea-flexibility/index.html

White House. (2015). The Every Student Succeeds Act. Retrieved from

https://www.whitehouse.gov/the-press-office/2015/12/10/white-house-report-every-student-succeeds-act

Cite This Article as: Teachers College Record, Date Published: January 05, 2016 https://www.tcrecord.org ID Number: 18983, Date Accessed: 2/8/2022 2:39:30 PM Purchase Reprint Rights for this article or review