Understanding Teacher Pay for Performance: Flawed Assumptions and Disappointing Results

by Karen Phelan Kozlowski & Douglas Lee Lauen - 2019

Background: Policy makers are increasingly adopting performance incentives to spur underperforming teachers as a way to improve teaching and student performance. However, much of the experimental research fails to find meaningful effects of performance incentives on either student achievement or teacher practice.

Purpose/Objective: Using the “principal–agent problem” as the theoretical motivation for the study, this research examines why performance incentives have not worked in American schools. The principal–agent problem suggests that in the absence of a perfect system to monitor agents, (e.g., teachers), there must be an incentive based on some measurable outcome to ensure maximal effort. The underlying assumptions about why performance incentives should work for teachers are that (1) teachers are primarily motivated by money, (2) teachers are not currently working hard enough, and (3) teachers know how to be more effective but are choosing not to put forth the necessary effort to do so. The purpose of this research is to examine whether these assumptions hold for teachers.

Research Design: We conducted qualitative analysis of interviews and focus groups with approximately 150 teachers and 20 administrators from 13 of the lowest performing school districts in North Carolina to understand how educators perceived performance incentives in the context of their own practice.

Findings: Three key themes emerged from our study. First, teachers report being motivated by service to their students instead of opportunities to maximize income. Second, teachers think they are already working as hard as they can and find little room in their practice to work harder, whatever the financial reward. Third, when teachers do improve their practice, it comes from opportunities to learn new strategies and techniques.

Conclusions: The empirical research presented in this paper suggests that performance incentive programs rest on a set of flawed theoretical assumptions. Performance incentives assume that teachers (1) are primarily motivated by financial rewards, (2) are not working as hard as they can, and (3) know how to be more effective. However, these assumptions do not comport with what teachers and administrators report about their motivation and practice. Therefore, performance incentives will likely do little to improve teacher effectiveness overall.

In an unequal society, equal access to educational opportunity is a particularly important goal. When the Coleman Report was written 50 years ago, policy makers focused on the importance of providing students equal access to quality schools. Today, the focus has shifted to providing equal access to quality teachers. With options to reassign teachers to students limited by seniority rights and collective bargaining agreements, teacher performance pay has become an increasingly attractive policy approach to increase student access to quality teaching. Teacher performance pay programs are spreading rapidly throughout the country because of the federal Teacher Incentive Fund (TIF) and Race to the Top funding, Gates Foundation funding, and changes in state law. It is therefore critical to understand the theory of action behind such programs. Currently, approximately 3.5% of districts use some form of merit or performance pay (Buck & Greene, 2011), but polls suggest that more than half of the public supports performance pay policies for teachers (Henderson, Peterson, & West, 2016).

Many policy makers suggest that teachers ineffectiveness stems from employment practices that do not sufficiently incentivize teachers to do good work. For example, the single salary schedule sets pay by years of experience and degree rather than by student test score growth or performance as judged by a lead teacher or administrator. Across the country, states and districts have attempted to rectify this by adopting models of performance pay, which reward teachers with extra compensationranging from as little as a few hundred dollars to as much as $15,000 or morewhen their students meet particular performance or growth targets.

Unfortunately, there has been little evidence that performance pay works in the United States. Experimental studies conducted in several states over the last decade have shown few to no effects of performance pay on either teacher behavior or student outcomes (Fryer, 2013; Springer et al., 2010; Yuan et al., 2013). Therefore, it is worth considering why performance pay does not seem to work in American schools (Rice & Malen, 2017). We argue that performance incentives are ineffective in American education because they rest on a set of flawed assumptions: that teachers (1) are primarily motivated by financial rewards, (2) are not working as hard as they can, and (3) know how to be more effective. We posit that performance pay systems have not worked in American schools because, as our research shows, teachers conceive of their jobs as callings rather than an opportunity to maximize income, they think they are already working as hard as they can, and when improvements in practice do occur, they come from opportunities to learn new strategies and techniques rather than a desire for bonus pay.

BACKGROUND AND RELEVANCE

Performance pay is not a new concept. Legislators have implemented small-scale performance pay programs for teachers since the early 20th century in attempts to reward particularly effective teachers, but the difficulty of measuring teacher effectiveness prevented widespread adoption of performance pay for teachers (Taylor, Hunnicutt, & Keeffe, 1991). In recent years, though, interest in performance pay has resurfaced, in part because of efforts to increase student, teacher, and school accountability. In 1983, A Nation at Risk argued that American students were falling behind the students of other industrialized nations. This suggested to policy makers that American education must be reformed, otherwise comparative underachievement may result in lost innovation, jobs, and, ultimately, dollars for the American economy. One of the ways that policy makers believed schools could be reformed was by increasing student, teacher, and school accountability through high-stakes testing. This was the foundation of No Child Left Behind (NCLB), which required all schools to make annual yearly progress based in part on student standardized test scores. NCLB significantly increased the number of students being tested each year and significantly increased the number of times those students were tested throughout their educational careers. One of the legacies of NCLB has been the persistence of standardized testing for students across the country, from elementary to high school.

As a result, many states have developed massive stores of student achievement data, and when those data are linked to particular teachers, new measures of teacher effectiveness can and have been developed. One such measure is called teacher value-added, which predicts how much a teacher adds to student learning based on students past achievement trajectories (Hanushek, 1992; Hanushek & Rivkin 2004). An early study examining the effects of teachers suggests that highly effective teachers produce a years worth of growth for students than the least effective teachers (Hanushek, 1992). More recent studies have found that a one-standard-deviation increase in teacher value-added is associated with significant student test score increases: .04 to .26 standard deviation increases in reading or English and .10 to .36 standard deviation increases in math (Chetty, Friedman, & Rockoff, 2014a; Nye, Konstantopoulos, & Hedges, 2004; Rockoff, 2004). In addition, teachers with high value-added scores tend to produce students who are more likely to attend college and to earn more, and less likely to have a teen birth (Chetty et al., 2014b). Because of studies like these, not only has it become clear to policy makers that effective teaching matters for student achievement, but it has also become possible to more clearly differentiate between, and hence differentially reward, effective teachers through performance pay on at least one measure of teacher effectiveness.¹

Policy makers have been optimistic about the adoption of performance pay in the United States because in some countries, performance pay has had significant positive effects on schooling outcomes for students. Lavy (2009), for example, found that in Israel, cash bonuses for student improvement led to increases in test-taking rates, conditional pass rates, and test scoresoutcomes that he argued were mediated through changes in teaching methods and practices. Similar results have been found in developing countries, including Kenya (Glewwe, Ilias, & Kremer, 2010) and India (Duflo, Hanna, & Ryan, 2012). In addition, a cross-national study using cross-sectional PISA data reported that performance pay structures are significantly (.25 SD higher) associated with math, science, and reading achievement (Woessmann, 2011).

But these promising results have not been replicated in recent studies set in American schools.² Experimental studies on traditional models of performance pay (i.e., systems in which the opportunity to earn a bonus is offered at the beginning of the year and rewarded at the end of the year if a teacher meets the performance criteria) in American elementary, middle, and high schools suggest that merit pay does not affector, at best, minimally affectsteacher practices and student outcomes (Chiang et al., 2015; Fryer, 2013; Gius, 2012; Glazerman & Seifullah, 2012; Springer et al., 2010, 2012; Yuan et al., 2013).

For example, a randomized experiment of a school-based pay-for-performance program in New York City elementary, middle, and high schools found no effect of incentives on students performance, attendance, high school graduation, or behavior, nor did it find evidence that incentives change teacher retention, absences, or reported behavior (Fryer, 2013). Springer et al. (2010) reported that Nashvilles three-year randomized experimental merit pay program of $5,000$15,000 individual teacher bonuses for Grades 58 had virtually no effect on teachers classroom practices or student achievement.

One recent study of 10 school districts (132 total schools) showed some positive effects of performance pay (Chiang et al., 2015). The National Center for Educational Evaluation report on the 20122013 TIF implementation showed small positive impacts of $1,700$1,800 bonuses on reading. Reading effects were quite small, though, increasing student performance only one percentile. Results were not significant for math, and student growth effects only occurred at the school level, not the classroom level. Despite the small positive impacts of performance pay on student reading, performance pay had no positive impacts on other aspects of teacher effectiveness or job satisfaction. Observations of teacher practice did not change, and more effective teachers were not recruited or retained as a result of performance pay (Chiang et al., 2015). In other words, the mechanisms that supposedly explain why performance pay should work showed no effects in the study. Finally, the report found that in the first year of the program, only half of the teachers in treatment schools realized they were eligible for a bonus, and that percentage only improved to 62% in the following year (Chiang et al., 2015). If so many teachers were unaware of the bonus or its terms, it seems that any positive increase in student achievement cannot be attributed to pay for performance alone.

It is unlikely that limited or null effects of performance incentives could be attributed to characteristics of the bonus itself. Whether bonuses are team based or individual, traditional performance incentives do not seem to have an impact on teacher effectiveness (Fryer, Levitt, List, & Sadoff, 2012). Additionally, it is unlikely that these performance incentives offered too little incentive to make a difference in teaching because the amount of money available to teachers in many of these experimental studies was quite substantialup to $15,000 per individual teacher in Nashville, and up to $6,894 for teachers in some of the TIF experiment districts.³

PERFORMANCE PAY AND THE PRINCIPALAGENT PROBLEM

 

Why have performance pay programs not worked in American schools? To understand this better, it must be noted that the roots of performance pay lie in what economists call the principalagent problem (Alchian & Demsetz, 1972; Holmstrom & Tirole, 1989; Murnane & Cohen, 1986; Sappington, 1991; Williamson, 1975). Here, principals refers to employers or others who have contracted with an agent to produce a service or good (Sappington, 1991). The principalagent problem arises when the interests of agents (e.g., teachers) diverge from those of the principal (e.g., a school principal, superintendent, or policy maker). More specifically, principals seek to maximize the output of their agents, whereas agents tend to act in their own self-interest, preferring to work less hard than their principals might want them to (Murnane & Cohen, 1986).

One way to encourage agents to maximize their effort is by monitoring them, given that the watchful eye of an employer may incentivize agents to work as hard as possible. However, it is impossible, or at least very costly, for principals to monitor the output and effort of all agents at all times (Murnane & Cohen, 1986), so principals cannot exert complete control over their agents. This is particularly true in education because teachers are agents with a great deal of autonomy, and their activities are difficult for principals and policy makers to monitor (Levačić, 2009). Because teachers, like other agents, are viewed as self-interested actors who value their leisure time and whose day-to-day performance is hard to monitor, they have an incentive to shirk in the performance of what the principals conceive of as their dutiesto put forth more effort toward raising student achievement test scores.

In the absence of a perfect monitoring system, principals must establish a principalagent contract on the basis of a performance outcome (Eisenhardt,1989), or, in other words, incentivize a performance outcome to ensure maximal work effort. Examples of incentives include piece-rates, commissions, bonuses, and performance measurement. In fields like sales, it is relatively straightforward to design a commission or bonus scheme because bonuses can be based on a percentage of the sale price, for example. In fields like education, bonuses schemes are fraught with complexity, because there are many goals of education (e.g., reading skills, creativity, ability to work on teams), there are many factors outside of teachers control that contribute to student achievement (e.g., parents, intrinsic student ability), and the nature of effective teaching practice is much harder to define than in other industries (Murnane & Cohen, 1986). However, merit pay programs are evidence that policy makers are willing to forgo this complexity to better align teachers interests with their own. In short, teachers are rewarded in bonus pay for meeting measurable objectives in student achievement. So, the solution of using bonus pay based on student test score achievement emerged from a policy problem: the perceived difficulty of monitoring and motivating workers in a field with very few managers and many teachers working in isolation. With some understanding of the origins of the policy problem, we now turn to the core argument of our article: the assumptions underlying bonus schemes.

We posit that policy makers, frustrated by the lack of educational progress in schools, and informed by an overly simplistic model of what motivates instructional changes in teacher practice, have developed performance pay programs as a solution to the principal agent problem: Rather than tighten monitoring by school principals and others, simply provide bonuses for teacher value-added. However, policy makers response to the principalagent problem are motivated by a set of assumptions about teaching that we believe are flawed:

1.

Teachers are primarily motivated by money.

2.

Teachers do not work as hard as they can.

3.

Teachers have the necessary knowledge to be most effective.

These assumptions are implicit and rarely, if ever, communicated directly by policy makers, so we are left to infer the assumptions on which merit pay programs are based. The principalagent problem suggests that, in the absence of a perfect monitoring system, agents must be incentivized with external rewards to ensure maximal effort. We therefore argue that policy makers implicitly assume that teachers must be motivated by money because the rewards are financial. As to the second assumption, the principalagent problem suggests that agents will shirk their effort in the absence of supervision or an external reward for output. Therefore, we infer that policy makers must believe teachers are not currently working hard, because the extra performance incentive would theoretically reward extra effort to meet principals demands.  Finally, the principalagent problem suggests that agents knowingly shirk their effort and know how to increase their output to principals specifications. Because we believe that policy makers strive for fairness, we therefore believe that policy makers trust that teachers have the knowledge required of them to teach effectively. If that were not the case, then policy makers would have created an unfair game to incentivize higher work effort, which we do not find very likely. As our empirical results will show, these assumptions do not comport well with what teachers say motivates them and helps them perform well on the job.  

To set the stage for our analysis, we now turn to the literature base on the nature of teaching to understand why the principalagent model may not apply to the teaching occupation. The first assumption suggests that teachers are primarily motivated by money, but the literature on the nature of the teaching suggests what most people understand intuitivelyteachers are not like car salesmen and hedge fund traders; they go into teaching with different motivations and expect different things in return. These differences may account for why performance pay has not mattered in increasing either teacher effectiveness or student achievement. According to a study by Farkas, Johnson, and Foleno (2000), an overwhelming majority (72%97%, depending on the indicator) of teachers with five or fewer years of experience reported that it was important for them not only to love their work and believe that it contributed to bettering society and others, but also to believe that teaching had those characteristics of their ideal job. This is consistent with past research on teaching. A review by Brookhart and Freeman (1992) noted that of the approximately 15 studies conducted on teachers motivation between the late 1970s and early 1990s, the common theme is that an overwhelming majority of teachers enter the profession for altruistic and service-oriented reasons. According to a detailed factor analysis, intrinsic value for teaching (i.e., always wanting to be a teacher), social utility (working with children, enhancing social equity, making a social contribution), and perceived ability to teach are the highest rated motivations for those entering the teaching profession, whereas teaching as a fallback career and the personal utility of teaching (i.e., job security, holidays off, time with family, transferability) were negatively or not correlated with teacher motivation (Watt & Richardson, 2007). These results echo those of Schipp (1999), who found that African American college students planning on entering the teaching profession value their contribution to society more than students not planning on entering the teaching profession, whereas nonteaching students were more concerned with salary, job security, prestige, and advancement opportunities. What these findings suggest is that the intrinsic motivation that teachers have for choosing and exceling in their profession is inconsistent with a performance pay structure that extrinsically rewards teachers with extra money. In fact, it may be antithetical. Incentivizing teachers with more pay may corrupt the intrinsic motivations of teachers to serve the needs of children, thereby lowering their motivation overall.

That performance pay incentivizes teachers on an easily measurable outcome, student test scores, may also not comport with what teachers view as an appropriate outcome for measuring their job performance. Murnane and Cohen (1986) noted that, in addition to teaching academic curriculum, schools are also supposed to teach students other parts of what Gracey (1975) called the hidden curriculum, like how to be good democratic citizens, how to realize their potential, and how to eliminate destructive behavior like drugs and alcohol. It is hard to measure and subsequently evaluate teachers individual contributions and progress on some of these other goals. Further, if teachers contributions to these outcomes are not valued as much as their contributions to student achievement, their intrinsic motivation to develop these skills may decrease from resentment.

The second assumption suggests that teachers are not working as hard as they canthat there is room to shirk some of their teaching responsibility. Aside from the occasional administrator observation as part of the teacher evaluation process, teaching is an isolated practice conducted behind closed doors and therefore are not subject to watchful monitoring at all times. It may be easy to assume that without the watchful eye of a supervisor, shirking is bound to happen. Yet even without consistent monitoring, there is evidence that teaching does not permit much shirking, especially in terms of time. According to a survey of 10,000 teachers by the Bill & Melinda Gates Foundation (Scholastic, Inc. & the Bill & Melinda Gates Foundation, 2012), teachers work an average of 10 hours and 40 minutes a day, which accounts for time in the classroom, time mentoring students, lesson planning, and grading outside regular school hours. Teachers who supervise extracurricular organizations or coach athletic teams spend an additional 90 minutes working beyond the 10 hours and 40 minutes (Scholastic, Inc. & the Bill & Melinda Gates Foundation, 2012). Data from the American Time Use Survey suggest that teachers work 38 hours per week during the year and 21.5 hours per week during the summer, though it is difficult to know from the data if those hours include unpaid hours or if summer hours in particular are dedicated to teaching or to a second job (West, 2014). If teachers are already exerting maximal effort, at least in terms of hours put into their work, then performance incentives intended to maximize effort may not be particularly effective for teachers, despite the assumption that effort is what separates the most effective from the least effective teachers.

This is not to say that all teachers know how to teach well or use all this time wisely. As in any occupation, skill among teachers varies. Principalagent theory is based on the exchange of agents effort for a contracted product or service but says nothing of agents ability to create that product or service. Performance incentives inherently reflect this thinking, because they assume (see Assumption 3) that teachers already know how to be effective. However, research suggests that knowledge of how to be effective is not something that all teachers have. For example, according to Rivkin, Hanushek, and Kain (2005), it takes about three years to become an effective teacher, and the experience gained from zero years of experience to the end of the first year matters the most for student achievement (Boyd, Lankford, Loeb, Rockoff, & Wyckoff 2008). This is significant for a profession that has in recent years (since 2007) seen massive increases in novice teachers who do not bring the expertise required to be effective (Ingersoll, Merrill, & Stuckey, 2014). Viewed in this light, performance incentives may not be particularly useful, because if these new and inexperienced teachers do not know how to be effective, then increasing their effort will likely have minimal impact on students or their performance.

Using interview data from approximately 150 teachers in North Carolina, we argue that these three assumptions about performance pay and the teaching profession are flawed. First, we find that performance incentive structures are inconsistent with teachers and administrators primary motivation and best practices for teaching. Specifically, teachers say they are not primarily motivated by the opportunity to earn extra money, but rather an intrinsic desire to help students. Additionally, teachers report that collaboration, which they think serves the best interests of the students, is actually undermined by performance incentives, especially at the individual level. Second, we find that teachers believe that they and their colleagues are working as hard as they can. Last, we find that when teachers do improve their practice, it results not from working harder, but working smarter. That is, teachers report becoming better practitioners, or increasing their teaching ability, when they have opportunities to learn new strategies and techniques through regular training, teaching mentors, and professional development.

NORTH CAROLINA RACE TO THE TOP PERFORMANCE INCENTIVE

To understand why performance pay has not had the desired impacts on teacher practice and student achievement, we interviewed teachers and administrators in schools that were eligible for the Race to the Top (RttT) performance incentive in North Carolina. The RttT performance incentive was designed to increase the performance of North Carolinas lowest performing schools. Therefore, the schools in North Carolina that were eligible for the incentive included the lowest 5% of elementary, middle, and high schools (118 total schools), as determined by their performance composites (percent of students at grade level according to standardized test scores) and graduation rates.  

The bonus itself was distributed from 2011 through 2014. For the first two years of the grant period, the bonus was a $1,500 schoolwide bonus made available to all certified staff if a school met high growth,⁴ as defined by the North Carolina school accountability model. In 2013, the structure of the incentive changed somewhat to reflect the change in North Carolinas accountability model. Though the schoolwide bonus was still available, teachers of tested grades and subjects were eligible for an additional $500 bonus on top of the $1,500 schoolwide bonus if their classrooms exceeded expected growth. Even if teachers of tested subject areas taught in a school that did not make the $1,500 school-level bonus, these teachers were still awarded $2,000 for exceeding individual-level expected growth.⁵

It is important to note that North Carolina is a state with particularly low teacher salaries, ranking near the bottom of all states in average teacher pay, at about $45,000 (National Education Association, 2015). These bonuses, therefore, represent between 3% and 4% of base pay. Although these bonuses are certainly not as high as those tested in some districts (in Nashville, bonuses were as high as $15,000), they are consistent with the size of bonuses tested in New York ($1,500$3,000) and the average bonus in TIF evaluation districts ($1,700$1,800) (Chiang et al., 2015; Fryer, 2013; Springer et al., 2010). Although there is some evidence that larger bonuses matter for teachers to buy in to performance incentive structures (Rice & Malen, 2017), even the highest bonuses have produced minimal to no impact on student achievement and teacher performance (Springer et al., 2010). This suggests that teachers and administrators perceptions of the North Carolina bonuses will offer meaningful insight into why performance incentives fail to produce significant gains in student achievement or changes to teacher practice, more generally.

Our data cover the first three years of the RttT performance incentive. In December 2011, 23 schools earned schoolwide bonuses based on growth attained during the 20102011 school year, and the following December, 35 schools earned schoolwide bonuses for growth during the 20112012 school year. There was not a high degree of overlap among winners between 2011 and 2012, with only eight of the 23 winners from 2011 earning a bonus again in 2012. In December 2013, $1,500 schoolwide bonuses were awarded to certified staff in 35 schools. Twelve of the 35 bonus-winning schools in 2013 had received a bonus in one of the two prior years, and three of the 35 bonus-winning schools received the bonus in all three years. Also in 2013, a total of 232 individual teachers earned the individual classroom bonus for exceeding expected growth. Approximately 48% of those teachers taught in a school that earned the schoolwide bonus, and 52% of them taught in a school that did not earn the schoolwide bonus.

METHODS

This study is part of a larger qualitative evaluation of the RttT performance incentive for the state of North Carolina, which examined teachers awareness of the program; perception of program changes; and their perception of the performance incentives role in motivation, practice, continued commitment to the profession, and student achievement. Because this study had the potential to produce nuanced, complex, and unforeseen responses, qualitative interviews and focus groups were necessary to understand how and why teachers perceived performance incentives in particular ways (Lofland, Snow, Anderson, & Lofland, 2006).

We draw on data from semistructured focus groups and interviews with approximately 150 teachers and 20 administrators in 13 districts of North Carolina. Members of the RttT evaluation team conducted interviews between May 2012 and January 2014. All our study participants taught in one of the 118 bonus-eligible schools.

Of the 150 teachers interviewed, approximately 33 were interviewed individually, and 117 were interviewed in focus groups. Although we would have ideally liked to interview more teachers separately to increase the likelihood of teachers responding openly and candidly (Hatch, 2002), we elected to conduct some of the interviews as focus groups in order to maximize the number of teachers we could speak with at each school. To increase the likelihood that teachers in focus groups shared their opinions honestly and to ensure that all participants in the group were eligible (or not eligible) for the same bonuses, we conducted focus groups of tested and untested teachers separately. It is possible that teachers in focus groups, in front of their colleagues, felt pressure to limit their responses to those consistent with the norms of the teaching profession; however, we found very few differences in responses between teachers who were interviewed individually and teachers who were interviewed in focus groups. This suggests that the results do not simply reflect teaching norms or norms management, but teachers actual perspectives on the role of performance incentives in their practice. Further, teachers had an interest in sharing their honest opinions with us because we informed them that we would be presenting their (anonymous) perceptions of the program to the North Carolina Department of Public Instruction in order to inform future policy on the issue.

SAMPLE

Interviews were conducted over three separate time periods. The first set of interviews occurred between May and June 2012, six months after bonus winners had received the first RttT bonus, which had been awarded for meeting high growth in the 20102011 academic year. Of the 118 eligible schools, we randomly selected 24 to participate in the initial assessment of the program, and 12 schools agreed to participate. Six of those schools had won the bonus, and six of them had not. Seven were elementary schools, two were middle schools, and three were high schools (see Table 1).

The next year, we conducted a follow-up assessment of the program and selected 20 of the bonus-eligible schools to participate in follow-up interviews. Half of the schools in our selected sample were schools where we had interviewed in the first year of the program evaluation, and half of the sample included schools where we had not previously interviewed. Willingness to participate in the follow-up evaluation was lower than it had been in the first year, resulting in a total Year 2 sample of seven schoolsthree elementary, one middle, and three high schools. Two of the schools we interviewed had won the bonus twice, one had never won the bonus, one won the bonus in the first year but not the second, and two won the bonus in the second but not the first year. Additionally, four of the schools we interviewed in Year 2 were schools that we had interviewed in Year 1.

Table 1. Sample Selection

	Year 1		Year 2
	Winner	Nonwinner	Two-Time Winner	One-Time Winner	Two-Time Nonwinner
Elementary	4	3	1	1	1
Middle/High	2	3	1	3	0

We conducted Year 2 interviews in two waves. The first occurred between May and June 2013, about six months after bonus winners had received the second RttT bonus, and the second occurred between September 2013 and January 2014, 913 months after bonus winners had received the second RttT bonus.

INTERVIEWS

All interviews and focus groups were audio-recorded and conducted during the school day during a teacher workday, a prep period, or a time when teachers were able to get coverage for their classes. To be considerate of time and coverage constraints for teachers, interviews with teachers and administrators lasted a maximum of 30 minutes, and focus groups lasted a maximum of 45 minutes, with few exceptions.

Interviews were semistructured, which allowed us to explore unanticipated themes as they arose. We asked questions about teachers awareness of the program, if and how it affected their motivation or teaching practice, and their general perceptions of the program and performance pay. We also asked questions about general practices, changes, programming, and school climate or culture, particularly when we discovered from participants that significant changes, like an administrative change or the adoption of a new program or curriculum, had occurred in recent years.

DATA ANALYSIS

In Year 1, the members of the evaluation team who conducted the interviews created detailed site visit summaries (similar to field notes) for all interviews and focus groups conducted in a particular school. Site visit summaries included details about the school building, the affect of interview participants, general patterns in what respondents shared, and some direct quotes. Initial coding involved reading and coding these reports for themes about teachers awareness, understandings, and perceptions of the incentive, as well as other emergent themes, such as opinions about measurement or general barriers to student growth. We organized text associated with frequently used codes into tables, then wrote interpretive memos to find the consistent patterns among interviewees responses.

In Year 2, other members of the evaluation team transcribed interviews and focus groups after conducting them instead of summarizing them into reports. We uploaded full transcriptions of Year 2 interviews and focus groups into Dedoose qualitative analysis software, which allows for systematic coding and analysis of codes (e.g., which codes are most often used, excerpts attached to certain codes). We pulled lists of text excerpts by frequently used codes and organized those excerpts into interpretive memos. For both years, we paid careful attention to confirming and disconfirming evidence of themes by school type (elementary, middle, or high school), subject/grade taught (tested or untested), and interviewee type (principal or teacher).

RESULTS

Because it is impossible for principals to oversee all aspects of teachers (agents) practice, the principalagent problem suggests that teachers will knowingly shirk their effort when they can unless an external reward for meeting target outputs incentivizes them otherwise. Based on the themes that emerged from the coding of our interviews and focus groups, we argue that performance incentives have had, and will continue to have, little impact on changing teachers practice or effectiveness in increasing student achievement because the implicit assumptions of the principalagent problem do not apply to education.

The principalagent problem suggests that agents primary motivation for their work is money and that they knowingly shirk their effort unless extra money is available for maximal effort. However, three themes emerged from our interviews that contradict these assumptions: (1) performance incentive structures are inconsistent with teachers and administrators primary motivation for and best practices of education, (2) teachers perceive themselves to be working as hard as they can already, and (3) teachers improve their practice through training and other opportunities to learn strategies for success, not incentives to work harder.

Because administrators can be seen as the principals who are trying to maximize teachers effort, we also present administrators views on performance incentives separately from teachers. However, it is important to note that teachers and administrators did not differ meaningfully in their responses; the themes that emerged from teachers were echoed in administrators responses and vice versa.

INCOMPATIBILITY OF MOTIVATION: ITS NOT ABOUT THE MONEY, ITS ABOUT THE STUDENTS

The principalagent problem suggests that agents cannot be effectively monitored to ensure maximal effort and therefore must be incentivized with external rewards to do so. Because student achievement and growth are key outputs in education, policy makers suggest that extra money for meeting these target outputs will ultimately increase teacher performance. Performance incentives therefore assume that teachers, like any other agent, are primarily motivated by financial interests; however, our first finding suggests otherwise. A majority of teachers and administrators (about 70%90% of teachers and 80%90% of administrators, depending on the year) believed that the performance incentive played no role in their practice, often because they said that what teachers do is not for financial reward.

Teachers

When we asked teachers about the role of the school-level performance incentive, teachers responded in an overwhelmingly similar waythat the money is nice to receive, but it did not affect how they were going about their practice. Said one teacher, I think for me personally more of the here and now of what I do would be more incentive. . . I mean we all care very deeply and want to move our kids, so that when they move to the next grade theyre the best prepared children.

For this teacher and most of the teachers we interviewed, the primary motivation for teaching was the learning goals teachers had for the students. Many of the teachers even called the students their (or our, my, etc.) children, suggesting a level of care that extends deeper than a typical transaction between a principal and agent. What this suggests is that teachers are intrinsically motivated by the needs of their students, not by an extrinsic financial reward. Even if everyone (including nontested teachers) is able to receive the schoolwide incentive, teachers regularly shared that it played no role in their everyday practice, because, as the mentioned teacher put it, the here and now of the students mattered more to them than a potential financial reward down the road.

It is possible that the schoolwide incentive, although appreciated if received, was not a primary motivation for teachers if they believed they had less control over the outcome than in their own personal classroom. We therefore asked teachers if individual performance incentives would motivate or change their practice in any way. One untested teacher (Spanish) in the focus group groaned and then later explained,

Why do they teach their subject? Why do they think this is important? And if you have educators with a heart for their subject, and they are passionate about what they teach, it doesnt matter if theres an incentive because I dont have integrity in my classroom because I get paid extra money. [Another teacher in the focus group agrees.]

Before we even had a chance to finish our question, the first teacher in this focus group groaned in clear frustration. Although at first, her frustration seemed to stem from the fact that her subject would not be eligible for an individual performance incentive (she taught Spanish, and only core subjects like math, English, and science would be eligible), she became more heated as she talked about her integrity as an educator. She seemed to believe that her practice was motivated by preparing students adequately for college, for learning and appreciating Spanish, and for helping students become better people. She was quite clear that money did not motivate her practice.  Even though a performance incentive would represent, to some degree, student learning and growth (two values this teacher suggested were part of what gave her professional integrity), this teacher believed that preparing her students for life was her primary motivation as a teacher; her motivation was not financial.

Among tested and nontested teachers alike, teachers regularly reported that performance incentiveseven individual, classroom-based oneswould not dramatically impact their practice because they did not align with their primary motivation for teaching. However, some teachers reported that other teachers (i.e., some of their colleagues) might be motivated by extra money. According to one elementary school teacher,

I think a lot of us here are just, we want to be teaching and we want the kids to learn so Im not sure how much the money really changes that. But Im just saying for some teachers, like, if theyre not very motivated to be here, maybe thats whats gonna motivate them, a big bonus at the end.

This teacher recognized that a big bonus could motivate some teachers to work harder. She later explained that a bonus might make some teachers more likely to incorporate more reading into their curriculum or hold students more accountable for turning in homework. Other teachers suggested that tested-area teachers who would be eligible for an individual incentive might focus on test prep at the expense of a good classroom experience. Said one teacher, It will be the most boring class in the world, [but] when you have three and a half months to get them ready for this test, you do test prep. However, even one of the teachers who was most critical of her colleagues motivation simultaneously recognized that most of the teachers she knows at the school would not put the money first, suggesting that working for the money is not the normal or preferred motivation for teaching.

Though a vast majority of teachers interviewed said that money was not the primary motivating factor in their practice, about 20% of teachers and administrators did recognize that extra money could be a motivator or affect teaching in a positive way either for themselves or for others. This was particularly true of high school and, to a lesser extent, middle school teachers in the second year of data collection, when awareness of the program was greater and there was an additional individual incentive for exceeding growth at the classroom level. However, several teachers who reported that the money could be extra motivation had a hard time explaining how that would translate into actual practice. Take for instance, this interchange between the interviewer and a high school teacher:

Interviewer: In what way does the performance incentive help you or motivate you? How does it help you achieve growth for your students?

Teacher: Hm. . . [pause], thats an interesting question. Um, like I said, it just gives me something to um, work towards in addition to the kids being successful.

Although this teacher seemed to believe that the incentive did provide some kind of motivation for her, she was unable to say how it would motivate her or how it would translate into tangible practices that promote growth effort in her classes.

It is important to note that money did matter to teachers to some extent. Teachers were appreciative of opportunities to earn more money, wished that they could earn more money regularly, and were thankful for the bonuses if/when they received them. But financial interests were not most teachers primary motivation for their practice, and the performance incentive, many believed, did little, if anything, to change their teaching.

Administrators

It is perhaps easy for teachers to say they are not primarily motivated by money, so we asked administrators if they believed performance incentives played a role in their agents teaching. If so, this would suggest that teachers may have been more motivated by money than what they wanted us to believe. Here is what one principal said: It just didnt seem to make a difference. I never heard them talk about it, as a motivating factor . . . I dont think it ever came up after that initial meeting that I told them about it. When this principal shared information about eligibility for the performance incentive, she seemed to assume that if teachers were interested in or supported it, they would discuss it openly. However, this is not what happened. Instead, she never heard teachers talk about the incentive, suggesting that the incentive was not a particularly meaningful motivator to the teachers.

The principal went on to explain why she thought the performance incentive was not relevant:

Because . . . for me what education is, is a moral purpose. Its a moral imperative. And, I think [the teachers at this school] understand that . . . and I think thats become part of our [school] culture, that its a moral imperative that we do our best by these children.

Similar to teachers, this principal also describes personally being motivated by a moral purpose rather than a monetary one, and other principals recognize that teachers feel the same way. One principal put it very bluntly:

You have to realize that these people [teachers] do not do what they do for the money, because if they did theyd be looking for another job. This school has been touted as being possibly one of the most challenging schools. I mean its been identified as such. And we know that the challenges are great and to be willing to come into this school every day and do what you do, its not because you think youre gonna get $1,500 or $2,000 extra bonus. Thats not why you do it.

This principal recognizes that, particularly in so-called challenging schools, where student growth is slower and teachers are faced with incredible challenges (such as student homelessness, violence, the need for social services, etc.), financial motivation is simply not the reason that teachers come to work every day. This is meaningful because the schools with the most challenging teaching conditions are also the ones that require the most academic improvement. These are the schools where teachers are most heavily criticized for not meeting achievement and/or growth targets. If, in these schools, teachers are not going to be motivated by extra financial incentives to increase student growth, performance incentives will likely do little to improve teaching across the profession.

At one high school, the administration team works so hard not to encourage a work-for-the-money mentality that they do not even share with their teachers the incentives for which they could be eligible. One of the assistant principals, in explaining why they do not share performance incentive eligibility information, said,  

Heres the thing. You want everybody to be lifelong learners. The kids, the teachers, so, so what if money may dry up? And so, do they quit trying when that money dries up like everything else, and thats why I think our approach is the best . . . its not about the pay, its about producing that doctor, that lawyer, that whoever you produce to be successful.

Like many of the teachers we interviewed, this high school principal recognizes that working for extra money is not what should motivate teachers; instead, the focus should be on the growth and development of students, whatever the financial reward. This is not just a philosophical position, but a practical one, because this assistant principal also recognizes that monies designated for performance incentives in North Carolina have been inconsistent or even disappeared in the past (in part, perhaps, because these incentives were not vastly changing the effectiveness of the teaching population in North Carolina).

In sum, most teachers and administrators believed that teaching stemmed from an intrinsic moral commitment to students and their growth. Performance incentives would therefore have little impact on teachers, because their practice is not driven by extrinsic financial reward.

INCOMPATIBILITY OF BEST PRACTICES

In addition to performance incentives being incompatible with teachers and administrators primary motivation for teaching, performance incentives that reward individual teacher efforts tend to be at odds with what teachers and administrators view as best practices in teaching. Specifically, teachers and administrators point to the importance of collaboration among faculty and staff members to best identify and share strategies that lead to increased student learning.

Administrators

Ideas about collaboration and its usefulness seemed to stem from the administration at each school, given that teachers regularly reported ideas that reflected their principals philosophies. Said one elementary school principal,

We work very very hard to create an atmosphere of unity and collaboration. And get away from that old mindset of going into my classroom, closing my door, and these are my 20 or 25, and Im only concerned about what they do . . . it doesnt matter if that 25 are successful, and the rest of the 500 or so are not . . . we are a team . . . when we all arent successful, then none of us are successful.

This principal recognizes that collaboration is necessary for a whole schools success, and though this school has yet to receive a bonus, when they started implementing strategies to increase collaboration, the school made growth for the first time in their five-year history and has been identified as one of the best elementary schools in the district. A similar story was shared by two coprincipals at a two-time bonus-winning elementary school. They illustrated exactly how a collaborative and team-based model works at their school:

Coprincipal 1: Effective teachers, come out of your classrooms. Stop closing your door. Because we have kids across the hall that can benefit from the same information that youre giving in your homeroom.

Coprincipal 2: Share best practices. So she would go around and geometry was her specialty. Everybody rotated through her for geometry. Division was my specialty so everyone got me for division . . . its a collaboration, and so this bonus . . . takes away the need to collaborate in order to be effective. It goes back to me me me me me . . . now youre gonna penalize the teachers because of the model were using.

Like the previous elementary school, this elementary school has had great success with its efforts to move away from a model of teaching in which teachers are only responsible for their own classrooms. Instead, teachers divide up their areas of expertise and rotate all the students in that grade through each teacher. That way, all students can benefit from one teachers effective lesson on geometry and a different teachers effective lesson on division, for example.

In theory, the schoolwide performance incentive could increase collaboration. In a schoolwide incentive, all educators (typically certified teachers and administrators, but not necessarily staff) receive a bonus if the school as a whole meets specified targets. Because all teachers would have a stake in the financial reward, it is possible that teachers would be more likely to work together to meet those targets. One principal (and the teachers in his school) believed that collaboration and a general sense of were all in this together did improve when the teachers were aware of the schoolwide performance incentive. This principal was also one of the only principals to advocate for performance incentives (he even wished that teachers base salaries were structured by performance targets). The other principals adoption of more collaborative styles of teaching occurred before and regardless of the schoolwide incentive, and they believed that the incentive did nothing to change their practice.

Administrators were most critical of individual, or classroom-based, performance incentives (added midway through the RttT period) because they believed these incentives did not foster a culture of collaboration and sharing. They believed that individual performance incentives would encourage teachers to think of their own students achievement rather than the whole grade level or school. This was expressed by one high school principal, who said,

[Performance incentives] will give a false impression for some. I mean if you have, we have a couple of teachers who are outstanding and they happen to teach a tested area, and they, word gets out theyre the ones getting incentives . . . they may perceive a peer of theirs whos just as good a teacher but may not be in tested area, that theyre not as good a teacher cuz they dont get that recognition.

This principal is worried that the incentives will start to make some teachers think they are better than others, which he believes will be damaging to the culture and morale of the school.

Teachers

A high school teacher in a different school echoed this concern, saying that money can do strange things to people, [even] people who are best friends. This teacher said that a previous experience with individual performance incentives, especially with bad communication about who received it, made her see that clearly in her school. She perceived the school to be a tight, close-knit community, but because of this bad experience with a performance incentive, the school turned ugly. Gossip, crying, and at least one teacher leaving early were all cited as part of that ugliness.

One way to avoid divisions between those who are eligible for extra money and those who are not is to adopt a schoolwide incentive instead of an individual incentive. Just as one administrator believed that schoolwide incentives encouraged collaboration, so too did a few teachers in several schools. This small minority of teachers reported a greater sense of schoolwide responsibility for students test performance; reported incorporating elements of math, English literacy, and science more into their untested curriculum; and even admitted to holding each other more accountable for student progress as a result of the incentive. Often, though, when teachers reported collaborating to earn the performance incentive, it involved students missing time in untested subjects to emphasize tested skills. Specifically, teachers (with administrative support) worked together to pull students away from untested classes for extra enrichment or intervention (tutoring) time in tested subjects. Ultimately, this is not the kind of best practice collaboration (i.e., sharing effective lesson plans or other resources) described by principals and teachers.

It is possible that an individual performance incentive could also promote collaboration. If, as some of the administrators described, teachers came together to divide up lesson plans by area of expertise, and all students improved, then teachers might have a better chance at earning the individual performance incentive. Some teachers did seem to think that performance incentives could increase collaboration in their school. For example, one focus group reported that their math department (tested area teachers who would be eligible for individual incentives) had started collaborative professional learning communities (PLCs) as a result of the performance incentive. PLCs consist of teachers who teach similar subjects or grade levels and meet regularly to share best practices, lesson plans, and professional coaching. If a PLC had been created because of the possibility of earning a performance incentive, then collaboration would have increased in the school regardless of whether the teachers earned them. However, it is questionable as to whether the performance incentive was actually the impetus for the change in this school, because three or four years ago, when the group reported they started PLCs, the RttT bonus was in its beginning phases of implementation, and awareness of the program was quite low statewide.⁶

The vast majority of teachers did not seem to think collaboration would increase as a result of the performance incentive, especially individual ones. In fact, in one school, the individual RttT performance incentives seemed to drive teachers apart. In particular, they seemed to create friction and resentment toward teachers of tested core subjects among teachers of untested electives and other courses. One middle school teacher in a focus group of other teachers from tested grades and subjects said,

I feel like last year when everybody knew that everybody would benefit together, that we were helped more by teachers that were not in a tested area. Um, our resource teachers . . . PE and art and music, they were more helpful. They pushed the students a little more last year than they have this year. This year if you ask for help, you may or may not get it.

The teachers in this school were very aware of their eligibility for performance incentives, and as a teacher in a tested grade, this teacher would be eligible for an individual performance incentive for high student growth if her students scored well enough on state assessments. However, other teachers in the school who regularly worked with the students in her classroom, like resource teachers or even PE and art teachers, would not be eligible for those incentives. Several interviewees from different schools discussed having received help from elective and other untested teachers. Those teachers might incorporate some extra reading or math into a PE or art activity, which could help students associate a tested concept with an activity they enjoyed or remembered more clearly. Resource teachers might offer enough supplemental support and tutoring to struggling students to increase their test scores. In each of these cases, though, the untested teachers efforts would go unrewarded because the teacher to whom the student was regularly assigned would get the credit for that students scores. The teacher interviewed here seemed to think that the kind of help she had received from untested teachers before, when performance incentives were at the school level, was not available when performance incentives became awarded at the individual level.

When I talked with ineligible untested teachers in that same school, they did in fact report less willingness to help:

Teacher A: I really do think eventually that it, once people realize where the money is going, that it might. There might be some professional jealousy. Lets use that term,

Teacher B: We might not be so apt to--

Teacher C: Help.

Teacher B: help with reading, right?

Teacher D and Teacher C: Yeah.

Teacher A: No. Because why am I gonna help you get $2500 when I maybe get a bag of peanuts?

Not many teachers were so blunt about their unwillingness to help as the teachers in this school. Of the 20% of teachers who did say that extra money would motivate them, this group of teachers was one of the only groups to clearly and emotionally express that motivation. What makes the teachers from this school worth featuring, though, is that they had a hyperawareness, albeit slightly misinformed, of the incentive program, and the principal in their school was one of the only administrators to appear to value the incentive program and express support for it. Teachers in other schools had little to no awareness of either the incentive program in general or the midprogram addition of an individual bonus for tested-area teachers only.

In short, teachers and administrators reported that one of the best practices for student learning was collaboration between faculty, which included sharing resources and lesson plans, and even moving students around to different teachers to hear the same lesson in a different way. Although schoolwide incentives may theoretically incentivize this best practice, few administrators and teachers actually reported an increase in collaborative efforts from the incentive itself, likely because performance incentives are still at odds with teachers primary motivation for teaching. Individual performance incentives were antithetical to this best practice, promoting an older model of teaching that emphasizes one teachers complete responsibility for the knowledge acquisition of her students. In schools that emphasized individual incentives, ineligible teachers seemed to have resentment toward eligible teachers; these schools were less likely to use best practices (collaboration), and individual incentives generally seemed to reduce the morale and overall quality of teaching.

WERE ALREADY WORKING AS HARD AS WE CAN

The principalagent problem suggests that agents will shirk their effort in the absence of supervision or an external reward for output. Therefore, one of the assumptions underlying adoption of teacher performance incentives is that teachers are not currently working hard; the idea is that additional money would reward the extra effort the agent would expend to meet the obligations set by the principal.

Teachers

Most teachers we interviewed over our 18-month period claimed to be working as hard as they could, regardless of whether an incentive was available to them. One high school teacher noted, If we get a $30,000 raise it would make it easier to come to work every day, but Im not going to work any harder because I feel like I work as hard as I can every day. For this teacher, any amount of money, even a $30,000 raise (which would nearly double her salary), would not have impacted how hard she was working because she felt there was no room in her practice to work any harder. Teachers in every school where we interviewed shared similar sentiments about working hard in their jobs. They described giving up free periods to tutor students, the hours they spent on lesson plans, the before- and after-school service to student clubs and academic support, the personal money they spent to buy supplies for their students, and the ever-increasing administrative paperwork they had to do in order to meet district and state requirements. In short, these teachers sounded as if they had little time left to give such that they could work any harder. As one teacher put it, Youre not short on motivated teachers [at this school.] I see people who are here before the sun comes up and are here till the sun goes down, and theyre doing everything they can possibly do to motivate students.

For this teacher, his colleagues effort and motivation could not possibly increase. The teachers in his school spent so much time working that they were there before the sun rose and after the sun set. During those hours, they were working with students, planning lessons, serving on committees, filing administrative paperwork, and doing everything else they could be doing to serve and motivate students. Further, this teacher suggested that it was somewhat common to see teachers dedicating this amount of effort to their students; he said the school was not short on teachers who worked this way.

It is possible that when teachers claim they are already working as hard as they can, they are failing to recognize or admit to their own failings to exert more effort. Therefore, to get a better sense of the degree to which teachers in general were exerting or not exerting enough effort, we asked them how many of their colleagues they thought were ineffective or might be motivated to work harder by a performance incentive. In general, teachers identified very few of their colleagues as being ineffective, which they characterized as prizing efficiency and their own time above all else, not doing enough to update their teaching materials, or generally not caring as much as they believed teachers should. This was particularly true in schools with a culture of collaboration and team-teaching, where teaching practice was more visible to teacher colleagues and administrators.

Most teachers described a bell curve of teaching effectiveness at their school. Some teachers identified only one or two teachers in their school as being ineffective, whereas others believed up to 10% or 15% of their colleagues were ineffective. There was only one teacher out of 150 interviewees who claimed that as many as 25% of her colleagues were slacker teachers (25% was the highest percentage any teacher reported as being ineffective in a school). However, regardless of how many teachers participants claimed were on the least effective end of the distribution, only three of the teachers and one of the principals interviewed believed that a performance incentive would increase the efforts or effectiveness of the lowest performing teachers in their schools. In other words, it seems unlikely that anything, let alone a performance incentive, would make the least effective teachers work harder or be more effective at their jobs.

Administrators

When we asked administrators how many of their teachers were low performing, ineffective, or not working hard, the same story emerged. Administrators talked about a few, a couple, or as many as 8 of 50 (16%) teachers who needed to move on. Dedication to teaching, lack of motivation, or unwillingness to spend extra time researching new teaching strategies are all characteristics that administrators discussed about teachers they perceived to be ineffective. Importantly, though, principals, whom we may think of as principals in the principalagent problem, perceived that most of their agents (teachers) were working as hard as they could. This invalidates the need for a performance incentive as a way to improve the teaching profession as a whole, given that there is little extra effort for principals to extract from agents.

Although most teachers believed that they were working as hard as they could already and that a performance incentive would do nothing to affect their teaching practice, a small group of teachers and administrators (about 10% of all those interviewed) did report some tangible and meaningful changes to their practice that they credited to the performance incentive. Most often, teachers reported doing extra tutoring with students before, after, or during school. Administrators reported structuring the school day such that more time was spent in tested subject classes and less time was spent in elective classes. They also said they started more interventions with struggling students earlier in the school year, including pulling students out of elective classes for extra tested-subject practice.

It is important to note that although all these strategies are real changes to practice, it is unlikely that they alone are responsible for significant, long-term changes in student growth. We interviewed teachers and administrators several months before the announcement of the years bonus winners, and once the bonus winners became public, we checked whether those who reported being motivated by the bonus had actually received the bonus. Only two teachers who had done extra tutoring or early interventions with students received the bonus, suggesting that even if teachers are actually motivated by the money, and they do have some room to work harder, performance incentives may not be particularly effective at increasing teaching effectiveness across the entire teaching force because student performance is still somewhat outside of teachers own control.

HOW TEACHERS BECOME MORE EFFECTIVE

The principalagent problem suggests that agents knowingly shirk their effort, which suggests that they would know how to work harder if they were to do so. Yet teachers, in addition to reporting that they are already working as hard as they can, also report that when they do improve their practice, it is not by working harder, but by learning to work smarter. That is, teachers were learning to increase their effectiveness by learning new strategies for teaching from professional development, collaboration with each other, and other training opportunities, not by exerting more effort in their current practice. In the lowest performing schools, RttT provided professional development workshops for teachers and principals, as well as learning coordinators from the North Carolina Department of Public Instruction (NCDPI), who would work one-on-one with teachers, help develop their lesson plans, observe, and provide feedback. Several of the principals and teachers we spoke with credited their students growth to professional development.

Administrators

One middle school principal said,

It is phenomenal. And I bring a lot of it back to our teachers and . . . it gives me an opportunity to network with the other principals that are part of the Race to the Top and you can learn strategies and bounce ideas off of them. Thats pretty neat.

As a result of the opportunities to learn new ideas and strategies from other principals, this principal said, [we] went in and changed [teachers] way of teaching . . . we made our teachers more knowledgeable of the strategies that they need to use within their particular content.

This principal later discussed several specific strategies that he learned and then taught to his teachers through professional development, including: runners for reading (a mnemonic device students use for reading comprehension), thinking maps for brainstorming ideas, cooperative learning (a strategy that involves students being active participants in their own learning and instruction of peers), and the 80-20 rule (making sure teachers do not lecture for more than 20 minutes at a time before doing an activity that engages the students).

Because some of the schools in this study are some of the lowest achieving schools in the state, they are also eligible for additional grants and opportunities. One high school principal reported using funding from a different grant source to hire a learning coordinator for his staff and found that, more than anything else, the learning coordinator helped his teachers become more effective:

Really the way that weve improved teachers the best is we hired . . . a learning development coordinator. And his only role is really to provide on the job, just in time, professional development . . . hes in the classroom, planning, modeling, coaching, um, being a really, being a true instructional leader. . . . Because they go in the classroom and theyll model it to the student or to a teacher how to teach a lesson and theyll watch the teacher do it, offer feedback, theyll lesson plan, theyll look at data, theyll look at room setup, I mean everything. Its probably been the strongest thing that Ive seen in 14 or 15 years of education that weve been doing the last couple a years at the high school.

As this principal explained, a dedicated instructional learning coach who is able to provide on-site training and feedback to teachers on the lesson plans they are actually using in class seems to be one of the most beneficial initiatives he has seen in his experience as an educator. Having a dedicated staff member as a learning coach could be helpful for teachers in a number of ways. First, it may be difficult for teachers to admit that they need help. But, if they have a relationship with another staff member, they might be more likely to ask for help and trust that they are receiving good quality assistance. Second, if the learning coordinator is on-site, she will understand the needs of the students better than an outsider, so her strategies might speak more effectively to the particular school audience. Third, she will be available to teachers more quickly than a quarterly or semiannual professional development session might be to solve real-time issues and challenges in teaching. And finally, she will be able to continue to work with and follow up with teachers who need more training and practice over time.  

In addition to on-site learning coaches, other schools describe different kinds of professional development that they have found useful for increasing teacher effectiveness. One elementary school with the opportunity to receive funding from a School Improvement Grant credits the growth of teachers and, by extension, students to differentiated professional development for teachers:

Coprincipal 2: With the academic coaches, we were able to differentiate . . . so our Wednesdays, professional development. . . We take 10 minutes, but we have 3 or 4 sessions going at a time, and thats only because we have academic coaches or the additional support staff that provide service for K2, service for 35, because those grade levels look different, or service for a beginning teacher or service for a teacher thats been teaching, you know 3 or 4 years, and then service PD sessions for the teachers that have been teaching 5 or more yearsthey have to look different. . . . When we provide customized professional development for the teachers then the teachers in return provide customized lesson plans . . . for their students.

For this school, a one-size-fits-all model of professional development did not seem to be adding value to teachers practice. A novice kindergarten teacher, for example, requires different kinds of coaching and assistance than a teacher who has taught third grade for 10 years. To remedy this problem, this school used extra funding from an outside grant to hire academic coaches who could provide different kinds of professional development for teachers of different grade and experience levels. The administrators in this school have found that with this targeted training, teachers can learn the most relevant curricular and age-appropriate strategies for their classroom. It is important to note that this school conducts these differentiated professional development programs weekly for teachers, which means that teachers likely benefit from the same consistency and real-time problem-solving that the school with the on-site learning coach does.

In addition to professional development and academic coaching, other schools found that teacher effectiveness increased through collaboration between teachers in professional learning communities. In one school, the principal received training from one of the NCDPI coaches and replicated it in weekly PLC meetings centered on team building, data analysis, and identifying and sharing strategies for success.

Im a firm believer of professional learning communities as a way of collaborating, as a way of teaching and learning as well, so I decided that one of the strategies in trying to increase academic achievement here would be to have the weekly PLC meetings with me. So each Monday I devoted all day to meeting with each grade level. . . . We first began our first four or five sessions on what it meant to have a collaborative work environment. What it meant to have a collaborative team . . . then we started talking about how the collaborative environment would transfer into successful planning, successful, better lesson plans, which would then in turn mean better delivery in the classroom, which would hopefully ultimately lead to more academic success for the children.

In a systematic way, this principal began creating a culture of collaboration and sharing between the teachers at the school. She started by separating each grade level into a team whose members would all meet with her together. Like the mentioned school that created differentiated professional development by grade and experience, this principal recognized that collaborative communities ought to be differentiated to some degree in order to share the most relevant curricular materials and strategies for students of particular age groups. However, before the PLC could dive into sharing strategies and offering critical feedback, the principal first had to build with the teachers a shared understanding about what it meant to collaborate effectively and what it meant to take a team-based approach. Having the teachers buy in to this idea seemed to be a necessary step in creating an effective PLC. Later in the interview, the principal said she went out on a limb further and asked the teachers if she could be transparent with their students test score data. She believed it was necessary to understand how the students were performing as a grade level and in individual teachers classrooms. When students did well in one teachers classroom on a certain concept, but others in a different classroom did poorly on the same one, she believed it was important to talk about what a teacher did well in the first classroom so it could be applied to the second. Teachers seemed to recognize that the professional development and new strategies that they were learning were helping as well. Teachers in this elementary school described those weekly PLC meetings as meaningful and credited them with becoming more effective at teaching. Had the principal not built a safe, comfortable environment for teachers, the PLC might not have been so successful.

Teachers

Many teachers in these schools seemed to find their professional development and PLCs to be as useful as administrators claimed they were. A high school English teacher from one school found value in the professional development: I think we have a clearer definition of what teaching is, what it looks like, what it feels like, what it sounds like. This teacher also shared that outside of the on-site learning coach at their school, there were other professional development opportunities throughout the summer and periodically during the year. These sessions were teacher led, and they shared new strategies with one another. Teachers had an option to go to two out of every three sessions offered, and this teacher said that every teacher went. This suggests that other teachers in her school also found the professional development to be valuable.  

Other teachers at one elementary school discussed how significant changes to their practice occurred through professional development with an NCDPI learning coordinator.

Interviewer: So what kind of resources have they given you?

Teacher: Oh god, first they started by reconstructing our lesson plan. Two years ago? Yeah. They totally reconstructed our lesson plan, beginning with the end in mind. At first it was kind of like, gosh [expresses annoyance, resentment], but it really has helped us.

At first, the teacher seemed to express resentment toward the learning coordinator for coming in and providing critical feedback on their practice. But after seeing personal improvement, as well as other colleagues improvement, she became thankful for the coaching on more effective lesson planning.

However, not all teachers recognized the utility of the professional development. Two teachers in one high school found little value in the programming and found it to be a waste of their time. Additionally, some teachers, particularly untested area teachers in two different schools, claimed that the learning coaches targeted most of their time and resources to tested grade teachers or teachers with whom they were personally friends. However, these opinions were not representative of the majority of teachers who discussed their perceptions of professional development, learning coaches, and practice improvement.

DISCUSSION

Results from this study challenge traditional principalagent assumptions about teachers motivation and practice. Principalagent theory assumes that teachers are not working as hard as they can and will be motivated by money to do so; however, results from this study suggest that not only are many North Carolina teachers working as hard as they can, but they are also not so self-interested that they can be motivated by a performance incentive to maximize their own profits. If the opportunity to maximize profits were a key motivator for teachers practice, we would likely have seen those sentiments arise in interviews. At the time of data collection, North Carolina was 48th in the country in teacher pay, and because of budget cuts and changes in the priorities of state government, teachers in North Carolina had not had a pay increase in five years. If any teacher would be responsive to a performance incentive for extra pay, it would likely be a teacher in North Carolina.

This is not to say, of course, that teachers are completely unmotivated by money. Between 2008 and 2013, 14% of teachers in North Carolina quit the teaching professionmany because they could no longer support themselves or their families on their salaries (BEST NC, 2015). In 2014 alone, about 19% of teachers left the teaching profession, citing low pay as the number one reason for leaving (Childcare Services Association, 2014). In about half of North Carolinas counties, the teacher turnover rate is more than 35%. Two teachers we interviewed in border regions shared that they would be seeking employment in Virginia and South Carolina, where the pay for teachers is far better than the pay they would continue to receive in North Carolina. A principal shared that she would be retiring early because she felt powerless to help the teachers and teaching assistants in her school, some of whom had to steal food from the cafeteria to feed their families because their salaries were too low to support them but too high to qualify for assistance. Another teacher shared that if she had one more baby, she could qualify for assistance, and though she said she realized she sounded ridiculous for thinking it, the thought of having another child to increase her monthly take-home pay had crossed her mind. This is common across the state; 42% of teachers and 48% of teaching assistants received public assistance between 2011 and 2014 (Childcare Services Association, 2014). Obviously, money matters to teachers, and particularly for teachers in North Carolina. However, even in this kind of environment, incentives were not motivating changes to teachers practice. Rather, most teachers described being intrinsically motivated to serve their students, a result that is consistent with prior research on teachers and their motivation (Finnigan & Gross, 2007). In all, these results seem to suggest that performance incentives will not have the kind of broad impact on the teaching labor force that policy makers believe they will.

Additionally, the practices principals and teachers consider to be most effective for increasing student learning are inconsistent with the reward structure of performance incentives. Principals and teachers describe student learning as benefitting from teamwork and collaboration on the part of teachers, but performance incentivesat least those that are individually baseddo not incentivize the practices that teachers and principals claim to increase student learning the most, and if student learning is the primary motivation of teachers practices, the incentives will have little utility.

Because principalagent assumptions do not appear to apply to the teaching profession, performance pay is probably an ineffective strategy in developing higher quality teachers, at least in the United States. NCLB and state accountability systems have generated vast archives of student test score data that can often be linked to individual teachers. Teacher value-added modeling has emerged as the second wave of accountability, but with obvious limitations given that it only covers students in tested grades and subjects. Linking teacher value-added and performance pay seems like natural fit: Marry a somewhat objective measure on an important aspect of teachers work (raising student test scores) and then pay them based on moving this metric. But our results show that the assumptions undergirding the second piece of this linkage are flawed. So even if teacher value-added models are validand there is much debate about that (Harris, 2009; Hill, 2009; Koedel & Betts, 2011; Rothstein, 2010)the use of these measures to rate teachers into performance categories may be a fruitless exercise. That is not to say, however, that these measures could not be used in other useful ways. For example, our findings suggest that teacher quality may be improved when teachers have opportunities to increase their human capital through training and professional development. Examination of student test score growth, when available, could be an important measure to discuss within critical conversations about practice.

Professional development has been criticized by some for not being particularly useful in increasing teacher effectiveness (Ball & Cohen, 1999), but professional development can take many forms, from watching an online module or PowerPoint presentation, to on-site strategy instruction. If most forms of professional development tend to be passive information-sharing meetings, it is possible that teachers will not find much value in it. However, instructional coaching by a dedicated staff member or effective colleague who provides one-on-one observation, feedback, and assistance is perhaps a more useful form of professional development. As principals and teachers have described here, student learning and growth seems to improve when teachers learn new strategies and best practices from training, coaching, and even collaboration with each other.

As we have argued, performance pay takes as a given that teachers work in isolation, in competition with each other, immune to the monitoring of managers and school principals. But increased awareness of the important role of teacher quality and the development of teacher accountability policies have created a push for greater instructional leadership by principals, observation of teacher lessons, and instructional coaching. Therefore, instruction is likely to improve when student performance data inform and are coupled with observation, coaching, and collaboration; instruction is less likely to improve when student performance data are used simply as the metric for performance pay.

Like all research, this study is not without limitations. One is that, particularly in the first year of data collection, there was limited awareness of the incentive program. Therefore, teachers opinions of the program reflected their opinions of performance pay in general rather than their personal experiences being incentivized. However, we believe that the conclusions of this study are minimally affected by this limitation because awareness of the program was greater during the second year of data collection, and a majority of teachers (66%) still believed that the incentive played no role in their own teaching. Of the 33% (22 total teachers) in that year who did believe performance incentives played a role in their or their colleagues teaching, 13 of them (approximately 59%) believed the role the incentive played was negative. In all, the percentage of teachers who reported being positively impacted by the performance incentive was quite small, suggesting that for a majority of teachers interviewed in this study, performance incentives were inconsistent with teachers primary motivation for and practices of teaching.

Notes

1. Measuring teacher effectiveness with value-added is not without its detractors. Rothstein (2010) for example, noted that value-added models that rely only on test scores as predictors are biased and misleading. For defenses of the validity of value-added models, see Chetty et al. (2014a, 2014b) and Kane, McCaffrey, Miller, and Staiger (2013).

2. That the U.S. experience with performance incentives differs from Indias or Kenyas is perhaps not surprising given the many differences between the labor markets in developed and developing economies. But the positive effects in Israel and null effects in the United States are harder to explain away because both Israel and the United States are developed economies. That said, the U.S. GDP per capita is roughly 50% higher than Israels, and without comparing the percentage increase to base pay in each experiment and differences in the purchasing power of a similar basket of goods with typical teacher pay, it is difficult to fully assess the reasons for these differences.

3. One exception has been a loss aversion study conducted by Fryer and colleagues in Chicago Heights, Illinois (Fryer et al., 2012). In this random-control trial, bonuses were given to teachers at the beginning of the school year, and if teachers did not meet the performance criteria at the end of the year, they were forced to return the money. This loss-aversion type of bonus did produce large and statistically significant gains of .2 to .4 standard deviations in math for students. The research is still in progress at the National Bureau of Economics Research.

4.  Growth was calculated by standardizing students academic change, or changes in test scores (students normalized test score minus the average of scores from the two prior years, adjusted for the mean) on the mean and standard deviation from the first year the test was used. Expected growth is defined as having a collective change ratio of 0.0 or better based on results from all the students who take the end-of-course or end-of-grade tests, and high growth is defined as having a change ratio of 1.5 or better.

5. Growth is determined by EVAAS value-added scores, which is a measurement of how much students achieved relative to how much the students were expected to achieve in a particular course or grade. Meeting growth translates into students, on average, meeting the predicted scores associated with one years worth of more growth at the school. The change in language reflects North Carolinas switch from the ABCs Accountability model, which ended in 2012, to the READY accountability model, which began in 2013. The READY accountability model was adopted to align with the Common Core standards, which were also adopted for North Carolina beginning in the 20122013 academic year.

6. It is possible that the person in this focus group was referring to the ABC bonuses available to schools in North Carolina before RttT. However, these bonuses were also discontinued in 2009 out of fiscal necessity. It is therefore unlikely that any bonus had been available at the time the PLCs were established in this teachers school.

References

Alchian, A. A., & Demsetz, H. (1972). Production, information costs, and economic organization. American Economic Review, 62, 111195.

Ball, D. L., & Cohen, D. K. (1999). Developing practice, developing practitioners: Toward a practice-based theory of professional education. In G. Sykes & L. Darling-Hammond (Eds.), Teaching as the learning profession: Handbook of policy and practice (pp. 332). San Francisco, CA: Jossey-Bass.  

BEST NC. (2015). Education in North Carolina: Facts and figures January 2015. Retrieved from http://best-nc.org/wp-content/uploads/2015/02/EducationinNC-1.pdf

Boyd, D., Lankford, H., Loeb, S., Rockoff, J., & Wyckoff, J. (2008). The narrowing gap in New York City teacher qualifications and its implications for student achievement in high-poverty schools. Journal of Policy Analysis and Management 27, 793818.

Brookhart, S., & Freeman, D. J. (1992). Characteristics of entering teacher candidates. Review of Educational Research, 62, 3760.

Buck, S., & Greene, J. P. (2011). Blocked, diluted, and co-opted. EducationNext, 11(2). Retrieved from http://educationnext.org/blocked-diluted-and-co-opted/

Chetty, R., Friedman, J. N., & Rockoff, J. E. (2014a). Measuring the Impacts of Teachers I: Evaluating bias in teacher value-added estimates. American Economic Review, 104(9), 25932632.

Chetty, R., Friedman, J. N., & Rockoff, J. E. (2014b). Measuring the Impacts of Teachers II: Teacher value-added and student outcomes in adulthood. American Economic Review, 104(9), 26332679.

Chiang, H., Wellington, A., Hallgren, K., Speroni, C., Herrmann, M., Glazerman, S., & Constantine, J. (2015). Evaluation of the Teacher Incentive Fund: Implementation and impacts of pay-for-performance after two years (NCEE 2015-4020). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/pubs/20154020/pdf/20154020.pdf

Childcare Services Association. (2014). The early care and education workforce in North Carolina. Retrieved from http://www.childcareservices.org/2014-north-carolina-workforce-study/

Duflo, E., Hanna, R., & Ryan, S. P. (2012). Incentives work: Getting teachers to come to school. American Economic Review, 102, 12411278.

Eisenhardt, K. M. (1989). Agency theory: An assessment and review. The Academy of Management Review, 14, 5774.

Farkas, S., Johnson, J., & Foleno, T. (2000). A sense of calling: Who teaches and why. New York, NY: Public Agenda.

Finnigan, K. S., & Gross, B. (2007). Do accountability policy sanctions influence teacher motivation? Lessons from Chicago's low-performing schools. American Educational Research Journal, 44(3), 594629.

Fryer, R. G. (2013). Teacher incentives and student achievement: Evidence from New York City Public Schools. Journal of Labor Economics, 31, 373427.

Fryer, R. G., Levitt, S. D., List, J., & Sadoff, S. (2012). Enhancing the efficacy of teacher incentives through loss aversion: A field experiment (NBER Working Paper No. 18237). Cambridge, MA: National Bureau of Economic Research.

Gius, M. (2012). The effects of teacher merit pay on academic attainment: An analysis using district-level data. Journal of Economics and Economic Education Research, 13, 93108. Retrieved from http://www.alliedacademies.org/articles/the-effects-of-teacher-merit-pay-on-academic-attainment-an-analysis-using-districtlevel-data.pdf

Glazerman, S., & Seifullah, A. (2012). An evaluation of the Chicago Teacher Advancement Program (TAP) after four years. Washington, DC: Mathematica Policy Research.

Glewwe, P., Ilias, N., & Kremer, M. (2010). Teacher incentives. American Economic Journal: Applied Economics 2, 205227.

Gracey, H. L. (1975). Learning the student role: Kindergarten as academic boot camp. In H. R. Stub (Ed.), The sociology of education: A source book (pp. 8295). Homewood, IL: Dorsey Press.

Hanushek, E. (1992). The trade-off between child quantity and quality. The Journal of Political Economy, 100, 84117.

Hanushek, E., & Rivkin, S. E. (2004). How to improve the supply of high quality teachers. In D. Ravitch (Ed.), Brookings papers on education policy (pp. 725). Washington, DC: Brookings Institution Press.

Harris, D. N. (2009). Teacher value-added: Don't end the search before it starts. Journal of Policy Analysis and Management, 28(4), 693699.

Hatch, A. (2002). Doing qualitative research in education settings. Albany: State University of New York Press.

Henderson, M. B., Peterson, P. E., & West, M. R. (2016). The 2015 EdNext poll on school reform. EducationNext, 16(1). Retrieved from at http://educationnext.org/2015-ednext-poll-school-reform-opt-out-common-core-unions/

Hill, H. C. (2009). Evaluating value-added models: A validity argument approach. Journal of Policy Analysis and Management, 28(4), 700709.

Holmstrom, B. R., & Tirole, J. (1989). The theory of the firm. Handbook of Industrial Organization, 1, 61133.

Ingersoll, R., Merrill, L., & Stuckey, D. (2014). Seven trends: The transformation of the teaching force, updated April 2014 (CPRE Report No. RR-80). Philadelphia: Consortium for Policy Research in Education, University of Pennsylvania.

Kane, T. J., McCaffrey, D. F., Miller, T., & Staiger, D. O. (2013, January). Have we identified effective teachers? Validating measures of effective teaching using random assignment (Research Paper, MET Project). Seattle, WA: Bill & Melinda Gates Foundation.

Koedel, C., & Betts, J. R. (2011). Does student sorting invalidate value-added models of teacher effectiveness? An extended analysis of the Rothstein critique. Education Finance and Policy, 6(1), 1842.

Lavy, V. (2009). Performance pay and teachers effort, productivity, and grading ethics. American Economic Review, 99, 1979–2011.

Levačić, R. (2009). Teacher incentives and performance: An application of principal–agent theory. Oxford Development Studies, 37, 33–46.

Lofland, J., Snow, D., Anderson, L., & Lofland, L. H. (2006). Analyzing social settings (4th ed.). Toronto, Ontario, Canada: Wadsworth.

Murnane, R. J., & Cohen, D. K. (1986). Merit pay and the evaluation problem: Why most merit pay plans fail and a few survive. Harvard Educational Review, 56, 117.

Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004). How large are teacher effects? Educational Evaluation and Policy Analysis, 26, 237257.

Rice, J. K., & Malen, B. (2017). Performance-based pay for educators: Assessing the evidence. New York, NY: Teachers College Press.

Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools, and academic achievement. Econometrica, 73, 417458.

Rockoff, J. E. (2004). The impact of individual teachers on student achievement: Evidence from panel data. The American Economic Review, 94, 247252. Retrieved from http://www.jstor.org/stable/3592891

Rothstein, J. (2010). Teacher quality in educational production: Tracking, decay, and student achievement. Quarterly Journal of Economics, 125, 175214.

Sappington, D. E. (1991). Incentives in principal-agent relationships. The Journal of Economic Perspectives, 5(2), 4566.

Schipp, V. H. (1999). Factors influencing the career choices of African American collegians: Implications for minority teacher recruitment. Journal of Negro Education, 68, 343351.

Scholastic, Inc. & the Bill & Melinda Gates Foundation. (2012) Primary sources 2012: America's teachers on the teaching profession, a project of Scholastic and the Bill & Melinda Gates Foundation. New York, NY: Scholastic, Inc. Retrieved from http://www.scholastic.com/primarysources/pdfs/Gates2012_full.pdf

Springer, M. G., Ballou, D., Hamilton, L. S., Le, V., Lockwood, R., McCaffrey, D. F., . . . Stecher, B. M. (2010). Teacher pay for performance: Experimental evidence from the project on incentive in teaching. National Center on Performance Incentives at Vanderbilt University. Retrieved from http://www.rand.org/pubs/reprints/RP1416.html

Springer, M. G., Pane, J. F., Vi-Nhuan, L., McCaffrey, D. F., Burns, S. F., Hamilton, L. S., & Stecher, B. (2012). Team pay for performance: Experimental evidence from the Round Rock Pilot Project on team incentives. Educational Evaluation and Policy Analysis, 24, 367390.

Taylor, R. L., Hunnicutt, G. G., & Keeffe, M. J. (1991). Merit pay in academia: Historical perspectives and contemporary perceptions. Review of Public Personnel Administration, 11, 5165.

Watt, H. M. G., & Richardson, P. W. (2007). Motivational factors influencing teaching as a career choice: Development and validation of the FIT-Choice scale. The Journal of Experimental Education, 75(3), 167202.

West, K. L. (2014). New measures of teachers work hours and implications for wage comparisons. Association for Education Finance and Policy, 231263.

Williamson, O. E. (1975). Markets and hierarchies: Analysis and antitrust implications. New York, NY: Free Press.


Woessmann, L. (2011). Cross-country evidence on teacher performance pay. Economics of Education Review, 30, 404–418. Retrieved from http://dx.doi.org/10.1016/j.econedurev.2010.12.008

Yuan, K., Le, V. McCaffrey, D. F., Marsh J. A., Hamilton, L. S., Stecher, B. M., & Springer, M. G. (2013). Incentive pay programs do not affect teacher motivation or reported practices: Results from three randomized studies. Educational Evaluation and Policy Analysis, 35, 3–22.

Cite This Article as: Teachers College Record Volume 121 Number 2, 2019, p. 1-38 https://www.tcrecord.org ID Number: 22587, Date Accessed: 10/12/2021 12:52:40 PM Purchase Reprint Rights for this article or review