Principal Turnover Under No Child Left Behind Accountability Pressure

by Hajime Mitani - 2019

Background/Context: The No Child Left Behind (NCLB) Act was a performance-based accountability policy designed to motivate educators and administrators to change their behaviors and improve school and student outcomes. The simple logic behind this accountability policy was that they would change their behaviors to avoid sanctions. Many studies have investigated the impact of NCLB on students and teachers; however, little research has examined its impact on school principals even though they were a prime target of NCLB.

Purpose/Objective: This study fills the gap in the literature and investigates the impact of NCLB sanctions on principal turnover. It answers whether NCLB’s informal and formal sanctions influenced principal turnover behaviors and whether the influence was moderated by principal and school characteristics. It also examines patterns in principal transfers and position changes.

Research Design: This study uses longitudinal administrative data and detailed school-level assessment data obtained from Missouri Department of Elementary and Secondary Education from 2001–02 to 2009–10. It constructs a distance variable to the adequate yearly progress threshold and uses it as a key matching variable in propensity score matching to identify comparable schools not facing NCLB sanctions. Postmatching logistic regression models identify the impact of NCLB sanctions.

Findings/Results: Although I find no evidence that informal sanction affected principal turnover, the impact is significantly moderated by principals’ job experience, Title I school status, and the percent of non-White students. The first-year NCLB sanction does not appear to have affected principal turnover. However, this finding needs to be interpreted with caution because of the way the NCLB sanction system is structured and the small sample sizes. A descriptive analysis of the relationship between the second-year NCLB sanction and beyond, and principal turnover suggests that principals tend to leave their schools when they face NCLB sanctions. Finally, I find that principals transfer away from Title I schools, transfer to schools with a smaller number of high-needs students, and take positions at district central offices, regardless of whether they face NCLB sanctions. Collectively, NCLB sanctions appear to have impacted principal turnover.

Conclusions/Recommendations: The results from this study have policy implications. They suggest that policy makers should provide professional support and adequate resources for principals, especially inexperienced principals, who work at low-performing schools and face sanctions. Moreover, policy makers should develop and embed a policy in new accountability systems that addresses inequity in the distribution of principal quality.

The federal No Child Left Behind Act of 2001 (NCLB) was a performance-based accountability policy designed to motivate educators and administrators to change their behaviors and improve school and student outcomes. Schools faced sanctions when they did not meet adequate yearly progress (AYP) goals set by each state education agency. A logic behind this accountability policy was that educators and administrators would respond to sanctions.

A number of researchers have examined this assumption and, more broadly, the impact of NCLB. For example, studies have found that NCLB, in general, raised overall student performance and the performance among minority and disadvantaged students (e.g., Ballou & Springer, 2011; Dee & Jacob, 2011; Lauen & Gaddis, 2012; Reback, Rockoff, & Schwartz, 2014). Other studies have reported that teachers viewed the NCLB system negatively and claim that it increased the amount of pressure, which raised their job stress levels and, in some cases, led them to burn out or become demoralized (e.g., Berryhill, Linney, & Fromewick, 2009; Center on Education Policy, 2006; Dee, Jacob, & Schwartz, 2013; Mertler, 2011; Santoro, 2011; Sunderman, Tracey, Kim, & Orfield, 2004). A more recent empirical study found that although NCLB negatively influenced teachers perceptions of cooperation with their colleagues, it did not change their job satisfaction or commitment levels (Grissom, Nicholson-Crotty, & Harrington, 2014).

Although the primary focus of NCLB studies is on students and teachers, school principals were, in fact, a prime target of NCLB. They were required to make a number of important decisions to respond to NCLB sanctions, and, at a certain sanction stage, they were the target of replacement as a result of low performance. Anecdotal evidence suggests that NCLB made principalship more challenging and that many principals could not meet the growing demand of the law, which increased their job stress (Brown, 2006). Principals reported that high-stakes accountability systems in general created a culture of fear among school leaders (McGhee & Nelson, 2005). In some cases, NCLB pressure drove the principals to tamper with test scores (Barry, 2015; Berman, 2015; Gabriel, 2010). Survey reports show that a majority of principals viewed NCLB unfavorably and sometimes perceived it as an unfair system (Educational Testing Service, 2008; Lyons & Algozzine, 2006; Salazar, 2007). These stressful working conditions could have led to frequent turnover among principals, especially those working at low-performing schools, which had a higher probability of facing the NCLB sanctions (Ahn & Vigdor, 2014; Li, 2012; White & Agarwal, 2011).

Principal turnover, particularly frequent turnover within a school, has been found to cause conflict between the teachers and new principals, disruption of the instructional program, and changes in school vision, which are likely to delay and destabilize school reforms and may negatively impact teacher morale (Hargreaves & Fink, 2006; Meyer, Macmillan, & Northfield, 2009). As a result, principal turnover could cause a decline in student performance (Béteille, Kalogrides, & Loeb, 2012; Miller, 2013). Ensuring sustainable leadership is therefore crucial for school reform and improvement (e.g., Hargreaves & Fink, 2004, 2006; Hargreaves, Moore, Fink, Brayman, & White, 2003).

Despite compelling anecdotes and the importance of stability and sustainability in school leadership, few empirical studies have examined this relationship, and it remains unclear whether the NCLB sanctions affected principal turnover. This study fills this gap in the literature by analyzing longitudinal administrative data and detailed school accountability data that I obtained from the Missouri Department of Elementary and Secondary Education (DESE). More formally, I first examine whether NCLB sanctions affected principal turnover and then test whether the relationship is moderated by principals professional characteristics and characteristics of schools that they serve. Moderation effects might be present because these characteristics are likely to affect principals decisions. Finally, I investigate patterns in principals transfer behaviors and position changes because NCLB sanctions could cause principals to leave for different schools or change their positions to avoid future sanctions. Results from this study may encourage policy makers to reexamine new state accountability plans under the Every Student Succeeds Act and amend them if necessary.

This study proceeds as follows. The next section overviews the NCLB sanction system and specific features of AYP decision rules that the Missouri DESE applied. I then discuss a conceptual framework and review the studies on NCLB and principal turnover. Following that section, I describe data and methods used to answer my research questions, after which I report the results. I conclude with the implications and limitations.

NCLB SANCTION SYSTEM

GENERAL DESCRIPTION

NCLB required all public schools to make AYP in reading and math from Grades 3 to 8 and once at the high school level for the all-student group and all student subgroups that met a minimum subject-by-subgroup cell size requirement.¹ It also required the schools to meet one additional academic indicator requirement, such as attendance rate for elementary and middle schools and graduate rates for high schools. Annual measurable objectives (AMOs) for AYP were determined by each state education agency, and they were typically based on the percent of students performing at or above the state-defined proficient level on state assessments. Student subgroups included race and ethnicity, eligibility for the federal free/reduced lunch program, English language proficiency status, and disability status. In addition to the academic performance requirements, the law required schools to meet a 95% participation rate for all student groups.

If a school missed any of these requirements, it failed to make AYP. Missing AYP in the same subject or in the same academic indicator for two consecutive years placed the schools in the first year of the NCLB sanctions. It is important to note that only Title I schools were subject to these federal sanctions. Non-Title I schools were only required to make school improvement plans. Table 1 presents a summary of the sanction system. Although the first sanction started after missing AYP for two consecutive years, schools faced informal sanction when they missed AYP once, regardless of whether they were Title I schools. Informal sanction means that schools were exposed to public criticism and scrutiny over school activities because state education agencies released school report cards to the public every year. The sanction system allowed schools missing AYP to still make it through alternative routes, which include confidence interval, safe harbor, student growth model, and uniform averaging across years.²

Table 1. NCLB Sanction System

Number of consecutive years that school missed AYP in same subject in at the end of year t-1	Sanction imposed in the following school year (year t)
0	No sanction imposed.
1	No sanction imposed, but the failure is publicly announced.
2	School Improvement Year 1 (SIY 1). District must offer Public School Choice (PSC) with transportation to all students to transfer to another public school or charter school within the same district that has not been identified as "in need of improvement. It also needs to provide technical assistance.
3	School Improvement Year 2 (SIY 2). District must offer supplemental educational services (SES) to students from low-income families, in addition to PSC and technical assistance.
4	School Improvement Year 3 (SIY 3). District must take at least one of the following corrective actions on failing schools: (1) replace school staff, (2) implement a new curriculum (with appropriate professional development), (3) decrease management authority at the school level, (4) appoint an outside expert to advise the school, (5) extend the school day or year, and (6) reorganize the school internally. District continues to offer PSC, SES, and technical assistance.
5	School Improvement Year 4 (SIY 4). District must initiate plans to fundamentally restructure failing schools. It continues to offer PSC, SES, and technical assistance.
6	School Improvement Year 5 (SIY 5). District must implement the school restructuring plan. It includes one or more of the following actions: (1) reopen the school as a public or charter school, (2) replace all or most of the staff who are responsible for the failure to make AYP, (3) enter into a contract with an entity, such as private management company, to operate the school, (4) turn the operation of the school over to the state department of education, and (5) other major restructuring. District continues to offer PSC, SES, and technical assistance.
7 and beyond	School remains in restructuring until it meets AYP for two consecutive years.
Notes: If a district does not have PSC options because all schools are identified as "in need of improvement," it offers SES instead. Data source: U.S. Department of Education

IMPLEMENTATION OF THE NCLB SANCTION SYSTEM IN MISSOURI

The Missouri DESE used Missouri Assessment Program (MAP), MAP-Alternative Assessment (MAP-A), and End-of-Course (EOC) assessments for AYP determination (Missouri DESE, 2010).³ DESE set a proficiency cut score for each grade in each subject, on which its annual school-level proficiency target for every subject was based. Table 2 reports the annual measurable objectives from 200102 to 201314 in communication arts and math.⁴ The state required a minimum of 30 students in a subject-by-subgroup cell in order to determine AYP. This cell-size requirement was 50 for students with disabilities and English language learners until 200607. For the all-student group, if the number of students tested was lower than 30 and the school did not meet AYP in a given year, the state aggregated the number for the most recent three years to determine AYP (i.e., uniform averaging procedure). The state determined AYP even if the number was still under 30 after the three-year aggregation.

Table 2. Annual Measurable Objectives (AYP Targets) by Subjects in Missouri

Year	Communication Arts (%)	Math (%)
200102	18.4	8.3
200203	19.4	9.3
200304	20.4	10.3
200405	26.6	17.5
200506	34.7	26.6
200607	42.9	35.8
200708	51.0	45.0
200809	59.2	54.1
200910	67.4	63.3
201011	75.5	72.5
201112	83.7	81.7
201213	91.8	90.8
201314	100	100
Participation rate	All schools	95
Attendance rate	Elementary/middle schools	93
Graduation rate	High schools	85
Notes: These are school-level targets. The U.S. Department of Education started requiring additional academic indicators from 200506.

DESE used the 99% confidence interval, safe harbor, safe harbor with the 75% confidence interval, and student growth models to determine AYP.⁵ The confidence interval was applied to the actual percent of proficiency achieved. It was also applied to both the all-student group and student subgroups, whereas the safe harbor was applied to student subgroups only. No alternative route was applied unless the participation rate and cell size requirements were met. DESE was approved to use student growth models in 20072008 (Missouri DESE, 2010).    

It is important to note that DESE did not have its own sanction system independent of the NCLB sanction system. DESE used a single accountability system and followed the NCLB system. No schools, whether Title I schools or otherwise, received additional state sanctions. Thus, the influence of NCLB sanctions is not confounded with that of the states sanction system.

NCLB SANCTION SYSTEM AND PRINCIPAL TURNOVER

CONCEPTUAL FRAMEWORK

Principal turnover results from many factors. For instance, principals may leave their schools because their salaries are not competitive enough, or the salaries do not compensate for the tough job demands or poor working conditions. They may transfer to other schools because accountability pressure in their current schools is so high that it increases their job stress and dissatisfaction levels. Or, from the demand side, district administrators may relocate them to new schools for strategic reasons. To disentangle factors affecting principal turnover and simplify the analysis of the relationship between the NCLB sanctions and principal turnover, this study uses a labor market framework that is similar to one that Grissom and Andersen (2012) applied from the study of administrative turnover.

In this framework, both principals and district administrators are viewed as actors in the principal labor markets, and principal turnover results from a two-sided decision. From the principals perspectives, they make turnover decisions by evaluating their current jobs and comparing them with the alternative jobs available to them, including principal jobs at different schools, jobs at district central offices, and jobs outside the field of education. They calculate net benefits of staying in the current schools and compare them with those of leaving those schools. If net benefits of leaving schools exceed the net benefits of staying, they depart.

Similarly, from the district administrators perspectives, they determine whether to retain principals at current schools, reassign them to different schools, or terminate their job contracts by calculating the net benefits of each option. If net benefits of one option exceed those of the other, district administrators would choose to take the most beneficial option to them.

Situating principals and district administrators in the principal labor markets also raises a need to consider their personal and professional characteristics, the characteristics of the schools they serve, and the characteristics of school districts in which they work because these factors also play an important role in principal turnover. Prior studies have found that principals age, race/ethnicity, gender, professional experience, education level, and salary are associated with principal turnover (Akiba & Reicardt, 2004; Baker, Punswick, & Belt., 2010; Cullen & Mazzeo, 2008; Farley-Ripple, Solano, & McDuffie, 2012; Fuller & Young, 2009; Gates et al., 2006; Loeb, Kalogrides, & Horng, 2010; Solano, McDuffie, Farley-Ripple, & Bruton, 2010). They also suggest that student demographics, school enrollment size, school level, and school location are all correlated with principal turnover. The number of schools in a district is also an important factor because a larger district would have more job openings than the smaller ones (Farley-Ripple et al., 2012).

Although the two-sided nature of the turnover suggests the use of structural models, I use reduced-form models to estimate the relationship between the NCLB sanctions and principal turnover. These models do not differentiate between decisions made by the principals from decisions made by the district. As a result, estimates on the influence of NCLB sanctions on principal turnover could be driven by either actor. Later, I will descriptively explore whether principal turnover was driven by principals or district administrators through examining the characteristics of the new hires.

NCLB AND PRINCIPAL TURNOVER

The research base on school principals responses to the NCLB sanction system has been thin. Among the limited number of studies on NCLB and principal turnover, Li (2012) has examined whether NCLB affected school principals mobility by using longitudinal administrative data from North Carolina. She found that NCLB lowered the average quality of school principals among the schools serving students from underserved populations. Principals with higher value-added scores tend to transfer to the schools that are less likely to face NCLB sanctions. In other words, NCLB shifted the distribution of principal quality across schools by affecting the principal mobility and hiring patterns.    

Although the focus of their study was on student outcomes, Ahn and Vigdor (2014) examined the relationship between school restructuring under the NCLB sanction system and principal exit behaviors (i.e., leaving the public school system) by using similar data from North Carolina. They found that school restructuring raised the probability of principal exit by 618 percentage points. White and Agarwal (2011) found that, even after controlling for confounding factors, schools that failed AYP experienced higher principal turnover rates in Illinois.     

Studies on state and local accountability systems provide additional insights about principal turnover patterns under accountability pressure. For example, Cullen and Mazzeo (2008) examined principal turnover by using data from the state of Texas. They found that schools rated as low performing were 16.3 percentage points more likely to have different principals the following year when compared with schools that were rated as exemplary. Similarly, a different study that examined the distribution of principal quality in Miami-Dade County Public Schools found that principals at the schools that received D or F in Floridas accountability system were more likely to turn over (Loeb et al., 2010).

These studies generally suggest that principals might leave schools when they faced NCLB sanctions. Yet, principals responses to NCLB sanctions could be different by their professional qualifications and school characteristics. For example, experienced principals may possess a better set of skills to cope with accountability pressure, whereas inexperienced principals could struggle with the pressure and/or conflict with NCLB requirements. Because Title I schools were subject to formal NCLB sanctions, Title I school principals might feel stronger accountability pressure than nonTitle I school principals. Student demographics could also influence principal turnover because principals serving multicultural, low-income student populations typically face more difficult educational tasks than those serving predominantly White schools (Gardiner & Enomoto, 2006; Jencks & Phillips, 1998). All these factors could increase the accountability pressure and influence principals responses to it.

Conclusively, this review of the literature suggests that it may be equally important to investigate moderation effects by principal qualifications and school characteristics, along with examining the relationship between NCLB sanctions and principal turnover. My study answers these research questions by using a unique distance variable to the AYP threshold and, subsequently, provides new empirical evidence.   

DATA

This study relies on multiple data sources to investigate the research questions in the state of Missouri. The Missouri Department of Elementary and Secondary Education (DESE) operates 523 regular public school districts and 2,430 public schools that enroll approximately 913,000 students. It is a relatively rural state, with 40% of the schools located in rural areas, compared with 34% in a median state. Missouri is, however, nationally representative in other important school characteristics. It serves 22% Black or Hispanic students (with a median state being 25%) and 45% students who are eligible for the federal free/reduced lunch program (with a median state being 48%).⁶ Student performance in the state is close to that of a median state. The average student test score for the state on the National Assessment of Educational Progress is within 2 points of the national median on fourth- and eighth-grade math and reading tests.      

The primary data source is a longitudinal administrative data file obtained from DESE, which includes all personnel employed in the state education system between 199394 and 200910. Although all the analyses but one in this study are based on the data from 200102 to 200910, I use data before 200102 to construct a variable in order to measure years of principal experience. The data file includes information about age, gender, race/ethnicity, highest degree, and job/position code, which allowed me to identify school principals, position full-time equivalency, years of experience as an educator in the state, name of the undergraduate institutions attended, and annual salary. The annual salary was converted to 2009 constant dollars by using consumer price index for all urban customers (CPI-U) from the Bureau of Labor Statistics and adjusted for regional differences in the cost of living by using the Comparable Wage Index.⁷ I converted the adjusted annual salary into salary that is relative to the average salary in a labor market where a principals school is located after controlling for principal and school characteristics. Information on undergraduate institutions was matched to Barrons ratings of college selectivity in order to measure the principals academic qualifications.⁸ From this information, I create a variable that indicates whether a principal attended a competitive undergraduate institution. The data file does not include information about years of experience as a principal or the length of tenure as a principal in the current school. I create these variables for principals I observed entering the principalship after 199293. Using the administrative data file, I create a binary turnover variable that indicates whether a principal returned to his or her school next year in the same position.

These administrative data were merged with the data on school characteristics obtained from the Common Core of Data (school level) between 200102 and 200910, maintained by the National Center for Education Statistics (NCES). These data include school locale (i.e., urban, suburban, town, and rural), student demographics (i.e., percent of non-White students, and percent of students eligible for the federal free/reduced lunch program), school level, and school enrollment size. I calculated the number of schools per school district. In addition, I merged these data with labor market information in Missouri that was obtained from the Comparative Wage Index data file, maintained by the NCES.⁹

I obtained detailed school-level assessment data used for AYP determinations from DESE, the Barnard/Columbia No Child Left Behind Database, 200203 and 200304 (Reback, Rockoff, Schwartz, & Davidson, 2011), and the National Longitudinal School-Level State Assessment Score Database (NLSLSASD) created by the American Institutes for Research. DESEs accountability data file includes data on the number of students accountable, the number of students tested, the number and percent of students at or above the proficient level, attendance rates, graduation rates, and AYP results by subjects and subgroups, from 200405 to 200910. Using subject-by-subgroup AYP results and the information on additional academic indicators, I create a binary variable for school-level AYP results from 200405 to 200607. For years from 200708 to 200910, school-level AYP results are available from a school improvement status data file that I obtained from DESE.

The Barnard/Columbia No Child Left Behind Database includes data on the number and percent of students tested, the percent of students at or above the proficient level, and AYP results by subjects and subgroups in 200304. It also includes school-level AYP results. Although the database includes accountability data for 200203, it does not include the data on the number of students tested as per subjects and subgroups, which prevents me from creating a variable to measure a distance to making AYP (see the methods section). Fortunately, the NLSLSASD includes detailed school-level assessment data by subjects and subgroups, which include the number of students tested. Thus, for 200203, I use their data and merge them with AYP data from the Barnard/Columbia database. Similarly, for 200102, I use the NLSLSASD data. AYP results for that year were directly downloaded for each school from DESEs AYP Grid website.

I obtained a list of schools in school improvement status from DESE in 200809. For years from 200304, which was the first year that the schools faced NCLB sanctions, to 200708, I collected a list of schools from DESEs websites and the Consolidated State Performance Report published on the U.S. Department of Education website. The appendix summarizes the data sources for school-level assessment data.

METHODS

CONSTRUCTION OF THE DISTANCE VARIABLE

Identifying the effect of NCLB sanctions on principal turnover is challenging because schools that faced the sanctions could be systematically different from those that did not face them, in both observable and unobservable ways. An important step is to identify comparable schools that did not face the sanctions but were similar to schools that faced them. Standard regression-based techniques will bias the estimates to the extent that unobservable factors such as parental involvement, community support, and school culture are associated with both NCLB sanction status and principal turnover. One way to find comparable schools would be to measure the distance to make AYP. Theoretically, if two schools, one making AYP and the other not, are located at similar positions in the distribution of the distance to the AYP threshold and have similar observable school and principal characteristics, they would be statistically identical. The difference in AYP status between them would be due largely to the AYP determination rules not incorporated into the construction of the distance variable, petitions filed by schools, and/or other idiosyncratic factors. I use this variation to identify the effect of NCLB sanctions on principal turnover.

The idea of the distance variable comes from recent studies that used regression discontinuity designs to estimate local average treatment effects of the NCLB and its sanctions (e.g., Ahn & Vigdor, 2014; Chakrabarti, 2014; Fruehwirth & Traczynski, 2013; Hemelt, 2011). I modify a distance variable employed by Fruehwirth and Traczynski (2013) and use it as one of the matching covariates in a propensity score matching (PSM) method to identify the comparable schools.¹⁰

Following Fruehwirth and Traczynski (2013), I create a distance variable for each school in each year in the following way. First, I compute the number of students in each subject-by-student cell group, including the all-student group that had to perform at or above the proficient level for the school to make AYP, incorporating safe harbor and safe harbor with the 75% confidence interval. I also calculate the number of students in each subject-by-student cell group that actually performed at or above the proficient level, incorporating the 99% confidence interval, the student growth models, and uniform averaging. Next, I subtract the first number from the second number. If the difference is all positive across subjects and subgroups, I aggregate the differences at the school level. If it is negative in any subjects in any subgroups, I aggregate the negative differences across the subjects and subgroups for each school. As a final step, I convert this measure into a percentage measure by dividing it by the total number of students tested in each subject-by-subgroup cell that exceeds the minimum cell size requirements set by DESE.¹¹ I use this percentage as the distance to make AYP. Theoretically, when a school makes AYP, the distance variable is 0 or positive; otherwise, it is negative. In practice, however, there are some false positives (i.e., missing AYP but the variable is positive) because the distance measure does not incorporate other AYP requirements, such as participation rates, attendance rates, and/or graduation rates. There are also some false negatives (i.e., making AYP but the variable is negative) due to the appeal processes that might have been in process while data were collected, exceptional decisions made by DESE, and other idiosyncratic factors.

PROPENSITY SCORE MATCHING PROCEDURES

The distance variable plays a critical role in identifying the comparable schools except AYP or sanction status. However, it alone is not sufficient to do so because these schools could be still different in other dimensions. To improve the matching quality, I use a PSM method. This matching is based on year t or a baseline year because the actual treatment is given next year (year t+1). Figure 1 shows when the matching is performed and when the treatment (i.e., sanction) is given.

Figure 1. Implementation of the propensity score matching

First, I estimate the probability of missing AYP in a baseline year or the propensity scores as a function of characteristics of principals, schools, and districts through standard logistic regression techniques. More formally, I estimate the following logistic regression model:

(1)  

where

The probability that a school s where a principal i works misses AYP in a baseline year t is a function of principal characteristics P_isdt (i.e., age, female, race/ethnicity, selectivity of undergraduate institutions that a principal attended, holding an education specialist or doctorate degree, years of principal experience, years in current school, years in the Missouri education system, and relative salary), school characteristics S_sdt (i.e., distance to making AYP, urbanicity, school level, school enrollment size, percent of non-White students, percent of students eligible for the federal free/reduced lunch program, and Title I school status), district characteristics D_dt (i.e., number of schools), and a random error term ε_isdt. Principal characteristics may not be directly associated with the schools’ AYP status. Yet, because they affect turnover or the outcome variable, I include them to increase the precision of the estimates (Brookhart et al., 2006; Stuart, 2010). The baseline year varies from principal to principal because it is the year when the school missed AYP. The last year of the baseline year is 200708 because the treatment year becomes 200809 in this case, and I need administrative data for 200910 (which is the last year of the data) to identify whether a principal left at the end of 200809.

After estimating the propensity score for each principal, I perform one-to-five nearest neighbor matching within the common support region with replacement.¹² I set a caliper width at a quarter (0.25) of the standard deviation of the logit of the propensity scores (Austin, 2011; Rosenbaum & Rubin, 1985). I assess the matching quality based on standardized bias (or difference) in the covariates between the treatment and comparison groups and use 25% or below as a criterion to determine whether the covariates are balanced (Harder, Stuart, & Anthony, 2010; Rubin, 2001; Stuart, 2010).¹³ If any one of the covariates exceeds this threshold, I include a combination of various forms of variables (e.g., quadratic and cubic forms) and/or the interaction terms among the covariates exceeding the threshold until all standardized biases become below the threshold.

Based on this set of matched schools, I estimate the probability of principal turnover at the end of the treatment year, year t+1, using an indicator variable for the sanction status and the same set of the covariates used in Equation 1, except that I drop the distance variable to avoid the issue of statistical overcontrol. Year fixed effects and labor market region fixed effects are also added to the equation.¹⁴ The model takes the following form:

              (2)

where

The probability that a principal i in school s in district d in the treatment year, year t+1, turns over at the end of year t+1 is a function of NCLB sanctions Sanction_sdt+1, baseline principal and school characteristics X_isdt, S_sdt, year fixed effects γ_t, labor market region fixed effects π_t, and a random error term ε_isdt+1. If the odds ratio on the sanction variable is equal to or greater than 1, that indicates that principals were more likely to leave their schools when they faced sanctions.

For my entire analysis, I exclude those principals who did not stay in the same schools in the baseline year and the treatment year because the PSM is based on the baseline year. To estimate the effect of each sanction stage, I first restrict the sample to the principals working at schools in a given sanction stage in the baseline year. Then, I perform the PSM described earlier. The treatment group faces the next stage sanction in the treatment year; the control group does not. Because of small sample sizes in later sanction stages, I focus on the effect of informal sanction and School Improvement Year 1 (SIY 1). Moderation effects are tested through interaction terms between the sanction status variable and moderators.

RESULTS

EFFECT OF INFORMAL SANCTION/SIY 1 ON PRINCIPAL TURNOVER

For this research question, I examine two different samples. For informal sanction, my sample includes both Title I and nonTitle I school principals. The sample for SIY 1 only includes Title I school principals. I first take a descriptive look at the prematching samples of principals. Table 3 reports the descriptive statistics on characteristics of principals, schools, and districts by sanction status. The first set of four columns shows descriptive statistics of principals who did not face a sanction in the baseline year. I divide this sample into two groups: principals facing no sanction (comparison group) and principals facing information sanction (treatment group) in the treatment year. The second set of columns displays the descriptive statistics of principals who faced informal sanction in the baseline year. Like the first sample, I divide the second sample into two groups: principals facing no sanction (comparison group) and principals facing SIY 1 (treatment group) in the treatment year. In each sample, characteristics of treatment and comparison groups are compared by using a series of t tests.  

The table shows that these two groups of principals are quite different in many characteristics, especially their school characteristics in both samples. For instance, I observe a notable difference in the distance variable between the two groups in both samples. Many principals facing the sanction tend to work at urban schools, secondary schools, larger schools, and schools enrolling a large number of students from underserved populations. These differences highlight the importance of the use of PSM to balance the differences between these two groups of principals.    

Table 3. Descriptive Statistics

	No sanction in the baseline year				Informal sanction in the baseline year
	No sanction treatment year		Informal sanction treatment year		No sanction  treatment year		SIY 1 treatment year
Variable	N	Mean	N	Mean	N	Mean	N	Mean
Principal characteristics
Age	4,577	46.4	3,424	47.8***	323	47.7	408	48.7*
Female	4,577	0.56	3,424	0.42***	323	0.64	408	0.56**
Black	4,577	0.05	3,424	0.16***	323	0.16	408	0.28***
Other non-White	4,577	0.01	3,424	0.00	323	0.00	408	0.00
Attended selective undergraduate institutions	4,553	0.17	3,416	0.20**	322	0.17	408	0.15
Education specialist or doctoral degree	4,577	0.38	3,424	0.40*	323	0.40	408	0.38
Total years of principal experience	4,577	6.29	3,424	6.17	323	6.58	408	6.30
Years in current school as principal	4,577	4.66	3,424	4.77	323	4.94	408	4.81
Total years of experience in education	4,577	18.35	3,424	19.43***	323	19.12	408	19.52
Relative salary ratio	4,541	1.01	3,396	1.03***	321	1.00	407	1.00
School characteristics
Distance to making AYP (percent)	4,569	26.76	3,415	3.78***	323	24.63	408	3.21***
Urban	4,577	0.15	3,424	0.24***	323	0.22	408	0.35***
Suburban	4,577	0.29	3,424	0.30	323	0.25	408	0.24
Town	4,577	0.11	3,424	0.17***	323	0.15	408	0.14
Rural	4,577	0.44	3,424	0.29***	323	0.38	408	0.27***
Elementary school	4,577	0.72	3,424	0.32***	323	0.74	408	0.62***
Middle school	4,577	0.11	3,424	0.30***	323	0.14	408	0.25***
High school	4,577	0.16	3,424	0.34***	323	0.11	408	0.13
Other grade configuration	4,577	0.01	3,424	0.04***	323	0.01	408	0.01
Title I school	4,577	0.61	3,423	0.42***	NA	NA	NA	NA
Percent non-White students	4,577	15.27	3,403	30.54***	323	29.36	408	46.07***
Percent free/reduced lunch eligible students	4577	39.60	3403	45.97***	323	54.84	408	60.19***
School enrollment size	4577	392	3404	617***	323	389	408	462***
District characteristics
Number of schools	4577	14.84	3424	23.76***	323	19.93	408	34.03***
Notes: Data include years from 200102 to 200708. Principals who did not stay in the same schools in either the baseline year or the treatment year are not included. The last baseline year is 200708 because the treatment year is 200809 in this case, and I need data on 200910, the last year of the data, to identify whether principals turned over at the end of 200809. Both Title I and nonTitle I schools are included for the first sample; only Title I schools are included in the second sample. The means of these variables are compared using a series of t tests. *p < 0.10. **p < 0.05. ***p < 0.01.

Figure 2 descriptively shows principal turnover rates in the same way as Table 3 by the treatment status in the prematching sample. In the first sample, 19% of principals facing no sanction left their schools, whereas 25% of them facing informal sanction departed from their schools. The difference is statistically different from zero. On the other hand, in the second sample, I did not observe a notable difference between the two groups.

Figure 2. Turnover rates by sanction status

Notes: Data include years from 200102 to 200708. The first sample includes all principals who did not face any sanction in the baseline year, whether they worked at Title I or nonTitle I schools. The second sample is limited to principals working at Title I schools who faced informal sanction in the baseline year. Principals who did not stay in the same schools in either the baseline year or the treatment year are not included. The last baseline year is 200708 because the treatment year is 200809 in this case, and I need data on 200910, the last year of the data, to identify whether principals turned over at the end of 200809. The means are compared using a series of t tests.
* p < 0.10. ** p < 0.05. *** p < 0.01.

Next, I performed the PSM and checked the covariate balance between the two groups in the two samples. Table 4 reports results from the balance tests. The results are displayed in the same way as Table 3. The first two columns of each sample show the means of the covariates for the comparison and treatment groups. The third column displays the standardized biases. All the differences fall within a range from −25 to 25, which suggests that the PSM achieved a balance in all the covariates.

Table 4. Balance Test Results for the Sample of Principals

	No sanction in the baseline year			Informal sanction in the baseline year
Variable	No sanction treatment year	Informal sanction treatment year	Standardized bias (%)	No sanction treatment year	SIY1 treatment year	Standardized bias (%)
Principal characteristics
Age	46.1	47.5	-16.0	47.0	47.9	-11.2
Female	0.47	0.55	-16.5	0.55	0.60	-10.6
Black	0.10	0.06	14.0	0.20	0.20	1.1
Other non-White	0.00	0.00	1.3	0.00	0.00	.
Attended selective undergraduate institutions	0.19	0.24	-13.5	0.17	0.26	-23.2
Education specialist or doctoral degree	0.38	0.45	-13.9	0.47	0.47	0
Total years of principal experience	5.42	6.07	-17.3	5.78	6.31	-14.2
Years in current school as principal	3.94	4.52	-21.3	4.13	4.01	4.3
Total years of experience in education	18.03	18.76	-9.3	18.21	19.71	-18.8
Relative salary ratio	1.02	1.04	-14.1	0.99	1.01	-18.9
School characteristics
Distance to making AYP (percent)	5.89	7.40	-11.5	7.01	7.50	-3.8
Urban	0.19	0.17	5.8	0.25	0.22	6.5
Town	0.17	0.11	19.2	0.16	0.08	20.9
Rural	0.36	0.31	11.4	0.35	0.39	-9.8
Middle school	0.24	0.16	21.8	0.24	0.16	20.9
High school	0.27	0.29	-6.2	0.12	0.08	15.3
Other grade configuration	0.02	0.01	6.0	0.01	0.03	-21.1
Title I school	0.50	0.45	10.6	NA	NA	NA
Percent non-White students	22.00	22.61	-2.5	34.80	35.05	-0.7
Percent free/reduced lunch eligible students	43.92	38.89	22.5	56.65	55.13	6.6
School enrollment size	501	571	-22.7	443	397	21.6
District characteristics
Number of schools	18.16	17.10	4.4	23.16	20.30	8.3
Others
Interaction terms/different forms of variables	NA	NA	NA	NA	NA	<|25|
Notes: Data include years from 2002 to 2008. Principals who did not stay in the same schools in either the baseline year or the treatment year are not included. The last baseline year is 200708 because the treatment year is 200809 in this case, and I need data on 200910, the last year of the data, to identify whether principals turned over at the end of 200809. Both Title I and nonTitle I schools are included for the first sample; only Title I schools are included in the second sample. NA means (1) that mean calculations are not applicable, (2) that interaction terms or different forms of variables are not included in the model, or (3) that the variable is not included in the model.

Now, I turn to estimation results. Panel A of Table 5 reports results from logistic regression of the probability that a principal turns over at the end of the treatment year. Odds ratios and standard errors are reported. It shows that, although the main effect is insignificant, the relationship is significantly moderated by Title I school status. The predicted probability of turnover among Title I school principals is 23% when they face informal sanction, whereas it is only 16% when they do not. The sign of inequality flips among nonTitle I school principals. The predicted probability is 27% when they do not face informal sanction, but it decreases to 24% when they face it.¹⁵ I did not observe a moderation effect by high school. Panel B displays results for the relationship between SIY 1 and turnover. I found no evidence that SIY 1 affects principal turnover.¹⁶

Table 5. Relationship Between Informal Sanction and SIY 1 and Principal Turnover

Panel A: Informal sanction	Model 1	Model 2	Model 3
Facing informal sanction	1.12	0.84	1.06
	(0.23)	(0.23)	(0.24)
Sanction × Title I		1.94**
		(0.65)
Sanction × high school			1.22
			(0.53)
Year fixed effects	Yes	Yes	Yes
Labor region fixed effects	Yes	Yes	Yes
Observation	2074	2074	2074
Pseudo R-squared	0.10	0.11	0.10
Panel B: SIY 1	Model 4	Model 5
Facing SIY 1	0.94	0.90
	(0.36)	(0.37)
Sanction × high school		1.41
		(1.59)
Year fixed effects	Yes	Yes
Labor region fixed effects	Yes	Yes
Observation	384	384
Pseudo R-squared	0.32	0.32
Notes: New principals in current schools are excluded from the analysis. All models include characteristics of principals, schools, and districts. For Panel B, the sample includes Title I school principals only. Schools whose SIY 1 status is delayed are not included. Odds ratios are reported. Standard errors are clustered at the district level and reported in parentheses. *p < 0.10. **p < 0.05. ***p < 0.01.

Next, I examined moderation effects by principal qualifications and student demographics. Because I did not find any notable moderation effects by college selectivity, the highest degree level, or percent of FRL students, I focused on the total years of principal experience and the percent of non-White students. Table 6 reports results. In Panel A, I found strong evidence that the effect of informal sanction is negatively moderated by years of principal experience and positively moderated by the percent of non-White students.

Table 6. Moderation Effects by Years of Principal Experience and Percent of Non-White Students Informal Sanction and SIY 1

Panel A: Informal sanction	Years of principal experience		Percent non-White students
	Model 1	Model 2	Model 3	Model 4
Facing informal sanction	2.62**	2.75***	0.80	0.89
	(1.05)	(0.88)	(0.19)	(0.19)
Sanction × yrs prin exp / pct non-White	0.89**		1.02***
	(0.05)		(0.01)
Sanction × yrs prin exp (46 years) / pct non-White (25%50%)		0.30***		2.05
		(0.13)		(1.19)
Sanction × yrs prin exp (79 years) / pct non-White (50%75%)		0.64		6.89***
		(0.33)		(3.98)
Sanction × yrs prin exp (10 years or longer) / pct non-White (75%100%)		0.18***		3.51***
		(0.09)		(1.34)
Year fixed effects	Yes	Yes	Yes	Yes
Labor region fixed effects	Yes	Yes	Yes	Yes
Observation	2074	2074	2074	2074
Pseudo R-squared	0.11	0.13	0.11	0.11
Panel B: SIY 1	Years of principal experience		Percent non-White students
	Model 5	Model 6	Model 7	Model 8
Facing SIY 1	0.84	1.02	1.10	1.03
	(0.63)	(0.65)	(0.51)	(0.46)
Sanction × yrs prin exp / pct non-White	1.02		1.00
	(0.10)		(0.01)
Sanction × yrs prin exp (46 years) / pct non-White (25%-0%)		1.73		2.20
		(1.75)		(2.36)
Sanction × yrs prin exp (79 years) / pct non-White (50%75%)		0.62		0.01***
		(0.53)		(0.02)
Sanction × yrs prin exp (10 years or longer) / pct non-White (75%100%)		0.77		1.08
		(0.69)		(0.96)
Year fixed effects	Yes	Yes	Yes	Yes
Labor region fixed effects	Yes	Yes	Yes	Yes
Observation	384	384	384	384
Pseudo R-squared	0.32	0.33	0.32	0.35
Notes: New principals in current schools are excluded from the analysis. All models include characteristics of principals, schools, and districts. For Panel B, the sample includes Title I school principals only. Schools whose SIY 1 status is delayed are not included. Odds ratios are reported. Standard errors are clustered at the district level and reported in parentheses. *p < 0.10. **p < 0.05. ***p < 0.01.

To ease the interpretation of these results, I plotted the predicted probability of turnover, which is based on Models 2 and 4 by sanction status over the four principal experience/percent non-White student categories in Figures 3 and 4, respectively. Principals with two to three years of principal experience are about 13 percentage points more likely to leave their schools when they face informal sanction, compared with their colleagues with the same experience level but not facing the sanction (24% and 11%, respectively). In the last experience category, on the other hand, principals facing informal sanction are about 15 percentage points less likely to turn over than those not facing it (27% and 42%, respectively). As Table 6 indicates, the difference in the differences between the two experience categories (i.e., about 28 percentage points) is statistically significant.

Figure 3. Predicted principal turnover by years of principal experience

Figure 4. Predicted principal transfer by percent of non-White students

Figure 4 displays a stark moderation effect. The difference in the predicted probability of turnover between principals facing the informal sanction and those not facing it widens as the percent of non-White students increases. At the lowest level, the difference is about negative 2 percentage points and statistically insignificant; the sign of the difference flips in the next non-White category, and the difference increases to about 7 percentage points (19% and 12%, respectively). The difference jumps to about 17 percentage points in the next two levels. This suggests that informal sanction affects principals serving a larger number of non-White students more strongly than those serving a small number of such students.¹⁷ In contrast, in Panel B, I found little evidence on the moderation effect.

FALSIFICATION TEST

All the analyses so far assume that the PSM method and the subsequent postmatching logistic regression models account for bias because of potential endogeneity in the sanction status variable. However, it is still possible that the variable is correlated with other unobservable factors, which are also associated with turnover. For example, if these omitted variables are positively correlated with the sanction status variable and negatively correlated with the turnover variable, the estimates are biased downward. If this is the case, the null results for the main analyses may be misleading.

To test this possibility, I perform a falsification test, which examines whether the sanction status variable in post-NCLB years predicts principal turnover behaviors in the pre-NCLB years. Theoretically, NCLB sanction cannot predict turnover behaviors in the pre-NCLB years. However, if there are omitted variables, the test will show the significant relationship between the sanction status and principal turnover in the pre-NCLB years. This suggests that the sanction status is endogenous, and the main results are misleading. I conduct this test only for the main analyses.

For this test, I use schools in each matched sample and predict principal turnover as a function of the sanction status in the post-NCLB years; characteristics of principals, schools, and districts in the pre-NCLB years from 199394 to 19992000; year fixed effects; and labor market region fixed effects.¹⁸ Most of the principals in these years would not be the same as those in the post-NCLB years. I found little evidence that the sanction status, whether informal sanction or SIY 1, predicts the turnover outcomes in the pre-NCLB years, alleviating some concern that the sanction status variable is endogenous (results not reported).      

ANALYSIS OF PRINCIPAL TRANSFERS AND POSITION CHANGES

My analysis, so far, focused on whether the NCLB sanction affects principal turnover. This subsection takes a closer look at the patterns in principal transfer and position changes. NCLB sanction may have influenced principals transfer patterns and career trajectories in a systematic way.

Transfer Patterns

As discussed in the conceptual framework section, when principals choose whether to move to different schools, they perform costbenefit analyses by comparing current schools with future schools in terms of both unobservable and observable characteristics. Principals may prefer to work at schools with certain characteristics, particularly those with a lower probability of missing AYP or facing a sanction. Although my empirical models cannot incorporate future job characteristics, I can provide descriptive information about the principals transfer behaviors by examining the characteristics of sending schools and receiving schools that are potentially associated with AYP results and testing whether the differences are statistically indistinguishable from zero. I examine changes in the following school characteristics: distance to making AYP, percent of students eligible for the free/reduced lunch program, percent of non-White students, school enrollment size, Title I status, and school level. For this analysis, I focus on two groups of principals who did not face any sanction in the baseline year because of the sample size issue: (1) principals who did not face sanction in the treatment year, and (2) principals who faced informal sanction during that year. The left side of Table 7 reports results for the first group; the right side reports results for the second group. The last two columns show differences in the differences between the two groups and associated p values. Panel A includes all principals; Panel B includes Title I school principals. Because sample sizes are small, which yield relatively large p values, I do not discuss the results from t tests.

Table 7. Changes in School Characteristics Before and After Transfer by Sanction Status (No Sanction and All Informal Sanction)

Panel A: All schools	No sanction in the baseline year
	No sanction in the treatment year					Informal sanction in the treatment year
	N	Sending schools	Receiving schools	Diff. (R-S)	p value	N	Sending schools	Receiving schools	Diff. (R-S)	P value	D informal - D nosanction	P value
Distance to making AYP	47	18.85	19.57	0.72	0.82	25	14.75	15.01	0.26	0.95	-0.46	0.93
Percent low-income students	55	42.31	37.02	-5.29	0.09	27	49.41	45.78	-3.62	0.29	1.67	0.72
Percent non-White students	57	15.75	17.49	1.74	0.52	34	17.63	20.33	2.70	0.28	0.96	0.77
School enrollment size	55	382	540	158	0.00	27	393	485	93	0.16	-66	0.34
Title I schools (percent)	57	0.58	0.46	-0.12	0.11	34	0.56	0.44	-0.12	0.29	0.01	0.97
Elementary/middle schools	57	0.81	0.82	0.02	0.66	34	0.76	0.76	0.00	1.00	-0.02	0.78
High schools	57	0.18	0.16	-0.02	0.57	34	0.21	0.24	0.03	0.66	0.05	0.52
Panel B: Title I schools	No sanction in the baseline year
	No sanction in the treatment year					Informal sanction in the treatment year
	N	Sending schools	Receiving schools	Diff. (R-S)	p value	N	Sending schools	Receiving schools	Diff. (R-S)	p value	D informal - D nosanction	P value
Distance to making AYP	26	20.17	21.08	0.91	0.84	13	13.90	17.56	3.66	0.50	2.75	0.69
Percent low-income students	29	48.16	37.29	-10.88	0.02	15	57.40	47.39	-10.00	0.05	0.87	0.89
Percent non-White students	29	22.13	22.54	0.41	0.91	20	25.84	28.31	2.47	0.54	2.06	0.71
School enrollment size	29	419	495	76	0.11	15	399	469	69	0.41	-6	0.94
Title I schools (percent)	29	1.00	0.62	0.38	0.00	20	0.95	0.50	0.45	0.00	-0.07	0.61
Elementary/middle schools	29	1.00	1.00	0.00	1.00	20	0.90	0.85	0.05	0.58	0.05	0.57
High schools	29	0.00	0.00	0.00	1.00	20	0.10	0.15	-0.05	0.58	-0.05	0.57
Notes: School characteristics for sending schools are based on the data in the treatment year; school characteristics for receiving schools are based on data for next year. Principals without valid data for both receiving and sending schools are excluded. Principals who did not stay in the same school in either the baseline year or the treatment year are not included. I used bivariate regressions for continuous variables to perform statistical tests. For the binary variables, I used paired t tests. *p < 0.10. **p < 0.05. ***p < 0.01.

I observed three important patterns. First, whether they face informal sanction or not, principals are more likely to transfer away from Title I schools. For instance, in Panel A, among all principals who faced informal sanction and transferred at the end of the treatment year, 56% worked at Title I schools before transfer. The percent declines to 44% after transfer. Similarly, in Panel B, among Title I school principals facing the sanction, 95% of them worked at Title I schools before transfer¹⁹; it declines to 50% after transfer. It is clear that principals tend to avoid Title I schools. Along with this move, principals tend to transfer to schools that serve a smaller number of students who are eligible for the federal free/reduced lunch program. On the other hand, the percent of non-White students does not change very much before and after transfer. Third, principals do not appear to be very concerned about the distance to the AYP threshold. Changes in the distance variable are minimal, whether principals face the informal sanction or not and whether they work at Title I schools or not. Overall, I did not observe notable differences in the transfer patterns between those facing informal sanction and those not facing it.

Patterns in Position Changes

Next, I examine what kind of positions principals took after they faced informal sanction and whether the patterns in their position changes are systematically different from those who did not face the sanction but changed their positions. I focus on the same two groups of principals as in the transfer analysis. Table 8 displays results by sanction status.

Table 8. Position Principals Took by Sanction Status (No Sanction and All Informal Sanction)

	No sanction in the baseline year
Position	No sanction in the treatment year	Informal sanction in the treatment year	Total
Principal	18.52	20.59	19.52
Assistant principal	8.33	12.75	10.48
Teacher	12.04	14.71	13.33
Central office	42.59	39.22	40.95
Other school administrator	12.04	6.86	9.52
Supervisor	4.63	2.94	3.81
Other	1.85	2.94	2.38
Total	100	100	100
Chi-squared proportion test: Chi-squared = 3.6447, p = 0.725
Notes: If the full-time equivalency changes from 0.75 or above to below 0.75, I treat it as a position change.

 

A chi-squared proportion test shows that there was no significant relationship between the sanction status and types of positions that the principals took. However, there are some notable patterns. First, principals tended to take positions at the district central office, regardless of whether they faced informal sanction. For instance, about 43% of principals who did not face the sanction took central office jobs; about 40% did so among those facing the sanction. Second, about 1 in 5 principals changed their positions full-time equivalency from 0.75 or above to below 0.75. This means that their job duties were reduced and/or that they were assigned to other jobs at the same time. Third, principals facing the informal sanction tended to become assistant principals or classroom teachers, when compared with those not facing it (27% and 20%, respectively).

DISTINGUISHING VOLUNTARY AND INVOLUNTARY TURNOVER

All turnover models used for the analyses are reduced form models, and they do not distinguish between voluntary and involuntary turnover. Yet, it is important to know whether turnover was initiated by principals or district administrators because it affects the interpretation of the results. To test whether turnover was systematically initiated by the principals, I descriptively compared professional characteristics of principals who left their schools and those of new hires. If turnover was initiated by district administrators, I would expect that principal qualifications systematically improved after turnover. I examined the following principal qualifications: college selectivity, highest degree attained, and years of principal experience. Table 9 reports results. Panel A displays results for informal sanction and Panel B for SIY 1. The first set of three columns in each panel compares principal qualifications for schools that did not face informal sanction or SIY 1. The second set compares the same qualifications for those schools facing them. The last column reports the differences in the differences and indicates whether the differences are statistically significant.

Table 9. Descriptive Analysis of Changes in Principal Qualifications Before and After Turnover

Panel A: Informal sanction	Not facing informal sanction			Facing informal sanction
	Leaving principal	New principal	Difference	Leaving principal	New principal	Difference	D informal - D nosanction
College selectivity	0.18	0.17	-0.01	0.23	0.18	-0.05	-0.04
Highest degree attained	0.43	0.29	-0.14***	0.38	0.26	-0.12**	0.01
Years of principal experience	6.64	2.61	-4.02***	6.73	2.82	-3.90***	0.12
Panel B: SIY 1	Not facing SIY 1			Facing SIY 1
	Leaving principal	New principal	Difference	Leaving principal	New principal	Difference	D SIY 1 - D nosanction
College selectivity	0.20	0.20	0.00	0.15	0.12	-0.03	-0.03
Highest degree attained	0.36	0.28	-0.08	0.52	0.27	-0.24**	-0.16
Years of principal experience	6.44	2.68	-3.76***	6.97	2.55	-4.42***	-0.66
Notes: Leaving principals include those who transferred to different schools, changed their positions, and exited the system. Duplicates are dropped. A series of t tests and bivariate regression models are used to examine whether the differences are statistically distinguishable from zero. *p < 0.10. **p < 0.05. ***p < 0.01.

Table 9 shows no evidence that the principal qualifications systematically improved after the turnover, whether facing the sanction or not. Rather, schools systematically hired principals with weaker qualifications. Furthermore, I found no evidence that schools facing the sanctions hired principals with stronger qualifications than schools not facing them. These results suggest that principal turnover was not systematically initiated by district administrators, but school principals themselves.  

DISCUSSION AND CONCLUSIONS

Despite the critical role that principals play in improving student performance and the importance of stability in school leadership for successful school improvement (e.g., Branch, Hanushek, & Rivkin, 2012; Brewer, 1993; Grissom, Loeb, & Master, 2013; Hallinger & Heck, 1998; Hargreaves & Fink, 2004; Hargreaves et al., 2003; Waters, Marzano, & McNulty, 2003), few studies have examined whether NCLB sanctions have influenced principals turnover behaviors. This study fills this gap in the literature by using longitudinal administrative data and detailed school-level assessment data for AYP determinations from Missouri. It constructs a distance variable to the AYP threshold to identify statistically comparable schoolsthose facing the sanction and those not facing itthrough a PSM method. It offers important findings about principals turnover behaviors under the NCLB sanction system and provides implications for the Every Student Succeeds Act (ESSA), which took full effect in the 201718 school year.

First, although I found no evidence that informal sanction affected principal turnover, the impact of informal sanction is significantly moderated by principals job experience, Title I school status, and the percent of non-White students. For example, principals with two to three years of principal experience are about 13 percentage points more likely to leave their schools when they face informal sanction as compared with their colleagues with the same experience level but not facing the sanction (24% and 11%, respectively). In contrast, experienced principals (i.e., 10 years of experience or more) facing the informal sanction are about 15 percentage points less likely to turn over when compared with those not facing it (27% and 42%, respectively). The difference in these differences (i.e., 13% and −15%) is statistically significant. Similarly, the difference in the predicted probability of turnover by the informal sanction status among principals serving a larger number of non-White students is much wider than the difference among principals serving a smaller number of such students. The difference in the former group is about 17 percentage points, whereas it is about 2 percentage points in the latter group. As Figure 4 clearly shows, the difference becomes wider as the percent of non-White students increases.

SIY 1 does not appear to influence principals turnover behaviors. I found no evidence that SIY 1 is correlated with principal turnover or that the relationship is moderated by years of principal experience or other principal qualifications. However, as explained in the earlier section, SIY 1 exposes principals to competition with charter schools, whereas SIY 2 provides principals with supplemental educational resources. This null finding may suggest that some principals facing SIY 1 might have viewed SIY 2 as educational resources they can receive from the districts rather than a sanction, which could have caused them to stay for an additional year (Mintrop, 2004). These principals decisions might have offset the impact of SIY 1. In fact, the sign of the relationship flipped to negative.²⁰ Because the number of principals in the sample facing this sanction is relatively small, the finding may not suggest that the impact of SIY 1 is nonexistent. A large sample size may allow me to detect the impact.

Although I did not examine the impact of SIY 2 and beyond in depth for the small sample size issue, a descriptive comparison (see Note 16) suggests that SIY 2 and beyond might have increased turnover rates. I found that the turnover rate among principals facing SIY 2 was about 34%, whereas it was only 24% among those facing SIY 1, a difference of 10 percentage points. The difference became larger as the sanction level elevated. Collectively, NCLB sanctions, whether informal or formal, appear to have affected principal turnover in some way.

Finally, I examined whether the patterns in transfer and position changes were systematically different by sanction status. Although I found no evidence this is the case, I found some notable patterns. First, principals tended to transfer from Title I schools to nonTitle I schools, regardless of whether they faced informal sanction. Along with this pattern, they tended to move to schools enrolling a smaller number of students who are eligible for the federal free/reduced lunch program. For position changes, principals tended to take central office positions, regardless of whether they faced the informal sanction. Around 40% of principals who changed their positions took jobs at the central office. Their positions full-time equivalency also tended to be reduced to less than 0.75. On the other hand, those facing informal sanction were more likely to become assistant principals or teachers as compared with those who were not facing it.  

Not surprisingly, these results mirror teacher turnover patterns. Teachers are more likely to leave or consider leaving their schools when they face accountability pressure or shock (e.g., Clotfelter, Ladd, Vigdor, & Diaz, 2004; Feng, Figlio, & Sass, 2010; Ingersoll, Merrill, & May, 2016; Reback et al., 2014; Sunderman et al., 2004). Teachers working at these schools often face poor working conditions, toxic school climate and culture, and ineffective school leadership (e.g., Boyd et al., 2011; Johnson, Kraft, & Papay, 2012; Ladd, 2011; Simon & Johnson, 2015). Studies suggest that districts improve working conditions and provide teachers with additional support, including mentoring and training, to retain them (e.g., Ingersoll et al., 2016; Simon & Johnson, 2015).

In much the same way, the results from this study suggest that policy makers need to design new accountability systems under the ESSA that provide adequate resources and support for school leaders, especially given that the main purpose of the systems is to improve student and school outcomes. A poorly designed system will encounter unintended consequences, as did the NCLB system. Policy makers should, therefore, include in new accountability plans a provision of professional support and adequate resources for principals, especially inexperienced principals, who work at low-performing schools and are facing sanctions. These principals may not possess a set of skills and experience to turn around persistently low-performing schools. Support may include mentoring and coaching by experienced principals and district administrators, development of principal networks, assigning more professional staff to their schools, providing more funding to start new educational programs, and offering professional development programs for leadership skills and school improvement.

Along with these support programs, policy makers should develop and embed a policy in new accountability systems under the ESSA that addresses inequity in the distribution of principal quality. Research has found that principal quality and performance are inequitably distributed across schools and districts, and high-needs schools tend to have principals with weaker qualifications and lower performance ratings (Clotfelter, Ladd, Vigdor, & Wheeler, 2007; Grissom, Bartanen, & Mitani, 2018; Loeb et al., 2010). My findings suggest that performance-based sanctions only exacerbate the distribution. A combination of well-designed accountability systems and policies addressing the equity issue would be necessary to improve overall student and school performance. Examples of such policies include pay for performance for school principals, which the federal Teacher and School Leader Incentive Program promotes, and retention and signing bonuses.

This study faces several limitations. First, as discussed in the conceptual framework section, it examines the relationship between NCLB sanctions and principal turnover through a reduced form approach, which does not allow for making a distinction between voluntary turnover and involuntary turnover. Although I descriptively provide evidence that principal turnover might be largely initiated by principals themselves, the analysis does not examine principal performance measures. It is still possible that the district administrators systematically replaced low-performing principals with high-performing ones. Second, one of the key assumptions behind the PSM is strongly ignorable treatment assignment assumption, or conditional independence assumption. This means that the assignment to the treatment (e.g., informal sanction, SIY 1) is independent of the outcomes (i.e., principal turnover), conditional on the covariates. A threat to this assumption is unobservable confounders that are correlated with both the outcomes and the treatment. Although the empirical models control for as many observable principal, school, and district characteristics as possible, my estimates can still be biased because of the unobservable factors, such as support from parents and local community organizations, and school climate. Use of the distance variable in the matching model would reduce bias from these factors but would not be able to remove it completely.

Another limitation is that this study focuses on turnover at the end of the treatment year. It is possible that principals may have left their schools a couple of years after facing the sanction. To the extent that this is true, my estimates fail to capture the true effect of NCLB sanctions. Future research can examine the influence of NCLB sanctions dynamically using the same set of matched samples, but through a survival model that tracks the same principals over time.

 

My estimation models are also subject to anticipation effects (see Malani & Reif, 2010). If principals who are not facing any sanctions changed their turnover behaviors based on the anticipation that they would face sanctions near future, it may bias my parameter estimates downward because principals in the control group also responded to the sanction, making turnover rates between the two groups more similar. Finally, small sample sizes affected statistical power. More data would help estimate the impact more precisely.

The literature would benefit from future work that replicates the study by using different data sets. Given that Missouri is a rural state, it is worth investigating whether the results hold in larger states with more urban areas. It would also benefit from studies that examine principal turnover from both the demand and supply sides simultaneously and distinguish involuntary turnover from voluntary turnover. Because it is complicated to incorporate both sides into a regression-based statistical model, research that uses an explanatory sequential mixed methods design would be necessary to answer this question (Creswell & Clark, 2010). Future work might also include some studies that use a dynamic approach to investigate a cumulative influence of the sanctions.

Notes

1. All the information is based on the U.S. Department of Educations desktop reference for NCLB (U.S. Department of Education, 2002) unless indicated otherwise.

2. Confidence interval was applied to either schools actual performance or AMOs. Safe harbor allowed schools to make AYP if they reduced the percentage of students below the proficient level by 10% from the previous year. Some states combined confidence interval with safe harbor. Student growth model was first approved the USDOE in 200506. States were allowed to determine whether individual students could be classified as on-track to meet their individual growth targets. If so, the states could add these on-track students to the total number of students performing at or above the proficient level. Uniform averaging allowed schools to make AYP if the average of their last two to three years of performance exceeded AMOs in the current year. Not all states used all of these alternative routes.  

3. DESE tested students in Grades 4, 8, and 11 in communication arts, and those in Grades 3, 7, and 10 in math between 200102 and 200405. In 200506, DESE expanded the testing grades and started testing students in Grades 38 and once in high school in both subjects.

4. Unlike in most states, the Missouri DESE started making AYP determinations in 200102 (Duncan, personal communication, June 18, 2015).

5. DESE started using both the confidence interval and the safe harbor in 200304.

6. 6: These numbers and percentages are based on the authors calculations using the Common Core of Data 201213 (district and school levels) maintained by the National Center for Education Statistics.

7. I converted annual salary into 2009 constant dollars because the last year of the data (i.e., 200910) is used only to determine principals turnover status. The Comparable Wage Index (Taylor, Glander, & Fowler, 2007) is available from http://bush.tamu.edu/research/faculty/Taylor_CWI/.

8. I use Barrons ratings in 1993 because a majority of principals attended undergraduate institutions in the late 1980s and the 1990s.

9. The data file defines labor markets based on place-of-work as defined by the Census Bureau (Taylor et al., 2007). Based on this variable, I divided the state into the following 12 labor market regions: Saint Joseph, Northeast, Kansas City region, West Central, Joplin/Springfield, South Central, Cape Girardeau, Saint Louis Region, Columbia/Jefferson City, Lake of the Ozarks, Kansas City, and Saint Louis City. I treat Kansas City and Saint Louis City as single-district labor market regions. Taylor updated the data file recently, which is available from http://bush.tamu.edu/research/faculty/Taylor_CWI/.

10. Initially, I planned to implement a regression discontinuity design. However, the distance variable, which I will describe shortly, did not pass a density test (McCrary, 2008). Although it is unlikely that the distance variable was manipulated at the cutoff point, it is not clear what caused a discontinuity at the point, except the nature of AYP determination rules. To avoid a potential bias in the estimates caused by this discontinuity, I decided not to use a regression discontinuity design.

11. Students are counted more than once if they belong to multiple subgroups.

12. I also performed 1-to-1 nearest neighbor matching with a caliper width restriction, radius matching with a caliper width restriction, and normal kernel matching, but the matching quality was the best for 1-to-5 nearest neighbor matching with a caliper width restriction in terms of standardized bias and the number of units that lie within the common support region. I also used these three matching methods to check robustness of the main results, which are reported in the next section. I find similar patterns across the four different matching methods with minor differences due largely to the differences in the matching quality and sample sizes. Results are available on request.

13. The standardized bias (or difference) is the mean difference as a percentage of the average standard deviation. See Rosenbaum and Rubin (1985) for the formula to calculate standardized bias.

14. Because most of the principals were matched across districts, district fixed effects are not used.

15. For informal sanction, I also examined whether principals behave differently when they face informal sanction for the first time. Results are similar to those found in this section.

16. I also descriptively explored the relationship between sanctions beyond SIY 1 and principal turnover. Although sample sizes are small and thus results are suggestive, I found that the average turnover rate among principals facing SIY 2 was 34%, whereas it was only 24% among those facing SIY 1 delayed. The difference in the average turnover rate between principals facing SIY 3 and SIY 2 delayed is about 19 percentage points. These differences are not statistically indistinguishable from zero due largely to the small sample sizes. Larger sample sizes may turn them into significant.

17. Although not reported here, I also examined the relationship by turnover type and found similar patterns. In addition, I found that the moderation effect by years of principal experience came from transfers and position changes, whereas the moderation effect by the percent of non-White students largely came from transfers and exits.

18. Because data on students eligible for the federal free/reduced lunch program and Title I school status become available from 199899 in the Common Core of Data (school level), I dropped the Title I status variable from the models and replaced the percent of students eligible for the free/reduced lunch program with the percent of students eligible for the free lunch program. In addition, the weights generated from the PSM were not used for this analysis.

19. This is not 100% because Title I school status changed between the baseline year and the treatment year. Data on school characteristics among the sending schools came from the treatment year.

20. Although it is difficult to statistically test this hypothesis with the data I have, one way to investigate this hypothesis is to descriptively compare the difference in the turnover rates between principals facing SIY 1 and those not facing it, with the difference between principals facing SIY 1 (delayed) and those facing SIY 2. If the second difference is notably larger than the first difference, it agrees with the hypothesis. I found that the first difference was 0.74 percentage points (see Table 4), whereas the second difference was more than 10 percentage points. The difference in the differences was 9.4 percentage points. Although the difference was not attributable solely to a threat of facing SIY 3 or corrective actions next year, the large difference appears to support the hypothesis.

References

Ahn, T., & Vigdor, J. (2014). The impact of No Child Left Behinds accountability sanctions on school performance: Regression discontinuity evidence from North Carolina (NBER Working Paper No. 20511). Cambridge, MA: National Bureau of Economic Research. Retrieved from http://www.nber.org/papers/w20511

Akiba, M., & Reichardt, R. (2004). What predicts the mobility of elementary school leaders? An analysis of longitudinal data in Colorado. Education Policy Analysis Archives, 12(18), 120.

Austin, P. C. (2011). Optimal caliper widths for propensity-score matching when estimating differences in means and differences in proportions in observational studies. Pharmaceutical Statistics, 10(2), 150161.

Baker, B., Punswick, E., & Belt, C. (2010). School leadership stability, principal moves, and departures: Evidence from Missouri. Educational Administration Quarterly, 46(4), 523557.

Ballou, D., & Springer, M. G. (2011). Has NCLB encouraged educational triage? Accountability and the distribution of achievement gains (Unpublished manuscript). Vanderbilt University, Nashville, TN.

Barry, S. (2015, January 9). Former Robert Hughes academy principal gets 1 year of probation for MCAS cheating scam at Springfield charter school. MASSLIVE. Retrieved from http://www.masslive.com/news/index.ssf/2015/01/former_robert_m_hughes_academy.html

Berman, M. (2015, April 1). Former educators found guilty in Atlanta school cheating scandal. Washington Post. Retrieved from https://www.washingtonpost.com/news/post-nation/wp/2015/04/01/nearly-a-dozen-found-guilty-in-atlanta-school-cheating-trial/

Berryhill, J., Linney, J. A., & Fromewick, J. (2009). The effects of education accountability on teachers: Are policies too-stress provoking for their own good? International Journal of Education Policy and Leadership, 4(5). Retrieved from http://www.ijepl.org

Béteille, T., Kalogrides, D., & Loeb, S. (2012). Stepping stones: Principal career paths and school outcomes. Social Science Research, 41, 904919.

Boyd, D., Grossman, P., Ing, M., Lankford, H., Loeb, S., & Wyckoff, J. (2011). The influence of school administrators on teacher retention decisions. American Educational Research Journal, 48(2), 303333.

Branch, G. F., Hanushek, E. A., & Rivkin, S. G. (2012). Estimating the effect of leaders on public sector productivity: The case of school principals (Working Paper No. 17803). Cambridge, MA: National Bureau of Economic Research. Retrieved http://www.nber.org/papers/w17803

Brewer, D. J. (1993). Principals and student outcomes: Evidence from U.S. high schools. Economics of Education Review, 12(4), 281292.

Brookhart, M. A., Schneeweiss, S., Rothman, K. J., Glynn, R. J., Avorn, J., & Stürmer, T. (2006). Variable selection for propensity score models. American Journal of Epidemiology, 163(12), 11491156.

Brown, P. F. (2006). Preparing principals for todays demands. Phi Delta Kappan, 87(7), 525526.

Center on Education Policy. (2006). Ten big effects of the No Child Left Behind Act on public schools. Washington, DC: Center on Education Policy. Retrieved from http://www.cep-dc.org

Chakrabarti, R. (2014). Incentives and responses under No Child Left Behind: Credible threats and the role of competition. Journal of Public Economics, 110, 124146.

Clotfelter, C. T., Ladd, H. F., Vigdor, J. L., & Diaz, R. A. (2004). Do school accountability systems make it more difficult for low-performing schools to attract and retain high-quality teachers? Journal of Policy Analysis and Management, 23(2), 251271.

Clotfelter, C., Ladd, H. F., Vigdor, J., & Wheeler, J. (2007). High-poverty schools and the distribution of teachers and principals. North Carolina Law Review, 85, 13451379.

Creswell, J. W., & Clark, V. L. P. (2010). Designing and conducting mixed methods research. Thousand Oaks, CA: SAGE.

Cullen, J. B., & Mazzeo, M. J. (2008). Implicit performance awards: An empirical analysis of the labor market for public school administrators. Working paper.

Dee, T. S., & Jacob, B. (2011). The impact of No Child Left Behind on student achievement. Journal of Policy Analysis and Management, 30(3), 418446.

Dee, T. S., Jacob, B., & Schwartz, N. L. (2013). The effects of NCLB on school resources and practices. Educational Evaluation and Policy Analysis, 35(2), 252279.

Educational Testing Service. (2008). Standards, accountability, and flexibility: Americans speak on No Child Left Behind reauthorization. Princeton, NJ: Educational Testing Service.

Farley-Ripple, E. N., Solano, P. L., & McDuffie, M. J. (2012). Conceptual and methodological issues in research on school administrators career behavior. Educational Researcher, 41(6), 220229.

Feng, L., Figlio, D. N., & Sass, T. (2010). School accountability and teacher mobility (NBER Working Paper No. 16070). Cambridge, MA: National Bureau of Economic Research.

Fruehwirth, J. C., & Traczynski, J. (2013). Spare the rod? The dynamic effects of failing accountability on schools (Unpublished manuscript). University of Cambridge, Cambridge, England.

Fuller, E., & Young, M. (2009). Tenure and retention of newly hired principals in Texas. Austin: University Council of Educational Administration, Department of Educational Administration, University of Texas.

Gabriel, T. (2010, June 10). Under pressure, teachers tamper with tests. The New York Times. Retrieved from http://www.nytimes.com/2010/06/11/education/11cheat.html?_r=0

Gardiner, M. E., & Enomoto, E. K. (2006). Urban school principals and their role as multicultural leaders. Urban Education, 41(6), 560584.

Gates, S. M., Ringel, J. S., Santibañez, L., Guarino, C., Ghosh-Dastidar, B., & Brown, A. (2006). Mobility and turnover among school principals. Economics of Education Review, 25(3), 289302.

Grissom, J. A., & Andersen, S. (2012). Why superintendents turn over. American Educational Research Journal, 49(6), 11461180.

Grissom, J. A., Loeb, S., & Master, B. (2013). Effective instructional time use for school leaders: Longitudinal evidence from observations of principals. Educational Researcher, 42(8), 433444.

Grissom, J. A., Bartanen, B., & Mitani, H. (2018). Principal sorting and the distribution of principal quality: Documenting inequities in state and national data. Manuscript submitted for publication.

Grissom, J. A., Nicholson-Crotty, S., & Harrington, J. R. (2014). Estimating the effects of No Child Left Behind on teachers and their work environment. Educational Evaluation and Policy Analysis, 36(4), 417436.

Hallinger, P., & Heck, R. H. (1998). Exploring the principals contribution to school effectiveness: 19801995. School Effectiveness and School Improvement, 9(2), 157191.

Harder, V. S., Stuart, E. A., & Anthony, J. C. (2010). Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research. Psychological Methods, 15(3), 234249.

Hargreaves, A., & Fink, D. (2004). The seven principles of sustainable leadership. Educational Leadership, 61(7), 813.

Hargreaves, A., & Fink, D. (2006). Sustainable leadership. San Francisco, CA: Jossey-Bass.

Hargreaves, A., Moore, S., Fink, D., Brayman, C., & White, R. (2003). Succeeding leaders? A study of principal succession and sustainability. Toronto, Ontario, Canada: Ontario Principals Council.

Hemelt, S. W. (2011). Performance effects of failure to make adequate yearly progress (AYP): Evidence from a regression discontinuity framework. Economics of Education Review, 30(4), 702723.

Ingersoll, R., Merrill, L., & May, H. (2016). Do accountability policies push teachers out? Educational Leadership, 73(8), 4449.

Jencks, C., & Phillips, M. (1998). The Black-White test score gap. Washington, DC: Brookings Institution Press.

Johnson, S. M., Kraft, M. A., & Papay, J. P. (2012). How context matters in high-need schools: The effect of teachers working conditions on their professional satisfaction and their students achievement. Teachers College Record, 114(10), 139.

Ladd, H. F. (2011). Teachers perceptions of their working conditions. Educational Evaluation and Policy Analysis, 33(2), 235261.

Lauen, D. L., & Gaddis, S. M. (2012). Shining a light or fumbling in the dark? The effects of NCLBs subgroup-specific accountability on student achievement. Educational Evaluation and Policy Analysis, 34(2), 185208.

Li, D. (2012). School accountability and principal mobility: How No Child Left Behind affects the allocation of school leaders (Working paper). Harvard University, Cambridge, MA.

Loeb, S., Kalogrides, D., & Horng, E. L. (2010). Principal preferences and the even distribution of principals across schools. Educational Evaluation and Policy Analysis, 32(2), 205229.

Lyons, J. E., & Algozzine, B. (2006). Perceptions of the impact of accountability on the role of principals. Education Policy Analysis Archives, 14(16), 116.

Malani, A., & Reif, J. (2010). Accounting for anticipation effects: An application to medical malpractice tort reform (NBER Working Paper No. 16593). Cambridge, MA: National Bureau of Economic Research.

McCrary, J. (2008). Manipulation of the running variable in the regression discontinuity design: A density test. Journal of Econometrics, 142(2), 698714.

McGhee, M. W., & Nelson, S. W. (2005). Sacrificing leaders, villainizing leadership: How educational accountability policies impair school leadership. Phi Delta Kappan, 86(5), 367372.

Mertler, C. (2011). Teachers' perceptions of the influence of No Child Left Behind on classroom practices. Current Issues in Education, 13(3). Retrieved from http://cie.asu.edu/

Meyer, M. J., & Macmillan, R. B., & Northfield, S. (2009). Principal succession and its impact on teacher morale. International Journal of Leadership in Education, 12(2), 171185.

Miller, A. (2013). Principal turnover and student achievement. Economics of Education Review, 36(1), 6072.

Mintrop, H. (2004). Schools on probation: How accountability works (and does not work). New York, NY: Teachers College Press.

Missouri Department of Elementary and Secondary Education. (2010). Understanding your adequate yearly progress (AYP) report 2010-11. Jefferson City: Missouri Department of Elementary and Secondary Education.

Reback, R., Rockoff, J., & Schwartz, H. L. (2014). Under pressure: Job security, resource allocation, and productivity in schools under No Child Left Behind. American Economic Journal: Economic Policy, 6(3), 207241.

Reback, R., Rockoff, J., Schwartz, H. L., & Davidson, E. (2011). Barnard/Columbia No Child Left Behind Database, 200203 and 200304. Columbia University, New York, NY. Retrieved from http://www.gsb.columbia.edu/nclb

Rosenbaum, P. R., & Rubin, D. B. (1985). Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. American Statistician, 39(1), 3338.

Rubin, D. B. (2001). Using propensity scores to help design observational studies: Application to the tobacco litigation. Health Services and Outcomes Research Methodology, 2, 169188.

Salazar, K. (2007). Senator Salazars No Child Left Behind survey results. Retrieved from https://web.archive.org/web/20070131231836/http://salazar.senate.gov/other/nclbresults.html#who

Santoro, D. A. (2011). Good teaching in difficult times: Demoralization in the pursuit of good work. American Journal of Education, 118(1), 123.

Simon, N. S., & Johnson, S. M. (2015). Teacher turnover in high-poverty schools: What we know and can do. Teachers College Record, 117(3), 136.

Solano, P. L., McDuffie, M. J., & Farley-Ripple, E. N., & Bruton, J. (2010). Principal retention in the state of Delaware. Newark: University of Delaware, Center for Community Research and Service.

Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statistical Science, 25(1), 121.

Sunderman, G. L., Tracey, C. A., Kim, J., & Orfield, G. (2004). Listening to teachers: Classroom realities and No Child Left Behind. The Civil Rights Project. Cambridge, MA: Harvard University.

Taylor, L. L., Glander, M., & Fowler, W. J. (2007). Documentation for the NCES Comparable Wage Index Data File, 2005 (EFSC 2007-397). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics.

U.S. Department of Education. (2002). No Child Left Behind: A desktop reference 2002. Washington, DC: United States Department of Education.

Waters, T., Marzano, R. J., & McNulty, B. (2003). Balanced leadership: What 30 years of research tells us about the effect of leadership on student achievement (Working paper). Aurora, CO: Mid-continent Research for Education and Learning.

White, B. R., & Agarwal, P. K. (2011). The principal report: The State of School Leadership in Illinois. Edwardsville: Illinois Education Research Council, Southern Illinois University Edwardsville.

APPENDIX

Data Sources for AYP-Related Information

Year	Assessment Data	AYP		School Improvement
2002	NLSLSASD	DESE AYP Grid		NA
2003	Columbia/Barnard	Columbia/Barnard		NA
2004	Columbia/Barnard	Columbia/Barnard		DESE website
2005	DESE AYP data file	DESE AYP data file		DESE website
2006	DESE AYP data file	DESE AYP data file		DESE website
2007	DESE AYP data file	DESE AYP data file		DESE website
2008	DESE AYP data file	DESE school improvement data file		DESE school improvement data file
2009	DESE AYP data file	DESE school improvement data file		DESE school improvement data file
Note: NA stands for not applicable.

Cite This Article as: Teachers College Record Volume 121 Number 2, 2019, p. 1-44 https://www.tcrecord.org ID Number: 22562, Date Accessed: 10/2/2021 5:55:03 PM Purchase Reprint Rights for this article or review