Linking Principal Leadership to Organizational Growth and Student Achievement: A Moderation Mediation Analysis

by James Sebastian & Elaine Allensworth - 2019

Background: Although there is a substantial body of literature on school leadership and its relationship with student achievement, few studies have examined how change in leadership is related to organizational growth and school improvement. Also less well studied is the influence of contextual conditions on how leadership and organizational processes evolve to constrain/augment school outcomes.

Focus of Study: In this study, we use moderation mediation analysis to examine how change in principal leadership relates to achievement growth, mediated via change in multiple organizational processes—parent-teacher trust, school climate (measured by school safety), and professional capacity. We further examine how these mediational relationships are moderated by initial school conditions.

Research Design: We apply moderation mediation analysis to administrative and survey data of elementary schools from a large urban school district to examine if initial school conditions moderate mediational relationships between school leadership and student outcomes.

Conclusions: Our results show that improvements in school leadership are related to student learning gains only through improvements in school climate; this relationship is consistent regardless of whether schools initially had strong or weak leadership and regardless of whether schools initially had safe or unsafe school climates.

School leadership is considered one of the critical levers for school improvement and student learning, but research linking school leadership to student outcomes has proven difficult for a number of reasons. Grissom and Loeb (2011) highlight two important challengesdata availability and the complex nature of principal leadership work. Past research on school leadership has consistently shown that the effects of leadership are largely indirect, mediated via school organizational factors. Therefore, studies of school leadership need to collect data on school leadership as well as multiple school processes through which leadership is hypothesized to exert influence. Researchers are constrained in collecting data on a vast array of organizational processes and often focus on a few aspects of school organization (Louis, Dretzke, & Wahlstrom, 2010; Louis, Leithwood, Wahlstrom, & Anderson, 2010). Another challenge is that leadership work itself comprises many aspects that researchers can focus one.g., instructional management, organization management, relationship building, and administration (Barnes, Camburn, Sanders, & Sebastian, 2010; Camburn, Spillane, & Sebastian, 2010; Grissom & Loeb, 2011). Heck and Hallinger (2009) identify another challenge in conducting school leadership researchthe importance of studying change over time to assess the role of school leaders in influencing school improvement over time. Most studies of school leadership use cross-sectional data and are unable to provide evidence on how change in school leadership is linked to the improvement of student learning (Heck & Hallinger, 2009).

The changing context of a school plays an important role in influencing how leadership work evolves, introducing specific challenges, different priorities, and levels of resources that can facilitate or constrain leaders actions (Hallinger, 2003, 2005; Hallinger & Heck, 1996a, 1996b, 1998). For this reason, Hallinger and Heck (1996a, 1996b) argue for using contingency models of school leadership, wherein the work and effectiveness of school leaders depends on the particular school context. The challenges of studying longitudinal change in leadership and the role of school context add to the challenge of data availability. In addition to collecting information on school leadership and a comprehensive set of mediating organizational processes, longitudinal data also need to be collected on school contextual variables and student outcomes. There are few datasets with the information complexity of this nature, which is a limitation facing researchers interested in examining how school leadership relates to student outcomes. Additionally, where appropriate longitudinal data are available, sophisticated models are required to examine leadership effects correctly. These include: (i) mediational models to study indirect effects of school leadership, (ii) growth models to capture change in student outcomes, leadership inputs, and mediating organizational processes, and (iii) moderation models, which are best suited to capture contingency effectshow indirect effects of leadership are in turn dependent on contextual conditions.

In this study, we utilize a longitudinal database developed by the University of Chicago Consortium on School Research (UChicago Consortium) that has collected data since 1999, at regular intervals, on student outcomes, including achievement and grades (GPA), survey measures on school leadership, a comprehensive set of mediating organizational processes, and school contextual demographic information. This dataset provides a unique opportunity to address some of the challenges in studying school leadership research discussed above. We combine growth modeling and structural equation modeling (SEM) to examine how change in school leadership is related to student achievement growth via changes in multiple mediating organizational factors. We also use moderation mediation analysis to examine if the mediated effects of leadership may depend on initial conditions of the school. The application of moderation mediation analysis in educational leadership research is new and could help further research examining contingency effects in school organizations.

OVERVIEW OF LITERATURE

The many challenges inherent in conducting school leadership research using quantitative methods include: (i) the complex nature of principal leadership work, (ii) indirect effects of leadership work, (iii) the importance of contextual effects, and (iv) the importance of examining change. In the following sections, we provide a brief overview of prior research on school leadership research to highlight these challenges. We then provide a description of the data developed by the UChicago Consortium and describe the theoretical framework underlying the development of this database. In the subsequent section, we provide an overview of the methodology used in this studymoderation mediation analysisand describe the potential this methodology has in advancing research on school leadership effectiveness.

PRINCIPAL LEADERSHIP WORK

Early research on school principals described principals as lone rangers whose work was characterized by brevity, fragmentation, great diversity, constant interruption, frenzied pace, and long hours (Kmetz & Willower, 1982; Martin & Willower, 1981; Wolcott, 1973). More recent research finds that many of these descriptions still characterize principal work, although there is limited evidence for the lone ranger image (Camburn et al., 2010; Spillane & Hunt, 2010). With conceptualizations of principal work involving attention to numerous tasks and responsibilities still holding true (see Camburn et al., 2010), one particular focus of leadership research has been on the role of principals in directly or indirectly supporting good instructional practices, also termed instructional leadership (Hallinger, 2005; Hallinger & Heck, 1996a, 1996b, 1998; Hallinger & Murphy, 1986). The concept of instructional leadership includes a large array of tasks that could be classified into three dimensions: defining the mission and vision of the school, managing the instructional program, and promoting a positive learning climate (Hallinger, 2005). What this means for the allocation of principals time is not clearly defined, as it could include a very broad array of efforts such as observing teacher practice, coordinating professional development, developing teacher-parent relationships, contracting with external providers around student supports, initiating new discipline strategies, and so on. In a typical workday, principals allocate their time across a range of functions and interact with multiple personnel, making it difficult for researchers to tease out which aspects of their work are essential for school instructional improvement and ultimately lead to student learning growth.

INDIRECT EFFECTS OF LEADERSHIP

A considerable body of quantitative work on school leadership has shown that the effects of leadership on student outcomes are statistically significant and that these effects are largely indirect, mediated by school organizational features (Hallinger, 2003, 2005; Hallinger & Heck, 1996a, 1996b, 1998; Hallinger & Murphy, 1986; Leithwood, Louis, Anderson, & Wahlstrom, 2004; Louis, Leithwood et al., 2010). However, researchers disagree on which aspects of school organization are most important to study to examine leadership effects (Louis, Leithwood, et al., 2010). There is potentially a very large number of mediating variables, and researchers focus on different sets of variables depending on their respective theoretical models. Analysis of mediation effects using models that include only isolated organizational factors could be influenced by omitted variable bias. For example, many researchers have focused on professional community (e.g., Louis, Dretzke, & Wahlstrom, 2010; Louis, Leithwood, et al., 2010), or aspects of professional community such as teacher collaboration as key mediators (Dumay, Boonen, & Van Damme, 2013; Supovitz, Sirinides, & May, 2010). Yet Sebastian and Allensworth (2012) found that professional community mediated the relationship between principals instructional leadership abilities and student achievement only when other mediators were not included in SEM modeling. For example, when school climate was also included as a mediator, the mediation effect of professional community was no longer statistically significant (Sebastian & Allensworth, 2013).

Hallinger and Heck utilized a global school organizational variable in their studies, which they termed academic capacity (see, for example, Hallinger & Heck, 2009, 2010a, 2010b; Heck & Hallinger, 2009, 2010). Academic capacity included multiple aspects of school organization, including an emphasis on standards and its implementation, focused and sustained action on improvement, quality of student support, and professional capacity. Using a global organizational factor has advantages that include simultaneously capturing multiple aspects of the school organization and allowing for modeling simplicity. At the same time, it becomes difficult to disentangle which aspects of school organization or academic capacity must be given emphasis for school improvement, as working on all aspects together could be overwhelming. Moreover, other studies have shown that central aspects of school organization, such as professional capacity and the school climate, are related but distinct constructs (Sebastian & Allensworth, 2012; Sebastian, Allensworth, & Huang, 2016).

THE IMPORTANCE OF CONTEXT

Scholars, policymakers, and practitioners have long emphasized the importance of context in understanding the work of school leaders and their effectiveness (Louis, Leithwood, et al., 2010). Louis, Leithwood, et al. (2010) note that prior research on school leadership has been scattered in examining the importance of context:

At one extreme, researchers have claimed that local context trumps all other factors. Claims of this sort often are based on single or small-number case studies. In contrast, researchers working from quantitative studies treat contextual variables as factors to be controlled in inquiries about leadership effects. This approach essentially dismisses context as a substantive problem. (p. 96)

In order to test the idea that leadership effectiveness is dependent on context, quantitative studies could consider contextual factors as moderators instead of controlling for them. Further, since leadership effects are indirect, a combination of moderation and mediation models seems necessary. The increasing availability of longitudinal datasets on school systems, together with the development of computing capabilities and sophisticated SEM modeling techniques, will likely see quantitative researchers adopting different combinations of moderation and mediation techniques to understand the work and effectiveness of school leaders.

EXAMINING CHANGE IN LEADERSHIP WORK

One of the key limitations of existing research on school leadership research is lack of longitudinal studies examining how change in leadership is related to change in student outcomes; most studies are based on cross-sectional data (Hallinger & Heck, 1996a, 1996b, 2010a, 2010b; Heck & Hallinger, 2009, 2010). Hallinger and Heck (2010b) note that empirical studies of school leadership must employ longitudinal designs because by definition, school improvement involves some changes in an organization over a period of time. A few studies have combined longitudinal data on school outcomes with cross-sectional measures of leadership and school organization (see, for example, Louis, Dretzke, & Wahlstrom, 2010; Sebastian & Allensworth, 2012; Sebastian et al., 2016; Supovitz et al., 2010). Dumay et al. (2013) used longitudinal data over a six-year period and utilized a weighting strategy to uncover causal relationships between principal leadership, teacher collaboration, teacher efficacy and student learning growth (Principal Leadership ® Teacher Collaboration ® Teacher Efficacy® Student Learning Growth). However, the leadership and mediating variables were measured at single points in time; the lack of information on leadership and mediating organizational variables at multiple points of time is a limitation with this approach (Dumay et al., 2013). The focus on a few selected mediating organizational factorsteacher collaboration and efficacyis another limitation.

SEM analyses by Hallinger and Heck can perhaps be considered as among the most advanced longitudinal work to examine school leadership effects. They combined growth modeling and structural equation modeling to study how change in leadership was related to growth in student achievement via change in a global organizational constructacademic capacity (see Hallinger & Heck, 2009, 2010a, 2010b; Heck & Hallinger, 2009, 2010). The use of a global mediating organizational factor carries some of the disadvantages discussed earlier. Nonetheless, Hallinger and Hecks work provides a benchmark in conducting quantitative studies of leadership that focuses on change and accounts for the mediated nature of leadership work. Longitudinal data were used to link change in both school outcomes and school organizational processes, multilevel SEM was used to estimate direct and indirect (mediation) paths, and school contextual characteristics such as student composition variables and school size were appropriately accounted forthey were included as covariates.  

The present study can be considered an extension of the work done by Heck and Hallinger (2009). Similar to their work, we examine how change in school leadership is related to student achievement growth using similar growth models. Here, we explore multiple mediating mechanisms based on the theoretical framework proposed by Bryk, Sebring, Allensworth, Luppescu, and Easton (2010) rather than one global organizational construct. We go beyond accounting for school contextual characteristics as covariates and take a step toward examining contingency effects by exploring how mediational relationships may depend on (be moderated by) a few initial school conditions. For example, if a school already has a safe learning environment, does it matter if it makes further improvements? Or does a school just need to have a safe learning environment to begin with? Do schools that start out with weak leadership show larger gains in test scores if leadership improves over time, in comparison to schools that start out with strong leadership in the first place? For this purpose, we use moderation mediation analysis. To our knowledge, the work is the first of its nature in school leadership and organizational research. A strength of the present study is the unique database organized by the UChicago Consortium. Their use of consistent items in multiple survey administrations and Rasch modeling to anchor and scale measures makes it possible to compare data on leadership and school organization across multiple survey administrations and different school contexts (see Bryk et al., 2010; Luppescu & Ehrlich, 2012). The two main extensions of this study, the inclusion of multiple mediators based on the Bryk et al. (2010) theoretical framework and the use of moderation mediation analysis, are discussed in the following sections.

THE FIVE ESSENTIALS FRAMEWORK

The theoretical framework used to conduct our SEM analysis is the Bryk et al. (2010) framework, also commonly referred to as the five essential supports framework (Sebring, Allensworth, Bryk, Easton, & Luppescu, 2006). A practical reason for adopting this framework is that the UChicago Consortium database itself was developed based on the Bryk et al. (2010) model; the items on the student, teacher, and principal surveys, and the measures developed from those surveys, correspond to this theoretical framework. Hitt and Tucker (2016) identified this framework as one of three main frameworks that has informed past research on school leadership and the only one based on empirical evidence. Detailed descriptions of this framework have been provided elsewhere (e.g., Bryk et al., 2010; Sebastian & Allensworth, 2012; Sebastian et al., 2016; Sebastian, Allensworth, & Stevens, 2014; Sebring et al., 2006); we only provide a brief overview here (see Figure 1).

Like most models of school organization, it identifies school leadership as the prime mover of school improvement. The focus of leadership in this framework is principal leadership, specifically instructional leadership. Distributed leadership is also considered important, as evidenced by the involvement of teachers in school policy and work. Leadership works indirectly to improve classroom instruction and student learning through three organizational processesprofessional capacity of staff (which includes professional community and professional development), the school learning climate, and parent-community ties. In this study we adapt the Bryk et al. (2010) framework for SEM modeling (see Figure 2) to test for direct and indirect effects of school leadership on student achievement. Previous work using the UChicago Consortium data and the Bryk et al. (2010) framework has shown complex interactions between principal and teacher leadership in direct and indirect effects on instruction and learning; these effects also depend on the contextwhether elementary or secondary (Sebastian et al., 2016; Sebastian, Huang, & Allensworth, 2017).

Prior work using the Bryk et al. (2010) framework to examine leadership effects on student outcomes has relied on SEM techniques to examine direct and indirect pathways from leadership to student test scores and GPA via mediating organizational factors. This work consistently showed that school climateschool safety and academic focuswas the strongest mediator linking leadership and achievement when multiple pathways were included in the model. At the high school level, school climate emerged as the only significant mediator. At the elementary level, school climate emerged as the only significant mediator in one specification of the model, but in another specification it was one of two significant mediators; program quality (defined as the quality of teachers' professional development and program coherence) was another significant path (e.g., Sebastian & Allensworth, 2012; Sebastian et al., 2016; Sebastian et al., 2014; Sebastian et al., 2017). However, while the outcome was based on longitudinal information on student outcomes for these models, the leadership and organizational variables used in these studies were cross-sectional. In the present study, we use longitudinal data for leadership and mediating variables as well as outcomes. Using SEM mediation models, we examine how change in leadership is related via changes in mediating organizational processes to student achievement growth. We use a multiple mediation model as the Bryk et al. (2010) framework proposes: parent-community ties, school climate, and professional capacity. Based on prior research using cross-sectional data, we hypothesize that change in school climate will also be the most important mediator linking leadership and student outcomes.

Figure 1. Five essential supports for school improvement

Professional development and learning community are two critical aspects of professional capacity. Adapted from Bryk et al. (2010).

A further extension explored in this paper is how mediational effects themselves may be dependent on certain initial conditions. For this, we used moderation mediation analysis, which combines estimation from both moderation and mediation analysis:

The goal of mediation analysis is to establish the extent to which some putative causal variable, X, influences some outcome, Y, through one or more mediator variables&. In contrast, an investigator conducting a moderation analysis seeks to determine whether the size or sign of the effect of X on Y depends in one way or another on (i.e., interacts with) a moderator variable or variables&A more fine-grained understanding of a phenomenon comes from uncovering and describing the contingencies of mechanismsthe when of the how.&Described using easily confused terms such as moderated mediation and mediated moderation, the goal is to empirically quantify and test hypotheses about the contingent nature of the mechanisms by which X exerts its influence on Y. (Hayes, 2013, p. vii)

We used the SPSS macro PROCESS version 2.15 (Hayes, 2013) for conducting the moderation mediation analysis. While other software such as Mplus (Muthén & Muthén, 2007, 2013) can also be used, PROCESS is relatively easy to use and provides a large catalogue of built in model templates. From these templates, researchers can select the model of moderation mediation or multiple mediation that is most useful to answer their research questions.

As noted before, preliminary quantitative work with the Bryk et al. (2010) framework has shown that leadership and school climate are consistently linked to student achievement, whereas there was limited empirical support for the importance of the other essential supports when school climate was included in the model. Therefore, we decided to focus on the initial levels of principal leadership and initial school climate as the two key moderators for this study (see Figure 2). We hypothesized that schools with low levels on these two variables would have greater room to grow and would show higher mediated effects. The version of PROCESS we used could include up to two moderators for the particular moderation mediation model we wished to study. With the development of moderation mediation modeling, future research can examine the moderating influence of other contextual factors such as demographic information or other factors such as principal and staff turnover. This study can be viewed as an illustration of moderation mediation analysis for further school organizational research that could test additional moderators in the future.

Figure 2. Multiple mediation model representing the Bryk et al. (2010) framework

METHOD

SAMPLE

The data for this study include Chicago Public Schools (CPS) administrative data on elementary school students in Grades 3 to 8 and surveys conducted by the University of Chicago Consortium on School Research. CPS administrative data included information on student achievement measured by performance on the Illinois Standards Achievement Test (ISAT). Using data for all third to eighth grade students who took the ISAT between 20062007 and 20122013, we modeled linear growth on ISAT test scores and estimated school-to-school variation on this growth. We then matched student achievement data to survey data collected by the UChicago Consortium. The survey data provided information on school leadership and other organizational processes specified in the Bryk et al. (2010) framework. The UChicago Consortium surveys were administered to all CPS teachers and students in the years 20062007, 20082009, 20102011, 20112012, and 20122013. Only teacher survey data were used for this study. Student surveys collect extensive information on the quality of classroom instruction and noncognitive factors and represent an additional mediating factor that exists between school organization and student learning. However, those measures were not used in the present study, as serial mediation modeling was not possible with the PROCESS software (Hayes, 2013) we used for this study. Table 1 provides the response rates on the teacher surveys for each year of data collection.

Table 1. Number of Teachers Providing Survey Information and Test Score Information

Year			Teacher Surveys	Student Achievement (ISAT gains)
20062007	N		12,181	155,322
	N-Schools		465	492
	Response Rate		93%	NA
20082009	N		9,119	167,543
	N-Schools		444	495
	Response Rate		84%	NA
20112012	N		9,165	162,980
	N-Schools		507	509
	Response Rate		92%	NA
20122013	N		12,263	151,945
	N-Schools		510	464
	Response Rate		83%	NA

MEASURES

Dependent Variable

The outcome variable for this study was gains made by students on reading and mathematics ISAT scores as they moved from third grade to eighth grade. CPS administers the ISAT in the spring semester of every year. As multilevel modeling was not possible with PROCESS version 2.15, we aggregated ISAT gains of individual students to the school level. This aggregation was done with HLM growth modeling and then saving empirical Bayes (EB) residuals from those models. The HLM growth models predicted linear growth on ISAT scores in reading and mathematics controlling for student and school characteristics. The nesting structure of the data consisted of repeated observations of students as they moved from third grade to eighth grade (level-1), nested within individual students (level-2), nested within schools (level-3). At level-1 we stacked ISAT math and reading scores and included a dummy variable to indicate whether an observation was a reading or math score. Also at level-1, we included a trend variable which was coded as 0 for a third grade ISAT score, 1 for fourth grade, 2 for fifth grade and so on, to capture linear growth in test scores. This linear trend variable was allowed to vary at level-2 and also level-3, to capture variation in achievement growth between students and between schools, respectively. In this model, the school level EB residuals of the trend variable captured average achievement growth of students of each school as they moved from third to eighth grade. All students who attended CPS elementary schools in between 20062007 and 20122013 were included in the model. We used the software HLM 7.01 (Raudenbush, Bryk, & Congdon, 2011) to run these multilevel models.

The following equations describe the regression model used to obtain school level estimates of student growth on ISAT scores.

              

Y_ijk denotes achievement on ISAT math or reading for all students in third grade to eighth grade attending CPS elementary schools from 200607 to 201213. Trend is the linear variable described in the preceding paragraph that tracked the grade of student;  is its coefficient. X is a vector of level-1 control variables including subject (mathematics as a dummy variable), retention, and school change (whether the student changed schools);  is the corresponding coefficient for X. We allowed the level-1 intercept and the slope of the trend variable to be random at level-2.

             

W represents a vector of student level covariates that include each students cohort year, socioeconomic status, concentration of poverty, special education needs, age, race, gender, grade, and prior reading achievement. The measure of concentration of poverty was created from census data to reflect the percent of unemployed adults and percentage of families below the poverty line living in a students census block. The measure of socioeconomic status was also created from census data and reflected the mean level of education of adults and percentage of adults employed as managers or professionals in the students census block. Cohort year represented the year in which the student joined CPS as a third grader; we coded this as a linear variable with the most recent cohort having a value of zero.

Z is a vector of school contextual variables including school size, socioeconomic status, and concentration of poverty. All of the control variables, at each levels, except for the variables trend and cohort, were grand mean centered.

The EB residuals for  were saved to get variation at the school level in average student growth in test scores as students moved from third to eighth grade. It is possible that the average test score gains increased or decreased with each successive cohort, and there was variation among schools on this estimate. Therefore, the interaction of the trend coefficient and the cohort variable was allowed to vary at the school level in order to capture this variation; this was captured with . Both  and  were potentially of use as outcomes, as they both represented change in test scores over the time period being analyzed. However, the variation of the cohort interaction term was small, although statistically significant, and highly correlated with average test score gains (0.91). Thus, the two variables were not meaningfully distinguishable from each othergains were increasing across cohorts at the same schools where students were showing the most year-to-year improvements in scores. Therefore, we retained the average test score gains made by schools as the sole outcome for the SEM analyses (20072013) as this variable had the greater variation of the two.

Leadership and School Organizational Measures

Two measures were used to capture principal leadership. The first measured instructional leadership through eight questions on areas such as setting the school vision and goals and supporting effective classroom instruction. Supovitz et al. (2010) noted that despite challenges in identifying the key activities of principals in supporting teaching and learning, a review of the literature points to three commonly referenced themesdeveloping focused mission and goals, creating an environment of trust and collaboration, and actively supporting instruction. This measure is also consistent with the conceptualization of leadership developed by Hallinger and Murphy (1985). The second measure examined teacher-principal trust and reflected the extent to which teachers felt that their school principal had confidence in them, had trusting relationships, encouraged collaboration, and looked out for their personal welfare. These leadership measures were developed from previous empirical work using Rasch analysis (Bryk et al., 2010; Luppescu & Ehrlich, 2012). The specific survey items, their response scales, and their reliabilities are provided in Appendix A. The final principal leadership measure for each survey year was created with a simple mean of the instructional leadership and principal-teacher trust measure.

Mediating Processes

We used the Bryk et al. (2010) framework to guide the selection of mediating variables for analysis. This framework emphasizes three key mediating organizational processesprofessional capacity, parent-teacher ties, and school climate. Professional capacity is based on two aspects of the school organization school professional community and program quality. We measured professional community using three measuresnew teacher socialization, collective responsibility, and collaborative practice. Program quality was measured by averaging two measuresprofessional development quality in the school and a measure of program coherence. Program coherence measured whether the programs in the school were consistent, coordinated, and sustained. We captured learning climate with a measure of teacher perceptions of safety. At the high school level, learning climate also includes perceptions of teacher expectations of their students (Bryk et al., 2010). However, the elementary school surveys did not include a measure for teacher expectations. Prior research has found that teacher perceptions of safety and their expectations for students were strongly correlated (Sebastian & Allensworth, 2012). Other studies have shown a strong relationship (r=0.80) between teachers reports of safety on this measure and students reports of safety at school (Steinberg, Allensworth, & Johnson, 2013). This suggests that this measure of teachers perceptions captures the overall climate of safety in the school for students as well as teachers. Teachers perceptions of safety are also strongly and significantly related to the suspension rate of the school (p<.01, with a partial r=-.25 controlling for school achievement level, student demographics, crime, and poverty in students residential neighborhoods), which provides evidence that the measures capture more than just perceptions (Steinberg et al., 2013). The specific survey items for these measures, their response scales, and their reliabilities are provided in Appendix A.

As with test scores, the leadership measures and the mediating organizational measures were aggregated to the school level using three-level growth models. From the EB residuals of separate HLMs for each measure, we obtained school-specific estimates of linear change in leadership and organizational measures in between 20072013. A description of these HLM models is provided below. The nesting structure here consisted of anonymous teacher observations nested within school year, nested within school.

           

Y_ijk represents principal leadership and organizational measures such as professional capacity, school climate, and teacher-parent trust. At level-2 we specified a linear trend variable to model linear change across the survey years. Z represents a vector of school level covariates including school size, average prior achievement, and two indicators of the average socioeconomic status of students in the school. Both the intercept (starting point) and the subsequent growth (TREND) were allowed to be random at the school level. This variation is captured in the terms  and , for the intercept and slope respectively. The EB residuals for  were saved from each model and merged with the school level achievement data described in the earlier HLM models. Through these HLM models, data from student test scores and teacher survey reports were aggregated to the school level and all subsequent analyses were conducted at the school level.

Contextual Information

In the present study, we controlled for several school demographic variables so that the relationships observed between leadership, mediating variables, and student outcomes were not confounded with observable differences that exist across schools. These variables were included as covariates in the preliminary HLMs rather than the final moderation mediation analysis to reduce the number of variables included in the mediation models. The HLMs estimating average student growth on the ISAT and growth on measures of leadership and mediating processes all controlled for school contextual variables (see equations described above). By using contextual variables to adjust the HLM estimation of growth/change in organizational and achievement outcomes, we did not need to enter them in the moderation mediation analysis. Besides the EB residuals on student achievement growth and leadership and organizational change, we also saved the EB residuals of the intercepts of these models. For the student achievement model, the EB residual of the intercept represents the expected score on third grade ISAT for the typical student in each school. For the leadership and organizational variables, the EB residuals of the intercepts represent the initial level of these variables in 20062007 (the first year in the study where the TREND variable takes a value of zero). It is important to control for these initial variables when examining change in organizational variables and student achievement, as initial status and subsequent growth may be correlated. In brief, a fair bit of data preparation was conducted outside of the PROCESS program, using HLM models and EB residuals saved from those models.

Moderation Mediation Analysis

The analyses conducted for the present study utilized two of the 76 modeling templates provided by PROCESS. The first template (Figure 3) conducted a multiple mediation analysis and did not include any moderators. This model examines the direct and indirect pathways through which change in principal leadership (X) is related to student achievement growth (Y) via changes in mediating organizational factors (M_i). Here M_i represents multiple mediators changes in school safety (M₁), professional capacity (M₂), and parent-teacher trust (M₃).

Figure 3. Conceptual diagram of the multiple mediation model described in PROCESS
Indirect effect of X on Y through M_i = a_i b_i. Direct effect of X on Y = c'. Adapted from Hayes (2013).

The second template added two initial conditions as moderatorslevels of principal leadership and levels of school safety in the first year (20062007). PROCESS provides bootstrapped confidence intervals. For moderation mediation, the PROCESS routine calculates mediation effects at different values of the moderator: at the mean, 1 SD below the mean, and 1 SD above the mean. The moderation mediation model (see Figure 4) extends the multiple mediation model described above to estimate interactions of the mediation effects with initial levels of principal leadership (W) and initial levels of school safety (Z).

Figure 4. Conceptual diagram of the moderation mediation model described in PROCESS
Adapted from Hayes (2013).

As in the previous multiple mediation model, M_i represents three mediators. The mediation estimates are now conditional on the two moderatorsinitial levels of principal leadership (W) and school safety (Z). PROCESS calculates these mediational estimates at various points of each moderator. The conceptual and statistical diagrams are available with the PROCESS software documentation and are also described in Hayes (2013).

RESULTS

Table 2 shows the descriptive statistics for the study variables. The EB residuals that estimate initial levels and change in leadership, mediating factors, and the ISAT scores are centered around zero, hence the zero means for all variables shown in Table 2. The descriptive statistics show that the levels and change for each variable were negatively correlated. This means that schools with higher initial levels of a variable (in 20062007) had relatively lower growth in the subsequent years.

Table 2. Descriptive Statistics and Correlation of the Measures

	M	SD	1	2	3	4	5	6	7	8	9	10	11
1. Initial Principal Leadership	0.00	1.75
2. Change in Principal Leadership	0.00	0.29	-0.49
3. Initial Program Quality	0.00	0.83	0.79	-0.35
4. Change in Program Quality	0.00	0.18	-0.11	0.73	-0.16
5. Initial Professional Community	0.00	1.55	0.68	-0.40	0.71	-0.16
6. Change Professional Community	0.00	0.32	-0.18	0.64	-0.18	0.72	-0.37
7. Initial Teacher-Parent Trust	0.00	0.19	0.41	-0.22	0.45	-0.05	0.53	-0.11
8. Change in Teacher-Parent Trust	0.00	0.06	0.04	0.46	0.06	0.65	0.00	0.66	0.16
9. Initial School Safety	0.00	0.66	0.32	-0.19	0.36	-0.08	0.41	-0.16	0.53	0.00
10. Change in School Safety	0.00	0.13	-0.03	0.42	0.00	0.52	-0.08	0.55	-0.05	0.58	-0.25
11. ISAT Intercept	0.00	5.70	0.09	0.00	0.05	0.09	0.08	0.07	0.01	0.14	0.04	0.15
12. Growth in ISAT	0.00	2.30	-0.08	0.08	-0.09	0.07	-0.08	0.07	0.02	0.04	0.03	0.17	-0.31

Table 3 shows the results of the first modelthe multiple mediation model in which no moderators were included. In these models, the initial levels of all variablesleadership, mediating factors, and the school averages of the ISAT interceptwere included as covariates in the PROCESS routine. Recall that the school contextual information such as school size and student compositional characteristics was included as covariates in the HLM models for student achievement, leadership, and mediating school organization, from which the residuals were estimated. This longitudinal model shows only one significant path from principal instructional leadership to student achievement, which is the path that is mediated by school climate (school safety). This is consistent with the cross-sectional models in earlier studies, which showed school safety as the only consistently significant mediator of school leadership on student achievement across various model specifications (Sebastian & Allensworth, 2012; Sebastian et al., 2016; Sebastian et al., 2017).

Table 3. Direct and Indirect Relationships between Change in Leadership and Test Score Growth in the Multiple Mediation Model

Mediator	Estimate	Bootstrap SE	Lower Level CI	Upper Level CI
Direct	-0.02	0.08	-0.17	0.13
(Indirect)
Change in Program Quality	-0.01	0.07	-0.15	0.13
Change in Professional Community	-0.03	0.05	-0.13	0.07
Change in Teacher-Parent Trust	-0.03	0.04	-0.13	0.04
Change in Teacher Safety	0.17	0.03	0.11	0.24
(TOTAL)	0.10	0.07	-0.03	0.23

Note. Model controls for initial levels of leadership, mediating factors, and initial average ISAT scores. CI = confidence interval.

Tables 4 and 5 describe the results of the moderation mediation analyses. The results are from a single moderation mediation model; they have been broken out for readability. Table 4 shows the direct relationship of change in principal leadership with achievement growth conditional on different values of the moderators (initial principal leadership and initial school safety). The results show that at all values of the two moderators, the direct relationship of principal leadership with achievement growth was not statistically significant.

Table 5 shows indirect relationships of change in leadership with achievement growth, again conditional on different values of the two moderators. The results show that for three mediatorsprogram quality, professional community, and teacher-parent trustthe initial levels of principal leadership and school safety do not matter. The mediational relationships for these mediators are not statistically significant at combinations of the moderator values at the mean, 1 SD below the mean, and 1 SD above the mean. Again, change in safety is the only statistically significant mediator. Further, the mediational relationships of safety are statistically significant at all values of the moderator variables. The results additionally show that holding initial principal leadership constant, higher initial levels of school safety do not increase the mediational effects due to improvements in safety. Holding school safety constant, higher levels of principal leadership do lead to a small increase in the mediational effect; a 1 SD increase in initial principal leadership (holding initial safety levels constant) increases the mediational effects of improvements in school safety by about 0.02 SD. That is, schools that initially had principals rating highly on instructional leadership showed slightly greater improvements in student learning gains concurrently with improvements in safety. In general, though, it appears that there are fairly even mediational effects due to change in school safety at all levels of initial principal leadership and school safety.

Table 4. Direct Relationships between Change in Principal Leadership and Student Achievement Growth Conditional on Initial Leadership and Safety

Moderators
Initial Principal Leadership	Initial Safety	Estimate	SE	Lower Level CI	Upper Level CI
-1.00	-1.00	0.03	0.11	-0.19	0.24
-1.00	0.00	0.04	0.09	-0.14	0.23
-1.00	1.00	0.06	0.12	-0.18	0.30
0.00	-1.00	-0.05	0.11	-0.27	0.17
0.00	0.00	-0.03	0.08	-0.20	0.13
0.00	1.00	-0.02	0.10	-0.22	0.18
1.00	-1.00	-0.13	0.14	-0.40	0.14
1.00	0.00	-0.11	0.11	-0.32	0.10
1.00	1.00	-0.09	0.11	-0.31	0.13

Note. The direct relationships shown here are net of the indirect mediated relationships described in Table 5.

CI = confidence interval.

Table 5. Conditional Indirect Relationships between Change in Principal Leadership and Student Achievement Growth

	Moderators
Mediator	Initial Principal Leadership	Initial Safety	Estimate	SE	Lower Level CI	Upper Level CI
Change in Program Quality
	-1.00	-1.00	-0.06	0.11	-0.26	0.14
	-1.00	0.00	-0.08	0.10	-0.27	0.11
	-1.00	1.00	-0.10	0.14	-0.35	0.18
	0.00	-1.00	0.03	0.11	-0.19	0.26
	0.00	0.00	0.00	0.08	-0.16	0.15
	0.00	1.00	-0.02	0.10	-0.22	0.19
	1.00	-1.00	0.11	0.16	-0.19	0.44
	1.00	0.00	0.09	0.11	-0.13	0.32
	1.00	1.00	0.06	0.11	-0.16	0.27
Change in Professional Community
	-1.00	-1.00	-0.08	0.07	-0.21	0.04
	-1.00	0.00	-0.05	0.06	-0.17	0.06
	-1.00	1.00	-0.03	0.09	-0.20	0.14
	0.00	-1.00	-0.06	0.09	-0.23	0.11
	0.00	0.00	-0.03	0.06	-0.14	0.08
	0.00	1.00	0.00	0.08	-0.16	0.18
	1.00	-1.00	-0.03	0.14	-0.28	0.24
	1.00	0.00	0.00	0.09	-0.17	0.18
	1.00	1.00	0.03	0.10	-0.17	0.22
Change in Teacher-Parent Trust
	-1.00	-1.00	0.09	0.06	-0.02	0.21
	-1.00	0.00	0.02	0.05	-0.06	0.12
	-1.00	1.00	-0.03	0.06	-0.15	0.08
	0.00	-1.00	0.05	0.07	-0.07	0.19
	0.00	0.00	-0.03	0.04	-0.11	0.05
	0.00	1.00	-0.09	0.06	-0.20	0.02
	1.00	-1.00	-0.01	0.11	-0.21	0.19
	1.00	0.00	-0.10	0.07	-0.23	0.05
	1.00	1.00	-0.17	0.07	-0.31	-0.03
Change in Safety
	-1.00	-1.00	0.15	0.04	0.07	0.25
	-1.00	0.00	0.14	0.04	0.08	0.22
	-1.00	1.00	0.12	0.05	0.05	0.26
	0.00	-1.00	0.17	0.05	0.09	0.29
	0.00	0.00	0.16	0.03	0.10	0.24
	0.00	1.00	0.15	0.04	0.08	0.26
	1.00	-1.00	0.20	0.08	0.06	0.38
	1.00	0.00	0.19	0.06	0.09	0.31
	1.00	1.00	0.17	0.05	0.08	0.29

Note. The indirect relationships shown here are net of the direct relationships described in Table 4. CI = confidence interval.

DISCUSSION AND CONCLUSION

 This study examined relationships between change in leadership, school organizational processes, and student achievement growth, building on past research by using longitudinal data and multiple mediation analysis. Results were consistent with previous cross-sectional research showing the school learning climate is the main organizational process through which leadership is linked to student achievement growth and provide stronger evidence that this is the case. These new analyses further showed that the indirect relationship of change in school leadership with student achievement growth remained largely similar over varying levels of initial conditionsinitial school leadership and initial school safety. These results were contrary to our expectations; we hypothesized that schools that started out with lower levels of school safety and school principal leadership would have larger mediated effects. Even schools that start out with safe climates and strong leadership show benefits in terms of test scores from further improvements; conversely, these schools show declines in test scores if their climates decline over time. It appears that the benefits of improving school safety for student achievement gains do not depend on initial organizational conditions.

Climate can often appear to be a low-level issuesomething to get under control to be able to get down to the "real" work of instructional improvement (e.g., working on curriculum or coaching teachers on their pedagogy). Improving school climate may seem like a goal fitting for schools with very poor climates, where there are frequent fights and chaotic classrooms. These results challenge such perceptions. School safety is not just about avoiding physical threats to safety, such as gangs, weapons, or crime, although it is essential to address such severe problems for learning to occur. Students behavior and their engagement in school are intertwined with their emotional safetywhether teachers and other students put each other down or treat each other with respect. Adults may underestimate the importance of both physical and emotional well-being for students to engage in learning. Evidence on the science of learning and development suggests that relationships, emotions, and social functioning are central to the process of learning (Immordino‐Yang & Damasio, 2007; Osher, Cantor, Berg, Steyer, & Rose, 2018). Leaders who are anxious to improve learning gains in their schools should consider how strongly they are working to improve students' sense of safety and well-being, relative to other work in the school.

The present study contributes to research in two ways. First, it extends work done by Heck and Hallinger (2009) that examined the effects of change in leadership on student achievement growth via school academic capacity. We included multiple mediators as theorized by the Bryk et al. (2010) framework and found that one organizational process mediated all of the influence of improvements in leadershipschool climate as measured by teacher safety. This suggests that improvements in school leadership only lead to improvements in test gains if leaders are successful at improving the climate of the school. This has considerable implications for the ways in which leaders decide on their priorities. As researchers aim to test increasingly sophisticated theories of school organization, we expect that the use of multiple mediation models to examine change in organizational inputs, processes, and outputs will become commonplace. We demonstrated one way involving the use of preliminary HLM analyses and subsequent SEM modeling using PROCESS. Although growth modeling and SEM can be done simultaneously using SEM software like Mplus, the PROCESS program offers an alternative approach and has the advantages of ease of use; the program is run as an add-on in SPSS or SAS, and the predetermined templates are straightforward to adopt.

Second, we examined how mediational relationships may be contingent on certain conditions through moderation mediational analysis. In this case, there were no moderating effects. To our knowledge, this is the first work of its kind in research examining the effectiveness of school leadership. Moderation mediation analysis is a natural fit for conducting leadership research and examining contingency effects. The work conducted here can be considered a simple illustration of the moderation mediation methodology and its application to school organization research. We expect that as discussion on moderation mediation analysis develops further among researchers and programs like PROCESS and Mplus become increasingly sophisticated, moderation mediation analyses will become a natural choice for school organization researchers.

The present study controlled for school demographic characteristics in the preliminary HLM analyses. Subsequent work could include select demographic information as moderating variables. For example, future work can examine if the mediational effects of school safety are different depending on the size of the school or the incoming achievement of students. In this study, although we linked changes in organizational variables to student achievement growth, these changes were estimated over the same time period, from 20062007 to 20122013. The causal ordering of these changes was based on the Bryk et al. (2010) framework. Whether there is empirical backing for this selection of causal ordering (versus other alternatives such as reverse ordering) can be tested with innovative new methods like direction dependence analysis (DDA; Wiedermann & von Eye, 2015a, 2015b) which uses distributions of observed variable and error terms from regression models as diagnostic tests. Alternately, longitudinal information on survey measures can be used in other ways that would provide stronger evidence for causal ordering. For example, models can use information on leadership and organizational variables collected in different points in time (based on the ordering suggested by theoretical models), while controlling for prior levels of the same variables. This is a natural next step for our research work. Finally, the data used in the present study were based on one large urban school district and may not generalize to other contexts. As data on school organization become available in other places, which is starting to occur (see Kraft, Marinell, & Shen-Wei Yee, 2016), these models could be replicated and compared across diverse contexts.

Acknowledgments

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant #R305A120706 to the University of Chicago. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.

References

Barnes, C. A., Camburn, E., Sanders, B. R., & Sebastian, J. (2010). Developing instructional leaders: Using mixed methods to explore the black box of planned change in principals' professional practice. Educational Administration Quarterly, 46(2), 241279.

Bryk, A. S., Sebring, P. B., Allensworth, E., Luppescu, S., & Easton, J. Q. (2010). Organizing schools for improvement: Lessons from Chicago. Chicago, IL: University of Chicago Press.

Camburn, E., Spillane, J., & Sebastian, J. (2010). Assessing the utility of a daily log for measuring principal leadership practice. Educational Administration Quarterly, 46(5), 707737.

Dumay, X., Boonen, T., & Van Damme, J. (2013). Principal leadership long-term indirect effects on learning growth in mathematics. Elementary School Journal, 114(2), 225251.

Grissom, J. A., & Loeb, S. (2011). Triangulating principal effectiveness: How perspectives of parents, teachers, and assistant principals identify the central importance of managerial skills. American Educational Research Journal, 48(5), 10911123. doi:10.3102/0002831211402663

Hallinger, P. (2003). Leading educational change: Reflections on the practice of instructional and transformational leadership. Cambridge Journal of Education, 33(3), 329351.

Hallinger, P. (2005). Instructional leadership and the school principal: A passing fancy that refuses to fade away. Leadership and Policy in Schools, 4(3), 221239.

Hallinger, P., & Heck, R. H. (1996a). The principals role in school effectiveness: An assessment of methodological progress, 198095. In K. Leithwood, J. Chapman, D. Corson, P. Hallinger, & A. Hart (Eds.), The international handbook of educational leadership and administration (pp. 723784). Dordrecht, The Netherlands: Kluwer.

Hallinger, P., & Heck, R. H. (1996b). Reassessing the principals role in school effectiveness: A review of empirical research, 19801995. Educational Administration Quarterly, 32(1), 544.

Hallinger, P., & Heck, R. H. (1998). Exploring the principal's contribution to school effectiveness: 19801995. School Effectiveness and School Improvement, 9(2), 157191.

Hallinger, P., & Heck, R. H. (2009). Distributed leadership in schools: Does system policy make a difference? In A. Harris (Ed.), Distributed leadership: Different perspectives (pp. 101117). Amsterdam, The Netherlands: Springer.

Hallinger, P., & Heck, R. H. (2010a). Collaborative leadership and school improvement: Understanding the impact on school capacity and student learning. School Leadership & Management, 30(2), 95110.

Hallinger, P., & Heck, R. H. (2010b). Leadership for learning: Does collaborative leadership make a difference in school improvement? Educational Management Administration & Leadership, 38(6), 654678.

Hallinger, P., & Murphy, J. (1985). Assessing the instructional management behavior of principals. Elementary School Journal, 86, 217247. doi:10.1086/461445

Hallinger, P., & Murphy, J. (1986). Instructional leadership in effective schools. Retrieved from http://files.eric.ed.gov/fulltext/ED309535.pdf

Hayes, A. F. (2013). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New York, NY: Guilford Publications.

Heck, R. H., & Hallinger, P. (2009). Assessing the contribution of distributed leadership to school improvement and growth in math achievement. American Educational Research Journal, 46(3), 659689.

Heck, R. H., & Hallinger, P. (2010). Collaborative leadership effects on school improvement: Integrating unidirectional- and reciprocal-effects models. Elementary School Journal, 111(2), 226252.

Hitt, D. H., & Tucker, P. D. (2016). Systematic review of key leader practices found to influence student achievement: A unified framework. Review of Educational Research, 86(2), 531569. doi:10.3102/0034654315614911

Immordino‐Yang, M. H., & Damasio, A. (2007). We feel, therefore we learn: The relevance of affective and social neuroscience to education. Mind, Brain, and Education, 1(1), 3–10.

Kmetz, J. T., & Willower, D. J. (1982). Elementary school principals' work behavior. Educational Administration Quarterly, 18(4), 6278.

Kraft, M. A., Marinell, W. H., & Shen-Wei Yee, D. (2016). School organizational contexts, teacher turnover, and student achievement. American Educational Research Journal, 53(5), 14111449. doi:10.3102/0002831216667478

Leithwood, K., Louis, K. S., Anderson, S., & Wahlstrom, K. (2004). Review of research: How leadership influences student learning. Retrieved from https://www.wallacefoundation.org/knowledge-center/Documents/How-Leadership-Influences-Student-Learning.pdf.

Louis, K. S., Dretzke, B., & Wahlstrom, K. (2010). How does leadership affect student achievement? Results from a national US survey. School Effectiveness and School Improvement, 21(3), 315336.

Louis, K. S., Leithwood, K., Wahlstrom, K., & Anderson, S. (2010). Learning from leadership project: Investigating the links to improved student learning. Retrieved from http://www.wallacefoundation.org/KnowledgeCenter/KnowledgeTopics/CurrentAreasofFocus/EducationLeadership/Documents/Learning-from-Leadership-Investigating-Links-Final-Report.pdf

Luppescu, S., & Ehrlich, S. B. (2012). UChicago Consortium Rasch Measurement Model Primer. Retrieved from https://consortium.uchicago.edu/sites/default/files/2019-05/Overview%20of%20Analyses.pdf

Martin, W. J., & Willower, D. J. (1981). The managerial behavior of high school principals. Education Administration Quarterly, 17(1), 6990.

Muthén, L. K., & Muthén, B. O. (2007). Mplus user's guide. Los Angeles, CA: Muthén & Muthén.

Muthén, L. K., & Muthén, B. O. (2013). Mplus statistical analysis (Version 7.11). Los Angeles, CA: Muthén and Muthén.

Osher, D., Cantor, P., Berg, J., Steyer, L., & Rose, T. (2018). Drivers of human development: How relationships and context shape learning and development. Applied Developmental Science. doi:10.1080/10888691.2017.1398650

Raudenbush, S. W., Bryk, A. S., & Congdon, R. T. (2011). HLM 7 (Version 6). Lincolnwood, IL: Scientific Software International.

Sebastian, J., & Allensworth, E. (2012). The influence of principal leadership on classroom instruction and student learning: A study of mediated pathways to learning. Educational Administration Quarterly, 48(4), 626663.

Sebastian, J., & Allensworth, E. (2013). How do secondary principals influence teaching and learning? Principal's Research Review, 8(4).

Sebastian, J., Allensworth, E., & Huang, H. (2016). The role of teacher leadership in how principals influence classroom instruction and student learning. American Journal of Education, 123(1), 69108.

Sebastian, J., Allensworth, E., & Stevens, D. (2014). The influence of school leadership on classroom participation: Examining configurations of organizational supports. Teachers College Record, 116(8), 136.

Sebastian, J., Huang, H., & Allensworth, E. (2017). Examining integrated leadership systems in high schools: Connecting principal and teacher leadership to organizational processes and student outcomes. School Effectiveness and School Improvement 28(3), 463488. doi:10.1080/09243453.2017.1319392

Sebring, P. B., Allensworth, E., Bryk, A. S., Easton, J. Q., & Luppescu, S. (2006). The essential supports for school improvement. Research report. Retrieved from https://consortium.uchicago.edu/publications/essential-supports-school-improvement

Spillane, J. P., & Hunt, B. R. (2010). Days of their lives: A mixed-methods, descriptive analysis of the men and women at work in the principal's office. Journal of Curriculum Studies, 42(3), 293331.

Steinberg, M. P., Allensworth, E., & Johnson, D. W. (2013). What conditions jeopardize and support safety in urban schools? The influence of community characteristics, school composition and school organizational practices on student and teacher reports of safety in Chicago. Retrieved from https://www.civilrightsproject.ucla.edu/resources/projects/center-for-civil-rights-remedies/school-to-prison-folder/state-reports/copy2_of_dignity-disparity-and-desistance-effective-restorative-justice-strategies-to-plug-the-201cschool-to-prison-pipeline/steinberg-conditions-support-ccrr-conf-2013.pdf

Supovitz, J., Sirinides, P., & May, H. (2010). How principals and peers influence teaching and learning. Educational Administration Quarterly, 46(1), 3156.

Wiedermann, W., & von Eye, A. (2015a). Direction of effects in mediation analysis. Psychological Methods, 20(2), 221244. doi:10.1037/met0000027

Wiedermann, W., & von Eye, A. (2015b). Direction of effects in multiple linear regression models. Multivariate Behavioral Research, 50(1), 2340. doi:10.1080/00273171.2014.958429

Wolcott, H., F. (1973). The man in the principals office. New York, NY: Holt, Rinehart, & Winston.

APPENDIX

Survey Measures and Corresponding Items

Measures		Item	Value
Leadership	Principal Instructional Leadership Reliability = .79 ICC = .16	The principal at this school: 1. Makes clear to the staff his or her expectations for meeting instructional goals; 2. Communicates a clear vision for our school; 3. Understands how children learn; 4. Sets high standards for student learning; 5. Presses teachers to implement what they have learned in professional development; 6. Carefully tracks student academic progress; 7. Knows whats going on in my classroom; 8. Participates in instructional planning with teams of teachers	1. Strongly Disagree; 2. Disagree; 3. Agree; 4. Strongly Agree
	Teacher-principal Trust Reliability = .84 ICC = .22	Please mark the extent to which you disagree or agree with each of the following: 1. The principal has confidence in the expertise of the teachers; 2. I trust the principal at his or her word; 3. It is okay in this school to discuss feelings, worries, and frustrations with the principal; 4. The principal takes a personal interest in the professional development of teachers; 5. The principal looks out for the personal welfare of the faculty members; 6. The principal places the needs of children ahead of personal and political interests; 7. The principal at this school is an effective manager who makes the school run smoothly	1. Strongly Disagree; 2. Disagree; 3. Agree; 4. Strongly Agree
Professional Community	Collective Responsibility Reliability = .78 ICC = .16	How many teachers in this school: 1. Help maintain discipline in the entire school, not just their classroom; 2. Take responsibility for improving the school; 3. Feel responsible to help each other do their best; 4. Feel responsible that all students learn; 5. Feel responsible for helping students develop self-control; 6. Feel responsible when students in this school fail	1. None; 2. Some; 3. About half; 4. Most
	Collaborative Practice Reliability = .74 ICC = .18	This school year, how often have you: 1. Observed another teacher's classroom to offer feedback; 2 Observed another teacher's classroom to get ideas for your own instruction; 3. Gone over student assessment data with other teachers to make instructional decisions; 4. Worked with other teachers to develop materials or activities for particular classes; 5. Worked on instructional strategies with other teachers	1. Never; 2. Once or twice; 3. 3 to 9 times; 4. 10 or more times;
	New Teacher Socialization Reliability = .71 ICC = .14	1. Experienced teachers invite new teachers into their rooms to observe, give feedback, etc.; 2. A conscious effort is made by faculty to make new teachers feel welcome here	1. Strongly Disagree; 2. Disagree; 3. Agree; 4. Strongly Agree
Quality of Programs	Professional Development Quality Reliability = .79 ICC = .18	Overall, my professional development experiences this year have: 1. Been sustained and coherently focused, rather than short-term and unrelated; 2. Included enough time to think carefully about, try, and evaluate new ideas; 3. Been closely connected to my schools improvement plan; 4. Included opportunities to work productively with colleagues in my school; 5. Included opportunities to work productively with teachers from other schools	1. Strongly Disagree; 2. Disagree; 3. Agree; 4. Strongly Agree
	Program Coherence Reliability = .82 ICC = .23	1. Once we start a new program, we follow up to make sure that its working; 2. We have so many different programs in this school that I cant keep track of them all; 3. Many special programs come and go at this school; 4. Curriculum, instruction, and learning materials are well coordinated across the different grade levels at this school; 5. There is consistency in curriculum, instruction, and learning materials among teachers in the same grade level at this school	1. Strongly Disagree; 2. Disagree; 3. Agree; 4. Strongly Agree
Learning Climate	Teacher Safety Reliability = .88 ICC = .22	To what extent is each of the following a problem at your school: 1. Physical conflicts among students; 2. Robbery or theft; 3. Gang activity; 4. Disorder in classrooms; 5. Disorder in hallways; 6. Student disrespect of teachers; 7. Threats of violence toward teachers	1. Not at all; 2. A little; 3. Some; 4. To a great extent

Cite This Article as: Teachers College Record Volume 121 Number 9, 2019, p. 1-32 https://www.tcrecord.org ID Number: 22783, Date Accessed: 10/22/2021 3:44:10 PM Purchase Reprint Rights for this article or review