Making Adequate Yearly Progress: Teacher Learning in School-Based Accountability Contexts

by Carol Rinke & Linda Valli - 2010

Context: This study addresses recent changes in professional development policy, practice, and theory, in which professional development has increasingly become continual, collaborative, and school based. We consider both traditional notions of structure and content as well as context in developing a more complete understanding of professional development for today’s teachers.

Purpose: We conducted this study to understand more fully the delivery of school-based professional development within a high-stakes accountability context. We build on the accountability, professional development, and school context literature and expand the explanatory framework to include teacher experience with, and use of, school-based professional development in instructional practices.

Research Design: We conducted case studies of professional development in three elementary schools with varying levels of pressure to make adequate yearly progress. Although these three school sites operated within the same federal, state, and district policy contexts, the school contexts varied considerably. We focused in particular on professional development around short constructed responses, a form of paragraph writing required for the state exam.

Conclusions: We argue that school leadership, culture, and resources, as well as the structure and content of professional development, filter policy initiatives before they ultimately shape teacher learning experiences. Although high-stakes accountability policies increased the quantity of professional development, the quality of teacher learning opportunities depended on the context in which it was delivered.

In recent years, American schools have been operating within an increasingly high-stakes accountability context. The growth in statewide standardized testing, coupled with pressure to make adequate yearly progress (AYP) under the provisions of the No Child Left Behind Act of 2001, has promoted widespread policy and practice changes at the state, district, and school levels. In addition to their impact on teacher hiring, curriculum, and instruction, these changes have major implications for professional development. Because of the importance of professional development to classroom practice and, ultimately, to student learning (Fishman, Marx, Best, & Tal, 2003), research on these implications is critical. The research discussed in this article examines the practice of professional development in three elementary schools that function within a high-stakes accountability context. Although a variety of definitions currently exist for professional development, from the instruction of teachers in research-based practices (No Child Left Behind Act, 2002) to experiential opportunities for problem-oriented learning (Smylie, 1995), we consider professional development broadly as opportunities for teachers to learn about and improve their teaching and student learning.

As in many school districts, the current high-stakes accountability context generated a number of policy changes in the school district under study. First, to comply with NCLB regulations for student testing, the state had to change its testing program in fundamental ways. This change influenced the redesign of the school districts curriculum, which was already being realigned with state content standards. With new curriculum frameworks and instructional guides rolling out from the district to the schools on a regular basis, significant professional development was necessary if teachers were to keep up with changing demands. Second, federal Title I money allowed for greater support and mentorship for classroom teachers in high-needs schools, providing funds to supplement the investment the school district had already made in school-based staff developers and curriculum specialists. Third, the pressure to make AYP prompted school and school district leaders to make scheduling and other adjustments to allow for greater teacher learning, collaboration, and support. This confluence of factors put increased emphasis on school-based professional development, warranting a closer look at its implementation.

In the school district we studied, approaches to professional development were being implemented that looked strikingly different from traditional practices, in which professional development is delivered by the school district, universities, or other providers (Hawley & Valli, 1999; Little, 1989). An increased understanding of how teachers learn (Bransford, Brown, & Cocking, 2000), coupled with a clearer conception of what constitutes effective professional development models (Hawley & Valli, 1999, 2007; Novick, 1996), began to change how professional development was implemented. What used to be one-shot became more continual; what used to be top-down became more collaborative; and what used to be conducted by experts outside the school setting became more school based. For example, in this school district, teachers attended summer workshops followed by ongoing in-school support throughout the year, and each of the case study schools implemented some form of collective teacher-led learning that was integrated into the daily life of the school. Although the district continued to provide direction by training school-based leaders in a trainer-of-trainers model, most professional development took place among teachers at the school site.

In addition to these changes in policy and practice, an expanded view of professional development also began to emerge from research. Professional development has been traditionally theorized and studied along two dimensions: structure (the organization of learning activities) and content (the substance of those activities). According to extant models (e.g., Garet, Porter, Desimone, Birman, & Yoon, 2001; Kennedy, 1998; Thompson & Zeuli, 1999), the quality and effectiveness of professional development depends on these two key variables. However, recent work indicates that a third factorcontextalso plays a vital role. Lieberman and McLaughlin (2000), for example, found that the success of professional development opportunities, such as participation in the National Writing Project, is context specific and highly dependent on the extent to which teachers work in settings that allow them to incorporate what they have learned into their classroom practice (p. 233). In their reviews of the literature, Hawley and Valli (1999, 2007) also noted the importance of situating professional development within a comprehensive change context with supports for teacher learning. And the National Staff Development Council (NSDC, 2001) provides guiding standards for professional development, which include district and school context factors: leadership, learning communities, and resources.

This study builds on these changes in professional development policy, practice, and theory by asking, What is the nature of teachers learning opportunities when professional development is delivered within a high-stakes accountability policy context? To address that question, we look at three cases of school-based professional development in which teachers experience varying degrees of AYP pressure. We find that although the school district presumes similar opportunities to learn, teachers learning experiences are filtered through particular school contexts. Although teachers generally had more access to learning opportunities, the quality of their professional development was not necessarily better. Teacher experience with professional development varied with the schools leadership, culture, and resources, and pressure to make AYP. Based on this research, we argue for a more complex conceptual model for understanding teachers experience with school-based professional development, one that incorporates both policy and school contexts.

CONCEPTUAL FRAMEWORK

Three bodies of literature informed this article: literature on high-stakes accountability, professional development, and school context. Although prior studies in these areas do not directly ask the same research question, when examined as a whole, the literature sheds light on the delivery of school-based professional development within a high-stakes accountability environment. In this section, we highlight key features from these areas of scholarship and explain the ways in which we draw on related studies to conceptualize and examine teachers experience with school-based professional development in a high-stakes context.

HIGH-STAKES ACCOUNTABILITY

A high-stakes accountability context is characterized by public pressure to improve school performance, particularly on standardized exams (Malen & Rice, 2004), and has been shown to affect teachers, students, curriculum, and school culture (Craig, 2004; Dorgan, 2004; Jones et al., 1999). Research findings indicate that high-stakes accountability policies result in a test-driven school culture, a narrowed curriculum, and instructional methods that support test achievement (Boardman & Woodruff, 2004; Smith, 1991; Valli, Croninger, Chambliss, Graeber, & Buese, 2008). Boardman and Woodruff found, for example, that teachers use upcoming high-stakes tests as a reference point for instructional change. A study by Diamond and Spillane (2004) further shows how the impact of an accountability context varies with the degree of external policy pressure. Feeling pressure to meet accountability standards, low-performing urban schools target the achievement of particular students on a narrow range of content, whereas high-performing schools educate more broadly. Overall, teachers in high-stakes contexts tend to teach to the testthat is, they increase instruction on tested topics and eliminate instruction on nontested topics (McNeil, 2000; Moore, 1994).

Studies also indicate that standardized testing takes a personal and emotional toll on teachers, who often feel pressure to raise test scores but do not believe that the time and cost of the tests is worth the effort (Haladyna, Nolen, & Haas, 1991; Moore, 1994; Smith, 1991). Thus, high-stakes accountability can produce negative attitudes and low teacher morale. But the literature also points to the different ways in which school districts and schools respond to and shape teacher experiences with high-stakes accountability. District and school context appear vital to the ways in which accountability policies are implemented (Firestone, Schorr, & Monfils, 2004; Ingram, Louis, & Schroeder, 2004; Sipple & Killeen, 2004). The districts will to carry out reform initiatives, the schools use of student performance data, and the availability of human and organizational resources all affect local responses to high-stake accountability policies. Our study expands on these lines of inquiry by investigating teachers experiences with high-stakes accountability in district-initiated school-based professional development settings.

PROFESSIONAL DEVELOPMENT

As noted in numerous studies, various characteristics of the structure and content of professional development affect its quality. Structure is generally conceived of as the organization, mode of participation, and timing of professional development. Features such as sustained and intensive participation, coherence with the taught curriculum, and active involvement of teachers in one school or grade level have significant, positive effects on teacher knowledge, skills, and integration in the classroom (Hawley & Valli, 1999, 2007; Garet et al., 2001; Penuel, Fishman, Yamaguchi, & Gallagher, 2007). In their recent study of effective professional development, defined as strategies that foster curriculum implementation, Penuel et al. confirmed earlier studies of key predictor variables: coherence of the professional development with district goals for student learning, collective participation by teachers, and time to plan for implementation.

Similarly, content, defined as the substance or subject matter of professional development, is a critical part of its quality. Enhancing teacher knowledge of content, curricula, and students appears to be the most effective focus for professional development in terms of raising student achievement (Kennedy, 1998), and a strong focus on content and curriculum in professional development can, under the right circumstances, affect classroom practice and student learning (Cohen & Hill, 2000). There is also strong consensus about the importance of focusing on goals for student learning and creating cognitive dissonance about professional beliefs in order to generate sustained use in the classroom (Hawley & Valli, 1999, 2007; Gersten, Vaughn, Deshler, & Schiller, 1997; Thompson & Zeuli, 1999). Penuel et al. (2007) cautioned, however, that professional development must be localized and that providers must consider how their expectations for pedagogical change can be met within teachers own contexts. We draw on these notions of structure, content, and context to examine teachers experiences with professional development that is delivered in school-based settings.

SCHOOL CONTEXT

The literature on school context identifies several factors as relevant to school-based professional development, but we found three to be particularly salient because of the ways in which they work together to influence the structure and content of professional development. Together, a schools leadership, culture, and resources frame the context in which professional development is carried out. This literature suggests that these three factorsschool leadership, culture, and resourcesmatter not only for the outcomes of professional development, as indicated, but also for teachers immediate experiences with, and opportunities to learn from, professional development activities.

School and district leadership has been shown to support learning opportunities by developing trust (Youngs & King, 2002), focusing on particular goals (Desimone, Porter, Birman, Garet, & Yoon, 2002), negotiating administrative and teacher needs (Scribner, 1999), and generally supporting professional development initiatives (Desimone et al.; Newmann, King, & Youngs, 2000). Moreover, principals and district leaders also provide access to learning opportunities by creating structures for professional development (Firestone, Mangin, Martinez, & Polovsky, 2005; Youngs & King) and shaping professional development content (Spillane, 2002). In this study, we focus in particular on principal leadership in each of the three schools, defining and operationalizing leadership as the ways in which principals were able to provide teachers access to necessary resources for professional development and to positively shape school culture around the value of teacher learning.

School culture, understood broadly as school ethos, or school personnels shared meanings and patterns of behavior (Valli et al., 2008), is also central to school-based professional development. Although we recognize the reciprocal relationship between school culture and professional development (Gamoran, Secada, & Marrett, 2000), we are specifically interested in how school culture affects the delivery of professional development. The literature indicates that school culture shapes teacher response to professional development (Hamilton & Richardson, 1995; Scribner, 1999), affects the energy that teachers have available for professional development (Scribner), and influences teachers perceptions of the role of professional development (Scribner). Through normative expectations for teacher participation, interaction, and understandings, school culture becomes a powerful force in how teachers approach school-based professional development (Hamilton & Richardson). Based on this literature, we operationalize the term school culture as patterns in teachers behavior and language that reveal tacit understandings of how things are done around here.

Finally, school resources also appear to matter for professional development. Research on school resources indicates that time and money influence both the quality of, and access to, professional development in school settings (Scribner, 1999). Because school districts often work with schools to expand resources, research at the district level can shed light on this concept. Here, research shows the importance of both a wide range of resources available for professional development and the use of those resources. Money, time, and materials are widely regarded as invaluable resources. In addition, human capital, such as teacher commitments, dispositions, and knowledge, as well as social capital, such as professional relationships and teachers collective will to change practice, influence both the use of resources and teacher productivity (Spillane & Thompson, 1997). Because money, time, and materials were roughly equivalent at the three schools and determined at the district rather than the school level, this study focused on human and social resources, operationalized as the internal expertise and relationships available at each school site for conducting and engaging in professional development.

Considered individually, each component of the conceptual frameworkthe high-stakes context, the structure and content of professional development, and the leadership, culture, and resources of the school contextspeaks to the question of how school-based professional development is carried out in a high-stakes context. We expand on this framework by looking holistically at the implementation of high-stakes accountability policies in three distinct school contexts, teacher learning opportunities within these contexts, and the subsequent impact of professional development on classroom practice. The study builds specifically on research that  examines how high-stakes accountability policies play out in schools (Diamond & Spillane, 2004), looks at individual experiences with professional development (Scribner, 1999), and links policy to practice (Cohen & Hill, 2000). We combine these perspectives to explore ways in which high-stakes accountability policies are implemented in various school contexts and the role that implementation plays in teachers opportunities to learn from their professional development experiences.

RESEARCH METHODOLOGY

Our inquiry into school-based professional development is part of a longitudinal mixed-methods study of fourth- and fifth-grade reading and mathematics teachers. A basic assumption of that larger 4-year study is that understanding teaching requires looking not only at what teachers know and do but also at the context in which teachers work. This particular part of the study focuses on the policies and organizational factors that influence teaching. These three schools were selected for case studies because they represented the types of schools we targeted for study: They served large numbers of low-income students and had substantial populations of English language learners (ELLs). Each school also had a racially and ethnically diverse student population, which included African American, Hispanic, White, and Asian students. Beyond being representative of 16 participating schools, we also chose these three schools because they were at varying degrees of risk for meeting 20042005 AYP requirements.

DATA COLLECTION AND ANALYSIS

Data for this article were gathered primarily during the last year of the study (20042005) and consisted of three primary sources: interviews, observations, and artifacts. Principals were interviewed twice, at the beginning and end of the school year. Interviews were also conducted with mathematics and reading specialists, staff development teachers, English for speakers of other languages (ESOL) teachers, and special needs teachers. Grade-level teachers were interviewed in focus group settings, for a total of 31 interviews across the three schools. All interviews were audiotaped and transcribed. Altogether, interviews were conducted with all the administrators, specialists, and classroom teachers in the grade levels under study. Observations were conducted at grade-level, professional development, school improvement, and whole staff meetings throughout the school year, producing 135 sets of field notes. Relevant artifacts, 63 in all, were collected from both interview sessions and meeting observations. These artifacts included professional development materials, lesson plans, student worksheets, and school policies. This large data set was stored and managed on a Web-based platform to allow researchers joint access to the data.

Because the case studies focused on fourth- and fifth-grade teachers, members of the research team attended professional development sessions specifically designated for these teachers, some of which occurred in grade-level teams, some with specialists, administrators, and consultants, and some with teachers across grade levels. Thus, we were able to determine whether teachers experiences with professional development occurred mostly on a grade-level versus schoolwide basis. The analysis of observation field notes over the course of the full school year helped us to determine how professional development topics were treated at each school (e.g., repetitive or varied and coherent or disconnected). Interviews with teachers, specialists, and administrators, as well as observations in classrooms and professional development sessions, illuminated teacher experience with professional development and approaches to integrating new practices into the classroom. Although the intensive studies of these three case schools took place across only one school year, related data from previous years suggest that patterns we were seeing in all three schools were firmly rooted in leadership styles, school cultures, and resource use. The primary difference was the increased emphasis, in all three schools, on preparing students for the short constructed response (SCR) portion of the state test.

A number of analyses on the case study data were conducted using NVivo qualitative data analysis software. These analyses were guided by the overarching research questions aimed at understanding the policies and organizational factors that influence teaching. First, all the case study data were coded (Bogdan & Biklen, 2003) using broad categories such as curricula, assessment, school culture, accountability, school goals and priorities, professional development, resources, school community, and teaching. These primary categories were then broken into subcategories, which for professional development included time issues, curriculum rollouts, ELL issues, special education training, professional development for specialists, content of professional development, teachers response to professional development, and professional development planning. To ensure consistency of coding, we made operational definitions of ambiguous terms such as school culture and resources available within the NVivo project that we created to house and code the data.

Once the data were coded, we conducted within-case analyses (Miles & Huberman, 1994), developing matrices to understand the guiding policies, key players, and organizational impact on professional development at each of the three case study sites. We followed this with a cross-case analysis of variables that emerged as important during initial discussions, including professional development content, structure, focus, and goals, as well as teachers responses to professional development, responses to student needs, and opportunities for learning at the specialist and administrator level. Our goals were to understand what similarities and differences existed among study sites and to determine what factors might play a role in shaping this variation. By looking for strong patterns within schools and patterns of variation across schools, we determined the general categories and the within-school variations that would guide the case presentations. Throughout this analytic process, we looked iteratively between the data and the literature to understand professional development implementation at the three schools, ultimately developing a model of school-based professional development.

STUDY CONTEXT

The three case study schools were situated within one of the largest and most diverse school districts in the nation. At the time of the study, the school district had over 130,000 students, with fewer than 50% categorized as White.¹ More than 30% of students districtwide were enrolled in the Free and Reduced-Price Meals (FARMS) program, and 20% of all students had been in the ESOL program. Schools initially selected for participation in the study had moderate to high levels of poverty and higher than expected levels of achievement. From this initial set of schools, the three case study schools were chosen based on levels of poverty and relative success at making AYP. Two of the schools had been previously designated Title I schools; the third school was on the borderline (Table 1). Brookfield Elementary² was the smallest of the three schools and the only case study site not designed as Title I, although it became Title I the year after the study. Although Brookfield had the lowest FARMS rate among the three schools, it enrolled the highest combined percentage of special education and ESOL students. Hawthorne was the largest of the three schools, a Title I school with a highly diverse student body. With over 76% of its students enrolled in FARMS, Cherry Ridge served the most economically disadvantaged population of the three schools. More than 25 countries were represented and 16 languages spoken by Cherry Ridge students. Cherry Ridge also had a relatively inexperienced teaching staff, with a third of the professional staff in their first 5 years in the profession. At the fourth- and fifth-grade levels, more than half of the classroom teachers had less than 2 years of teaching experience.

Table 1. Case Study School Demographics

click to enlarge

The three schools differed with respect to student poverty levels and pressure to make AYP, but they operated within the same multilayered, high-stakes accountability context. Federally, NCLB requires states to implement accountability systems based on rigorous standards and to report those assessment results by poverty, race, ethnicity, disability, and limited English proficiency. Under NCLB, states establish a baseline and identify annual measurable objectives (AMO) for schools to demonstrate AYP on state-designated proficiency goals. The state determines the specific percentage of students from each demographic category that has to pass state assessments at proficient or advanced levels for that particular year. Schools that fail to make AYP are subject to improvement, corrective action, and, ultimately, restructuring. NCLB also shapes professional development in two ways: by authorizing grants at the state and district level for improving teacher quality, and by defining professional development as activities that produce measurable effects on student achievement and that are grounded in scientifically based research (NCLB, 2001). This approach differs from the definition of high-quality professional development in the academic literature in its emphasis on narrow but quantifiable outcomes.

At the state in which we conducted our study, these federal requirements were enacted and monitored through content standards across grade levels, a battery of statewide tests, a recommended state curriculum that linked standards and assessments, regulations regarding inclusion, and goals for AYP. The state test, a driving force for teaching and learning at each of the three school sites, was influenced heavily by NCLB assessment and accountability requirements. It was introduced during the 20022003 school year in Grades 3 and 5 in reading and mathematics and was expanded to include students in Grade 4 during the 20032004 school year.

The school district, in turn, developed comprehensive reading and mathematics curriculum frameworks by using state, national, and international content standards. During the 20042005 school year, the district mathematics curriculum was in its 3rd year of use, but the fourth- and fifth-grade reading curriculum was being rolled out to schools and teachers on a quarterly basis. The district was also in various stages of implementing several additional initiatives, including a data-based management system, new grading and reporting policies, and redesigned professional development expectations. Brookfield, Hawthorne, and Cherry Ridge operated within a complex policy environment in which the federal government, state department of education, and school district each operated in a loosely coordinated manner to emphasize accountability, assessment, and data-driven decision-making.

In response to changes in the accountability system, the school district issued several directives specific to professional development. First, the district placed a staff developer in each of the schools to support teachers. Second, the district provided ongoing training for specialist teachers in reading and mathematics, which they were expected to impart to classroom teachers at their school sites. And third, the district required a certain number of staff development hours from teachers, to be used in a flexible manner at their discretion. Together, these district directives resulted in ongoing and continual professional development at each of the school sites throughout the 20042005 school year. Structurally, this professional development was consistent with general recommendations regarding high-quality opportunities for teacher learning, including collective participation, extensive duration, a school-based setting, and integration with a larger reform process (Garet et al., 2001; Hawley & Valli, 1999, 2007).

Moreover, each of the three schools participated in professional development around the same content. In these cases, we look at professional development that supports teachers work with one particular type of test question that required a paragraph-length response, which we  termed a short constructed response (SCR). This question-answer format was critically important because students needed to be able to write this type of response on the state test in order for the schools to make AYP. Although professional development on SCRs was not a requirement, they were a district priority and an area of attention for all three principals. Teachers professional development around SCRs took a variety of forms, including modeling proficient SCRs, assessing and scoring SCRs, developing examples of SCRs for use in the classroom, and sharing strategies for the teaching of SCRs. Although some of these activities might not, in isolation, be considered professional development, they were all part of a larger yearlong effort to build teachers skills in working with SCRs. Therefore, we considered any joint activity focused on SCRs as meeting our definition of professional development because teachers had opportunities to learn about and improve their teaching and student learning. These activities were largely consistent with recommendations around professional development content, including a specific emphasis on enhancing teacher knowledge about curriculum and a focus on student learning (Gersten et al., 1997; Hawley & Valli, 1999, 2007; Kennedy, 1998).

Despite common district directives and a common focus on SCRs, professional development differed quite notably at the three school sites. In the following cases, we try to provide thorough descriptions of the delivery of SCR professional development. For each of the three schools, we describe the high-stakes accountability context, the structure and content of professional development, and the schools leadership, culture, and resources. We then examine the ways in which teacher understanding and practice differed at the three school sites (see Table 2). Our discussion makes evident our own bias toward professional development that values theoretically based, collaborative work over more routine skills training for teachers.  

Table 2. Policy and School Contexts

click to enlarge

CASE 1: BROOKFIELD ELEMENTARY SCHOOL

Of the three case study schools, Brookfield Elementary was experiencing the least pressure to make AYP. However, even though the school had been meeting the states AYP standard for all eight subgroups of students, the administration and staff still remained concerned about making adequate progress, particularly in reading. The principal described meeting AYP as the highest priority for her school, and the math content coach explained, Were always looking at AYP and the next steps to get toward it. This sense of pressure, even at a school like Brookfield, is understandable when one considers the federal goal of 100% proficiency by 2014. Furthermore, the states AMOs increased for the 20042005 school year by approximately 10% in both reading and mathematics, making AYP considerably more difficult to achieve as compared with the previous year. Within this policy context, Brookfields professional development was structured into half-day grade-level professional development sessions, bimonthly staff meetings, and weekly grade-level instructional planning meetings.

The structure of Brookfields professional development differed from that of the other schools regarding its use of whole-school staff meetings. Brookfield students at all grade levels were expected to work toward developing SCR skills, with even the youngest students participating. The reading specialist and staff developer discussed this orientation at one Instructional Leadership Team meeting, arguing that teachers at earlier grade levels needed to look at what was expected of fifth graders in order to determine and model expectations at each grade level. They put considerable emphasis on vertically aligning what teachers were doing at all grades to prepare students for SCRs. Later, at a January staff meeting, teachers across all grade levels brainstormed age-appropriate approaches to preparing students for SCRs. Some of their questions and strategies included helping students understand the question, teaching students how to transfer their thinking while using manipulatives into a written explanation, and modeling a prewriting response. Brookfields professional development seemed almost ideally designed to foster comprehensive schoolwide participation around SCRs.

In addition, the SCR-related content of professional development at Brookfield was delivered using a variety of strategies, including analyzing SCR characteristics, modeling instructional methods, assessing sample responses, and reflecting on progress. For example, at one staff meeting, teachers completed a Venn diagram comparing SCRs in reading with SCRs in mathematics. The teachers distinguished between content-specific aspectssuch as using information from the text in reading, and using equations in mathematicsand general features of SCRs like critical analysis and specific vocabulary. The math specialist explained,

At that staff meeting . . . we kind of related our math SCRs to reading SCRs, so that its not a totally different beast that were trying to tackle; its the same instructional strategies that we would use to approach reading SCRs, and then . . . we took it back to looking at SCRs of the kids and grading of SCRs, sharing that.

Brookfield teachers also had opportunities to have SCR instruction modeled. At one staff meeting aimed at developing a common understanding of SCRs, teachers were taught using activators or trigger words, an information packet, a treasure hunt, and a summarizer, all strategies that could be translated into classroom instruction for students. In addition, teachers shared strategies they had found to be successful in teaching SCRs. The observer noted,

The teachers then began to share how theyre teaching their students to do SCRs. One said she has the students underline components of the text that relates [sic] to the paragraph response in different colors, another talked about examining the question with her students, and another said it was a good idea to have students check each others work because they tended to be more objective about each others work than their own. The discussion was very collaborative and thoughtful. The strategies the teachers devised to teach students how to answer SCRs were specific, enabling other teachers to interject comments about each others ideas.

Classroom teachers monitored both student progress on SCRs and their own understanding throughout the school year. The reading specialist explained how student monitoring occurred:

[The students] wrote . . . SCRs . . . and then they were scored and sometimes we used to get together and everybody would score together, other times they all score and then we come back here and talk. What do you see? Sometimes they looked at certain types of questions. . . . Maybe well focus and work on that and then give that question again next time and see if we see improvement.

The teachers also used metacognitive skills to assess their own understanding of SCRs. One fall meeting began with teachers being asked to assess and declare their understanding. With the help of lighthearted graphic, a stop light, that they might use with their students, teachers did not hesitate to signal that they needed support. The reading specialist instructed teachers to cut out a go, slow, or stop sign from the sheet of paper at their table indicating where they were with their knowledge of SCRs and post it on the bulletin board by the appropriate signal light on the graphic. Later, at the conclusion of the staff meeting, teachers were asked to answer the following question: Using what you know about SCRs, explain how your knowledge of SCRs has changed or stayed the same. Use information from our staff meeting to support your answer. They were then told that this summarizer was itself an SCR-type question, further modeling SCR use in professional development. Overall, teachers across grade levels at Brookfield engaged with SCRs using a range of strategies, including analyzing, teaching, assessing, and reflecting on SCRs.

As described next, these professional development activities took place within a school context that functioned like a strong professional community, defined by its clear expectations for teacher work and collaborative effort to improve practice (McLaughlin & Talbert, 2001). Moreover, this school had the feel of a close-knit family. The principal was supportive of school change and inclusive of specialists and classroom teachers in the planning of professional development. The school culture was one of active engagement, with strong reliance on internal resources. Classroom teachers and teacher leaders conducted most of their own professional development. Ultimately, Brookfield teachers appeared to have in-depth understandings of SCR use, instruction, and assessment, implementing SCRs into their classrooms using an analytic approach.

Brookfields principal could be considered supportive of the process of reform and the professional development necessary to create change. She held the district superintendent in high esteem and believed that immediate change was indeed necessary for continued success, including the introduction of a new curriculum and additional professional supports for teachers. She commented, I think that the school district has really been very proactive. I have to compliment Dr. Barnes on that. I mean, he really set up that sense of urgency, weve got to change this, weve got to move, weve got to get this done. Moreover, she believed that the school district was headed in the right direction by giving teachers the support they needed to enact change: I think that the teachers have felt supported by some of the additional staff that has been added in order to help them become better teachers . . . and they feel that the system really is looking at their profession and making them better teachers.

The principal also viewed teachers as stakeholders in their own learning and designed professional development around their feedback. She took teacher time commitment, teacher feedback on reforms, and even parent concerns into account in the ongoing structuring of professional development schoolwide. She directed the Instructional Leadership Team to turn to teachers in assessing progress on the SCR goal. With respect to the creation of the professional development calendar, she commented,

You know, you have to continuously look at what youre doing and how youre going to make it mush. And you have to get the temperature of the staff. . . . I try to get it [the calendar] set through December and then well look in November/December, where are we? How are we doing? What do we need to change? What do we need to add?

The staff developer reiterated the same sentiment, explaining that she and the other specialist teachers try to get the input from staff, what it is that they need in order to be able to do their job as far as professional development and try to base the years staff development [on their needs].

Moreover, the principal encouraged Brookfield teachers to help structure their own learning experiences around SCRs. During one Instructional Leadership Team meeting, whose membership included grade-level team leaders, the observer noticed a substantive discussion around preparing for SCR professional development:

Teachers discussed ideas for a staff development at a staff meeting . . . Leslie Gabriel [the fifth-grade team leader] suggested that they have examples of SCRs from prior years. Examples of what is a 1 response, 2 response, and 3 response. The principal suggested that they start a staff meeting by examining a fourth grade 3 response and then discuss what it might look like at each grade. Teachers identified other sources of examples of SCRs that could be used.

At this meeting, teacher leaders and administrators were equally involved in the planning process. At the same meeting, an observer wrote that the principal requested a teacher volunteer to help, saying it is always more powerful if teachers participated in presenting. Believing that the overall success of professional development depended on teachers investing themselves in the process, the principal was careful to include them in the planning.

As suggested by the preceding paragraphs, Brookfields culture was characterized by active engagement in, and positive perceptions of, professional development. Teachers at Brookfield shared patterns of behavior characterized by participation in their own learning rather than the passive reception of information. During one professional development session, teachers across the school worked in teams to develop action plans for teaching SCRs. The fourth-grade team, for example, came up with the following steps:

Do you [students] understand the question?

Show the students the rubric that they [teachers] developed.

Give the students the five steps to a better answer.

Student checking of their own work and student scoring of SCRs

During . . . test prep, they do SCRs together in a whole group rather than independently.

In another situation, the teachers took their ownership of professional development as far as challenging official thinking on SCRs. The reading specialist, Karen McNeil, and the staff developer, Lottie Breman, openly disagreed with district policy even though they were generally strong supporters. During one staff meeting, they took exception to a district handout that said, When modeling a written response, do not begin the response by restating the question[;] . . . [this restricts] a students response. Ms. McNeil adamantly disagreed with this directive. She said that some students needed to restate the question. When teachers prepared students for the previous state test, they were encouraged to have students restate the answer, but now that the new state test restricts the space in which students have to write their response (because the tests are scanned), the district is telling teachers that this previous technique is no longer endorsed. Ms. Breman and Ms. McNeil told the teachers that students should be encouraged to recycle the question in their responses. They did not see evidence in the previous years test administration that recycling the question made the students answers too long.

In addition to assuming ownership, Brookfield teachers developed common meanings and orientations toward their learning opportunities. They appeared to view professional development as a serious learning experience, listening, taking notes, and working immediately and intently during professional development sessions. They seemed to see value in teaching SCRs. One fifth-grade teacher, who was in her 35th year of teaching, commented, I think that [the students] have to be able to express in writing to show their understanding because its the only way people know what theyre doing. In another instance, she joked, Oh yes, I practically invented SCRs [because] students need help writing. Brookfield teachers also looked for opportunities to continue learning. Despite numerous professional development and staff meetings, some of which ate away at personal time, teachers commented that they would like more time to visit one anothers classrooms and learn from each other.

Brookfield was also characterized by strong internal resources for professional development. Professional development sessions were almost exclusively teacher led, from grade-level meetings to whole-group staff meetings. The reading specialist explained how she and her colleagues took ownership over professional development: So we wanted to make sure that teachers had an understanding of what an SCR is and how would you provide instruction to your students so that they would be able to accomplish that goal of writing an SCR and looking at student work through that way. Brookfield was the only one of the three schools that did not bring in an outside consultant to guide professional development, relying instead on its own staff for leadership.

Brookfield teachers experiences with professional development were characterized by an understanding of SCR instruction and assessment. Teachers demonstrated the ability to think deeply about complex issues related to SCRs. The staff developer commented,

We spent a lot of time developing and working with the teachers on the use of the SCRs the past two years throughout the whole school and fourth and fifth in particular have a good handle on it. I mean, they have gone over the kinds of questions that need to be asked and answered and how they should be answered and the expectations for the kids, as well as how to score.

During one professional development session, fifth-grade teachers practiced using the scoring rubric and assessing sample SCRs. The observer noted, About five minutes were allotted for the teachers to review and score the sample student responses from Sarahs Elapsed Time. They then looked at their scoring as a group and [the math specialist] explained discrepancies between the states scores and the teachers. What stood out in this conversation was not necessarily teachers scoring accuracy but their ability to defend their responses in response to official scorer reasoning. Later, the teachers debated the relative advantages and disadvantages of putting the rubric in kid-friendly language, again showing their ability to reason through complex issues related to SCR instruction.

Teacher experiences with professional development also translated into the classroom. Brookfield teachers reported working with SCRs extensively in their classroom using an analytic approach. The special education teacher in particular commented on her ability to concentrate on SCRs with her smaller groups of students. Classroom teachers also explained that they taught their students how to understand the scoring rubrics and to analyze SCR responses. One fourth-grade teacher explained her strategy:

We would put them in groups and give them three choices for the SCR that would match the reading and they had to score them. We had gone over the rubric. What would you give this? A three? Why? And so they had to . . . justify it so they turned into analysts. It kind of told them youre the teachers now, to flip it around so that they could start revising and analyzing their own work.

The staff developer further described how Brookfield teachers involved the students in their own learning about SCRs:

Its always been number one, know where your students are at all times. But now [the teachers] do something about it in that they let the students in on it. In other words, you didnt really do well on this. Lets see how you can make it better. And they do take the time to go back and have them make those corrections so that now you dont have a one on this SCR. Youre going to have a three. What is it that you can do to make it a three? Lets talk about this. . . . And I think that thats, you know, really helped a lot at our school.

Classroom observations confirm that Brookfield teachers took an analytic approach when teaching SCRs. One fourth-grade teacher, for example, was observed as she generated a classroom conversation around scoring SCRs. In small groups, students applied the scoring rubric to sample SCRs, working to reach consensus by explaining their reasoning to one another. In the whole-group discussion that followed, the teacher encouraged students to be specific with the reasons behind their grading. What we see in this case study, then, is teacher experience of  professional development within a comparatively low-pressure policy environment and a professional and collaborative school culture. In this context, professional development experiences facilitated deep thinking about SCR-related issues. By carefully analyzing SCR goals and processes themselves, teachers enabled their students to be analytic as well.

CASE 2: HAWTHORNE ELEMENTARY SCHOOL

Although Hawthorne, the largest of the case study schools, met AYP for all eight subgroups of students, their success was tempered by being on the borderline in two areasESOL and special education readingwhich were identified for close monitoring during the 20042005 school year. Hawthorne was also designated Title I for the first time in 2004 because of the number of students receiving free or reduced-price meals. Compared with the two other schools, Hawthorne faced a moderate amount of external policy pressure. During the 20042005 school year, the Hawthorne administration continued to be concerned about meeting AYP. The principal commented, The other part that was uppermost in my mind was AYP, of course, and how we are going to achieve it. We made it, kind of made it by the, you know, safe harbor helped us out a lot last year. Given this history, Hawthorne teachers were not entirely confident that they could reach their goal with all subgroups of students. One fifth-grade teacher commented, I have nine ESOL students. I mean theres no way in six months theyre going to be where they need to be. Its just not going to happen. And I mean I try my best. A special educator also expressed concern about the achievement level of her students, saying, Its not going to happen for many of those children. Its not going to happen. And sometimes, you know, thats a source of anxiety for me. Despite some misgivings among the staff about the reality of meeting AYP, it remained a priority at Hawthorne.

Professional development at Hawthorne was organized around 45-minute to 1-hour weekly grade-level meetings and monthly grade-level meetings that were a half day or whole day long. In addition, the math specialist explained that she modeled what she learned in district trainings for classroom teachers:

Any time I go to my meetings and we learn anything about the state test, such as how to write an SCR, how to answer an SCR, what the rubric is for an SCR on the state test, I bring it back and I work with the teachers so they can implement that in their classrooms. . . . For one thing, this year we did get more information on the SCR and how the children can answer a mathematics SCR. I was trained on that and then I went in and modeled every class, modeled in every classroom how to do this so I was modeling for the teachers as well as showing the children how to do this.

But despite a structure designed for ongoing school-based professional development, we as researchers had few opportunities to see that professional development in action. This paucity of observations had three potential sources. First, as indicated by the math specialist, some professional development occurred as individual modeling in classrooms. Because we observed formal meetings but did not shadow specialists, we did not capture this form of professional development. Second, meeting planning at Hawthorne tended to be last minute and haphazard, making it logistically difficult for researchers to attend. Third, it is likely that group professional development at Hawthorne happened less frequently than at the other two schools. In fact, the principal intern commented at the end of May, after the state test and close to the end of the school year, that some of the teachers had just begun learning about SCRs. He explained that the fifth-grade teachers started their SCRs and common scoring a little bit. From what we were able to determine, professional development on SCRs at Hawthorne was relatively infrequent.

In addition, professional development around SCRs appeared to be restricted to assessment of sample student responses. On several occasions, grade-level teams met with Helena Norton, one of the districts Title I specialists, and reviewed high, middle, and low student responses on a variety of SCRs. An observer described what took place at January meetings, just a few weeks prior to the state test:

Helena asked the teachers questions about the SCRs to make sure they were all assessing the same things. Helena asked, What is the indicator were assessing? Answer from teachers: The main idea.  Helena: What is proficiency? . . . .The teachers questioned whether students needed to reference the text in order to receive a good answer on the SCR.

At this meeting, just a few short weeks before the state test, Hawthorne teachers appeared to be learning the most basic vocabulary related to SCRs. Their opportunities to learn about SCRs seemed limited to looking at student responses and being introduced to foundational concepts.

The limited nature of teachers learning opportunities was not hard to explain. Professional development at Hawthorne took place in a school context that could be described as both hierarchic and chaotic and that included challenges such as extended principal absences for family reasons and last-minute changes to test preparation methods. Resenting top-down change imposed by the district, the principal promoted an isolated, individualistic approach to professional development but was forced to rely on external expertise as a resource. This context shaped teachers experiences with professional development such that they ultimately demonstrated only minimal understanding of SCRs and implemented them superficially in classrooms.

In response to a question about the district support provided because of their Title I designation, Hawthornes principal commented,

It doesnt give you the resources that you need. You know now I just have my little Helena Norton [Title I specialist], you know driving me crazy all the time. Its like having this person around all the time did you do this did you do this? . . . . Yeah, like get out of my face. I mean, Im not that friendly of a person anyway. You know leave me alone. . . . Youre asking me for stuff that I need to work on this stuff now. I dont need to meet with you. Youre not my boss.

It might be noted that this same Helena Norton appeared to be the sole provider of professional development on SCRs in reading for Hawthorne teachers, but the principals lack of regard for her as a resource seems to have contributed to the scant professional development that her teachers received in this area.

In addition to Hawthornes professional development sessions being few and far between, at least one of the meetings intended for work on SCRs was derailed by an unexpected visit from the principal. Arriving close to the beginning of a fourth-grade meeting, the principal consumed the entire meeting soliciting feedback on a recent data meeting. At these regularly scheduled data meetings, teachers were asked to discuss the progress of individual struggling students with a panel of specialists and administrators. The principals intrusion on this fourth-grade meeting was significant for two reasons. First, the conversation itself sheds light on the leadership and culture at this school. The observer wrote that words like witch hunt were used to describe how some of the teachers viewed the data meetings, which some teachers left in tears, feeling as though they were being accused of not doing their jobs. The principal was attempting to rectify this situation by making the rounds of grade-level meetings, where she listened to teachers concerns and listed positive and negative feelings about the data meetings. The second reason that this intrusion was significant was that it supplanted a potentially valuable professional development session about SCRs with Ms. Norton just 4 weeks before the state test. Given the limited exposure that most Hawthorne teachers had to SCRs, and their lack of even the most fundamental knowledge, taking this time for debriefing data meetings was a questionable use of resources.

As we have depicted, Hawthornes school culture promoted SCRs as an isolated test preparation strategy rather than a comprehensive instructional reform approach. Teachers shared this common perception of SCRs as an instructional method. Professional development around SCRs was largely crammed into the month preceding the state test and combined with other test preparation strategies. The reading specialist described how Hawthorne rejected the districts advice and instead crammed for the state exam:

The county, at least the Reading Department, is saying if you teach from the instructional guides you will be prepared for the state exam. [The principal] feels like wed be doing a disservice to our children because weve done so well with this last-minute, one month, like four straight weeks of just boom! Test, prep, practice. And its SCRs, and its selected response, and we just go and go and go.

Many teachers at Hawthorne wanted to use SCRs with students throughout the year but had little time to implement them thoughtfully. One fourth-grade teacher noted that SCRs were good because they helped students learn how to explain their thinking and back up their answers. But the problem was their concentrated use as a test preparation strategy. A teammate echoed that teachers sentiment:

See, I feel very comfortable teaching SCR. . . . But what I dont feel comfortable is, teaching this concept, this concept, this concept, this concept, now youre gonna write a paragraph response. This is all in the same ninety minutes of reading . . . I found I didnt have enough time to go over it . . . I would rather have seen it done once a week, and say, okay, you know, Harry, heres your paragraph response . . . here are all the anchors  . . . now how can you improve it? Please rewrite it and improve it. I had no time for redos.

This teacher wished that this type of writing in response to reading had been embedded throughout the new curriculum so that teachers did not have to make up their own questions and just do crunch time SCRs. But although some teachers appeared eager for greater integration, the established culture at Hawthorne prevented this from happening.

In contrast to Brookfield, resources for professional development around SCRs at Hawthorne were primarily external, provided by Ms. Norton, the districts Title I specialist, in fourth- and fifth-grade team meetings. These sessions were scheduled in late winter, just weeks before the state exam, and they followed a prescribed protocol of walking through an assessment of high-, middle-, and low-quality sample responses. From what we could ascertain, apart from the individual classroom modeling by the math specialist, this was the only concrete SCR-related skill that was presented.

Given limited professional development opportunities, Hawthorne teachers lacked even the most fundamental knowledge about SCRs. One Hawthorne teacher described overall understanding as merely being aware of SCRs. In late January, only a few weeks before the state test, Hawthorne teachers did not even realize that a standard rubric existed for the scoring of SCRs. The observer noted, [The teachers] were concerned that there didnt seem to be any set criteria or rubric for SCRs. One of the teachers believed that the rubric was a three point scale but no one seemed to have an interpretation of the three points. One teacher commented that she had been using a rubric from the district curriculum, but later discovered that this rubric was intended for writing rather than reading. At one team meeting, teachers developed a plan for teaching SCRs in reading, but visibly absent from that plan was any reference to using the state rubric for scoring SCRs or sample responses for modeling what a proficient response looks like. These tools could have been useful for both students and teachers and were used extensively at the other case study sites. Furthermore, teachers criteria for determining quality responses included organization and word choice, attributes of writing quality, not reading comprehension.

Just 2 weeks before the state test, Hawthorne teachers still did not seem to link SCRs with reading comprehension. When Ms. Norton asked the fifth-grade team what types of strategies students could use to respond to an SCR, teachers responded with suggestions like organize, be specific about details, use descriptive language, and vary sentence structure. Again Ms. Norton pointed out that these were writing qualities and that the SCR was to assess reading. Although the problem was identified, it persisted throughout the meeting. When Ms. Norton prompted teachers to identify what middle-level students should do on an SCR, the observer noted, The teachers continued to describe the written structure of the students answers, give some description, order the paper. Helena continued to push them about the reading topic, Were these kids really able to find the main idea? Given limited learning opportunities, Hawthorne teachers appeared to build only minimal understanding of SCRs.

From what we could gather, Hawthorne teachers implementation of SCRs in their classrooms seemed to be superficial. There were mixed reports on the frequency of SCR use in the classroom. The reading specialist explained that teachers used a variety of reading strategies, including SCRs, in their classrooms every day:

They are getting in all of the components of balanced reading. . . . So, theres a read aloud going on on a regular basis. And theres guided reading groups going on every day and . . . theyre doing SCRs . . . and theres comprehension questions and theyre doing all of those components on a daily basis.

However, one fourth-grade teacher explained that although students occasionally worked with SCRs, they did not use them on a regular basis. Reflecting on the upcoming school year, she commented,

There is a difference between exposing them and getting them into a routine. I think once we start the year out, get them into this routine, do it once a week. They could hear the words, SCRs, they can hear the words, all the . . . words [for the state exam].

Nevertheless, another fourth-grade teacher did see progress from students over time, explaining, I pulled out my students SCRs. It was three in a row, and . . . it was in order of dates. And clearly, you can see, her answers becoming more descriptive and growing, and you could see her progress.

As apparent in that last comment, when Hawthorne teachers talked about using SCRs in the classroom, they did so superficially: Evidence of SCR improvement is simply more descriptive. During one fourth-grade team meeting, teachers devised a plan to prepare students for the upcoming state test using reading centers. The observer wrote,

They decided that they would start with only three centers. One would be test preparation work in which students would work on SCRs and multiple-choice questions. One would be word work and one would be teacher.  The teacher center would be the only one in which the students worked with the teacher, the other two would involve independent work. The conversation around planning was concerned mostly with how to plan when students were being pulled-out in the afternoons, the number of books each teacher was currently reading with their students, and student grouping (high, middle, and low).

This conversation is noteworthy because of its emphasis on activity format rather than students or curriculum content. One teacher was particularly positive about reading centers because she had learned about them in her teacher preparation program. Thus, Hawthorne teachers planned to implement SCRs without careful consideration of objectives or purpose. Overall, the combination of mixed reports of use, lack of evidence of SCRs in classroom observations, and superficial discussions of SCR implementation led to a picture of limited, and perhaps off-target, SCR instruction in Hawthorne classrooms.

CASE 3: CHERRY RIDGE ELEMENTARY SCHOOL

In contrast to Brookfield and Hawthorne, Cherry Ridge was a longtime Title I school that struggled to meet AYP requirements. Cherry Ridge failed to make AYP 2 years in a row in one category, special education reading. School personnel faced tremendous pressure to meet AYP requirements during the 20042005 school year in order to have the school improvement status lifted and avoid corrective action by the district. The staff repeatedly identified AYP as a major priority at Cherry Ridge for the 20042005 school year. The principal explained that a major focus of the school was making AYP and making sure that the teachers know what it means, what the kids have to know and be able to do to get there. She made this goal abundantly clear to her staff, sharing the schools AYP data with them in great detail. Given heavy pressure to make AYP, Cherry Ridge teachers engaged in the largest number of professional development activities. At Cherry Ridge, teachers spent 1 hour per week in reading grade-level meetings, 4050 minutes weekly in math grade-level meetings, and 5 hours per month in curriculum implementation meetings, which had a mandatory professional development component. Teacher leaders were also involved in the school improvement team, which required 3 hours of professional development per month. Almost all teachers were intensely involved in long hours of professional development around SCRs.

In these meetings, Cherry Ridge teachers had a prescribed method for working with SCRs. Week after week, Cherry Ridge teachers generated sample SCRs and evaluated instructional methods for using them in the classroom. One fifth-grade teacher explained their strategy:

What weve been doing is weekly, in reading, we pick an SCR that the kids do and we pick whatever our objective is, and we match it up to the recommended curriculum, and then we decide what were looking for from the kids to write in their SCR, and then the following week well bring it back to the meeting, and bring it to the table, and we all discuss it. And kind of look at, we usually pick a high, medium, and low student, we bring the same student each week, and then well kind of just have a dialogue going between the whole team about strategies that we might use. So thats something that I think has really influenced our teaching, just to really go back, for myself, to really make sure that we are re-teaching and revisiting those SCRs, and not when they finish it, that were moving on right away.

Despite intense involvement with SCRs at Cherry Ridge, professional development was often routine. Week after week, teachers used the same strategies, spending considerable time generating sample SCRs. One fifth-grade teacher commented,

Couldnt they [the districts curriculum developers] . . .  have written more SCR questions so we wouldnt be spending a whole team meeting on one SCR question. We would have it already, then we could have addressed . . . what would we expect from our students based on the question they gave us.

Although Cherry Ridge teachers had numerous opportunities to learn about SCRs, the same methods over and over became routine, prompting little learning over the course of the school year.

Unlike Brookfields collegial and Hawthornes chaotic work environments, the school context at Cherry Ridge was an efficient, tightly controlled bureaucracy. As the primary and solitary decision-maker at the school, the principal skillfully and single-mindedly managed policy requirements. School culture at Cherry Ridge felt dutiful with respect to professional development, with teachers conscientiously, but somewhat mechanistically and halfheartedly, making use of the limited available resources. Given this combination of school-based factors, Cherry Ridge teachers often seemed confused by SCRs but implemented them strategically, for improved test results, into classroom practice.

The principal at Cherry Ridge could best be characterized as pragmatic about district initiatives. Rather than supportive or resentful, she simply considered ways to make the existing rules benefit her school. Well-versed in the requirements for Title I spending, she knew how to use regulations to the schools advantage, as indicated by her thinking about the schools budget and curriculum implementation meetings:

When your school is in need of improvement, you have to spend 10% of your overall Title I budget, including salaries, for professional development. . . . Most of the schools used their 10% money to buy a half-time teacher. So, the way I look at it you get a half-time teacher who might impact what six people, 10 people and maybe 25 kids, 30 kids, 40 kids. Not a huge span of influence. . . . We offered a trainer planning stipend which at that time was $20 a night for our grade-level teams to stay from 4:30 to 7 . . .  Its two and a half hours on a Wednesday and two weeks later another two and a half hours. Thats when they did these two-day plans, thats when they look at the students work.

Moreover, the principal was clear that although she supported professional development, she did so for the purpose of meeting AYP, commenting, Hopefully well make our 10% and 11% AYP and wont have to spend 10% of funds on PD (Professional Development). This practical approach defined her orientation to professional development.

However, an unanticipated problem developed in January 2005, when the staff discovered that their use of funds for curriculum implementation meetings was not allowed under new federal guidelines, which specified using funds only for professional development that increased professional knowledge. Technically, teachers had been using these sessions for joint planning, but in practice, much of that planning involved conversation around the instruction and implementation of SCRs. Teachers frequently selected examples of SCRs to be used with students and as a group generated a proficient response to use as an example. Through this process, they collaboratively engaged with the complexities of SCRs to better understand it themselves. A team might discuss characteristics of historical fiction, how setting might affect events of a story, or what conclusions might be drawn about characters from dialogue. They would then jointly construct SCRs about these topics. Moreover, teachers brainstormed strategies in these sessions for helping their students improve in SCRs and shared resources they had found or developed for doing so. Thus, although teachers were not officially engaged in increasing their professional knowledge about SCRs, in practice, they were doing exactly that.

Teachers were upset by the impending change in the purpose of these curriculum meetings. As one of the support staff said, Im ready to retire! . . . The federal government has changed the definition of high-quality professional development to what increases teacher professional knowledge. We know that when you get teachers together to plan that thats what happens. But bureaucrats havent been in schools, so they have no idea. Wanting to support their use of the curriculum implementation meetings, the principal encouraged teachers to think liberally about what constituted professional development, including a discussion of student work under that umbrella. She told teachers to follow federal rules but to use them in a manner that made sense to them. This pragmatic approach to professional development appeared to successfully reconcile policy regulations with current practice and established approaches to making AYP.

This dutiful approach to policy regulations pervaded the school culture, with teachers sharing an approach to professional development as one of many obligations. Cherry Ridge teachers appeared to merely go through the motions during professional development sessions, an orientation that was closely linked to accountability requirements at the school. Each time grade-level teams did work around SCRs, teachers were required to fill out capture sheets in which they wrote down what they had learned for submission to the principal. Each capture sheet asked for the following information: What does student know? What does student need to know? What will I do next with the student? This exercise seemed to foster a sense among teachers that professional development sessions were simply hoops to jump through rather than authentic learning experiences. One fourth-grade teacher commented, I dont like rushing so we can show on paper that were doing this. Observers at these meetings noted that teachers were indeed rushed. Another teacher hurried her students through an SCR exercise in class, explaining that she needed their responses for her after-school team meeting. Although Cherry Ridge teachers supported professional development, they tended toward perfunctory, rather than active, engagement.

Even though an outside consultant was hired to help teachers develop expertise in teaching SCRs, resources at Cherry Ridge remained quite limited. As at Brookfield, professional development on SCRs was often carried out by members of the staff. But, perhaps because of their inexperience, the staffs expertise in this area was relatively weak. In contrast to Brookfield, where teachers engaged in a variety of learning opportunities, Cherry Ridge teachers participated in the same type of activity repeatedly with little visible growth. Although they worked together in weekly team meetings to generate reteaching strategies for SCRs, their repertoire remained fairly limited. As one example, the fifth-grade team leader attempted an instructional strategy with a struggling student and faced the same difficulty the following week, but decided to use the same failed instructional strategy. The observer explained,

Lisa began the SCR discussion by saying that she had pulled out the same kids whose paragraph response had been discussed last week, to see if there was any progress in their performance on this new paragraph response, as a follow-up. She shared that for this one student last weeks identified problem was that he did not pull out evidence, and that the diagnosis had been to highlight text. She said that in this weeks paragraph response he again did not use evidence from text; that he generalized it well this time by referring to the idea of the text, but again he did not use text examples. She said that for next time she would ask him to highlight where there are examples in the story that say where they [the characters] are strong.

In this situation, although teachers aimed to develop their SCR skills, they repeated the same exercise with the same resources week after week. Cherry Ridge teachers became adept at identifying problems with SCRs but less skilled at solving those problems. Thus, Cherry Ridge teachers worked diligently with limited expertise, which ultimately resulted in minimal progress.

Their experiences with professional development left many Cherry Ridge teachers confused about certain aspects of SCRs. During one discussion of SCRs, for example, teachers struggled to explain the term predicting in an SCR prompt, offering comments such as: Facts from story, plus what you know, your personal experience, plus make prediction, draw a conclusion, Is that like drawing generalizations? I find that the kids, their prediction was off because they didnt make the connection with personal experience. The teachers also had problems with other aspects of SCRs. Although they had been working consistently with Melany Batori, the school improvement consultant, for several months, Ms. Batori still had to recommend substantially different wording for an SCR question that the teachers had developed. She also had to engage in lengthy probing to elicit an appropriate set of criteria for a proficient response.

Cherry Ridge teachers also appeared to be confused about whether students should include their own background knowledge in an SCR. In early January, the paraeducator commented, Were often telling them to draw on background knowledge and that usually the last sentence is an opinion. In response, the fifth-grade team leader explained, In the [state test], unless they specifically ask for background knowledge, students are not to include it in their answers. However, later in the month, another teacher explained that she wanted her students to make more connections with their own experiences, using their background knowledge. Despite intense involvement with SCRs over the course of the school year, they still struggled to define key terms and expressed confusion over SCR use.

Once again, teacher experiences with professional development played out in the classroom. Cherry Ridge teachers used a distinctly strategic approach to teaching SCRs. Specifically, Cherry Ridge teachers showed their students techniques for answering SCRs successfully. This approach was distinct from Brookfields analytic approach, in which students read and discussed sample responses in an effort to understand the underlying concepts and assessment expectations. At Cherry Ridge, teachers focused instead on giving students tools for generating an acceptable answer. Moreover, at Cherry Ridge, SCRs were a major focus of classroom instruction. One fifth-grade teacher told us that she worked with SCRs in the classroom every day. As the reading specialist explained, Thats basically where we focus our writing, is being able to write a summary and being able to answer an SCR. This again contrasts with Brookfield, where a complex writing program was in place, incorporating SCRs as only one piece of the overall writing curriculum.

Cherry Ridge teachers described the approaches they used to support their students in SCR writing. One fourth-grade teacher modeled SCR writing for her students:

One day this week I will model how to write a summary SCR. . . . [The students] will have an organizer and itll be filled out and Ill say, okay, well the first thing you need to have in your summary is that and Ill model it all up there and then theyre going to write a summary as their independent work . . . I model the SCR and take them through step by step.

Similarly, a fifth-grade teacher explicitly taught rules for writing SCRs. During one lesson, the class jointly constructed a model SCR. In a follow-up lesson, they prepared to write their own SCR by reviewing several strategies for SCR writing, including:

Strategy 1: Read the question and understand what it is asking you to do

Strategy 2: Be aware of the important words or phrases in the text

Strategy 3: Organize your thoughts mentally

Strategy 4: Write your answer

Strategy 5: Read over your answer to make sure you have included everything and it makes sense

This strategic approach consisted of modeling and explaining a particular sequence of steps for SCR writing. Overall, in Cherry Ridges high-pressured and bureaucratic context, teachers remained somewhat unclear about SCRs despite intense professional development. Use of SCRs in classrooms was strategic, geared toward test taking rather than helping students themselves develop analytic thought processes.

DISCUSSION

Professional development has changed significantly in recent years. High-stakes accountability policies have increased the amount of professional development opportunities that occur, providing funds for additional support and initiating curricular changes that necessitate intensive training. Professional learning opportunities have shifted from one-time workshops outside of the school setting to ongoing collaborative teamwork in comprehensive change contexts. Recent theoretical developments indicated that structure and content, as well as context, are central to understanding professional development. The increased quantity, altered setting, and more complex picture of professional learning opportunities mean that traditional ways of analyzing professional development, namely its structure and content, are only partially adequate. The changed context necessitates a new look at teacher experience with school-based professional development in a high-stakes accountability environment.

In looking closely at professional development around SCRs, we found that school contextdefined as leadership, culture, and resourcesvaried among the three school buildings. Although all three principals were experienced leaders who had been at their schools for some time, their attitudes toward district initiatives and increased professional development requirements differed considerably. Brookfields principal was supportive, Hawthornes principal was resentful, and Cherry Ridges principal was dutiful toward top-down initiatives in general. These underlying orientations played a role in how each school leader approached and ultimately shaped school-based professional development. Likewise, the school culture in each buildingengaged, isolated, and perfunctoryrepresented shared meanings and patterns of behavior toward professional development and were associated with distinct teacher experiences in each building. Finally, the human and social resources varied across school sites, influencing the expertise available for teacher learning. Perhaps stronger networks at Brookfield, the result of more inclusive leadership, more engaged school culture, and stronger human and social resources, lent themselves to more extensive expertise for professional development (Bryk & Schneider, 2003; Frank, Zhao, & Borman, 2004). This contrasted with Hawthornes largely external, and Cherry Ridges more limited, expertise for professional development. Together, these factors contributed to distinct school contexts and variations in teacher understanding and practice. For example, Brookfield teachers gained in-depth understanding of SCRs and used analytic approaches in practice. At Hawthorne, teachers had only minimal understanding and used superficial instructional methods in the classroom. And at Cherry Ridge, teachers were confused about SCRs and implemented them strategically as test preparation.

In looking at the delivery of school-based professional development in this environment, we also found that professional development looked different in schools with varying levels of AYP pressure. Although its teachers maintained an eye toward test scores, Brookfield was under the least pressure to make AYP and was not (yet) classified as a Title I school. Brookfields entire staff was involved in learning about SCRs in a variety of ways. At the other extreme was Cherry Ridge, which faced tremendous pressure to make AYP. Jobs were on the line if the school failed to increase test scores. At Cherry Ridge, professional development around SCRs was extensive, but the content focus was often routine. Hawthorne, with a moderate amount of AYP pressure, conducted only infrequent professional development on SCRs with restricted content that simply introduced assessment and categorized sample responses. This pattern mirrors related research in which high-performing elementary schools aimed to increase the achievement of students across grade levels and subject areas, taking a comprehensive approach, whereas schools on probation responded only to specific policy requirements (Diamond & Spillane, 2004). Given their supportive school context, teachers in the low-pressure environment were able to sustain thoughtful and comprehensive professional development. In contrast, professional development in the high-pressure policy environment was intensely implemented but with a more routine approach and restricted results. These findings further research, such as Cohen and Hills (2000), that indicates that the content of teachers learning opportunities is a key link between policy and classroom practice. We find that not only the content but also the context of teachers experiences with professional development critically linksor weakens the link betweenpolicy and practice.

Moreover, extreme pressure to make AYP did not necessarily mean better teacher experience with professional development. At Brookfield, teacher experience was characterized by in-depth understanding and an analytic approach to practice. At Hawthorne, teachers had only minimal understanding of SCRs. And at Cherry Ridge, the school with the greatest amount of professional development, teachers were ultimately confused about SCRs but integrated them into their practice by teaching specific test-taking strategies. Instead of a direct connection between AYP pressure and teacher experience, it appears that professional development was filtered through particular school contexts. Brookfield had inclusive leadership, an engaged school culture, and extensive resources. Within this particular school context, teacher experience with professional development was characterized by in-depth understanding and analytic implementation. With Hawthornes disruptive leadership, isolated school culture, and external resources, teachers developed minimal understanding of SCR instruction. And with Cherry Ridges pragmatic leadership, perfunctory school culture, and limited resources, teachers were unclear about how to approach SCRs but implemented them in a strategic manner for the limited purpose of improved test performance. Although our data do not suffice for a causal explanation, it is evident that school context and teacher experience varied across the three schools. By showing patterns of variation, this study builds on existing understandings of the role of school context in policy implementation, expanding these notions to include a clearer picture of teachers opportunities to learn within school-based high-stakes accountability environments. Just as school districts interpret and support professional learning opportunities through a particular lens (Grossman & Thompson, 2004), so do school orientations shape teacher experience with professional development.

The experiences of teachers at these three case study schools inform existing notions about school-based professional development in high-stakes accountability contexts. The findings from this study suggest a nested model for the delivery of school-based professional development in a high-stakes accountability environment (see Figure 1). Although the notions of nestedness (e.g., Hightower, Knapp, Marsh, & McLaughlin, 2002) and local implementation of policy (McLaughlin, 1987) are well established, this study contributes by examining the interpretation and enactment of high-stakes policy initiatives at the school level and identifying resulting experiences for teachers. Teachers experiences with SCRs occurred within particular school contexts, with specific professional development structure and content, in a larger policy environment. This model helps clarify why policy alone does not influence teacher experience and why so much professional development has limited impact. Policy initiatives are translated by professional development, school context, and, ultimately, school staff. Thus, although external reform initiatives such as AYP impact professional development, that impact is mediated by the context in which it is delivered. To improve teacher experience with professional development, and in turn its impact on students, the policy and school contextsnot just the content and structure of professional developmentneed to be carefully analyzed. These findings support related research on the importance of both input-based standards and output-based accountability (Mintrop, 2004).

Figure 1. Teachers nested learning experience

click to enlarge

Although this study strongly suggests the importance of both policy and school context for teacher experience with professional development, we do not make claims beyond the experiences of the teachers in these three elementary schools. First, although the three schools did face varying amount of AYP pressure, they were all located within the same urbanized but well-resourced school district and served only moderately different student populations. Moreover, the study is limited in its specific focus on one area of language arts instructionSCRsand its use of focus group interviews as a method for understanding teacher experiences with professional development (Greenbaum, 1993). In addition, although we were involved with these schools over a period of several years, these intensive case studies were restricted to the 20042005 school year. Finally, although we attempted to clarify the ways in which we understood such complex terms as school leadership, culture, and resources, in practice, these terms are difficult to define and operationalize. Despite these limiting factors, this research provides a new, in-depth look at professional development processes and outcomes in a school-based high-stakes accountability context.

Overall, this study finds that although pressure to make AYP influences professional development, teacher experience is mediated by the particular school context. Thus, although high-stakes accountability policies increase the quantity of professional development, the quality of teacher experience continues to be dependent on the environment in which those learning opportunities take place. Policy aimed at teacher development should thus address not only learning opportunities but also the nested nature of those opportunities within a particular school. Supporting teacher development by improving school leadership, culture, and resources has a substantially better chance of improving teacher and student learning than focusing on the improvement of professional development alone.

Acknowledgments

This article is based on work supported by the National Science Foundation under Grant No. 0115389. We would like to thank other members of the research team for their work in conceptualizing the overall study as well as collecting, coding, and analyzing data.

Notes

1. For purposes of this study, we use the school districts official classification language and categories.

2. All names are pseudonyms.

References

Boardman, A. G., & Woodruff, A. L. (2004). Teacher change and high-stakes assessment: What happens to professional development? Teaching and Teacher Education, 20, 545557.

Bogdan, R. C., & Biklen, S. K. (2003). Qualitative research for education: An introduction to theories and methods (4th ed.). Boston: Allyn and Bacon.

Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.). (2000). How people learn: Brain, mind, experience, and school (Expanded ed.). Washington, DC: National Academy Press.

Bryk, A., & Schneider, B. (2003). Trust in schools: A core resource for school reform. Educational Leadership, 60(6), 4044.

Cohen, D. K., & Hill, H. C. (2000). Instructional policy and classroom performance: The mathematics reform in California. Teachers College Record, 102, 294343.

Craig, C. (2004). The dragon in school backyards: The influence of mandated testing on school contexts and educators narrative knowing. Teachers College Record, 106, 12291257.

Desimone, L., Porter, A. C., Birman, B. F., Garet, M. S., & Yoon, K. S. (2002). How do district management and implementation strategies relate to the quality of professional development that districts provide to teachers? Teachers College Record, 104, 12651312.

Diamond, J. B., & Spillane, J. P. (2004). High-stakes accountability in urban elementary schools: Challenging or reproducing inequality? Teachers College Record, 106, 11451176.

Dorgan, K. (2004). A year in the life of an elementary school: One schools experiences in meeting new mathematics standards. Teachers College Record, 106, 12031228.

Firestone, W. A., Mangin, M., Martinez, M. C., & Polovsky, T. (2005). Leading coherent professional development: A comparison of three districts. Educational Administration Quarterly, 41, 413448.

Firestone, W. A., Schorr, R. Y., & Monfils, L. F. (2004). The ambiguity of teaching to the test: Standards, assessment, and educational reform. Mahwah, NJ: Erlbaum.

Fishman, B. J., Marx, R. W., Best, S., & Tal, R. T. (2003). Linking teacher and student learning to improve professional development in systemic reform. Teaching and Teacher Education, 19, 643658.

Frank, K., Zhao, Y., & Borman, K. (2004). Social capital and the diffusion of innovation within organizations: The case of computer technology in schools. Sociology of Education, 77, 148171.

Gamoran, A., Secada, W., & Marrett, C. (2000). The organizational context of teaching and learning: Changing the theoretical perspectives. In M. Hallinan (Ed.), Handbook of the sociology of education (pp. 3763). New York: Springer.

Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38, 915945.

Gersten, R., Vaughn, S., Deshler, S., & Schiller, E. (1997). What we know about using research findings: Implications for improving special education practices. Journal of Learning Disabilities, 30, 466476.

Greenbaum, T. L. (1993). The handbook for focus group research (Revised and expanded ed.). New York: Lexington Books.

Grossman, P., & Thompson, C. (2004). District policy and beginning teachers: A lens on teacher learning. Educational Evaluation and Policy Analysis, 26, 281301.

Haladyna, T., Nolen, S. B., & Haas, N. S. (1991). Raising standardized achievement test scores and the origins of test score pollution. Educational Researcher, 20(5), 27.

Hamilton, M. L., & Richardson, V. (1995). Effects of the culture in two schools on the process and outcomes of staff development. Elementary School Journal, 95, 367385.

Hawley, W., & Valli, L. (1999). The essentials of effective professional development: A new consensus. In L. Darling-Hammond & Gary Sykes (Eds.), Teaching as the learning profession: Handbook of policy and practice (pp. 127150). San Francisco: Jossey-Bass.

Hawley, W., & Valli, L. (2007). Design principles for learner-centered professional development. In W. Hawley (Ed.), The keys to effective schools (2nd ed., pp. 117137). Thousand Oakes, CA: Corwin Press.

Hightower, A. M., Knapp, M., Marsh, J., & McLaughlin, M. W. (2002). The district role in instructional renewal: Making sense and taking action. In A. M. Hightower, M. Knapp, J. Marsh, & M. W. McLaughlin (Eds.), School districts and instructional renewal (pp. 193201). New York: Teachers College Press.

Ingram, D., Louis, K. S., & Schroeder, R. G. (2004). Accountability policies and teacher decision making: Barriers to the use of data to improve practice. Teachers College Record, 106, 12581287.

Jones, M. G., Jones, B. D., Hardin, B., Chapman, L., Yarbrough, T., & Davis, M. (1999). The impact of high-stakes testing on teachers and students in North Carolina. Phi Delta Kappan, 81, 199203.

Kennedy, M. (1998). Form and substance in inservice teacher education. Madison: National Institute for Science Education, University of Wisconsin-Madison.

Lieberman, A., & McLaughlin, M. W. (2000). Professional development in the United States: Policies and practices. Prospects, 30, 255236.

Little, J. W. (1989). District policy choices and teaches professional development opportunities. Educational Evaluation and Policy Analysis, 11, 165179.

Malen, B., & Rice, J. K. (2004). Framework for assessing the impact of educational reforms on school capacity: Insights from studies of high-stakes accountability initiatives. Educational Policy, 18, 631660.

McLaughlin, M. W. (1987). Learning from experience: Lessons from policy implementation. Educational Evaluation and Policy Analysis, 9, 171178.

McLaughlin, M. W., & Talbert, J. (2001). Professional communities and the work of high school teaching. Chicago: University of Chicago Press.

McNeil, L. M. (2000). Creating new inequalities: Contradictions of reform. Phi Delta Kappan, 81, 728735.

Miles, M., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. Thousand Oaks, CA: Sage.

Mintrop, H. (2004). High-stakes accountability, state oversight, and educational equity. Teachers College Record, 106, 21282145.

Moore, W. P. (1994). The devaluation of standardized testing: One districts response to a mandates assessment. Applied Measurement in Education, 74, 343367.

No Child Left Behind Act of 2001, Pub. L. No. 107-110, "115, Stat. 1425 (2002).

Newmann, F., King, M. B., & Youngs, P. (2000). Professional development that addresses school capacity: Lessons from urban elementary schools. American Journal of Education, 108, 259299.

Novick, R. (1996). Actual schools, possible practices: New directions in professional development. Educational Policy Analysis Archives, 4(14).

NSDC. (2001). NSDC Standards for Staff Development. Retrieved November 3, 2006, from http://www.nsdc.org/standards.

Penuel, W. R., Fishman, B. J., Yamaguchi, R., & Gallagher, L. P. (2007). What makes professional development effective? Strategies that foster curriculum implementation. American Educational Research Journal, 44, 921958.

Scribner, J. P. (1999). Professional development: Untangling the influence of work context on teacher learning. Educational Administration Quarterly, 35, 238266.

Sipple, J. W., & Killeen, K. M. (2004). Context, capacity, and concern: A district-level analysis of the implementation of standards-based reform in New York State. Educational Policy, 18, 456490.

Smith, M. L. (1991). Put to the test: The effects of external testing on teachers. Educational Researcher, 20(5), 811.

Smylie, M. A. (1995). Teacher learning in the workplace: Implications for school reform. In T. R. Guskey & M. Huberman (Eds.), Professional development in education: New paradigms & practices (pp. 92113). New York: Teachers College Press.

Spillane, J. P. (2002). Local theories of teacher change: The pedagogy of district policies and programs. Teachers College Record, 104, 377420.

Spillane, J. P., & Thompson, C. L. (1997). Reconstruction conceptions of local capacity: The local education agencys capacity for ambitious instructional reform. Educational Evaluation and Policy Analysis, 19, 185203.

Thompson, C. L., & Zeuli, J. S. (1999). The frame and the tapestry: Standards-based reform and professional development. In L. Darling-Hammond & G. Sykes (Eds.), Teaching as the learning profession: Handbook of policy and practice (pp. 341375). San Francisco: Jossey-Bass.

Valli, L., Croninger, R., Chambliss, M., Graeber, A., & Buese, D. (2008). Test driven: High-stakes accountability in elementary schools. New York: Teachers College Press

Youngs, P., & King, M. B. (2002). Principle leadership for professional development to build school capacity. Educational Administration Quarterly, 38, 643670.

Cite This Article as: Teachers College Record Volume 112 Number 3, 2010, p. 645-684 https://www.tcrecord.org ID Number: 15771, Date Accessed: 10/20/2021 3:51:48 AM Purchase Reprint Rights for this article or review