|
|
|
|
Professional Learning for Using Data: Examining Teacher Needs and Supportsby Jo Beth Jimerson & Jeffrey C. Wayman - 2015 Background: In the last few decades, a focus on school accountability at the state and federal levels has created expectations for teachers to attend to data in increasingly structured ways. Although professional learning is often cited as an important facilitator of effective data use, research that focuses on the intersection of professional learning and data use is scarce. Examining teacher perceptions of data use supports, and contrasting assertions of what is desired in data-related professional learning with accounts of the ways in which this professional learning actually happens provide an avenue for exploring these issues and for building a research base that can inform the work of district and campus leaders as well as support providers. Focus of Study: This study aimed at examining teacher needs specific to data-related professional learning through a lens informed by knowledge-based organizational learning. We were guided by two broad questions: (a) What knowledge and skills do teachers need in order to engage in data-informed practice? (b) How do professional learning supports address these needs? Research Design: The qualitative study draws on document analysis as well as interview and focus group data collected from n=110 participants (teachers, school leaders, and district support staff) in three school districts in central Texas. Flexible a priori coding rooted in our conceptual framework was employed to examine data for themes common across district settings and across school levels (e.g., elementary, middle, high). Code counts were used to further examine areas of professional learning focus and/or apparent imbalance. Findings: Educators articulated professional learning needs related to data use in six main areas: (a) asking appropriate questions of data (to guide analysis and use); (b) accessing and operating district data systems; (c) data literacy/interpretation; (d) fitting data use with day-do-day practice; (e) sharing information via collaboration; and (f) knowledge codification. Of these, data capture via computer data systems was by far the most prominent focus reported by educators in each district. Clear plans for addressing data use capacity through professional learning supports were lacking. Recommendations: Taking into account teacher perspectives on what professional learning for data use was needed and on how such supports were, in reality, structured, we make three recommendations: (a) purposefully embed professional learning for data use in ongoing organizational routines; (b) mitigate the district level silos that separate training-on-computer-systems from professional learning focused on turning data into action at the classroom level; and (c) seek balance in supporting the constellation of knowledge and skills that contribute to data use capacity. In one way or another, educators have always been expected to use data. Even in the one-room schoolhouses of the past, teachers used data by providing assignments and issuing grades. In the last few decades, however, an increasing focus on school accountability at the state and federal levels has escalated the expectations for educators to attend to particular types of data in addition to the traditional forms of data already in use. This policy shift has resulted in increased attention to how teachers use data to inform practice (Anderson, Leithwood, & Strauss, 2010; Mandinach, 2012; Mandinach, Honey, Light, & Brunner, 2008; Means, Padilla, & Gallagher, 2010; Wayman, Cho, Jimerson, & Spikes, 2012). DATA USE THROUGH AN ORGANIZATIONAL LENS
This notwithstanding, there is a lack of research that focuses on professional learning specific to data use. In pursuing such a study, we noted research suggesting that a systemic approach may best facilitate effective data use (Datnow et al., 2007; Honig & Venkateswaran, 2012; Supovitz, 2010; Wayman et al., 2007; Wayman Jimerson, & Cho, 2012). Thus, our conceptual framework for this study employs an organizational approach to supporting individual capacity for data use. A CONCEPTUAL FRAMEWORK FOR DATA-RELATED PROFESSIONAL LEARNING In this section, we outline a framework that presents individual learning for using data as part of a greater organizational picture. In building this framework, we noted that the data use and professional learning research bases both emphasize that teachers learn well in small groups, or collectives. Thus, we posit that effective data-related professional learning includes what educators learn independently and what they learn in collectives. Further, we posit that these are reciprocal. Figure 1 depicts this relationship. Figure 1. Reciprocal and mutually reinforcing professional learning
Research from the fields of data use and organizational learning suggests that systemic processes will be necessary to support such learning throughout a district (e.g., Argyris & Schön, 1996; Means, Chen, DeBarger, & Padilla, 2011; Senge, 2006; Spillane, 2012; Supovitz, 2010; Wayman, Cho, et al., 2012; Wayman, Jimerson, & Cho, 2012). Supovitz (2010) framework for knowledge-based organizational learning provides such support. Supovitzs (2010) framework describes the types of processes a district would need to employ to make data use more effective. The framework is described as systems designed to collect, synthesize, and disseminate knowledge valuable to the organization as a whole (2010, p. 708). Figure 2 illustrates Supovitz framework. Figure 2. Framework for knowledge-based organizational learning
Note: From Knowledge-Based Organizational Learning for Instructional Improvement by J. Supovitz in A. Hargreaves, A. Lieberman, M. Fullan, & D. Hopkins (Eds.), Second International Handbook of Educational Change, p. 710. Copyright 2010 by Spring Science +Business Media B.V. Supovitz (2010) situates data use in a cycle of continuous improvement and posits that school organizations move through four ordered processes. First, in data capture, educators specify the types of data necessary to measure learning and other outcomes, along with a process to collect, store, and access these data. Second, educators engage in meaning making based in these data, using their own sense-making, social interactions, and district- or school-based routines to decide how the information contained in these data will affect practice. Third, information sharing occurs, allowing members of the organization to share the knowledge they have amassed with other areas of the organization (e.g., other educators, subject-level teams, schoolwide). Fourth, knowledge codification is the process that allows this knowledge to be embedded throughout school and district documents, processes, and everyday practice. The conceptual framework we employ for the present study uses Supovitzs framework to provide process support for individual and collective learning. The key to our framework is that individual learning (Figure 1) and organizational processes (Figure 2) reciprocally support each other. In Figure 3, we diagram our framework. Figure 3. Conceptual framework for data-related professional learning
Note: Lower cycle reproduced from Knowledge-Based Organizational Learning for Instructional Improvement by J. Supovitz in A. Hargreaves, A. Lieberman, M. Fullan, & D. Hopkins (Eds.). Second International Handbook of Educational Change, p. 710. Copyright 2010 by Spring Science +Business Media B.V. The first two components of our framework (data capture and meaning making) primarily address ways that individuals and collectives gain practical knowledge. The second two components (information sharing and knowledge codification) primarily address the transfer and preservation of that information. Research on data-related professional learning has dealt primarily with the former two. However, organizational research suggests that without the latter two, knowledge gains are typically isolated to the individual, with little benefit to the organization as a whole. Further, knowledge will often be lost when individuals leave the organization (Argyris & Schön, 1996; Brown & Duguid, 1991; Senge, 2006). Through this framework, we posit that professional learning is a reciprocal endeavor, in which the organization supports individual learning, and individual learning contributes to the organization. This reciprocity flows throughout the four phases of the framework, as described in the following sections. Data Capture Data capture is a preparation stage. Learners prepare by accessing and organizing data in understandable forms and often use this time to help them develop questions to guide their inquiry (Datnow et al., 2007; Cho & Wayman, 2014; Lachat & Smith, 2005; Schildkamp & Kuiper, 2010; Wayman & Stringfield, 2006). Importantly, data capture also helps learners understand their data: Accessing or preparing data has learning benefits, especially when done collaboratively (Datnow et al., 2007; Lachat & Smith, 2005; Mandinach & Jackson, 2013; Wayman, Cho, et al., 2012; Wayman & Stringfield, 2006). The organization can provide support to individuals and collectives through structures such as effective data systems, training, or dedicated time for educators to collaboratively create data (Datnow et al., 2007; Lachat & Smith, 2005; Marsh, McCombs, & Martorell, 2010; Wayman, Jimerson, & Cho, 2012; Wayman & Stringfield, 2006). Examples in which data capture activities contribute back to the organization include creation of new forms of data, the preparation of graphic displays, and validation of existing data (Datnow et al., 2007; Lachat & Smith, 2005; Wayman & Stringfield, 2006). Meaning Making In the meaning-making stage, individuals and collectives interpret data and determine subsequent changes in practice. Much professional learning research focuses on meaning making (e.g., Desimone et al., 2002; Gallucci, 2008; Lachat & Smith, 2005; Wayman & Stringfield, 2006). The organization can support meaning making in a variety of ways, such as time and space for collaboration or support staff to help with interpreting data and choosing next steps (Datnow et al., 2007; Earl & Katz, 2006; Mandinach & Gummer, 2013; Marsh, McCombs, & Martorell, 2010; Wayman, Cho, et al., 2012). Educator meaning making contributes back to the organization by creating more effective, informed practitioners (Bailey & Barley, 2011; Brown & Duguid, 1991). Information Sharing Learning is often either lost or isolated to particular individuals unless this knowledge is transferred and preserved throughout the organization (Argyris & Schön, 1996; Brown & Duguid, 1991; Senge, 2006). The final two steps of our conceptual framework describe these processes. Once individuals and collectives have acquired practical learning via data capture and meaning making, this learning should be shared with other entities throughout the organization. The organization can support information sharing by establishing structures, mechanisms, and processes that enable knowledge transfer (Brown & Duguid, 1991; Senge, 2006; Wayman, Cho, et al., 2012; Wayman, Jimerson, & Cho, 2012). Examples include activities that connect educators or schools around specific problems, or websites that enable educators to share practices (Cho & Wayman, 2014; Kerr, Marsh, Ikemoto, Darilek, & Barney, 2006; Wayman, Midgley, & Stringfield, 2006). Information sharing contributes to the organization by disseminating expertise to existing members and by stimulating innovation through the introduction of ideas novel to the collective (Senge, 2006; Wayman, Jimerson, & Cho, 2012). Knowledge Codification Even when learning is commonly shared throughout an organization, there are no guarantees it will be preserved unless it is codified. Examples of such codification include consistent creation of documents that describe new learning, or creation of processes that permanently embed new learning into everyday practice (Brown & Duguid, 1991; Senge, 2006; Wayman, Jimerson, & Cho, 2012). Individuals and collectives benefit from knowledge codification through the availability of new ideas and rigor that codification provides (Brown & Duguid, 1991; Senge, 2006). Organizations benefit because new learning is not isolated in the individual or collective that produced that learning, but is spread throughout the entire organization (Brown & Duguid, 1991; Senge, 2006). Transfer and preservation of information are critical. They not only enable educators to build on prior learning but also add richness to learning by drawing from ideas in other parts of the organization. By casting professional learning in terms of organizational structures, this framework enables a district to provide needed help to individuals and collectives as they navigate a complete data-using process. Reciprocally, this process enables the district as a whole to benefit from that learning. Learning about data use thus becomes a mutually reinforcing enterprise, and districts become learning organizations in the sense described by Senge (2006). THE PRESENT STUDY As noted in our review of the literature, data-related professional learning is considered important for an effective data initiative. Unfortunately, the research base has not provided depth or detail regarding teachers needs, nor how these needs are supported. Accordingly, the goal of the present study is to examine data-related professional learning provided to teachers by their districts. Consistent with our conceptual framework, we will examine two research questions: (a) What knowledge and skills do teachers need to engage in data-informed practice? and (b) How do professional learning supports address these needs? To answer these questions, we examined data-related professional learning in three school districts. We used our theoretical framework to categorize the data-related professional learning needs of teachers and to determine how district-enabled or -supported provisions fit these needs. Finally, we use the framework and prior research to discuss how data-related professional learning might be made more effective. METHOD INTRODUCTION The present study is drawn from a larger three-year project designed to help three school districts improve their use of data. Data were collected from September 2010 through February 2011 in three districts in Texas. Study districts were chosen from a pool of volunteer districts to provide diversity in terms of size, achievement, and socioeconomic makeup. Study districts were not selected for their success at using data; in fact, district leaders volunteered for this study to improve their districts data use. In the following sections, we describe the contexts within which the study districts operated and describe the procedures used in collecting and analyzing data for the study. THE STUDY DISTRICTS Boyer School District1 was a district of approximately 8,000 students that mostly served a non-Latino White population,2 less than 5% of whom were economically disadvantaged. Educators in Boyer spoke of data use primarily in terms of state test scores. Most students in Boyer easily met minimum state standards, so many educators believed data use to be inapplicable to their context. Boyer employed periodic curriculum-driven benchmark exams districtwide, but computer systems made these assessment results difficult to access. In a typical year, the percent of students meeting standards on the state exam was consistently greater than 95% in Boyer; the overall state rate was typically about 82%. Gibson School District was a district of approximately 25,000 students of various ethnic backgrounds,3 half of whom were economically disadvantaged. State test performance was important in Gibson, but educators spoke of these tests as one of many student achievement indicators. Much of data use in Gibson surrounded a set of locally developed benchmark exams tied to the curriculum. In a typical year, approximately 75% of Gibson students met standards on the state exam. Musial School District was a district of approximately 45,000 students of various ethnic backgrounds,4 a third of whom were economically disadvantaged. State test performance was a very strong focus throughout Musial. Musial employed districtwide benchmark tests that were intended to align to curriculum and to predict outcomes on the state achievement test. In a typical year, approximately 85% of Musial students met standards on the state exam. PROCEDURE Data were collected through interviews, focus groups, and document analysis. In line with our research questions, we focused on determining which skills participants believed teachers needed to be effective users of student data and how each district provided data-related professional learning. In the following sections, we describe our procedure, including data collection, protocols, and analyses. Data Collection Data were collected through focus groups and interviews of teachers, principals, support personnel, and central office administrators (see Table 1). All interviews and focus groups lasted approximately 1 hour and were recorded and transcribed. In addition, we collected school and district documents that triangulated our interviews (see Table 2). Here, we describe school- and district-level data collection. Table 1. Study Participants by Role and District
School level. We used a stratified random sampling process to identify one high school, one middle school, and one elementary school within each district. We aimed to include voices throughout a school, including campus administration, classroom teachers, instructional support personnel (e.g., instructional coaches, data coaches, curriculum specialists), and individuals perceived as expert in the use of data to inform classroom practice. We conducted interviews of campus principals at each site. Data from teachers and other roles were collected through focus groups. At each school, we conducted two focus groups, each ranging from 3 to 6 participants. The first focus group consisted of randomly selected teachers. The second focus group consisted of exemplary users (individuals seen as go-to persons for data use) selected through a peer nomination process. We chose this method in an attempt to ensure that we did not unwittingly exclude any data-able educators who could speak to effective professional learning for data use at a campus or in a district. District level. At the district level, we used a chain sampling method (Merriam, 2009), in which we asked participants to identify persons whose job responsibilities entailed planning or supporting professional learning or data use, to build our complement of informants. We followed this chain of informants until recommendations became redundant. We worked to triangulate interview data through analysis of district and campus documents specific to professional learning or data use. To facilitate the collection of appropriate documents, we searched the three district websites and district-level online policy manuals using terms such as data, data use, professional development, professional learning, and training. We similarly searched the nine campus websites for appropriate documents (though we found that most such information was located on district websites). We also asked district- and campus-level participants for copies of documents they referenced in the course of any interviews or focus groups. Of the numerous documents linked to these terms, we selected a total of 50 documents across the three districts (see Table 2) that addressed one or more specific elements of professional learning for data use. Table 2. Number and Types of Documents Selected for Final Analysis
Protocols Focus groups and interviews were conducted, relying on semistructured protocols that allowed discussion to flow while ensuring that critical elements were addressed. Protocols were constructed to ensure that respondents had opportunities to address the characteristics of effective professional learning described previously. Protocols addressed questions such as: What should teachers know to be effective users of data? How do you best learn any new skill? How is professional learning delivered in your district? and Describe some data-related professional learning in which you have participated. Questions changed slightly depending on participant role (teachers, principals, or central office administrators), but each protocol addressed the same major concepts. Analyses Qualitative analyses were conducted by examining educator comments from interviews and focus groups. Data from collected documents were used to triangulate interview data and provide descriptions of district-provided data-related professional learning. In conducting these analyses, we followed our conceptual framework. We began the analysis process by categorizing comments relating to perceived needs (Research Question 1) into the four stages of our framework (data capture, meaning making, information sharing, and knowledge codification). As analysis proceeded, we allowed subcodes to emerge that described how individuals and collectives operated within these categories. Two subcodes emerged from data capture (asking appropriate questions and accessing data systems), two subcodes developed from meaning making (data literacy and fit with practice), one subcode developed within information sharing (collaboration), and one subcode developed from knowledge codification (link to solutions). We then used these codes to categorize data relating to how districts supported the learning needs of teachers (Research Question 2). We also allowed space for the emergence of subcodes, mindful of the possibility that districts might support data-related professional learning in ways other than those outlined in Research Question 1, but no such subcodes emerged. FINDINGS In our conceptual framework, we posited that in order to empower teachers to use data to inform classroom practice, professional learning for data use must address the ways in which individuals and teams of teachers capture data, make meaning of these data, share information, and codify knowledge into lasting changes in practice. Teacher needs for data-related professional learning were first examined, followed by an examination of the ways that districts supported these needs. Accordingly, results are presented in two sections. RESEARCH QUESTION 1: TEACHER NEEDS FOR DATA-RELATED PROFESSIONAL LEARNING Returning to our conceptual framework, we culled the data collected from interviews, focus groups, and campus and district documents to identify the data-related knowledge and skills most frequently identified by educators in these districts as contributing to competence in data use. In the following narrative, we categorize results of this analysis by the four stages of Supovitzs (2010) cycle. Within each of these sections, subsections are provided, corresponding to themes that arose regarding the needs of individual and collective learning. Data Capture The first element in the cycle of data-informed inquiry involved the ways in which data are generated, captured, or selected for review or analysis. This was the most often used code, resulting in 92 instances (see Table 3). Across the three districts, this was most frequently reflected by two themes: (a) the importance of generating appropriate questions to guide data use; and (b) the ways in which teachers accessed data from various data systems. Asking appropriate questions of the data. Educators in each district talked about the importance of posing thoughtful, relevant questions of the data. As a Musial high school teacher noted, You can have the data and it can mean nothing to you if you dont know how to really start asking the right questions. Thirty-seven comments addressed this theme (see Table 3). Table 3. Number and Percentages of Comments Specific to Articulated Teacher Needs by District
A few educator comments were very direct (I think its about asking the right questions), whereas others were more indirect but noted the importance of having a focus for data use (What are the significant pieces youre looking for?). Often, this skill emerged as educators talked about being overwhelmed by copious amounts of data. As an antidote, some educators in each district suggested that using guiding questions could make data use more manageable and purposeful. One teacher leader in Boyer, who led a professional learning community (PLC) group, noted that the teachers she worked with needed to know, in her words, how to ask the right questions to gather the kind of data you want. They understand they need to gather data, but theyre like, What do I need to be gathering data on? What are the important things? . . . And so for them, I think [whats needed] is how to incorporate gathering the data every day, or specific observationswhat is the data Im trying to gather? What questions? How do I frame it? In a similar vein, a high school teacher in Musial noted, You have to be able to see the big picture, and then see the smaller pictures, and be able to break it up and ask the right questions. Accessing and operating data systems. The skill most commonly addressed by teachers in each district was the ability to navigate various data systems that enabled access to student data. In all, 55 comments related to leveraging data systems (see Table 3). We note that this is likely an underestimation of the participants who considered technical skills important: In group interviews, we often heard one or two teachers make a comment about such skills, only to witness several others nod in seeming assent. Teachers routinely commented on the importance of technical skills as foundational to effective data use. A teacher in Gibson shared that to be capable users of data, teachers needed computer skills, for sureand not just your basics, but knowledge of many systems. In Boyer, another said teachers need to be completely computer savvy. In Musial, a teacher asserted, Right now, with the current shift, [teachers] have to be able to use the technology thats available. And I think a lot of teachers are resistant to that and the teachers who are really successful with keeping good data are the teachers who arent hesitant to use the new tools that are available. Campus and district leaders similarly addressed data system skills as critical to teacher data use. To illustrate, a campus principal in Gibson noted that the ability to use the technology better was essential because, with the user-unfriendly data systems at his campus, its a frustration to be able to get [data], so [teachers] just decide, oh heck Ill march on without it. Similarly, a leader in Musial noted that even with the new data portal in place, some teachers still needed help learning to access the data. Meaning Making The second element in the cycle of data-informed inquiry speaks to the processes by which individuals and small collectives make sense of data and fit new understandings into an ever-developing understanding of practice. This element was the second most often mentioned by participants, resulting in 87 comments (see Table 3). It was primarily reflected by two broad themes: (a) the capacity of educators to analyze and interpret data (data literacy); and (b) the capacity of teachers to fit data-related learning into their classroom practice. Data literacy. Numerous educators in all three districts said that teachers needed skills related to analyzing and interpreting data. Forty-one comments fit this code (see Table 3). A Musial middle school teacher described these skills generally as needing to know what were looking at in such a way that the data were not just numbers on a page. When participants spoke of these skills, they said they wanted to have enough knowledge to feel comfortable that they were making appropriate decisions based on their data. Participants not only noted these skills as important but also frequently characterized themselves and others as deficient in these skills. Other educators wanted to know more about the capabilities of various types of assessments, such as classroom tests, reading inventories, and state test scores. Along these lines, several educators expressed a need to know more about the relative benefits and uses of various forms of measurement, such as raw scores or scale scores. In fact, several educators across the districts talked about wanting to know more about what makes an assessment valid, or reliable, and how they could ensure that they were making decisions rooted in good assessments. The need for these skills was not just noted by teachers, but even by some instructional coaches (who often functioned as leaders of teachers). In Boyer, a high school instructional coach noted how she was often called on to calm anxieties among concerned teachers, noting, They know how important the data is and theyre so petrified that theyre going to interpret it the wrong way. At least theyre using the data, but theyre always checking and recheckingAm I reading this right? What does this say?So at least theyre asking, but theyre very intimidated by it. Fit with practice. Many educators in each district believed it was important for teachers to learn how to integrate data use into the whole of curriculum, instruction, and assessment. That is, educators wanted to learn how to use data within their day-to-day scope and sequence so they could adjust their teaching or lesson plans in more timely ways. In all, 46 comments addressed the need to make data use fit with the broader aspects involved in the practice of teaching (see Table 3). Fit with practice involved issues such as being able to use data in connection with planning and revising assessments at the time of need, being able to adjust strategies or reorder curriculum priorities, and selecting from among instructional strategies as needs were indicated by various data. For example, educators in all three districts said that if they learned how to use data more formatively, they could better react to student needs. They noted that using data in this way could guide efforts to expand or contract teaching units during the year, or even make adjustments to the next days lesson. One teacher noted the lack of this fit at present: So when you get your printout, you know exactly which [standards] your child was deficient in. And I think our problem at this point is, Great. I know where theyre weak. How do I go back and teach that when I need to be moving on to new content? A story from a Gibson district leader exemplified the struggle that many teachers faced in fitting data use to everyday practice. In this example, teachers experienced a poor fit between the type of assessment and chosen instructional remedies. She told us, [My teachers] say, Im progress monitoring, Im progress monitoring, but reported little progress after adjusting instruction. The leader therefore conducted a series of observations and discovered a mismatch between assessment and instructional response. She recounted telling the teachers, These activities are fabulous[you are using a] research-based activity focused on phonological awareness, research-based activity focused on decoding, research-based activity based on reading comprehensionbut you are measuring fluency. In this instance, a non-fit with practice undercut the well-intentioned efforts of teachers to use data to inform instruction. Information Sharing The third element in the cycle of data-informed inquiry speaks to the processes by which individuals and small collectives share what they have learned, in the pursuit of continued improvements in teaching and learning. Across the three districts, this was the skill mentioned least frequently, with only 17 comments relating to information sharing. Only two comments came from Boyer educators. In most of the 17 comments, educators described how they collaborated with others and noted the benefits they received from bouncing ideas off one another and sharing ideas for practice. They also indicated that collaboration was enjoyable, so they looked forward to working with peers even when the only time to collaborate was outside the regular workday. For example, during a focus group with elementary teachers in Boyer, one teacher noted, We have so much data, that I think, as one person, its really hard to make sense of it all. She continued on to say the most useful examinations of data, for her, involved time being set aside for teams of teachers to look at various data and talk, sometimes with data experts available for help. Still, teachers also told us that collaboration around data could sometimes be tense or uncomfortable. Some teachers said they had no issues laying out data from their students for supervisors and teachers, but others were more wary. A middle school teacher in Musial noted that sometimes teachers became defensive talking about data because they know they could have done something different, but didnt, and theyre trying to defend themselves so they dont look bad. These sentiments were echoed by a district leader in Musial, who talked about the trust that imbues good collaboration around data: Those collaborative conversations are a skill and being able to have collaborative conversation around what our data says and being able to be open about why it says what it does and what we are going to do about it and how we are going to help whoever is strugglingstudents or teachers. I think those are really pieces thattheyre hard. I mean, its hard to get there. Its hard to get to that trust level. Knowledge Codification The fourth and final element in the cycle of data-informed inquiry relates to the manner in which individuals and small collectives codify new understandings into longer term changes in practice. Examples of codification include revisions of curricula, revisions of scope and sequence documents, and the creation of descriptive documents to facilitate the use of data or data systems. This was among the least prominent of the four cyclic components, with 27 comments. Across districts, this was most frequently reflected by a single broad themethe need to link data use to particular solutions that extended beyond addressing a one-time need. Several teachers in each district said they needed guidance in selecting from various solution options (e.g., special programs, remediation, differentiated grouping) and in evaluating these options for effectiveness. Some wished there were some sort of master matching list to help them supplement data analysis with action steps. A group of elementary teachers in Boyer discussed this issue, noting that although the district was flush with support programs, linking solutions to student needs was difficult. During that group, we noted the following exchange: Teacher 1: Im new to [the district] so for me, OK, we have all these data, and we have all these resources. And you just match them up. If this student has this need, then which of the many computer programs, books, whatever it iswhat can really help with that? Teacher 2: Theres so many choices its confusing. Teacher 1: Sometimes theres too many resources. Teacher 3: Yeahits developing a plan, an individual plan, for the students after the data is collected. Where do you go from there? Educators also lamented a lack of manuals that detailed and explained reports available in data systems. For example, Gibson used a data warehouse that was stocked with numerous reports. A middle school leader told us, We have hundreds of reports we can run, but theres no manual that goes along with it that says, This is what this report represents so its just kind of guessing. Youre just looking through [the reports] and you say, Oh! I see what this is saying to me. But you have to figure it out. Further, a group of elementary teachers in Gibson noted that data system training occurred months before the actual launch of the system. Without written directions, they struggled to remember how to navigate the system. Suggestions ranged from an online store of directions to a ring of laminated [directions]. RESEARCH QUESTION 2: PROFESSIONAL LEARNING SUPPORTS In each district, a range of formats were aimed at supporting professional learningin general and specific to data use. Each district used time-bounded learning events, such as summer seminars, workshops, and learning delivered within faculty or team meetings. Each district also employed a variety of school-based personnel, and part of their jobs was to support data use in embedded ways. These roles included instructional coaches, content-area specialists, and academic deans, among others. We note that there was variation in how these educators perceived their job duties, with some focused on developing independent users of data, whereas others worked to run reports and point out trends for teachers. We also note that some professional learning initiatives encouraged and supported by districts (such as the PLCs in Musial and Boyer) varied in how they were implemented at the campus level. Also, educators in each district talked about learning via informal channelsthat is, learning from colleagues as a result of serendipitous conversations or asking for help. To examine how districts supported the needs of teachers, data were analyzed using the same codes that emerged from Research Question 1. Similar to Research Question 1, the following narrative categorizes results by the four stages of Supovitzs cycle. Within each of these sections, subsections are provided, corresponding to themes that arose regarding the needs of individual and collective learning. Data Capture In examining supports for teachers data-related learning needs, data capture was the most often used code. A total of 160 comments were coded as data capture, almost all of which (151) dealt with accessing data systems (see Table 4). In the following sections, we discuss professional learning supports for data capture in terms of the two themes identified in our analysis for Research Question 1: supports for learning to ask appropriate questions and supports for accessing data systems. Table 4. District Support of Teacher Needs for Professional Learning, Categorized by Components of the Conceptual Framework
Asking appropriate questions of the data. Supports for framing data use through effective questioning were notable in their absence: Only nine comments across the three districts were specific to the formation or use of guiding questions to drive data use (see Table 4). No evidence was found in district documents that indicated any attention to this issue. Most comments came from central office educators, with only 2 teachers offering comments indicating that they received professional learning related to the use of guiding questions. Although teachers and leaders frequently noted that data use could be overwhelming in the presence of copious amounts of data, leaders rarely spoke about helping teachers use questioning techniques to deal with this problem (i.e., selecting data appropriate to questions rather than digging through all available data). Instead, when leaders spoke of addressing this issue, they often spoke of solutions in terms of the data themselves (repackaging the data in a way that it is accessible to people, as one district leader put it). Accessing and operating data systems. Supports for accessing and operating data systems was the most often represented theme, with 151 comments (see Table 4). We found few details about such supports in district plans, which were riddled with ambiguity when it came to data use and technologies; although planning documents in each district mentioned technology as a training priority, they left unclear whether this related to instructional technology or to the use of district data systems. Still, participants frequently described event-based structures (i.e., workshops, meetings, web trainings) focused on access and use of data systems. Two prominent threads ran through these comments: (a) supports were often focused on the use of the systems themselves rather than the use of data housed in the systems; and (b) many of these learning opportunities were ill timed for impact. Consequently, participants often sought out each other for information as needed. These issues were remarkably consistent across the study districts. Across the districts, efforts aimed at helping teachers use district data systems were plentiful, but participant responses were mixed as to the effectiveness of these supports. This was because trainings focused predominantly on the systems themselves rather than leveraging data to improve practice. A district leader in Gibson described this phenomenon as having compartmentalized professional learning. She used an example in which the data system could be used to support block scheduling and a new social studies curriculum, but professional learning on these three components was dealt with separately. She explained, In this district theres not necessarily an integration of those three things. Sometimes its, Heres one thing, heres another, heres another. She thought this was problematic: Maybe teachers think about that as three different things when really it might be three different components of being able to function and be effective with one thing. Participants in all three districts frequently complained about a lack of thoughtful timing for these supports. Participants routinely described data system training that was provided long before it was needed (e.g., at end of a school year or during the summer) or at a time when they were overwhelmed by other tasks (e.g., the week before school started). These issues were described in a discussion between a Musial elementary teacher and campus leader, in which they discussed the rollout of training on their new data system: Leader: Well, I think the timing of rolling out the teacher portal was very poor. [District leaders] rolled it out in May, and by that time, teachers didnt havethere wasnt a need for them to go on the teacher portal. So they went to this training, got their passwords. . . Teacher (interrupting): TeachersI dont think they went to a training. Leader: Oh youre right! They didnt even go to a training. They were given an online training where they at least had to go in and make sure their username and password and print out a certificate, which, once you did that youre so worried about wrapping up the end of the year, people forgot about it at the very beginning of the year. . . . I really think that teacher portal is an outstanding tool. I just think that teachers werent given proper training and the timing was not relevant. Poorly timed learning opportunities created a dual-pronged hardship. Because learning about data systems competed with other pressing issues (i.e., obtaining textbooks, class rosters, and preparing classrooms), teachers attention was divided. Or, as an elementary teacher in Musial noted, [Trainings] occur when youre kind of overwhelmed, and its not going to be useful at that time. But then when you need that information, several months down the road, you cant remember what you were doing. Also, because training on data systems occurred before teachers were engaged with colleagues in collaborative planning around their current students, teachers lacked a relevant framework for application of the new learning. For these reasons, several teachers noted that despite the training on data systems, they still lacked requisite skills to access needed data in timely ways. Because existing supports for using systems were minimally effective, many teachers across the districts relied on colleagues for learning about systems. This permitted some teachers to acquire needed skills in ways that coincided with points of need. A high school teacher in Boyer explained: You hear [teachers] talking about stuff that theyre using and then you decide to use the same thing. You realize that its helping them, so you say, Hey, I noticed you did thiscan I do it too? Can you show me where to get it? And its a lot of show and tell. For othersthose who did not happen upon colleagues able to pass along needed knowledge and skillsdata systems remained an obstacle to data use. Such was the case with an elementary teacher in Musial, who noted, A lot of times, I feel at a loss. And then I try to talk to other people about it, and they dont remember either. Meaning Making Each district provided a range of supports (e.g., workshops, meetings, district-provided specialists) aimed at enabling meaning making. This portion of the cycle was the second most frequent, with 112 comments (see Table 4). In the following sections, we discuss professional learning supports for meaning making in terms of the two themes identified in our analysis for Research Question 1: the capacity of educators to interpret and analyze data (data literacy) and the capacity of teachers to fit data-related learning with classroom practice. Data literacy. Forty-four comments addressed data literacy supports (see Table 4). This was somewhat surprising because our examination of district policies and professional learning documents yielded no mentions of formal plans or sessions explicitly linked to data literacy. Despite the presence of comments suggesting efforts to support data literacy among teachers, interview data also suggested that this need went unmet for many teachers in each district. First, 21 of the 44 comments came from educators in Boyer, with only 23 comments coming from educators in Gibson and Musial combined. This suggested to us that data literacy efforts were sparse in those districts. However, we think it is possible that this need was largely unmet for Boyer educators too: Despite a greater volume of comments dealing with data literacy efforts, most comments from Boyer educators dealt with one particular initiative geared at helping teachers think outside of the high-achieving Boyer context. Leaders noted that teachers struggled to identify low-achieving students because some students could appear to be low achieving when compared with other Boyer students, while they were actually outperforming national norms. This led to over-referral of students to special program support. In response, two leaders visited elementary campuses, meeting with teams of teachers to discuss the difference between in-district comparisons and comparisons with national norming data. The size of Boyer seemed to make this initiative possible because district leaders could meet with teachers at all campuses within a short span of time. Apart from comments related to this initiative, we found little evidence to suggest widespread efforts to improve data literacy among teachers. The few comments related to data literacy efforts in Gibson or Musial were linked to special programs. Specifically, comments were made specific to workshops aimed at Response to Intervention requirements (progress monitoring and reporting of data) and at monitoring outcomes for English language learners. In these instances, teachers talked about being able to read reports with accuracy and about being able to report out individual student data compared with the progress of larger groups. Fit with practice. Analyses revealed 68 comments that addressed supports for fitting data use within the everyday practice of teachers (see Table 4). These supports existed in two primary areas. First, each study district had allocated funding for campus-based support personnel (e.g., instructional coaches, coordinators). Second, each district provided leadership in the area of special programs (e.g., special education, bilingual education, English as a second language). Comments from teachers suggested that leaders in these special programs did facilitate dialogues that helped teachers match program-specific data with classroom strategies or individualized educational program (IEP) planning. Even in the presence of these district-initiated supports, comments in each district typically referenced learning from campus-based colleagues. For instance, formal district supports included PLCs and team meetings, but comments still indicated that data-related professional learning typically occurred in these meetings because like-minded colleagues chose to address data, rather than because meetings were structured around data. For example, a high school teacher in Gibson noted that administrators facilitated large-group discussion about accountability data before the first day of school but suggested that the real work happened among departmental members after the formal session, where department members worked together to break down specific data relevant to their content areas. A similar colleague-driven process served as a learning support in Boyer, where a high school teacher described administrative responsiveness to ad hoc data-related requests: I dont think weve ever been denied information, and thats one of the things I really enjoy about this whole district is were very transparentwe can call teachers from other schools. . . . We can ask [our principal], we can ask any of the assistant principals and say Hey, Im looking for this data and they might ask Why? but only to make sure theyre getting us the right information, and then people will sit and go through the data with you to make sure you understand it. As another example, teachers specifically mentioned asking colleagues about spiraling skills into lessons (to fit a reteach into a curriculum scope and sequence) and using data to identify teachers who were being successful with particular lessons so that those teachers could then share successful practices with the team. In a handful of comments, leaders indicated that the campus-based support personnel were expected to facilitate this kind of sharing among colleagues based on data. In fact, some indicated that the informal follow-up was integral to understanding how to fit data to daily practice. To illustrate, an elementary teacher in Gibson shared that she found summer professional development related to data use to be overwhelming and decontextualized, with 30 or 40 teachers in a room all examining a common data set on a bank of computers. She told us, As a new teacher I would have a tendency more to have to rely on mentor teacherssome of the other teachers at my grade level who I could identify who could show me how to use the data. And I think I would probably get more from the other teachers and other instructional coaches that I work with who could actually sit and show me [how to use the data]. Information Sharing Sixty-one comments related to supports for information sharing, the third most coded element of the cycle (see Table 4). As with Research Question 1, we saw most district supports dealing with collaboration, with few supports directed toward other forms of information sharing. To this end, we note that each district encouraged and (to some extent) supported the co-construction of team structures aimed at using data to inform practice. In Gibson, these took the form of data teams, while Boyer used the PLC terminology. In Musial, both lexicons emerged: District-level leaders modeled data teams, and this language was adapted on some campuses (primarily the elementary and middle school campuses), but campus educators also talked about engaging in PLCs (mostly at the high school level). We considered these co-constructed supports because although districts offered support for teams and PLCs on request, they were not shaped by strict district mandate, and campus leaders had much influence over how these functioned. For example, campus leaders in Musial carved out time for PLC meetings, albeit in different ways: The middle school built an additional period into the schedule for interdisciplinary team meetings, the high school started school 20 minutes late one day a week to allow for morning PLC meetings to run slightly longer, and the elementary school used a buddy system so that every other week, teachers absorbed additional students into their classrooms to provide grade-level teams additional collaborative planning time. In each district, teachers and leaders talked at length about these collaborative structures, although comments suggested that implementation often fell short of providing the data-rich dialogues envisioned by district leaders. In Musial and Gibson, district leaders talked at length about data teams, and district leaders had prioritized the formation of these teams: A leader in Musial guided principals through a book study on the formation of data teams, and a leader in Gibson regularly modeled data talks wherein he would periodically visit each campus to talk with teams about benchmark data. However, comments from teachers suggested that on only two of the six campuses studied in those districts were data-rich dialogues a common occurrence (the elementary and middle school teachers in Musial attested to regular dialogues involving data). In Boyer and Musial, teachers and leaders talked about using PLCs to facilitate data-rich dialogues. However, comments suggested that some PLCs had integrated data into regular team dialogues, whereas others were called PLCs but functioned as traditional teams (i.e., there was little to no use of evidence, and agendas were described as addressing meeting-like issues such as field trips and scheduling). Comments regarding departmental and grade-level teams suggested that some data were discussed in these structures (e.g., benchmark data, state exams) and that in some, teachers engaged in collaborative planning and lesson sharing. Knowledge Codification The least coded element of the cycle was knowledge codification, with only 14 comments describing professional learning structures aimed at helping teachers learn to codify what they learn from data use (see Table 4). Consistent with Research Question 1, we examined codification in terms of solutions that went beyond a single instructional decision and were formally recorded to permanently preserve this knowledge. The few comments that related to codification dealt with curriculum and were only peripherally related to data. Comments included involving teachers in summer revisions to district curricula and with district-based personnel examining data to make suggestions on how to thread the patterns we saw into the next curriculum sequence, as a leader in Gibson put it. Overall, our data show little evidence of district supports for knowledge codification. DISCUSSION The present study examined data-related professional learning in three school districts. Guided by an organizationally based conceptual framework, we examined two research questions: (a) What knowledge and skills do teachers need to engage in data-informed practice? and (b) How do professional learning supports address those needs? In comparing teacher needs with associated supports, we observed that each was distributed similarly across the four components of the cycle. However, there was often a mismatch in the detail of how districts were providing supports for data-related professional learning needs. We also observed an imbalance across the four components of the cycle. The data indicated that teachers and leaders were more focused on data capture and meaning making than on information sharing and knowledge codification. The results of this study paint a picture of data-related professional learning that is counter to organizational research. Organizational research has noted the importance of knowledge generation but also has noted that the transfer and preservation of this knowledge is critical to the health of a learning organization (Argyris & Schön, 1996; Senge, 2006). Through transfer and preservation, knowledge is made rich: It enables individuals to build on their own prior learning, and it provides new learning to other individuals. Knowledge is not restricted to use by a few individuals but serves the whole organization. Thus, the school or district becomes smarter. The results of this study are potentially important for improved data use because few studies have examined data-related professional learning as an organizational process. In the following sections, we discuss what these results may mean for practitioners and future research. Specifically, we discuss data-related professional learning as a process, examine the balance between teacher needs and district supports, and offer recommendations for further action. DATA-RELATED PROFESSIONAL LEARNING AS A PROCESS The conceptual framework guiding this study suggests that data-related professional learning should be a process, embedded in organizational routines, where educators and the district reciprocally contribute knowledge and support. Preparation for learning, transfer of learning, and embedding this learning into routine district practice are equally important as the actual moment in which learning takes place. Together, these are elements of a healthy learning organization, as conceived by organizational theorists (Argyris & Schön, 1996; Senge, 2006). Our data showed that data-related professional learning was not a process in the study districts. Instead, data-related professional learning was typically gained in specific, isolated occurrences that were not integrated with each other or within a larger plan. Data from the present study highlighted three issues around lack of process: (a) knowledge was not preserved, (b) articulated plans for data-related professional learning were lacking, and (c) supports were isolated. In the following sections, we discuss each of these issues. Knowledge Was Not Preserved Although data-related professional learning occurs throughout the entirety of our conceptual framework, the framework can be loosely divided into two stages: one more focused on the moment when knowledge is gained (data capture and meaning making), and the other more focused on preserving this knowledge (information sharing and knowledge codification). Data from the present study showed participants at every level far more focused on the moment when knowledge was gained than they were on preserving that knowledge and using this preservation to catalyze further learning. This is hardly surprising given that prior research also focused on the moment when learning was actually gained (e.g., Desimone et al., 2002; Means et al., 2011; Wayman & Stringfield, 2006). The data did reveal a degree of support for information sharing, nearly all of which focused on collaboration. This is positive because collaboration is nearly unanimously touted as an effective learning structure, particularly for data use (Datnow et al., 2007; Earl & Katz, 2006; Hamilton et al., 2009; Marsh, 2012; Means et al., 2011; Wayman et al., 2006). However, the singular focus on collaboration raised two problems. First, we identified no mechanisms to share knowledge outside of the collaborative entity. Thus, learning gained within a collective typically stayed within that collective and was not available for the benefit of other educators (Argyris & Schön, 1996; Brown & Duguid, 1991; Senge, 2006). Second, the focus on collaboration meant our districts were not tapping a rich store of other information sharing structures, such as reports or push notifications from the data system (Supovitz, 2010; Wayman, Cho, & Richards, 2010). On the other hand, support for knowledge codification was negligible. When knowledge was gained in our districts, there were no mechanisms to spread this knowledge throughout the district, to appropriately infuse it into the practice of other educators, or even just to make it available for access by other educators. Thus, our districts suffered from a problem of efficiency common to many organizations: Much knowledge died within an individual or collective in our districts, rather than enabling the district to continually build on and reap the benefits of this knowledge (Argyris & Schön, 1996; Brown & Duguid, 1991; Senge, 2006). Articulated Plans for Data-Related Professional Learning Were Lacking Another hindrance to conducting data-related professional learning as a process was a lack of formal plans for integrating data-related professional learning into various aspects of educational process and practice. That is, our data showed no formal or informal plans for integration of data-related professional learning with district learning aims or content area professional development, nor was there evidence of intentionality in structuring professional learning supports to ensure the development of skills and knowledge across a complete data use cycle (i.e., data collection through knowledge codification). As a result, data-related professional learning opportunities were isolated events rather than a tightly connected set of capacity-building opportunities. Research on educator data use has noted the importance of articulated plans to guide the work of the organization or district (Datnow et al., 2007; Supovitz & Klein, 2003; Wayman, Jimerson, & Cho, 2012). In light of this research, we think the lack of such planning with regard to data-related professional learning may have limited educators ability to link data use with practical solutions. For example, participant comments on data system training were highly critical of poor timing, yet a planning process inclusive of teacher input could have mitigated this issue (Cho & Wayman, 2014). Further, our data revealed no evidence that data-related professional learning offerings were connected to each other, either formally or informally. This is in direct contradiction to research on general professional learning, which states that professional learning is most effective when it is integrated with prior learning (Borko, 2004; Desimone et al., 2002; Penuel, Fishman, Yamaguchi, & Gallagher, 2007). Further, this disconnect is in contradiction to systemic perspectives on data use, which call for alignment of definitions, uses, and practice (Datnow et al., 2007; Supovitz, 2010; Wayman, Jimerson, & Cho, 2012). Supports Were Isolated Our data revealed supports aimed at facilitating data-related professional learning, such as collaborative structures (e.g., PLCs), data system training, and instructional support staff. However, we saw little evidence that these supports were linked to other supports. That these supports were isolated from each other likely hindered the abilities of these districts to conduct data-related professional learning as a process. A good illustration is seen in the initiative undertaken by two Boyer administrators to help teachers see student achievement in the larger national context, in addition to their high-achieving local context. By itself, this learning support was a positive one because it was conducted within educators contexts, it was relevant to their work, and it dealt with an immediate problem (Desimone et al., 2002; Gallucci, 2008; Wayman, Jimerson, & Cho, 2012; Wei et al., 2009). Unfortunately, the data did not show that this support was connected to anything else. Thus, although it was likely helpful in the moment, the lack of connection to other supports, initiatives, and the larger process of data-related professional learning probably rendered it less effective in the long term. BALANCING TEACHER NEEDS WITH PROFESSIONAL LEARNING SUPPORTS At first glance, it might appear that supports for data-related professional learning were well balanced with needs for data-related professional learningthe percentages of comments coded in each of the four parts of the cycle were similar for both Research Questions 1 and 2. However, a closer look at the details of district supports highlights some mismatches that contributed to the negativity with which our participants spoke of their data-related professional learning. First, almost half of the comments regarding district supports focused on computer data system training, but these opportunities were widely panned because participants did not find them useful. Participants reported that these trainings were ill-timed, often occurring months before the trained skill was to be used or when they were overwhelmed with other responsibilities. Perhaps most important, participants characterized data system training as focused on data system functions rather than their practice. Thus, data system training was not contextual or coherent, as advised by scholars of professional learning (Borko, 2004; Gallucci, 2008; Garet et al., 2001). Second, teachers demonstrated a perspective on questioning that professional learning supports did not address. Teachers needed the ability to frame useful guiding questions to facilitate the identification of relevant data from the copious amounts of data available. However, professional learning supports to help teachers learn or apply questioning techniques were largely absent. When leaders did mention questioning, it was in terms of browsing data to see what trends or patterns emerged, as though analysis should give rise to guiding questions rather than the other way around. Third, educators often spoke of data-related learning needs in terms of day-to-day practice: They wanted to understand what data meant so they could make practical changes. Further, some educators seemed trepidatious about the decisions they made based on data, worried that they might not understand the data well enough. These educators had a need to develop deep pedagogical data literacy, or the ability to turn data into actionable knowledge (Mandinach, 2012), yet we did not see evidence that districts were moving to support these needs. Although one third of district support comments pertained to meaning making, we heard little to indicate that supports were aimed toward practice or issues preceding practice. Instead, supports often dealt with problems in isolation, such as special programs or unique problems. Fourth, one area where district supports meshed well with teacher needs was collaboration. Not only were districts providing time for educators to collaborate, but they also sometimes provided support personnel to work with collaborative teams on various issues. Thus, they were dealing not just with collaborative time but also with what went on within that time (Hamilton et al., 2009, Kerr et al., 2006; Wayman, Jimerson, & Cho, 2012; Wayman et al., 2006). Even with these supports, some participants raised concerns that teachers were uncomfortable with and defensive about what went on in collaboration. It was not clear whether these concerns existed within teams that benefited from support staff or occurred in teams unsupported by these staff. FUTURE DIRECTIONS FOR PRACTICE AND RESEARCH The problems shared by our districts in the present study may well be common across the nation, given the struggles shown by recent research (Anderson et al., 2010; Kerr et al, 2006; Louis et al., 2010; Wayman, Cho, et al., 2012; Wayman et al., 2007; Young, 2006). We believe that the present study has provided information that can inform district leaders looking to create more data-informed districts and researchers looking to create new knowledge. First, district leaders may need to change their perspective on data-related professional learning, from a set of isolated events to a fully integrated process. It will also be important to consider how data-related professional learning fits within the organization as a whole, remembering that the relationship between the district and professional learner is a reciprocal one. Too, districts will change as they grow in individual capacity and organizational learning, so it is important to consider these processes as organic, revisiting them frequently. Perhaps most important, each learning-related decision should be vetted in how it supports the work of education in a particular context. Second, knowledge preservation appears to be an area ripe for improvement. In terms of information sharing, our study and previous research (Datnow et al., 2007; Hamilton et al., 2009; Wayman, Cho, et al., 2012) suggest that districts are comfortable fostering collaborative opportunities. While collaboration can be improved, districts should also look to implement other context-relevant structures that enable knowledge among various entities (e.g., individuals, roles, teams, schools). Examples might include activities to discuss common understandings about teaching, learning, and data (Senge, 2006; Wayman et al., 2006; Wayman, Jimerson, & Cho, 2012) or processes to share various data reports. Data systems can be a prime vehicle for sharing information, using social networking to connect educators working on similar problems or sending push notifications through the data system (Cho & Wayman, 2014; Wayman, Cho, et al., 2012). Knowledge codification is another preservation issue. There are many forms of codification that could help districts. For instance, teachers in the present study suggested that even simple forms such as a cheat sheet of data system functions would be helpful. Along these lines, it might be important to create methods to codify major sources of information, such as solutions to problems, analyses, or effective practices. Besides recording information, codification also includes embedding knowledge into practice. As an example, participants in the present study worried about balancing attention to data with maintaining curriculum pacein this instance, a district practicing codification might pool knowledge across teams or schools to better inform support positions in providing recommendations. A final point is important to note: Much professional knowledge is contextual, so district leaders should not restrict codification only to knowledge that is generalizable to the entire district. Codification need not equate to unification, so it is appropriate to codify contextual knowledge as long as it is clear when and where this knowledge is applicable. Third, data system training is another area for potential improvement. Our results highlighted the ineffectiveness of data system training; this was especially problematic given the large amount of focus the study districts placed on this training. Our data suggest that data system training could have been improved in a number of ways, for example, by targeting knowledge relevant to current problems teachers were solving and being watchful that training was conducted at times when teachers could most use it. Perhaps most important, data system training could have been improved by focusing on problems of practice with the data system as a data access support rather than focusing on system functionalities (Cho & Wayman, 2014; Wayman, Jimerson, & Cho, 2012). Fourth, district leaders would do well to require themselves to be intentional in planning for data-related professional learning. There was little evidence of concrete plans or processes related to the development of capacity for data use, which may have contributed to their data-related professional learning struggles. Thus, district leaders should look to articulatein writingplans for data-related professional learning. Further, these plans should articulate how data-related professional learning fits into larger plans for professional learning, teaching practice, and other district process. And at every turn, a balance of formal and informal structures should guide the work to be conducted (Wayman, Jimerson, & Cho, 2012). Finally, there are a number of avenues for future research that can be built from this study. First, there is an opportunity to extend this study by examining data-related professional learning in practice, as advocated by a growing number of researchers (see Coburn & Turner, 2012; Little, 2012; Spillane, 2012). The present study provides a needed first step toward understanding data-related professional learning as a process, but without a study that examines practice as it happens, much is left unknown about the detailed workings of various supports and processes. Second, the present study leaned on empirical data to provide subcodes within the four cyclic components of our framework. Undoubtedly, there are other manifestations of these components that would be revealed through a review of research and examining different samples. Third, we were only able to touch generally on actions that districts can take within this frameworkfuture research could be conducted that specifically describes sets of tools and practices that are potentially effective. For instance, what are some innovative forms of information sharing that might prove effective? How can teachers best learn questioning techniques? What are the various ways that codification may be employed to support data-related professional learning throughout the organization? Fourth, it is necessary to engage in detailed study of the reciprocal role between individuals/collectives and the organization. CONCLUSION In his book Imagine (2012), Jonah Lehrer asserted that creative moments and innovative ideas may seem like a flash of inspiration but are actually the sum of that moment plus all the hours one has spent working through a problem. Lehrer contended that the moment of inspiration is actually just the end result of a long process.5 In the present study, we have taken a similar approach, that data-related professional learning is not just the moment that one learns. Instead, it is the sum of preparation, the knowledge that has been preserved in the school or district, and the actual moment of learning. Thus, educator capacity for data use is not achieved in momentary flashes or engagement in sporadic professional development sessions. Instead, it is constructed through data-related professional learning that is reciprocal in nature and that edifies the individual as well as the organization. Guided by our framework, we posited that although individual learning is important, it is also temporary in an organizational sense. That is, when the focus of data-related professional learning is solely on the individual, knowledge often either stays within the learner or is forgotten by the learnerin either case, it does not provide benefit to the entire school or district. Preservation and accessibility of knowledge throughout an organization thus becomes critical. We believe that, similar to the districts in the present study, there are many districts struggling with data-related professional learning. Rethinking data-related professional learning in terms of the organizationthat when we learn, we learn for all of usmay well reduce these struggles and make data use a positive force in teachers educational practice. Acknowledgments The authors thank The Spencer Foundation for funding the project from which this article comes. The authors would also like to express our thanks to Vincent Cho, Daniel Spikes, and Virginia Snodgrass Rangel for their thoughtful assistance. We wish to extend special thanks and admiration to all of the educators in the three study districts, not only for their assistance in our project, but also for their commitment to education. Notes 1. Pseudonyms are used for each district. 2. 80% non-Latino White, 10% Latino. 3. 40% Latino, 30% non-Latino White, 20% African American. 4. 50% non-Latino White, 25% Latino, 10% African American. 5. Lehrers book has been discredited and removed from circulation because he fabricated and misused quotes relating to Bob Dylan (Moynihan, 2012). Although misquoting Dylan is certainly an egregious mistake, we note that none of the science behind the point used here has been shown to be poorly researched. References Anderson, S., Leithwood, K., & Strauss, T. (2010). Leading data use in schools: Organizational conditions and practices at the school and district levels. Leadership and Policy in Schools, 9(3), 292327. doi:10.1080/15700761003731492/ Argyris, C., & Schön, D.A. (1996). Organizational learning II: Theory, method, and Practice. New York, NY: Addison-Wesley. Bailey, D. E., & Barley, S. R. (2011). Teaching-learning ecologies: Mapping the environment to structure through action. Organization Science, 22(1), 262285. Borko, H. (2004). Professional development and teacher learning: Mapping the terrain. Educational Researcher 33(8), 315. Brown, J. S., & Duguid, P. (1991). Organizational learning and communities-of-practice: Toward a unified view of working, learning, and innovation. Organization Science, 2(1), 4057. Cho, V., & Wayman, J. C. (2014). Districts efforts for data use and computer data systems: The role of sensemaking in system use and implementation. Teachers College Record. Coburn, C. E., & Turner, E. O. (2012). The practice of data use: An introduction. American Journal of Education, 118(2), 99111. Datnow, A., Park, V., & Wohlstetter, P. (2007). Achieving with data: How high-performing school systems use data to improve instruction for elementary students. Los Angeles: University of Southern California, Rossier School of Education, Center on Educational Governance. Desimone, L. M., Porter, A. C., Garet, M. S., Yoon, K. S., & Birman, B. F. (2002). Effects of professional development on teachers instruction: Results from a three-year longitudinal study. Educational Evaluation and Policy Analysis, 24(2), 81112. Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915945. Goertz, M. E., Olah, L. N., & Riggan, M. (2010). From testing to teaching: The use of interim assessments in classroom instruction. Philadelphia, PA: Consortium for Policy Research in Education. Guskey, T. R., & Yoon, K. S. (2009). What works in professional development? Phi Delta Kappan, 90(7), 495500. Hamilton, L., Halverson, R., Jackson, S. S., Mandinach, E., Supovitz, J. A., & Wayman, J. C. (2009). Using student achievement data to support instructional decision making. Washington, DC: Institute of Education Sciences and the National Center for Education Evaluation. Honig, M. I., & Venkateswaran, N. (2012). Schoolcentral office relationships in evidence use: Understanding evidence use as a systems problem. American Journal of Education, 118(2), 143166. Ikemoto, G. S., & Marsh, J. A. (2007). Cutting through the data-driven mantra: Different conceptions of data-driven decision making. In P. A. Moss (Ed.), Evidence and decision making (pp. 74104). Malden, MA: Blackwell. Ingvarson, L., Meiers, M., & Beavis, A. (2005). Factors affecting the impact of professional development programs on teachers knowledge, practice, student outcomes & efficacy. Education Policy Analysis Archives, 13(10), 128. Kerr, K. I., Marsh, J. A., Ikemoto, G. S., Darilek, H., & Barney, H. (2006). Strategies to promote data use for instructional improvement: Actions, outcomes, and lessons from three urban districts. American Journal of Education, 112, 496520. Lachat, M. A., & Smith, S. (2005). Practices that support data use in urban high schools. Journal of Education for Students Placed At Risk, 10(3), 333349. Lehrer, J. (2012). Imagine: How creativity works. New York, NY: Houghton Mifflin. Little, J. W. (2012). Understanding data use practice among teachers: The contribution of micro-process studies. American Journal of Education, 118(2), 143166. Louis, K. S., Leithwood, K., Wahlstrom, K., Anderson, S., Michlin, M., Mascall, B., . . . Moore, S. (2010). Learning from leadership: Investigating the links to improved student learning. Final Report to the Wallace Foundation. New York, NY: The Wallace Foundation.
Merriam, S. B. (2009). Qualitative research: A guide to design and implementation. San Francisco, CA: Jossey-Bass. Moynihan, M. C. (2012, July 30). Jonah Lehrers deceptions Tablet. Retrieved from http://www.tabletmag.com/jewish-news-and-politics/107779/jonah-lehrers-deceptions Park, V., & Datnow, A. (2009). Co-constructing distributed leadership: District and school connections in data-driven decision-making. School Leadership and Management, 29(5), 477494. Penuel, W. R., Fishman, B. J., Yamaguchi, R., & Gallagher, L. P. (2007). What makes professional development effective? Strategies that foster curriculum implementation. American Educational Research Journal, 44(4), 921958. Schildkamp, K., & Kuiper, W. (2010). Data-informed curriculum reform: Which data, what purposes, and promoting and hindering factors. Teaching and Teacher Education, 26, 482496. Senge, P. M. (2006). The Fifth Discipline: The art and practice of the learning organization. New York, NY: Currency Doubleday. Spillane, J. P. (2012). Data in practice: Conceptualizing the data-based decision-making phenomena. American Journal of Education, 118(2), 113141. Supovitz, J. (2010). Knowledge-based organizational learning for instructional improvement. In A. Hargreaves, A. Lieberman, M. Fullan, & D. Hopkins (Eds.), Second international handbook of educational change (pp. 707723). New York, NY: Springer. Supovitz, J. A., & Klein, V. (2003). Mapping a course for improved student learning: How innovative schools systematically use student performance data to guide improvement. Philadelphia: Consortium for Policy Research in Education, University of Pennsylvania. Valli, L., & Buese, D. (2007). The changing roles of teachers in an era of high-stakes accountability. American Educational Research Journal, 44(3), 519558. Wayman, J. C., & Cho, V. (2009). Preparing educators to effectively use student data systems. In T. J. Kowalski & T. J. Lasley (Eds.), Handbook on data-based decision-making in education (pp. 89104). New York, NY: Routledge. Wayman, J. C., Cho, V., Jimerson, J. B., & Spikes, D. (2012). District-wide effects on data use in the classroom. Education Policy Analysis Archives, 20(25). Retrieved from http://epaa.asu.edu/ojs/article/view/979 Wayman, J. C., Cho, V., & Johnston, M. T. (2007). The data-informed district: A district-wide evaluation of data use in the Natrona County School District. University of Texas, Austin. Wayman, J. C., Cho, V., & Richards, M. R. (2010). Student data systems and their use for educational improvement. In B. McGaw, P. Peterson, & E. Baker (Eds.), The international encyclopedia of education (Vol. 8, pp. 1420). London, England: Elsevier. Wayman, J. C., & Jimerson, J. B. (2014). Teacher needs for data-related professional learning. Studies in Educational Evaluation. doi:10.1016/j.stueduc.2013.11.001 Wayman, J. C., Jimerson, J. B., & Cho, V. (2012). Organizational considerations in establishing the data-informed district. School Effectiveness and School Improvement, 23(2), 159178. Wayman, J. C., Midgley, S., & Stringfield, S. (2006, April). Leadership for data-based decision making: Collaborative educator teams. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA. Wayman, J. C., Snodgrass Rangel, V. W., Jimerson, J. B., & Cho, V. (2010). Improving data use in Gibson School District: Becoming a data-informed district. University of Texas, Austin. Wayman, J. C., & Stringfield, S. (2006). Technology-supported involvement of entire faculties in examination of student data for instructional improvement. American Journal of Education, 112, 549571. Wei, R. C., Darling-Hammond, L., Andree, A., Richardson, N., & Orphanos, S. (2009). Professional learning in the learning profession: A status report on teacher development in the United States and abroad. Dallas, TX: National Staff Development Council. Yates, S. M. (2007). Teachers perceptions of their professional learning activities. International Education Journal, 8(2), 213221. Young, V. M. (2006) Teachers' use of data: Loose coupling, agenda setting, and team norms. American Journal of Education, 112, 521548.
|
|
|
|
   |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||
![[39_17855.htm_g/00002.jpg]](/Html/39_17855.htm_g/00002.jpg)
![[39_17855.htm_g/00004.jpg]](/Html/39_17855.htm_g/00004.jpg)
![[39_17855.htm_g/00006.jpg]](/Html/39_17855.htm_g/00006.jpg)
Executive Summary
Full Text in PDF Format
Email this article
Print this article
Post a Comment
