This article focuses on how teachers make sense of classroom-embedded data in professional learning communities in ways that lead to improvement in mathematics instruction. The case study illustrates the developmental nature of one teacher’s growth and the important roles of dissonance, collegial discussion, and productive dissonance in that process.
This article details a mixed methods project focused on identifying the combination of programs, practices, processes, and policies that make some high schools in large urban districts particularly effective with low income students, minority students, and English language learners.
This article examines the challenges and limitations of a research alliance—between a university research center, a high school, and one of its feeder K–8 school districts—focused on improving school climate.
This article provides a review of literature on teachers’ use of assessment data to inform instruction. The article reviews research on the types assessment data teachers use to inform instruction, how teachers analyze data, and how their instruction is impacted. Although teachers are often asked to analyze data in a consistent way, agendas for data use, the nature of the assessments, and teacher beliefs all come into play, leading to variability in how they use data.
Drawing on data collected in six middle schools, this article finds that coaches and professional learning communities (PLCs) played an important role in mediating teachers’ responses to data. We find that dialogue and the dynamic relationship between two types of expertise may help explain the ways in which PLCs and coaches facilitated deeper-level changes in pedagogy, and that school leadership and district-level context shaped the possibility for such changes.
This study focuses on the factors influencing a professional development intervention for data-based decision making: the data team procedure. In this article, authors discuss how data characteristics (e.g., access to data), school organizational characteristics (e.g., shared goal), and individual/team characteristics (e.g., pedagogical content knowledge) influence the use of data in data teams, and how these factors are interrelated.
This essay describes the habits of mind that underlie data literacy courses offered by the Data Wise Project at the Harvard Graduate School of Education, and how those preparing educators can incorporate these habits into instructional design. The habits include: shared commitment to action, assessment, and adjustment; intentional collaboration; and relentless focus on evidence.
This article proposes a conceptual framework for school and district data use practices based on an analysis of current research. The author outlines considerations for professional learning for each of the five framework elements and closes with a set of questions that may help to highlight future research needs in the area of school-level data use.
In this article, we draw from document analysis as well as interview and focus group data in three school districts to examine teacher needs specific to building data use capacity. Informed by work on knowledge-based organizational learning, we conclude that teacher professional learning needs specific to data use were met only partially and that existing systems overemphasized data systems access and operations at the expense of other skills needed to turn data into action.
This article presents a conceptual framework for a new construct, data literacy for teachers, laying out the knowledge and skills teachers need to use data effectively and responsibly. The framework emerges from a domain analysis, but the complex construct requires additional discussions to refine and reorganize it.
This article summarizes some significant insights of articles in this issue from the perspective of public policy, emphasizing their potential resonance in today's policy environment in using data for program improvement as well as accountability purposes.
Introduction to the special issue on data-driven decision making and the components needed to enculturate data use in education. The article briefly examines the landscape of existing literature and positions the papers for the special issue.
The chapter examines youth participation within three intergenerational collectives using participatory action research (PAR) to address educational policies youth viewed as counterproductive to their education. Outlining the complexity of youth voice, the multiple vehicles within the arts through which youth voice is expressed, and the different ways in which youth voice is received by educators and policy makers, the chapter underscores the promise of youth involvement in developing, assessing, and fundamentally altering educational policy.
This concluding chapter examines how this book on student voice intersects with previous research about policy, especially policy implementation and sustainability. Mapping onto the themes of this volume, Discovering, Developing, and Demonstrating the power of student voice, I focus on three issues—legitimizing the role of young people in the policy and reform process; preparing adults to work with young people; and sustaining ongoing student voice work.
Commonly applied criteria for generalizing that focus on experimental design or representativeness of samples of the population of units neglect considering the diversity in the targeted populations of interest and uses of knowledge generated from the generalization. This paper (a) articulates the structure and discusses limitations of different forms of generalizations across the spectrum of quantitative and qualitative research; and it (b) argues for an overarching framework that includes population heterogeneity and uses of knowledge claims as part of the rationale for generalizations from educational research.
This article explores how educational researchers can use meta-analysis to “power-up” the findings of their existing, small-scale qualitative research studies. By triangulating data from three independently conducted studies of academically at-risk college students, this research contests “time-to-degree” as a valid criterion for measuring academic success in college.
This chapter presents an introduction to design-based implementation research (DBIR). We describe the need for DBIR as a research approach that challenges educational researchers and practitioners to transcend traditional research/practice barriers to facilitate the design of educational interventions that are effective, sustainable, and scalable. We examine antecedents to DBIR, including evaluation research, community-based participatory research, design-based research, and implementation research. The four core principles of DBIR are explained: (1) a focus on persistent problems of practice from multiple stakeholders’ perspectives; (2) a commitment to iterative, collaborative design; (3) a concern with developing theory and knowledge related to both classroom learning and implementation through systematic inquiry; and (4) a concern with developing capacity for sustaining change in systems. We close with an overview of the chapters contained in this NSSE Yearbook on DBIR and explain how each chapter contributes to the overall development of the DBIR approach.
This chapter reviews four projects that reflect the principles of design-based implementation research (DBIR) in an effort to highlight a range of relevant theoretical and methodological perspectives and tools that can inform future work associated with DBIR.
Applied researchers, whether working with the framework of design-based research or intervention science, face a similar implementation challenge: practitioners who enact their programs typically do so in varied, context-specific ways. Although this variability is often seen as a problem for those who privilege fidelity and standardization, we argue for the advantages of researcher-practitioner collaborations that encourage local adaptation and ingenuity. We develop this argument for adaptive interventions by discussing two design-based research projects, Critical Civic Inquiry (CCI) and Science Literacy through Science Journalism (SciJourn), which create opportunities for youth to develop civics and science literacy respectively. CCI and SciJourn aim to build curricula that will travel to new schools and districts, but not through standardization. This is a delicate combination: the program must be flexible enough to enable productive adaptation, without being so protean that practitioners’ implementations lack substantive commonalities. We present two cases that show how project designers have sought to distinguish between invariant principles that define the intervention and heterogeneous practices that vary across sites. The cases also show how the model has improved when teachers can adapt it to their institutional context and when teachers and researchers establish social norms that encourage dialogic interactions.
This chapter focuses on how researchers and practitioners negotiate the focus of their joint work within design-based implementation research (DBIR). Studying and facilitating successful negotiation of the problems that become the focus of work and the search for solutions is important for developing DBIR, because of its commitment to focusing on persistent problems of practice from multiple stakeholders’ perspectives. Case studies of two different research–practice partnerships provide a context for exploring two different perspectives on negotiation. In one case study, the notion of partnerships as forms of cultural exchange across institutional boundaries that differ with respect to goals, norms, and practices is used to analyze a design partnership focused on repurposing curriculum units in elementary science. In the second case study, the concept of framing as developed in social movement theory is used to illuminate issues of status and authority within a partnership between a district and researchers. The chapter concludes by describing the contributions of each perspective to an understanding of how teams jointly negotiate the focus of their work and by providing some recommendations for how teams can do so successfully.
This chapter describes a partnership with four urban districts that aimed to develop an empirically grounded theory of action for improving the quality of mathematics instruction at scale. Each year, we conducted a data collection, analysis, and feedback cycle in each district that involved documenting the district’s improvement strategies, collecting and analyzing data to assess how these strategies were being implemented, reporting the findings to the district, and making recommendations about how the strategies might be revised. We distinguish between two distinct levels
Educational interventions typically are complex combinations of human actions, organizational supports, and instructional resources that play out differently in different contexts and with different kinds of students. The complexity and variability of outcomes undermines the notion that interventions either “work” or “don’t work.” Under the design-based implementation research (DBIR) model, the implementation of an intervention in particular settings is itself an object of research and a critical part of understanding how to scale an intervention without diluting its effectiveness. In this chapter, we compare the approach to evidence implicit in the defining features of DBIR to the prevailing evidence standards for educational research promoted by national policy. Our aim is to provide a frame for knowledge building within DBIR that draws from the strengths of both design-based research methods and research designs that permit causal inference about program impacts. Moreover, we endeavor to show how DBIR challenges current thinking about what counts as credible research. We conclude by considering the ways in which DBIR is a departure from much educational research in terms of what it means to conduct research that is useful and usable in education settings.
Design-based implementation research offers the opportunity to rethink the relationships between intervention, research, and situation to better attune research and evaluation to the program development process. Using a heuristic called the intervention development curve, I describe the rough trajectory that programs typically follow as they evolve, and argue that research design considerations and methodological choices are best made in consideration of where interventions are along this curve. Further, I contend that, as programs develop, situational influences play a major role in their evolution and consequently require increased attention to design and methodological considerations. By viewing research as an integral part of a program’s development, by making design and methodological choices in consideration of where programs are in their development, and by considering that the situation in which programs evolve as a potential source of change in the nature of the program itself, we alter fundamental perspectives on how research can best contribute to the steady work of building robust programs for educational improvement.
Education researchers are increasingly working in practice-based partnerships in order to direct their research efforts toward important problems of practice. We argue for the creation of an infrastructure to support routine and sustained interaction among researchers, practitioners, and designers in order to make partnership efforts more efficient and effective over time. We describe SERP’s efforts to initiate such an infrastructure through the creation and operation of “field sites”—long-term research, development, and implementation partnerships with school districts. We describe principles for field site operation and for product design that are emerging through our “learning by doing” approach. And we argue that the field site partnerships have produced work that is fundamentally different in character and content because of the SERP rules of engagement. SERP work is also more coherent because of its interdisciplinary character, and the sophistication of the partnership’s work grows over time as related lines of work become integrated and as those engaged in sustained collaboration learn from each other.
In this chapter, we describe the evolution of an intermediary organization, the Institute for Learning (IFL) at the University of Pittsburgh, devoted to improving teaching and learning in some of America’s largest and most challenged school districts. Over the nearly 20 years of IFL’s life, we have learned that design-based researchers have to move beyond classrooms and focus on entire school systems if they hope their innovations will scale and sustain. Specifically, we believe the best way to develop capacity for sustaining innovation is through the careful design of practices intended to provide an ongoing forum for mutual engagement among actors from various levels of the education system. These practices, we have found, provide a productive way for actors from different levels of the system to understand and support the local activity of other levels.
This chapter discusses frameworks and conceptual lenses that help orient design-based implementation research (DBIR) work to the types of infrastructure required for success, while contributing to theories about the processes of educational improvement. Such infrastructures can be conceived as a framework: a set of interconnected elements that facilitate the integrated development of an initiative, provide a continuing narrative for this development, create shared responsibility for its implementation, and facilitate its sustainability. The chapter posits that several types of infrastructure are necessary for successful, sustainable DBIR, including conceptual frameworks that attend to issues of scale, human capacity, technical support, policies, organizational learning, and long-term funding. In addition, the chapter discusses the roles that feedback loops, intermediary organizations, and multiple timescales play in systemic educational improvement, and describes how attention to these different infrastructural needs can lead to sustained improvements and support their evolution over time. The chapter also comments on theoretical and practical advances in social policy that often are not integrated into learning sciences research but are crucial for evolutionary educational improvement, including using organizational learning as a measure of progress, attending to social feedback loops and networks inside and outside of schooling, and utilizing the tools of policy informatics.
The research project involved four campuses that chose to integrate the arts into tested subject areas. Digital storytelling technology was combined with the narrative inquiry research method to produce ‘digital narrative inquiries’—16-20 minute multimedia representations of each school’s experience of its change initiative. Video clips excerpted from these larger productions were then used to discuss the affordances and constraints of using the ‘digital narrative inquiry’ representational form to share research findings arising from innovative educational practices.
This investigation derives four quantifiable measures of school quality and subsequently tests them in an empirical model that relates school-level inputs to school-level outputs. By conducting this analysis on a longitudinal, school-level data set of all 175 elementary schools within the School District of Philadelphia, this research provides new evidence that a range of school-level inputs has a significant relationship with school quality.
This article introduces two-dimensional video-based animations of fictional classroom interactions as a new kind of video image for representing classrooms. The authors suggest that this addition to the existing repertoire for representing practice can support conversations about tactical and strategic dimensions of the work of teaching in ways that overcome some of the limitations of usual video footage.
I was invited to give the annual lecture that honors Lawrence Cremin, the historian of American education at Teachers College, Columbia University. To pay tribute to the way in which Cremin used an academic discipline to bring rigor and depth to educational research, I described the way in which I used an academic discipline—linguistics and its varied tools of discourse analysis—to conduct research at the College.