by Barry J. Fishman, William R. Penuel, Anna-Ruth Allen, Britte Haugan Cheng & Nora SabelliThis chapter presents an introduction to design-based implementation research (DBIR). We describe the need for DBIR as a research approach that challenges educational researchers and practitioners to transcend traditional research/practice barriers to facilitate the design of educational interventions that are effective, sustainable, and scalable. We examine antecedents to DBIR, including evaluation research, community-based participatory research, design-based research, and implementation research. The four core principles of DBIR are explained: (1) a focus on persistent problems of practice from multiple stakeholders’ perspectives; (2) a commitment to iterative, collaborative design; (3) a concern with developing theory and knowledge related to both classroom learning and implementation through systematic inquiry; and (4) a concern with developing capacity for sustaining change in systems. We close with an overview of the chapters contained in this NSSE Yearbook on DBIR and explain how each chapter contributes to the overall development of the DBIR approach.
by Jennifer Lin Russell, Kara Jackson, Andrew E. Krumm & Kenneth A. FrankThis chapter reviews four projects that reflect the principles of design-based implementation research (DBIR) in an effort to highlight a range of relevant theoretical and methodological perspectives and tools that can inform future work associated with DBIR.
by Milbrey Wallin McLaughlin & Rebecca LondonA societal sector perspective looks to a broad array of actors and agencies responsible for creating the community contexts that affect youth learning and development. We demonstrate the efficacy of this perspective by describing the Youth Data Archive, which allows community partners to define issues affecting youth that transcend specific institutional responsibilities and to ask and answer questions using combined administrative records in an effort identify opportunities for joint action.
by Ben Kirshner & Joseph PolmanApplied researchers, whether working with the framework of design-based research or intervention science, face a similar implementation challenge: practitioners who enact their programs typically do so in varied, context-specific ways. Although this variability is often seen as a problem for those who privilege fidelity and standardization, we argue for the advantages of researcher-practitioner collaborations that encourage local adaptation and ingenuity. We develop this argument for adaptive interventions by discussing two design-based research projects, Critical Civic Inquiry (CCI) and Science Literacy through Science Journalism (SciJourn), which create opportunities for youth to develop civics and science literacy respectively. CCI and SciJourn aim to build curricula that will travel to new schools and districts, but not through standardization. This is a delicate combination: the program must be flexible enough to enable productive adaptation, without being so protean that practitioners’ implementations lack substantive commonalities. We present two cases that show how project designers have sought to distinguish between invariant principles that define the intervention and heterogeneous practices that vary across sites. The cases also show how the model has improved when teachers can adapt it to their institutional context and when teachers and researchers establish social norms that encourage dialogic interactions.
by William R. Penuel, Cynthia E. Coburn & Dan GallagherThis chapter focuses on how researchers and practitioners negotiate the focus of their joint work within design-based implementation research (DBIR). Studying and facilitating successful negotiation of the problems that become the focus of work and the search for solutions is important for developing DBIR, because of its commitment to focusing on persistent problems of practice from multiple stakeholders’ perspectives. Case studies of two different research–practice partnerships provide a context for exploring two different perspectives on negotiation. In one case study, the notion of partnerships as forms of cultural exchange across institutional boundaries that differ with respect to goals, norms, and practices is used to analyze a design partnership focused on repurposing curriculum units in elementary science. In the second case study, the concept of framing as developed in social movement theory is used to illuminate issues of status and authority within a partnership between a district and researchers. The chapter concludes by describing the contributions of each perspective to an understanding of how teams jointly negotiate the focus of their work and by providing some recommendations for how teams can do so successfully.
by Meredith I. HonigThis chapter argues for the importance of design-based leadership research (DBLR) for advancing the research and practice of educational leadership, with a focus on school district central offices. DBLR, like other design-based research, calls on researchers to develop designs for practice. Unlike other such research in education that calls for designs for classrooms, DBLR focuses on designs for leaders. Researchers working in this mode develop designs for leadership practice that reflect the latest knowledge about how leaders matter for improved student results; they work alongside leaders to use that knowledge to design and engage in new forms of their own practice consistent with the knowledge and appropriate to their settings. Participants study the process to feed new knowledge into the partnership sites and the field. This chapter elaborates how such research differs from traditional scholarship on district central offices and forms of action research. Challenges to conducting DBLR include focusing practitioners on central offices (especially in tough budget times), capturing central office practice in DBLR knowledge-building activities, and growing and sustaining the work. Early experience illuminates how to address those challenges and advance DBLR partnerships that promise to significantly strengthen leadership practice in support of improved results for all students.
by Hilda Borko & Janette K. KlingnerTo meet the growing demand for teacher learning opportunities, the educational community must create scalable professional development models and study their effectiveness. In this chapter, we argue that design-based implementation research (DBIR) is ideally suited to these efforts, and we use two research projects in which we are currently involved as illustrative cases: CSR Colorado and Implementing the Problem-Solving Cycle (iPSC). The core of CSR Colorado is Collaborative Strategic Reading, an instructional approach designed to enhance reading comprehension in content classes. The focus of iPSC is the Problem-Solving Cycle, a mathematics professional development (PD) program designed to help teachers improve their instruction through closely examining mathematics problems, student thinking, and pedagogical practices. Each project works with a school district to bring a PD model to scale, and both projects are studying the structures and resources needed to build the district’s capacity to sustain the model beyond the duration of the research. The chapter describes each project and discusses the successes and challenges we experienced as we collaborated with the districts and schools to carry them out. By highlighting two very different projects we show how, through different means, it is possible to achieve the same ultimate end of a scaled-up program for improving instructional practices.
by Angela Haydel Debarger, Jeffrey Choppin, Yves Beauvineau & Savitha MoorthyProductive adaptations at the classroom level are evidence-based curriculum adaptations that are responsive to the demands of a particular classroom context and still consistent with the core design principles and intentions of a curriculum intervention. The model of design-based implementation research (DBIR) offers insights into complexities and challenges of enacting productive curriculum adaptations. We draw from empirical research in mathematics and science classrooms to illustrate criteria for productive adaptations. From these examples, we identify resources needed to encourage and sustain practices to promote productive adaptations in classrooms.
by Paul Cobb, Kara Jackson, Thomas M. Smith, Michael Sorum & Erin HenrickThis chapter describes a partnership with four urban districts that aimed to develop an empirically grounded theory of action for improving the quality of mathematics instruction at scale. Each year, we conducted a data collection, analysis, and feedback cycle in each district that involved documenting the district’s improvement strategies, collecting and analyzing data to assess how these strategies were being implemented, reporting the findings to the district, and making recommendations about how the strategies might be revised. We distinguish between two distinct levels
by Barbara Means & Christopher J. HarrisEducational interventions typically are complex combinations of human actions, organizational supports, and instructional resources that play out differently in different contexts and with different kinds of students. The complexity and variability of outcomes undermines the notion that interventions either “work” or “don’t work.” Under the design-based implementation research (DBIR) model, the implementation of an intervention in particular settings is itself an object of research and a critical part of understanding how to scale an intervention without diluting its effectiveness. In this chapter, we compare the approach to evidence implicit in the defining features of DBIR to the prevailing evidence standards for educational research promoted by national policy. Our aim is to provide a frame for knowledge building within DBIR that draws from the strengths of both design-based research methods and research designs that permit causal inference about program impacts. Moreover, we endeavor to show how DBIR challenges current thinking about what counts as credible research. We conclude by considering the ways in which DBIR is a departure from much educational research in terms of what it means to conduct research that is useful and usable in education settings.
by Jonathan SupovitzDesign-based implementation research offers the opportunity to rethink the relationships between intervention, research, and situation to better attune research and evaluation to the program development process. Using a heuristic called the intervention development curve, I describe the rough trajectory that programs typically follow as they evolve, and argue that research design considerations and methodological choices are best made in consideration of where interventions are along this curve. Further, I contend that, as programs develop, situational influences play a major role in their evolution and consequently require increased attention to design and methodological considerations. By viewing research as an integral part of a program’s development, by making design and methodological choices in consideration of where programs are in their development, and by considering that the situation in which programs evolve as a potential source of change in the nature of the program itself, we alter fundamental perspectives on how research can best contribute to the steady work of building robust programs for educational improvement.
by Suzanne Donovan, Catherine Snow & Philip DaroEducation researchers are increasingly working in practice-based partnerships in order to direct their research efforts toward important problems of practice. We argue for the creation of an infrastructure to support routine and sustained interaction among researchers, practitioners, and designers in order to make partnership efforts more efficient and effective over time. We describe SERP’s efforts to initiate such an infrastructure through the creation and operation of “field sites”—long-term research, development, and implementation partnerships with school districts. We describe principles for field site operation and for product design that are emerging through our “learning by doing” approach. And we argue that the field site partnerships have produced work that is fundamentally different in character and content because of the SERP rules of engagement. SERP work is also more coherent because of its interdisciplinary character, and the sophistication of the partnership’s work grows over time as related lines of work become integrated and as those engaged in sustained collaboration learn from each other.
by Jimmy Scherrer, Nancy Israel & Lauren B. ResnickIn this chapter, we describe the evolution of an intermediary organization, the Institute for Learning (IFL) at the University of Pittsburgh, devoted to improving teaching and learning in some of America’s largest and most challenged school districts. Over the nearly 20 years of IFL’s life, we have learned that design-based researchers have to move beyond classrooms and focus on entire school systems if they hope their innovations will scale and sustain. Specifically, we believe the best way to develop capacity for sustaining innovation is through the careful design of practices intended to provide an ongoing forum for mutual engagement among actors from various levels of the education system. These practices, we have found, provide a productive way for actors from different levels of the system to understand and support the local activity of other levels.
by Jonathan R. Dolle, Louis M. Gomez, Jennifer Lin Russell & Anthony S. BrykThis chapter is a case study of the Carnegie Foundation for the Advancement of Teaching’s Pathways™ program. The goal of the Statway®™ and Quantway®™ pathways is to improve the success rate of community college students who place into developmental mathematics. What makes these programs unique is their strategy of building a particular kind of professional network, what Carnegie refers to as a Networked Improvement Community (NIC), to organize and lead an array of continuous improvement processes. NICs are a social mechanism through which the collaborative designs and practical theories produced by design-based implementation research (DBIR) can become live resources for the improvement of systems. NICs are comprised of highly structured groups of education professionals, working in collaboration with designers and researchers, to address a practical problem. Driver diagrams are introduced as a tool for organizing the improvement work of NICs. After briefly describing several drivers behind the Pathways program, the chapter details the main elements of the network organization driver as a distinct approach to building communities aimed at improving education.
by Nora Sabelli & Chris DedeThis chapter discusses frameworks and conceptual lenses that help orient design-based implementation research (DBIR) work to the types of infrastructure required for success, while contributing to theories about the processes of educational improvement. Such infrastructures can be conceived as a framework: a set of interconnected elements that facilitate the integrated development of an initiative, provide a continuing narrative for this development, create shared responsibility for its implementation, and facilitate its sustainability. The chapter posits that several types of infrastructure are necessary for successful, sustainable DBIR, including conceptual frameworks that attend to issues of scale, human capacity, technical support, policies, organizational learning, and long-term funding. In addition, the chapter discusses the roles that feedback loops, intermediary organizations, and multiple timescales play in systemic educational improvement, and describes how attention to these different infrastructural needs can lead to sustained improvements and support their evolution over time. The chapter also comments on theoretical and practical advances in social policy that often are not integrated into learning sciences research but are crucial for evolutionary educational improvement, including using organizational learning as a measure of progress, attending to social feedback loops and networks inside and outside of schooling, and utilizing the tools of policy informatics.