New Directions in Education Policy Implementation: Confronting Complexity

“A theory should be made as simple as possible, but not simpler.” Often attributed to Albert Einstein (Calaprice 2000, p. 314), this quip would serve as a suitable epigraph for this volume tracing the frontiers of policy implementation research in education. Rather than shy away from the complexities of the American education system, says editor Meredith Honig, scholars should confront them head on. There are no simple answers to the question of “what works.”

This advice runs counter to prevailing currents shaping contemporary education research. With accountability systems and changes in the economy increasing the pressures on public school systems to improve student outcomes, demand has grown for unambiguous, actionable information about the effectiveness of alternative policies, programs, and reform strategies. The “What Works Clearinghouse,” launched with fanfare in 2002 by the federal government’s Institute for Education Sciences, is intended to disseminate exactly this type of information to educators, policymakers, and parents (http://www.whatworks.ed.gov/).

The problem, as Honig suggests in her insightful introduction, is that no single educational program can be expected to be successful regardless of the context in which it is implemented.   “Implementability” and “success” are “not inherent properties of particular policies” but rather “the product of interactions between policies, places, and people” (p. 3). Researchers therefore need to go beyond the question of what works to explore “what is implementable and what works for whom, where, when, and why” (p. 2). The essays that follow, each of which summarizes the findings of research published elsewhere, intend to illustrate the utility of various theoretical perspectives for answering these questions.  

The first three chapters tackle perhaps the thorniest problem in education policy implementation: changing what goes on behind closed classroom doors. Each concludes that understanding the effects of policy on teachers’ classroom practice requires shifting attention from individual teachers to groups of which they are a part. Cynthia Coburn and Mary Kay Stein, for example, identify distinct “communities of practice” within a school that mediate teachers’ responses to instructional mandates. James Spillane and colleagues elaborate on previous work demonstrating that the non-implementation of instructional policies often stems not from active resistance on the part of teachers but from a failure to understand policy demands. Cognition, they emphasize here, is an inherently social process shaped by interactions between school-level actors and their environment. Finally, Heather Hill argues that policymakers and teachers constitute separate “discourse communities,” and that their divergent understandings and usages of common terms can cause implementation to deviate from policy design.

Elsewhere in the volume, Betty Malen and Amanda Datnow offer complementary analyses of whole school reform policies. Malen argues persuasively that conflicts over implementation are an extension of the policymaking process in which competing bureaucratic actors and interest groups attempt to dilute, appropriate, amplify, or nullify the goals established by political authorities. She uses this perspective to explain the failure of an urban superintendent’s unilateral attempt to reconstitute several low-performing schools due to opposition from teachers and parents who he had effectively excluded from the decision-making process. Datnow, in turn, shows how the federal Comprehensive School Reform Program, though on paper a top-down Congressional mandate to be carried out by state education agencies, has in practice been “co-constructed” by actors at multiple levels of government through a process of mutual adaptation.

Two more chapters examine the growing role of outside organizations in the education reform process, a phenomenon Honig refers to as “collaborative education policy.” Honig describes a dynamic process of organizational learning within the Oakland Unified School District (CA) as it pursued a series of initiatives intended to facilitate school-community partnerships. In particular, district officials were forced to move beyond their traditional regulatory relationships with school leaders into a more supportive role. Mark Smylie and Andrea Evans discuss the role of social capital in policy implementation, arguing that strong social relationships within schools can both facilitate the faithful implementation of new policies and, if not aligned with underlying policy goals, undermine reform efforts altogether. Their research on the Chicago Annenberg Challenge, which provided funds to schools to work with external partners from other schools to promote school-level reform, illustrates both dynamics.

Michael Dumas and Jean Anyon broaden the book’s scope of vision still farther, arguing that New Jersey’s half-hearted response to the 1990 ruling in the Abbott v. Burke school finance adequacy case reveals deep connections between the education system and the political and economic context in which it operates. There is no doubt that housing, employment, health care, and tax policies all have important implications for the prospects of education reform. Dumas and Anyon draw on an eclectic mix of critical social theories to argue that the state’s failure to enhance educational opportunities for urban, predominantly minority students reflects the unequal distribution of power across lines of race and class.

The chapter by economists Susannah Loeb and Patrick McEwan stands out in the volume for its willingness to make simplifying assumptions. The economic approach to policy implementation is distinguished above all by its focus on individuals, who are assumed to make rational decisions based on their preferences subject to knowledge and resource constraints. Loeb and McEwan review the ever-growing body of research by economists on issues of teacher quality, test-based accountability, and school choice. Their analysis demonstrates how careful attention to the incentives facing individuals charged with implementing a given policy change can yield helpful predictions about its consequences. As they point out, economists have also been at the forefront in the development of rigorous empirical methods to test those predictions and inform the construction of new theories.

Milbrey McLaughlin’s concluding chapter draws lessons from the current generation of implementation research and proposes an agenda for the one to follow. She argues that education scholars should continue to expand the boundaries of their work to include new organizations such as community coalitions and advocacy organizations that work in and alongside schools to influence student learning. Rather than cataloguing the myriad ways in which policies as implemented fail to reflect the intent of those who crafted them, they should focus on how this entire system of organizations responds to the sometimes competing demands placed upon it by policy and the broader environment. In this way, researchers can heed Honig’s plea to delve deeper into complexity.

To be sure, this is good advice. Education policy implementation is a messy process, and the research compiled here contains many valuable insights into the factors shaping how it unfolds. These insights will be especially appreciated by graduate students looking for new theories to develop and apply in their own work.

Yet this approach to education research has its limitations, which are also evident throughout the volume. Virtually all of the studies described are based on small handful of cases, but they provide scant information as to whether they are representative of situations elsewhere. And little attention is paid to the effects of policies on student outcomes, perhaps because it would be hard to reach definitive conclusions with this case-study approach.

The authors are well aware of these issues, often noting that their work is intended to inform the development of theories to be tested more rigorously in the future. This aspiration, however, will hardly be satisfying to policymakers concerned with raising student achievement in the short run. For them, evidence on the average impact of alternative interventions based on experimental and quasi-experimental research designs may be more useful, even if it does not fully account for or anticipate the myriad variations in implementation that occur locally. The What Works Clearinghouse, despite its shortcomings, may have a role to play after all.

References:

Calaprice A., ed. (2000). The expanded quotable Einstein. Princeton: Princeton University Press.