The Politics of Data Use
by Jeffrey R. Henig - 2012
Background/Context: Many contemporary education reformers present themselves as reformers who, armed with data and evidence, are locked in battle against politics, the weapon of choice for entrenched defenders of the status quo. Although studies of school reform increasingly recognize that politics is inevitably intertwined with reform efforts, the conceptualization of politics is thin and largely divorced from the broader literature in the discipline of political science.
Purpose/Objective/Research Question/Focus of Study: This study reviews political science theories and findings to inform our understanding of how politics affects efforts in order to increase data usage in education policy and school reform and to lay a foundation for future research on the issue.
Research Design: This is a review and analytical synthesis of literature that for the most part has not looked at the education policy arena in order to derive insights relevant to contemporary efforts to infuse education policy with greater attention to data.
Conclusions/Recommendations: Rather than block the door to politics, those who hope to promote informed policy making might consider ways to use politics to protect and defend high-quality data and their informed application.
Evaluators often long for a world where rationality holds sway and decisions are made on the basis of evidence, not politics. (Weiss, 1998, p. 315)
Those who would like to increase the informed use of good-quality data in education policy and practice typically recognize that politics is an important force to reckon with. Politics is about ideology, which leads individuals subconsciously to filter out information that challenges their assumptions about how public education is and should be structured and delivered. Politics is about partisanship, which leads elected leaders to promote agendas based on what they think will get them reelected instead of what is accurate and what is right. Politics is about systemic power, which leads competing interest groups to manipulate data to make them appear to align with the programs they favor and to discredit opponents and their ideas. And politics is about personal power, which leads superintendents, principals, and teachers to hoard, dismiss, and manipulate data to better get their way. Politics, in a nutshell, is the enemy, to be kept at bay.
The argument that politics is the enemy to be kept at bay has been influential in shaping Americas thinking and its actions, both historically and on the contemporary scene. It informed and justified structural changes successfully promoted by the progressive reformers of the early 20th century. There is no Democratic or Republican way to pave a street was a slogan of the time, with the implication that there was instead an objectively correct way, best determined via technical and scientific expertise. Policies like teacher certification, civil service protections, and the formal assignment of education policy making to school boards independent from municipal governments and the political machines that often controlled them were portrayed as a way to empower the experts, who would both know and respect objective data and explicitly buffer them from political interference, patronage politics, and faddish and emotion-driven popular whims.
Many contemporary education reformers, in their enthusiasm for evidence-based policy and data-driven practice, are heirs to this progressive vision. Like their predecessors, they present themselves as reformers who, armed with data and evidence, are locked in battle against politics, the weapon of choice for entrenched defenders of the status quo. Joel Klein, reflecting on his tenure as New York City chancellor, described teacher union resistance to his efforts to use teacher value-added measures of student progress in evaluating teachers as one episode that still shocks me. Seemingly overnight, a budget amendment barring the use of test data in tenure decisions materialized in the heavily Democratic State Assembly (Klein, 2011). Terry Moe referred to this as the battle against data, suggesting that although the tide slowly is turning, unions resistance has for decades prevented the nation from simply putting objective information to reasonable use in trying to improve the public schools (Moe, 2011, pp. 319320). Whereas the historical progressives portrayed professionalism and bureaucracy as the solution to the problem of politics, the contemporary progressives see professional educators and district administrators as the problem. In place of education professionals and bureaucracy, they place their bets for better data use on strong executive leadership, strict accountability regimes, market forces, and technology (Moe, 2011).
There are reasons, though, to be wary of a keep politics at bay perspective. Historians came to challenge the early progressives claims to be apolitical apostles of scientific truths, suggesting that this mask of objectivity covered a set of class-based political interests (Bridges & Kronick, 1999; S. P. Hayes, 1964). Among contemporary analysts who study the utilization of data and research, similarly, there is growing skepticism about the existence of a sharp distinction between data and evidence, wearing the white hats on one side, and partisanship and politics wearing black hats on the other. Although evidence based decision-making is sometimes framed as an antidote for ideology-driven decision-making, Coburn, Honig, and Stein (in press) observed, people make decisions precisely by drawing on what might be considered ideologyincluding their prior knowledgeas a fundamental part of the decision-making process.
This revisionist view of the intersection between politics and data use shares the progressives view that politics is potent but frames it less as enemy to be vanquished than as one of a series of challenges to be acknowledged and taken into account. Researchers working in this vein find that the path from data to practice in general is complex, mediated by various factors, including characteristics of the data, the organizational culture and capacity, and the fiscal and political context (Coburn et al., in press; Coburn, Toure, & Yamashita, 2009; Honig, 2008; Honig & Coburn, 2007; Marsh, Pane, & Hamilton, 2006; Moe, 2011; Spillane, 2002). When data are not used, the explanation does not necessarily rest on outright resistance. Willful refusal to acknowledge these complicationsand the associated insistence that all roadblocks and deviations to data use are corruptions attributable to political maneuvering by defenders of the status quomay itself be best understood as a political phenomenon. Ravitch (2010), for one, characterized the champions of test-based accountability regimes as a Billionaire Boys Club comprising corporate and philanthropic elites unable or unwilling to acknowledge that their vision of value-free data and analysis is conveniently aligned with their own values and economic interests and is itself political in both its implications and many of its pursuits.
Those who see politics as a threat to be avoided and those who see it as a reality to be understood agree on one point: Politics is important. Despite this, their framing of the political dimension is thin. As Coburn and Turner (2010) noted, In spite of the fact that data use and politics are often intertwined, this area has largely been ignored by advocates and scholars of data use at the school and district level (p. 23). Despite offering a more sophisticated understanding of the role that politics unavoidably plays, even the revisionist analyses are largely disconnected from what the discipline of political science might have to offer. There are at least two reasons for this. First, the discipline of political science has been rather diffident about applying its insights to the education sphere; education policy scholars will not find much within the disciplines literature that explicitly connects to the issues with which they are wrestling. Second, the strongest message from the discipline is one that reinforces the notion that politics and data use do not mixat least if by data use, one means their application as part of a pragmatic effort to generate real solutions to real problems.
In this article, I review political science theories and findings that are not explicitly linked to data use in education policy in an effort to extract insights and deepen the base for future research and practice. I organize the presentation to underscore two points. First, the main thrust of the political science literature serves as a warning against idealized visions of pure data being applied in depoliticized arenas. Although generalizations about an entire discipline inevitably are oversimplifications, the center of gravity within the field encourages skepticism about proposals for a rational, comprehensive science of public policy making and regards data and information as sources of power first and foremost. Second, despite the cynicism that this stance might initially induce, the discipline also provides a foundation for reconceptualizing the relationship between data and politics. Rather than block the door to politics, those who hope to promote informed policy making might consider ways to use politics to protect and defend high-quality data and their informed application. I begin, though, with some definitional distinctions useful to the overview and critical to some of the conclusions I will draw.
DATA, DATA SYSTEMS, AND DATA REGIMES
To get leverage on understanding the translation of research into policy, Carol Weiss observed more than 30 years ago, It is essential to understand what using research actually means (Weiss, 1979, p. 426). The definitional confusion is perhaps even more striking today in the context of education policy, in which terms like research utilization, science-based, evidence-based decision-making, and data-based decision-making are employed without agreement on their precise delimitations (Coburn et al., 2009; Hess, 2008; Honig & Coburn, 2007; Mosteller & Boruch, 2002; Towne & Shavelson, 2002; Walters, Lareau, & Ranis, 2009). In keeping with the framing of the Spencer Foundations project on data usage and on the volume in which this article appears, I focus here on data usage, drawing a distinction between dataas collections of descriptive indicators of social conditions, policy processes, and practiceand research, which is a more processed form of data undertaken with the goal of testing for causal relationships or building testable theory. To illuminate and draw on the relevant research in political science, however, it is helpful also to distinguish among data, data systems, and what I will refer to as data regimes.
In the abstract, one can conceive of raw data, but in practice, even relatively unadorned descriptive information is framed by interpretation. The simple decision to collect particular kinds of data rests on judgments about organizational and societal priorities. And even elemental data collection efforts incorporate categorization schemes that are grounded implicitly in sense-making theoretical schema. Core elements of the U.S. Censusbasic counts of the number of persons and households, combined with information about their ages and racemight seem like raw descriptive data, but from its origin through today, the Census has been permeated by political considerations about who should count, how electoral power should be distributed, and what racial and ethic distinctions are legitimate (Prewitt, 2010).
Although what gets collected is not random and likely reflects the interests of more powerful groups and reigning ideologies, it may still be meaningful to think about basic data that are relatively untethered from particular political agendas. Descriptive data that are collected in uncorrupted ways and are generally made available to users of different types possess flexibility akin to that of stem cells in biology. Regardless of origin, they can be put to various and surprising uses, including uses that might end up challenging reigning paradigms and political cliques. From a societal standpoint, there are compelling reasons to protect the integrity of basic numbers, even if the interpretation and uses of those numbers unavoidably will be contested (Prewitt, 2010).
In the contemporary debates about data usage in education, though, the key indicators under discussion typically are not raw and isolated from one another but rather are linked to systems designed with specific instrumental applications in mind. Assessment and accountability systems use data for tracking and evaluating students, teachers, schools, and districts for the boarder purpose of allocating rewards and sanctions or targeting remedies. Although the data incorporated into those systems may have been collected previouslyfor example, most districts collected graduation and standardized test data for years before those became incorporated into accountability systemsone of the important contributions that political science can offer is the insight that data usage may be determined as much by the systems in which the data present themselves to key stakeholders as they are by the data themselves.
In drawing some broad conclusions, I will take this insight further by introducing the concept of data regimes. Political scientists use the concept of regimes to refer to the relatively stable political arrangementscomprising not only the formal institutions and powers of government but also supporting rationales and the informal-yet-patterned interactions among government, citizens, business, unions, and the nonprofit sectorthat shape policy priorities and the prospects for getting things accomplished. Regimes consist of relationships among various stakeholder groups and offices that tend to outlast particular individuals and even administrations (C. N. Stone, 1989). If data systems consist of raw data plus the arrangements for their collection and intended application, data regimes incorporate data systems plus the reigning ideas, values, and political contexts that shape how they actually are put to use. A data regime, in other words, comprises not just a particular set of indicators and the systems in which they might be embedded but also the array of animating ideas (e.g., theories about the relative influence of teachers or of nonschool factors), supporting interest groups (e.g., political and policy entrepreneurs who promote the data systems; for-profit and nonprofit organizations that collect, disseminate, analyze the data), and the governance institutions (e.g., mayoral control, federalism) that enable those to take hold and flourish.
CAVEAT EMPTOR: POLITICAL SCIENCE AS A SOURCE OF USEFUL SKEPTICISM ABOUT DATA USE INITIATIVES
From the standpoint of political science, data in the policy-making world are first and foremost a source of power and influence. Individuals and groups use data to increase the probability that policies will maximize their interests relative to others, in much the same way that they may use campaign contributions, or the capacity to mobilize votes, or tactical expertise. Because this is known to be the casebecause public office holders, interest groups, and even mildly attentive citizens understand that data can and will be manipulateddata are rarely reacted to as if they were raw bits of undigested truth. Just as Chekhov indicated that a gun on the wall in Act I of a play suggests that someone will fire that gun by Act III, the potential that others will use data as a weapon leads many to assume that their opponents will do so and encourages them to act preemptively based on that assumption.
Taken to an extreme, this perspective can seem cynical and unhelpful: If data are nothing more than political weaponry, there is little to say about how different structures and processes for utilizing data might improve the capacity of government to contribute to our collective well-being. In a more moderate dose, however, this perspective offers a valuable corrective to some of the more idealized and naïve portrayals of the potential for evidence-based decision making in education, portrayals thatprecisely because they are idealized and naïverisk setting us up for two types of failure. One type of failure stems from continual disappointment. Overselling the prospects for achieving better education through better use of data can lead to frustration when the results come up short, leading some to conclude that public education is hopelessly immune to sensible reform. A second type of failure stems from vulnerability to manipulation. Anticipating that data and their application will stir politically motivated responses can reduce the likelihood that some individuals and groups will co-opt the process in ways that promote their interests at the cost of the broader good.
In this section, I provide an overview of some of the important ideas and supporting evidence that can be found in the political science literature as applied generally to data and knowledge utilization. Reflecting the political science literature itself, these sections for the most part do not speak directly to education, although I will highlight some implications for education as I move along. The primary thrust of this section is to underscore the disciplines message of caveat emptor: the risks of politicization of data and the forms that risk might take. But this review can also be helpful as insight into why well-intentioned efforts to introduce more data- and research-based decision making frequently are met by what seems to be surprising and disproportionate resistance. Resistance, at times, is predicated not on opposition to data per se but on anxieties about how data will be used politically in ways that bolster the power and influence of some actors at the expense of others. Drawing on the distinctions drawn earlier, when it comes to understanding the dynamics around data usage, data systems and data regimes are as important as, or more important than, the data themselves.
LIMITS OF COMPREHENSIVE RATIONALITY
One important line of work in political science emerged as a challenge to idealized notions about the prospects for a rational, comprehensive, nonpartisan, data-based approach to setting public policy. Early progressives claim that government should be guided by rigorous application of scientific principles was picked up and amplified in the second half of the 20th century. Proponents of operations research, systems analysis, strategic planning, cost-benefit analysis, and program planning budgeting were buoyed by growing confidence in economic models and quantitative techniques. The movement reached what might have been its pinnacle, with the broad introduction of Planning-Programming- Budgeting Systems (PPBS), first into the Defense Department under Robert McNamara, and then more broadly infiltrating federal agencies. The vision held that social science could sweep away policies and practices favored simply because they were familiar, routine, or politically protected and replace them with broader, integrated initiatives continuously modified by evidence about effectiveness and efficiency (Aaron, 1978; Dror, 1967; Lasswell, 1970; Nathan, 2000; Novick, 1965).
Lindblom (1959) developed an influential critique of the rational-comprehensive model, which he argued was neither an accurate account of how policy actually is made nor a goal toward which we should aspire. In practice, policy advances in small steps and by trial and error. This is fine and good, he proposed, because we actually cannot collect enough objective data to allow us to make accurate predictions of the consequences of policy interventions. Muddling through by means of incremental steps and a process of trial and error is not only the best we can do; in light of the incomplete and often distorted data we have at our disposal, it would be reckless to undertake sharp changes in policy.
Lindbloms largely theoretical argument gained backing from several strains of research. Studies of public budgeting showed that the best predictors of the level and distribution of government expenditures typically are the level and distribution of spending in the preceding years (M. T. Hayes, 1992; Sharkansky, 1968; Tucker, 1982; Wildavsky, 1984). Critical assessments of PPBS and similarly ambitious efforts to rationalize governmental policy suggested that they both overestimated the analytical power of existing methodologies and underestimated the variety of ways in which politics and preconceptions would steer putatively neutral scientific techniques (Malbin, 1980; Nathan, 2000; Wildavsky, 1966). The limitations of dataeven comprehensive dataas a guide for policy making have been underscored as well by political scientists drawing on research in organizational and social psychology that challenged the notion that more and better data will make individuals or societies better decision makers (Jones, 1995, 1999; Simon, 1985). Cognitive limitations and emotions lead people to filter out information that does not fit with favored perceptual frames. Information is costly, and sometimes the benefits do not outweigh those costs (Downs, 1957), so we rely on various heuristics, habits, and mental shortcuts (Kahneman, Slovic, & Tversky, 1982).
The answer is not simply to pursue more and better data. Even when large quantities of information are available, processing limitations may make data overload a bigger problem than data lack. Hence, too much inquiry, as well as too much information, disables (Lindblom, 1990, p. 63).
DATA AS WEAPONRY IN INTEREST GROUP POLITICS
More threatening than incomplete utilization of data is the possibility that data will be used tactically in ways that exacerbate existing inequalities or that weaken the reins of democratic accountability. Political science most typically conceptualizes the policy process as a zero-sum game in which government action necessarily creates winners and losers, and interest groups jockey for access and influence precisely to increase the probability that they gain at some others expense. Data, in these battles, can be a valuable source of influence and power. Contestants for power are aware of this, moreover, and therefore take positions on policies relating to data production and dissemination at least in part based on their assessments of whether particular indicators or data systems will give them a political edge.
One facet of the data-as-weaponry framing relates to information asymmetries. Information asymmetries exist when some individuals and groups have greater access to information that political competitors lack. Privileged access to data can be an important form of political capital (Austen-Smith, 1997; Hula, 1999). Unequal access to data can exacerbate power inequalities, for example, by helping some actors more readily and reliably spot potential problems (stagnating property values; cancer clusters near waste sites; growing incidence of school absences due to asthma; falling test scores for important subgroups), giving them advantages in mobilizing support for government action or, conversely, launching strategies to preempt the same. According to some analysts, the selective provision of information is the most important resource that interest groups utilize in lobbying for benefits from government (Grossman & Helpman, 2001).
It is not just uneven access to data that can turn them into a political resource; political science reminds us that political stakeholders often use data in deliberate efforts to mislead or divert policy makers or competing interest groups. Research on political socialization, political communications, political uses of rhetoric and symbolism, and the politics of problem definition suggests that elites weave claims based on data into narratives designed to reduce the likelihood that they will be challenged (Bachrach & Baratz, 1963; Cobb & Kuklinski, 1997; Crenson, 1972; Edelman, 2001; Jacobs & Shapiro, 2000; Mutz, Brody, & Sniderman, 1996; Rochefort & Cobb, 1994; D. Stone, 2002).
This is a reminder that data and data systems can develop constituencies comprising individuals and groups that are advantaged by them in some way. Some data constituencies may be good government groups motivated by the sincere belief that better data will lead to better policy. But other constituencies include interest groups that see particular forms of data as serving their narrower tactical needs. Private providers of supplemental education, for example, might want more precise data on student performance and adequate yearly progress (AYP) status in planning how and where to market their services. Nonprofit organizations may want access to data that can buttress their grant proposals. Realtors famously exploit school data as a way to convince families that particular neighborhoods are desirable because of access to good schools. Interest groups support for public data collection may be a case of their wishing to push costs of gathering information they need for business or political advantage onto the public sector so they need not absorb those costs themselves. In addition, interest groups might support data systems because they believe a particular set of indicators is likely to show them in a good light (or distract attention from other indicators that might show them in a bad light). Data lend authority and legitimacy to claims about best practices but can just as likely lend force and cover to parochial claims by interest groups seeking favorable treatment at the expense of others or the public good.
The constellation of supporting interests groups, and the problem definitions and ideologies that help to sustain them, constitutes what I am referring to as a data regime. Precisely because they can be so established a part of the accepted landscape, data regimes represent what Bachrach and Baratz referred to as the second face of power (Bachrach & Morton, 1962). By treating some ideas and policies more favorably than others, they may stifle rival ideas and keep alternative policies off the public agenda.
DATA AND BUREAUCRATIC POWER
Standard notions about democratic accountability assume that executive agencies are the agents that elected leaders employ to implement their policy agendas, but this model breaks down if bureaucracies have their own set of interests and the power to substitute those for the principals to whom they are nominally responsible.
Researchers who study bureaucratic politics identify the ability to selectively collect and selectively disseminate data as two important weapons that bureaucracies use to maximize their budgets and protect their independence from oversight and control (Mitnick, 1975; Moe, 1984; Rourke, 1969). Monitors ultimately are dependent on the agencies for the very information they need to hold them in line: in a sense, the agency both keeps the books and performs the audit, McCubbins et al. observed. If agencies have important private information, not all of which can be obtained by external monitors, they can use this information to hide noncompliance (McCubbins, Noll, & Weingast, 1987, p. 251). Also contributing to bureaucratic power is the reputation for technical expertise that can lead the media, public, and politicians to defer to those in position to wield data with confidence and an air of authority.
Bureaucratic control over access to and interpretation of data has been seen as especially problematic in school districts and schools (McLaughlin, 1987; Spillane, 2002; Weick, 1976). The traditional portrayal of education as a loosely coupled system rested in part on the belief that it was impossible for central authorities to know with any certainty what was going on in classrooms. And the fact that the goals of education were multiple and complex led some to conclude that it was infeasible to shift the focus from compliance on process to compliance on outcomes. The result, in the eyes of some critical observers, was a system free to operate in the interests of its own members, selectively collecting and disseminating data that showed it in a good light.
One way to think about the standards movement in education is as a self-conscious effort by state and national policy makers to create new data and data systems to allow them to exercise better monitoring and more managerial control over those who ultimately implement policies at the district, school, and classroom levels.
The standards movement enabled the switch to an outcomes-oriented compliance regime in three ways. First, it pushed aside the traditional education is too complex to measure concern by elevating math and reading performance to the front of the pack of important values (doing so by posing them as prerequisites to gains on other fronts). Second, it mandated the collection of specific kinds of data that would allow policy makers (and others; see the text that follows) to monitor performance. Third, it created a range of specific sanctions, activated by data, that held agents accountable. Considered as such, the impetus behind the emphasis on increased data usage is seen as part of a battle to replace one data regimewhich both empowered local education bureaucracies and shielded them from close oversightwith a rival regime in which state and national political actors (including governors, state legislatures, Congress, and the White House) and the interest groups that had their ear had more central roles.
Understanding that data act within politics and not simply as a tether on it has implications for contemporary initiatives to promote data use. The overarching risk is that increased emphasis on data will end up augmenting the power and autonomy of an elite group that controls data and has the capacity to manipulate and disseminate them. Despite the intent to use data to better hold the education bureaucracy to account, one real possibility is that data initiatives will evolve in ways that make it harder for elected officials, parents, and voters to assert oversight. No Child Left Behind (NCLB) initially supported rather simplesome said simplisticmeasures for classifying schools by performance. But subsequent efforts to build in student-level value-added schemes and to apply these to teachers as well as schools are pushing the mechanics in much more technically complex directions.
Paradoxically, one result may be the re-empowerment of bureaucrats, within either public bureaucracies or private organizations. Labeling a school as either meeting or failing to meet AYP was straightforward. But with more complex, longitudinal, multivariate modeling coming into play, expert analysts may become the only ones able to understand, and possibly shape, what is really being measured. Although New York City has developed what seems like a straightforward letter-grade system for categorizing school performance, for example, the underlying algorithm on which the grade is based is extremely complicated, and elements and weightings have shifted from year to year, leaving principals, parents, and others dependent on experts to differentiate meaningful year-to-year grade changes from those that might be artifacts (Hemphill & Nauer, 2010).
The accountability movement hopes to use data as a tool for keeping district and schools in line, however, history suggests that bureaucracies are at least as likely to use data to their own advantage as they are to be tamed by it. Progressive reformers efforts to build bureaucracies as politically neutral guardians of data and expertise, for example, simply shifted the playing field on which pluralistic political competition took place. Bureaucracies did not neutralize the political dimension of data; they converted it into political resources of their own, making themselves more valuable as an ally to competing politicians and interest groups and making them less reliable extensions of the policy objectives articulated by legislatures (Lowi, 1967). If the past is prologue, district bureaucracies may find ways to complicate the new data regimes in a manner that once again gives them an effective monopoly on interpretation.
The experts that emerge as the critical guardians and interpreters of data may not end up being employees of the traditional public school system, however. The growing use of private providers and analysts could shift the locus of expertise and data control into testing and publishing companies, charter management organizations, and various service providers operating on a contractual basis. But shifting data-induced power from unmanageable public bureaucracies to unmanageable private ones is hardly the democratizing solution that most reformers are shooting for (Burch, 2009; Henig, 2010a).
MOVING BEYOND CYNICISM: WHAT DOES POLITICAL SCIENCE HAVE TO SAY ABOUT WHEN AND HOW DATA CAN IMPROVE GOVERNMENT PERFORMANCE?
It is one thing to be reminded that data use inevitably intersects with the pursuit of power and advantage. It is another thing to conclude that this is all that data use is or can be. The first is a valuable corrective on naïve idealism. The latter has the potential to fuel bleak cynicism about whether objective evidence, good research, and intelligent inferences based thereon can contribute in any measure toward making policy and governance more informed and effective.
Despite more typically emphasizing the dark side of data manipulation, the discipline of political science also offers insights into more constructive ways of thinking about and studying data use for better policy. For example, some studies of agenda setting have incorporated interviews with policy makers in an attempt to better understand what they need and want from data and research and how their receptivity depends on context and timing (DeBonis, 2009; Kingdon, 1995; Weiss, 1979). Studies of state policy outputs, the consequences of government structure, and policy innovation and diffusion indirectly and implicitly use rational decision making based on objective data as the default premise against which they empirically assess the consequences of political and economic variables that are their more obvious focus. Sifting through that literature reveals how contexts and institutions might facilitate the promotion of data use and suggests the possibility that data usage can exercise beneficial results by working through politics rather than around it.
CONTEXTS CONDUCIVE AND NOT CONDUCIVE TO DATA USAGE
Rather than conceptualize economics, demographics, and politics as rivals to rational data-based decisions, some of the studies of state and local policy outputs suggest the possibility of interaction effects that shed light on the role of context. Soss, Schram, Vartanian, and OBrien (2001) explicitly introduced problem solving as one possible explanation of how states used the new discretion given them after the 1996 federal welfare reforms, and juxtaposed that with other possible predictors, including liberalism, party competition, innovativeness, and race. Evidence of problem solving, as they framed it, would consist of a correlation between objective indicators of need and a policy response: for example, if states with higher welfare caseloads adopted more restrictive policies, or if those with higher rates of unmarried mothers adopted family cap requirements. Although they found some evidence of problem solving, a much more consistent finding was that polices differed according to the racial composition of the likely target population (Soss et al., 2001). They found that the tougher and more punitive policies were adopted in states where the percent African American was higher; in states with fewer Blacks, it seems, the racial attitudes of the general population were less likely to be activated (see also Gilens, 1995; Lieberman, 1998; Schneider & Ingram, 1993).
This suggests both a general and a more specific hypothesis about how local political context might affect whether data usage initiatives bear their intended fruit. Generally, objective data may more likely be pushed to the sidelines whenever emotions and symbolism run strong. Specifically, districts with histories of racial tension, those undergoing rapid change, and those in which the race and ethnicity of public school students differs from that of the general electorate (the other peoples children phenomenon) may be quicker to see data initiatives as portending regime realignmentshifting what government does and in whose benefitrather than apolitical efforts to make government work better. Somewhat paradoxically, these are the very settings that many data advocates are especially anxious to convert. This does not mean that reformers should shy away from promoting data use in such contexts, but it should alert them that data initiatives in such places are more likely to be regarded warily and against a historical context in which some groups past experiences make them suspicious of outsiders who claim that science and data will redound to their benefit. The history of school politics in racially changing urban areas offers instructive illustrations of the subtle ways in which race-based loyalties and experiences can frustrate well-intentioned efforts to institute reforms that, like data usage initiatives, are framed initially in apolitical terms (Henig, Hula, Orr, & Pedescleaux, 1999).
Three lines of research that provide evidence about whether particular institutional forms are more conducive to data-based decision making deal with reform versus machine-style electoral and governance institutions, legislative professionalism, and professional networks for information diffusion. Each of these examines the premise that some governance arrangements promote betteror at least differentpolicies by buffering key decision makers from electoral and interest group pressures.
Impact of progressive reforms.
Numerous studies compare spending patterns and policy adoption in local governments with institutions that had been promoted by the progressive reform movement (at large elections, nonpartisan elections, city manager systems), with those in places that retained the nonreform structures (ward-based elections, partisan elections, strong mayor systems) that some associated with the machine-style politics the progressives sought to undo. In a classic analysis, Lineberry and Fowler (1967) argued that one of the principal goals of the reform movement was to immunize city governments from artificial social cleavagesrace, religion, ethnicity, and so on (p. 708). Based on their study of expenditures of 200 of the nations largest cities, this appeared to be the case; race and ethnicity was less predictive of policies in cities with reformed structures than with those with unreformed institutions. Besides blunting the effects of racial contexts (as discussed earlier), this suggests the hypothesis that some governance structures may moderate the influence of interest group politics in general, including the political influence of teachers unions, which are often portrayed as a major source of resistance to data-based reform. The contemporary manifestation of the progressives confidence in governance institutions as a way to tame politics can be found in the arguments offered by proponents of increasing mayoral control of schools (Henig & Rich, 2004; Viteritti, 2009).
The concept of legislative professionalism generally refers to the extent to which legislatures have the resources, time, and expertise to grapple with complex issues without undue reliance on lobbyists and other self-interested parties (Mooney, 1994). There is considerable agreement that better paid, better staffed, and better informed legislatures make different policies from those comprising part-time amateurs with short terms in office, time-limited sessions, and few professional staff, with much of the literature suggesting that these differences promote better efficiency and coherence (Bowman & Kearney, 1988; Grumm, 1971; LeLoup, 1978).
Political scientists have disagreed about whether legislative professionalism affects policy by strengthening the links between democracy and policy outcomes or by filtering out some of the distorting pressures of interest group politics, and some recent research suggests that at least some legislative efforts to control their sources of information paradoxically might exacerbate inequalities generated by interest group politics. Reenock and Gerber (2008) found that legislative efforts to insulate themselves from political influence may have the unintended effect of increasing their reliance on more elite and mobilized interest groups that offer what is at least on the surface more extensive and sophisticated forms of information while weakening their accountability to smaller scaled citizen groups that also may have important and relevant information but of a different type and form. This serves as another reminder that it is not enough to ask how much policy makers rely on data, but also whose data and what kinds of data they attend to, factors that adhere to the broader regimes within which data and data systems reside.
Policy innovation and diffusion.
Most of the relevant studies here focus on the 50 states and attempt empirically to determine why some states are policy innovators, why some states are likely to be quick on the draw in adopting new ideas from the pioneers, and why some states tend to be laggards. The most replicated finding in the literature is that new policies diffuse spatially, with individual states more likely to adopt an initiative that has already been put in place by one of its neighboring states (Berry & Berry, 1990; Gray, 1973; Walker, 1969). Depending on the underlying mechanism, this tendency to key off of neighbors might have nothing to do with data at all. If this is due to simple mimicry or a casual and ad hoc presumption by policy makers that nearby states are somehow like them, it does not provide much sustenance to those who hope that knowledge and data are key leverage points for policy change. Walker, though, suggested that proximity worked at least in part via awareness, and data might offer a different channel toward awareness and one that communications and technological advances make increasingly capable of trumping geography.
Subsequent research strengthened the evidence that information was a key element of the process. Balla (2001), for example, in examining states adoption of legislation related to health maintenance organizations, found evidence that membership in professional associations provided a channel for the diffusion of information that is not reliant on proximity. Mintrom and Vergari, similarly, found that external networks are important in determining which states actively consider school choice policies (Mintrom & Vergari, 1998), and Cohen-Vogel, Ingle, Levine, and Spence, 2008, through interviewing political leaders and program administrators, found that professional associations played an important role in the adoption of merit-based college aid.
This attention to professional networks lends support to the notion that it is information that facilitates policy diffusion and not simple mimicry of neighbors. It also turns attention toward institutional facilitators of information flow and usage, which are more amenable to corrective intervention than geography and the social-psychological factors at play when citizens and policy makers identify with their neighbors. At least two other implications are relevant to the framing of future research and thinking about data usage in education. First, as throughout the literature considered here, information competes with other forces, including explicitly political ones, as an influence on policy; although its relative impact is higher under some conditions than others, it is rarely the dominating factor. Underscoring this point is the finding that entrepreneurs and professional networks are much better predictions of state consideration of choice policies than of states ultimate adoption of those policies (Mintrom & Vergari, 1998). Second, data traveling through regional or national networks may face skepticism from states or districts based on the flipside of the social-psychological forces that lie behind the imitation of neighbors pattern; locally collected data and studies may be more readily seen as relevant to us than those carried to a community from distant and less familiar settings.
FROM BUFFERING TO LEVERAGING: POLICY DESIGN TO MAKE POLITICS AND DATA WORK TOGETHER
Faith in apolitical data use as deus ex machina for good governance would get short shrift from James Madison. If men were angels, no government would be necessary, Madison wrote in The Federalist Papers #51. If angels were to govern men, neither external nor internal controls on government would be necessary. Rather than structure government around an idealized notion of evidence-based policy untainted by politics, one strain of political science lays the groundwork for a more Madisonian concept: Instead of designing institutions and processes to block politics, design them to use politics as its own check and balance, as its own restraint.
The concept is relatively straightforward but runs directly counter to some of the favored strategies for reform. If correct, instead of tightly controlling data, we should maximize access and utility. Instead of relying on those with the greatest expertise to monopolize the interpretation of data and the mapping of evidence onto applications, we should facilitate the creation of multiple nodes of data analysis as well as raise the general level of data sophistication in the citizenry at large. Instead of designing policies that elevate particular data indicators into triggers for automatic action (dispensing rewards or sanctions), we should regard data as grist for pluralist politics in which negotiation, argument, and ultimately judgment and democracy play the deciding roles.
This alternative perspective is illustrated with reference to two lines of theory coming out of political science. The first grows out of the question, raised by the principal agent literature, of whether legislative bodies can construct policies in ways that force bureaucracies to hew to their intent. The second grows out of perceived failures of past policies to promote greater transparency in government and asks whether transparency policies can be designed to be more politically sustainable.
POLICE PATROL AND FIRE ALARMS
As is common in social science research, the literature on bureaucratic power and democratic accountability is characterized by definable waves. Early studies tended to highlight the problems posed by a bureaucracy able and willing to substitute its own priorities and interests for those that emerged from electoral politics and legislative negotiation (Mitnick, 1975; Moe, 1984). A second wave of studies was more optimistic. Empirical work highlighted evidence that changes in the legislature and executive branch often do lead to consequent changes in bureaucratic practice, suggesting that the hypothetical autonomy was more theoretical than real (Wood, 1990). This wave also comprised work that took the additional step of mapping out institutional designs that could be used to solve or ameliorate the principal agent problem. This institutional design perspective provides some concrete ideas about how data usage can be encouraged despite possible bureaucratic resistance, as well as how data usage can be part of the answer to the question of how to keep bureaucracies on track.
Although executive agencies greater control over data production, interpretation, and dissemination contributes to the bureaucracys power and ability to pursue its independent interests, the principals (Congress in most of the literature, but mayors and school boards and superintendents by inference) have the capacity to structure policies in ways that make data their ally in keeping bureaucracies in line. One way to do this is for elected leaders to mandate the collection and reporting of data, which allows them to better monitor and assess bureaucratic behavior and performance. Both Congress and the OMB receive oceans of data and reports from offices within agencies about ongoing programs. And, through GAO and the General Services Administration (GSA), political actors impose rigid accounting requirements that can be used subsequently as the basis for sanctions (McCubbins et al., 1987, p. 250).
Highly pertinent to the politics of data use in school reform is a distinction between police patrol oversight and fire alarm oversight (McCubbins & Schwartz, 1984). Police patrol oversight is centralized and activated by the policy-making body. Just as a police car might cruise a precinct looking for suspicious activity, a district might mandate analysis of student-level gains by schools or teachers, conduct school visits to gather more qualitative data, or contract with independent researchers to evaluate program implementation. Like police patrol, these uses of data might help detect and remedy violations of expected behavior and simultaneously serve as a deterrent to teachers and schools that might otherwise substitute their own goals and referred behaviors for those pronounced from above. Most of the discussion about the role of educational data for standards-based accountability has been framed in this top-down, managerial, police patrol formulation.
Fire alarm oversight, on the other hand, involves establishing rules and procedures that empower individual citizens and organized interest groups to more easily and effectively spot poor or misdirected bureaucratic performance (see smoke) and bring this to the attention of higher authorities (ring the alarm) when they do. It reconfigures the data regime to broaden the range of groups and the types of data involved. NCLB, by mandating the public release of test score data and requiring that districts inform parents when their schools are failing to make AYP, has elements of the fire alarm approach. Data in this way are designed to trigger politicsto activate the mobilization of groups whose street-level perspective might otherwise be excludedrather than as a depoliticized source of guidance aimed primarily at elites. The fire alarm approach suggests that making data more broadly available could stimulate a healthier policy-making process by working through politics rather than around it.
Policies designed to expand access to government data are not new. Fung, Graham, and Weil (2007) described three generations. The first involved various right to know laws, including, most notably, the Freedom of Information Act (1966), intended to make available to citizens a broad range of information about how government operated as a matter of individual rights and democratic accountability. The second generation, targeted transparency, evolved from the firsts more open-ended pursuit of openness to mandate access to precisely defined and structured factual information with the goal of promoting particular public ends (p. 25). They described a third, nascent generation of collaborative transparency policies. Among the forces driving the evolution of transparency policies are: the accumulated lessons that data alone do not ensure that they will be used, that elites will continue to seek ways to monopolize information, and that data availability may change behaviors in unanticipated ways. Like fire alarm strategies, collaborative transparency policies are meant to broaden the data regimeenlisting a range of citizens and nongovernmental organizations to produce, publicize, and protect data systems that can be applied for public purposes (Fung et al., 2007).
The data systems promoted under state and federal educational accountability regimes resemble the second-generation approaches. Although freedom of information laws made it more possible for alert parents to obtain information that school districts already possessed related to their children and how they were treated within schools, targeted transparency policies stipulated which test data had to be collected as well as when and how they would be disseminated. Fung et al. envisioned a possible third-generation transparency system that would combine government-mandated school report cards that already exist with the active efforts of parents and students (Fung et al., 2007, p. 163). Not only would students and parents be given opportunities to input their own concerns and impressions into school ratings, but they also would be given a greater role in determining the goals and metrics by which school performance is measured (p. 163). As shapers of the data systems, parents and students would have a greater stake in protecting those systems from predictable efforts to erode transparency, whether those efforts come from bureaucracies seeking more autonomy and control or the growing private sector in education seeking to protect commercial and intellectual property
DATA IN THEIR POLITICAL CONTEXT: TOWARD A RESEARCH AGENDA
Basic descriptive data, untainted by association with particular evaluative systems and political agendas, are valuable, and their integrity needs to be protected. In practice, though, it is nearly impossible to disentangle data from the broader interpretative and political regimes that attach them to particular purposes. When it comes to understanding reactions to data usage initiatives, I have suggested that it is these broader contexts that determine reactions. Simply recognizing the complicated dynamic linking data, data systems, and data regimes could have beneficial effects. Advocates of greater data use should restrain their impulse to characterize all reticence as self-interested defense of the status quo. Critics of the reigning vision of test-based accountability should restrain their impulse to frame their resistance to a particular data regime so broadly as to suggest that data and quantification are inherently suspect.
But gaining real leveragewhether for the purpose of predicting or directing likely outcomes of data usage initiativeswill require a stronger base of research. Ideally, such research should bridge the gap that currently exists between political science and applied studies of education policy. The former offers valuable findings about how interest groups and institutions use and affect information but for the most part has yet to focus closely on education, an arena that may be distinct in some consequential ways. The latter has begun to provide a foundation of evidence about the actual role that politics plays in mediating the application of data to educational challenges but for the most part retains a view of politics as an obstacle to be banished or worked around; it has yet to embrace the possibility that policies might be designed with an eye toward building political constituencies to support and sustain the production, maintenance, and pragmatic application of data to better meet public needs.
Essential to building the necessary bridges is research that recognizes the datapolitics connection as being a two-way street. Politics can affect the kind of data collected, who has access to it, and the extent to which those data are applied to broad collective problems or the pursuit of narrower agendas. Data systems, and the broader regimes within which they are embedded, can alter the distribution of power and influence, pushing some groups and the values they hold to the margin while giving others stronger holds on the levers of policy change. I close by illustrating what such a research agenda might look like, presenting some examples that strike me as interesting and worthy of empirical probing. The discussion is meant to be suggestive and a catalyst for thinking and discussion, neither comprehensive in its coverage of the possibilities nor fully developed in the specifics of the examples put forth.
IMPACT OF POLITICS ON DATA USAGE
What political conditions make it more likely that government policy will promote and sustain the collection of high-quality data relevant to informing educational policy?
Explaining leaders and laggards in state administrative data systems. Some states, such as Florida, North Carolina, and Texas, have been national leaders in the establishing of data systems that allow them to track students as they move through school, across schools (including charters), and even into higher education. Both the Bush and Obama administrations have cited these as valuable tools for accountability and have taken steps to encourage other states to move in this direction. High-quality and extensive data systems can inform decisions, but they also can lead to political embarrassment for state leaders if they reveal overall failures or failures to meet the needs of specific subsets of the student population, or if they show that showcased programs do not make a difference. The Data Quality Campaign has developed a 10-point checklist of state progress in building longitudinal data systems, providing one immediately available way to operationalize relative achievement (http://www.DataQualityCampaign.org).
What political factors account for the creation, development, and maintenance of state data systems? How much is attributable to individual champions, and are such champions more commonly drawn from elected office or within state bureaucracies? Is adoption of data more likely in states with competitive party systems (where the minority party might have both the incentive and enough power to push these as a means for checking the party in power), or in more stable political systems where the state leaders feel more secure? In states with more professional legislatures or more extensive education agencies? Is support or opposition from key interest groups critical? Do states that respond to national government incentives differ from those that initiated such efforts on their own?
The politics of local data consortia. The Consortium on Chicago School Research is regarded by many to be a model through which to increase local data capacity and use, and with its former executive director now heading the federal Institute for Education Sciences, there is even wider interest in seeding similar institutions in other localities. But what are the political conditions that make it more or less likely that such an institution will garner enough civic support, not only to come into existence but also to have sufficient capacity and enough autonomy and authority to establish a research agenda that is not overly dominated by the local district and superintendent? What are the relative importance of a local philanthropic community, business community, and various constructed coalitions among elected leaders, parent organizations, teacher unions, and various good government groups?
One research design might be to undertake a comparative case study comparing the origins and development of the Chicago Consortium with similar efforts initiated elsewhere but with less satisfactory results. Part of the explanation for the Consortiums success may have to do with better funding or know-how; others may involve the ways in which supporters framed the issue and recruited allies. One key issue is keeping open access to the school districts data, which may depend on developing an element of trust, a sense on the part of the district that even if some unflattering outcomes are brought to light, the district will not be bludgeoned or caught by surprise. But local data organizations also need to earn the trust of other community stakeholders, including some who may think that research challenging the districts claims is exactly what is needed the most. How did the Chicago Consortium negotiate these issues, and are their strategies sufficient to ensure success in different political contexts?
What political conditions make it more likely that relevant and high-quality data will be available and used by a broad range of organizations and be incorporated into policy discourse and public deliberation?
Media deployment of school data. The mediaranging from traditional print and broadcast to the new media of the so-called blogosphereare important components of data regimes because of the role they can play in incorporating data into policy discourse (Henig, 2008). Important as they are in their own right, the media also provide an empirical window into difficult-to-measure aspects of data dissemination and distribution as well. Whether and how different forms of media will utilize educational data in their news coverage and editorials is likely affected by many things, including the objective existence and availability of data, journalists knowledge of and confidence in dealing with quantitative data, the structure of media markets, and the economic capacity of media organizations to pursue and analyze data. A distinctly political question is whether media will be more or less likely to pursue and deploy data in politically competitive environments (where stakeholders outside the educational bureaucracy will be more likely to demand and be capable of putting it to use) compared with those where there is less polarization (and therefore lower risk to the media of being seen as raising questions that challenge the dominant power structure or norms).
Broadening constituencies for data and their application. The fire alarm conception of democracy and accountability suggests that elected officials will be better able to hold bureaucracies accountable when systems are in place to allow multiple stakeholders to spot and publicize the smoke of emerging problems or bureaucratic nonperformance. Leaving aside for the moment important (but diminishing) concerns about the digital divide, publicly available websites provide a reasonably reliable and informative measure of what amounts and kinds of data are available to parents, concerned citizens, and community-based groups that lack the resources to generate data on their own. Researchers can use the Web to simulate the experience that interested parties would encounter in attempting to find various kinds of data, carefully coding the ease of search, the amount and detail of data, the quality and flexibility of site-linked analytical tools (e.g., can users generate their own tables and graphs?), and the ease of downloading data in useable formats. With this information constituting the dependent variable(s), researchers could then study the conditions associated with the availability of relevant and useable data. For example, does district willingness and capacity to extend access to data depend on support having been mobilized within the preexisting political context, or can it be injected into reluctant districts (e.g., via state mandates or private organizations)?
IMPACT OF DATA ON POLITICS
In her 2009 AERA presidential address, Lorraine McDonnell called for education researchers to pay more attention to policy feedback: the theory that policies enacted and implemented at one point in time shape subsequent political dynamics so that politics is both an input into the policy process and an output (McDonnell, 2009, p. 417). Policies affect politics by making costs and benefits of programs more apparent, by redirecting support so that some groups become stronger and others less so, by creating new inside-government allies comprising public employees and political sponsors who become more mobilized actors because they have jobs and reputations at stake, and by creating new interest groups that directly benefit from the programs and often become their most attentive and ardent proponents.
Paying attention to policy feedback is important for at least two reasons. First, whether and how new policies change political dynamics is critical to determining whether those policies are sustained or prove to be short-lived. Sometimes a new policy that is enacted by thin margins builds stronger constituencies over time. Sometimes a new policy sparks backlash among previously complacent interests, leading them to mobilize more aggressively and work for the policys speedy repeal (Patashnik, 2008). In the case of data usage, we should be especially attentive to the ways in which the availability of more relevant, precise, and comprehensive indicators reduces the probability of both Type 1 and Type 2 errors: sticking for too long to policies that are not working or too abruptly terminating helpful initiatives because their costs are readily apparent while their benefits are unmeasured. Second, paying attention to policy feedback can sensitize us to a range of possible unintended consequences; when policies alter political dynamics, they can set the stage for a wide range of subsequent shifts in policy and implementation that may not have been accounted for in the original design. In the case of data usage, we should be especially attentive to the issue of whether broadening the range and availability of educational data empowers new voices within collective policy debates or whether it augments the already substantial power of bureaucracies.
How does the generation and maintenance of a high-quality comprehensive data system affect regime stability and interests and alignments among stakeholder interest groups?
Data systems and the politics of policy churn. Analysts have identified policy churnthe tendency to rapidly cycle through shallow reform effortas an ingrained characteristic of American (particularly urban) education and a threat to more serious and sustained efforts at substantive change (Farkas, 1992; Hess, 1998; Shah & Marschall, 2005; C. Stone, Henig, Jones, & Pierranunzi, 2001). Does a richer data infrastructure exacerbate or moderate the politics of policy churn? On the one hand, a rich data infrastructure might slow and smooth out the politics of reaction and backlash, by, for example, making it easier to empirically benchmark small gains in young students skills and attitudes that would not register as performance gains until those students wrestled with the more demanding coursework in middle and high school. On the other hand, the availability of precise and varied indicators could be exploited by some groups to buttress their insistence on sharp shifts in policy direction because demonstrable gains are too weak, scattered, and costly.
Potentially valuable lines of research could involve cross-sectional or longitudinal analysis of policy churn in relation to relative changes in local, state, and national data systems. Do districts and states with more developed data systems experience different rates of program initiation, program termination, bureaucratic reorganization, or leadership change?
Data access and the politics of expertise and fire alarms. Political science has revealed that data can serve as a political weapon by bolstering the perceived legitimacy and authority of some interests, making it less likely they will be challenged, or by making it more likely that citizens and others will have the information to spot and report incipient problems, increasing their influence and helping legislatures and executives keep a check on their bureaucratic agents. Does development of centralized and comprehensive databases for accountability systems alter the balance of influence between superintendents and school boards? Central districts and principals? Parents and teachers?
Recent events in New York City school reform politics illustrate the potential for data systems to alter the dynamics of political events. During the 20082009 run-up to the state legislatures decision to renew mayoral control of schools, critics of the Bloomberg-Klein administration challenged the administrations claim that test scores were undergoing dramatic improvements attributable to their reforms. Although the Department of Education exercised tight control over its own data and analysis, the existence of statewide systems and national data systems made it possible for critics to raise credible challenges to the administrations data claims. Reference to statewide performance data enabled critics to argue that the citys gains on state tests were no greater than those in several other large urban districts, and comparison with other cities that, like New York City, participated in the NAEP Trial Urban District Assessment allowed them to show that the citys proficiency levels and gains were unexceptional when compared with other large districts around the nation (Brennan, 2009; Pallas & Jennings, 2009; Ravitch, 2009, 2010).
Measuring power and influence is among the most fraught areas in political science, giving rise, for example, to a long-running and still unresolved set of scholarly battles known collectively as the community power debate (Orr & Johnson, 2008; Scott, 1994). Rather than directly trying to operationalize influence, researchers might productively study whether and how various interests incorporate data claims into their organizational websites, public statements, or testimony and whether this varies in jurisdictions that utilize different data systems or have developed different data regimes. One of the frequent criticisms of elected school boards, for example, is that superintendents manipulate them in ways that prevent the boards from exercising meaningful oversight and control. Superintendents often are in position to selectively generate and selectively release data in ways that create an impression of success, with little possibility of challenge by school boards, which in many districts are part-time bodies with little or no professional staff. A longitudinal analysis of school board transcripts before and after the initiation of comprehensive data systems could reveal whether board members (or other nondistrict sources) become more likely to use their own presentation of data to challenge central office data claims, for example, by reducing their relative reliance on anecdotal reports.
Efforts to infuse better data and analysis into education decision making occur in the context of broad shifts in the institutional landscape of American education. These shifts are forcing changes in what have been defining characteristics of American education, including localism, deference to professionalism, and the strong American faith in public schools as vehicles for personal mobility and societal progress. Growing state and federal involvement in core elements of education policy has significantly encroached on localism, a powerful wellspring of support for schooling that is buttressed not only by norms and traditions but also by sentimental longing for a decentralized democracy of friends and neighbors, and by more hard-nosed assessments of the role that good schools can have in raising local property values. A spreading sense that educators may have placed their own interests above those of their students has fueled a backlash against teachers unions, education bureaucracies, and traditions of deference to professional expertise. And growing doubt that ours are the best schools in the world has punctured the American publics faith in the existing system, opening the field to a new set of private-sector educational organizations with their own claim to special expertise (Henig, 2009, 2010b).
Data usage is not simply affected by this institutional upheaval that is shifting the boundaries of education decision making; it is implicated in the upheaval itself. Databecause of their incorporation into systems and regimesare implicated in the structural changes altering the education policy landscape. This is most directly so because the promotion of data and data systems has so central a role in the accountability movement (McDonnell, 2005). This accountability movement is frequently framed as an apolitical application of administrative oversight, but it is also deeply entwined with sharp shifts in governance structures (e.g., accountability mechanisms are central to the expansion of the state and federal role) and changes in interest group influence (e.g., student performance gains as a challenge to teacher unions historical centrality in shaping systems for allocating salaries and rewarding tenure). Because the lines of cleavage involve competing values, ideas, interest groups, and institutions, the dynamics arguably are best understood as a function of competing political regimes.
Situating data usage within this broader political context, in which long-standing institutions and processes of educational decision making are being realigned, explains why resistance is often broader, more ingrained, and more intense than reformers anticipate or can readily understand. Conflicts over data use are surrogates for other battles in which the stakes are seen to be very high indeed. Failing to understand this macro-context may be one reason that micro-interventions (e.g., providing professional development to make teachers comfortable with data under the assumption that their reluctance is based on inadequate information and preparation alone) can widely miss the mark.
Aaron, H. (1978). Professors and politics: The Great Society in perspective. Washington, DC: Brookings Institution Press.
Austen-Smith, D. (1997). Interest groups: Money, information and influence. In D. C. Mueller (Ed.), Perspectives on public choice (pp. 296321). Cambridge, England: Cambridge University Press.
Bachrach, P., & Baratz, M. S. (1963). Decisions and nondecisions: An analytical framework. American Political Science Review, 57(3), 632642.
Bachrach, P., & Morton, S. B. (1962). Two faces of power. American Political Science Review, 56(4), 947952.
Balla, S. J. (2001). Interstate professional associations and the diffusion of policy innovations. American Politics Research, 29(3), 221245.
Berry, F. S., & Berry, W. D. (1990). State lottery adoptions as policy innovations: An event history analysis. American Political Science Review, 84, 395415.
Bowman, A. O. M., & Kearney, R. C. (1988). Dimensions of state government capability. Western Political Quarterly, 41(2), 341362.
Brennan, J. F. (2009). New York City public school performance before and after mayoral control. In L. Haimson & A. Kjellberg (Eds.), New York City under Bloomberg and Klein: What parents, teachers, and policymakers need to know (pp. 105114). New York, NY: Lulu.
Bridges, A., & Kronick, R. (1999). Writing the rules to win the game: The middle-class regimes of municipal reformers. Urban Affairs Review, 34(5), 691706.
Burch, P. (2009). Hidden markets: The new education privatization. New York, NY: Routledge.
Cobb, M. D., & Kuklinski, J. H. (1997). Changing minds: Political arguments and political persuasion. American Journal of Political Science, 41(4), 88121.
Coburn, C. E., Honig, M. I., & Stein, M. K. (in press). Whats the evidence on districts use of evidence? In J. Bransford, L. Gomez, D. Lam, & N. Vye (Eds.), Research and practice: Towards a reconciliation. Cambridge MA: Harvard Education Press.
Coburn, C. E., Toure, J., & Yamashita, M. (2009). Evidence, interpretation, and persuasion: Instructional decision making at the district central office. Teachers College Record 111(4), 11151161.
Coburn, C. E., & Turner, E. O. (2010, January 30). Data use and educational improvement: Toward a research agenda. Paper prepared for the Spencer Foundation Initiative on Data use and Educational Improvement, working draft.
Cohen-Vogel, L., Ingle, W. K., Levine, A. A., & Spence, M. (2008). The spread of merit-based college aid: Politics, policy consortia, and interstate competition. Education Policy, 22(3), 339362.
Crenson, M. A. (1972). The un-politics of air pollution: A study of non-decisionmaking in the cities. Baltimore, MD: Johns Hopkins University Press.
DeBonis, M. (2009, March 6). Fund and games: Inside Michelle Rhees official schedule. Washington City Paper. Retrieved from http://www.washingtoncitypaper.com/display.php?id=36893
Downs, A. (1957). An economic theory of democracy. New York, NY: Harper & Row.
Dror, Y. (1967). Policy analysts: A new professional role in government service. Public Administration Review, 27(3), 197203.
Edelman, M. J. (2001). The politics of misinformation. Cambridge, England: Cambridge University Press.
Farkas, S. (1992). Educational reform: The players and the politics. New York, NY: Public Agenda Foundation.
Fung, A., Graham, M., & Weil, D. (2007). Full disclosure: The perils and promise of transparency. Cambridge, England: Cambridge University Press.
Gilens, M. (1995). Racial attitudes and opposition to welfare. Journal of Politics, 57(4), 9941014.
Gray, V. (1973). Innovations in the states: A diffusion study. American Political Science Review, 67, 11741185.
Grossman, G. M., & Helpman, E. (2001). Special interest politics. Cambridge MA: MIT Press.
Grumm, J. (1971). The effects of legislative structure on legislative performance. In R. Hofferbert & I. Sharkansky (Eds.), State and urban politics: Readings in comparative public policy (pp. 298322). Boston, MA: Little, Brown.
Hayes, M. T. (1992). Incrementalism and public policy. New York, NY: Longman.
Hayes, S. P. (1964). The politics of reform in muncipal government in the Progressive Era. Pacific Northwest Quarterly, 55, 157169.
Hemphill, C., & Nauer, K. (2010). Managing by the numbers: Empowerment and accountability in New York Citys schools. New York, NY: The New School, Center for New York City Affairs.
Henig, J. R. (2008). Spin cycle: How research is used in policy debates: the case of charter schools. New York, NY: Russell Sage Foundation/Century Foundation.
Henig, J. R. (2009). The politics of localism in an era of centralization, privatization, and choice. NSSE Yearbook, 108(1), 112129.
Henig, J. R. (2010a). Portfolio management and diverse provider models as contracting regimes. In K. E. Bulkley, J. R. Henig, & H. M. Levin (Eds.), Between public and private: Politics, governance, and the new portfolio models for urban school reform (pp. 2752). Cambridge, MA: Harvard Education Press.
Henig, J. R. (2010b). The contemporary context of public engagement: The new political grid. In M. Orr & J. Rogers (Eds.), Public engagement for public education (pp. 5285). Palo Alto, CA: Stanford University Press.
Henig, J. R., Hula, R. C., Orr, M., & Pedescleaux, D. S. (1999). The color of school reform. Princeton, NJ: Princeton University Press.
Henig, J. R., & Rich, W. C. (Eds.). (2004). Mayors in the middle: Politics, race, and mayoral control of urban schools. Princeton, NJ: Princeton University Press.
Hess, F. M. (1998). Spinning wheels: The politics of urban school reform. Washington, DC: Brookings Institution Press.
Hess, F. M. (Ed.). (2008). When research matters: How scholarship influences education policy. Cambridge MA: Harvard Education Press.
Honig, M. I. (2008). District central offices as learning organizations: How sociocultural and organizational learning theories elaborate district central office administrators participation in teaching and learning. American Journal of Education, 114, 627664.
Honig, M. I., & Coburn, C. (2007). Evidence-based decision making in school district central offices: Toward a policy and research agenda. Educational Policy, 22(4), 578608.
Hula, K. (1999). Lobbying together: Interest group coalitions in legislative politics. Washington DC: Georgetown University Press.
Jacobs, L. R., & Shapiro, R. Y. (2000). Politicians dont slander: Political manipulation and the loss of democratic responsiveness. Chicago: University of Chicago.
Jones, B. D. (1995). Reconceiving decision-making in democratic politics: Attention, choice, and public policy. Chicago: University of Chicago Press.
Jones, B. D. (1999). Bounded rationality. Annual Review of Political Science, 2, 297321.
Kahneman, D., Slovic, P., & Tversky, A. (1982). Judgment under uncertainty: Heuristics and biases. New York, NY: Cambridge University Press.
Kingdon, J. W. (1995). Agendas, alternatives, and public policies (2nd ed.). Boston: Little, Brown.
Klein, J. (2011). The failure of American schools. The Atlantic. Retrieved from http://www.theatlantic.com/magazine/archive/2011/06/the-failure-of-american-schools/8497/
Lasswell, H. D. (1970). The emerging conception of the policy sciences. Policy Sciences, 1(1), 314.
LeLoup, L. T. (1978). Reassessing the mediating impact of legislative capability. American Political Science Review, 72(2), 616621.
Lieberman, R. C. (1998). Shifting the color line: Race and the American welfare state. Cambridge, MA: Harvard University Press.
Lindblom, C. E. (1959). The science of muddling through. Public Administration Review, 19, 7988.
Lindblom, C. E. (1990). Inquiry and change: The troubled attempt to understand and shape society. New Haven, CT, and New York, NY: Yale University Press and Russell Sage Foundation Press.
Lineberry, R. L., & Fowler, E. P. (1967). Reformism and public policies in American cities. American Political Science Review, 61, 701716.
Lowi, T. J. (1967). Machine politics: Old and new. Public Interest, 9, 8392.
Malbin, M. J. (1980). Congress, policy analysis, and natural gas deregulation: A parable about fig leaves. In R. A. Goldwin (Ed.), Bureaucrats, policy analysis, statesmen: Who leads? (pp. 6287). Washington DC: American Enterprise Institute.
Marsh, J. A., Pane, J. F., & Hamilton, L. S. (2006). Making sense of data-driven decision making in education: Evidence from recent RAND research. Santa Monica, CA: RAND.
McCubbins, M. D., Noll, R., & Weingast, B. R. (1987). Administrative procedures as instruments of political control. Journal of Law, Economics, and Organization, 3, 243277.
McCubbins, M. D., & Schwartz, T. (1984). Congressional oversight overlooked: Police patrols versus fire alarms. American Journal of Political Science, 28(1), 165179.
McDonnell, L. M. (2005). Assessment and accountability from the policymakers perspective. NSSE Yearbook, 104(2), 3554.
McDonnell, L. M. (2009). Repositioning politics in educations circle of knowledge. Educational Researcher, 38(6), 417427.
McLaughlin, M. J. (1987). Learning from experience: Lessons from policy implementaion. Educational Evaluation and Policy Analysis, 9(2), 171178.
Mintrom, M., & Vergari, S. (1998). Policy networks and innovation diffusion. Journal of Politics, 60, 126148.
Mitnick, B. M. (1975). The theory of agency: The policing paradox and regulatory behavior. Public Choice, 24, 2742.
Moe, T. M. (1984). The new economics of organization. American Journal of Political Science, 28(4), 739777.
Moe, T. M. (2011). Special interest: Teachers unions and Americas public schools. Washington, DC: Brookings Institution Press.
Mooney, C. Z. (1994). Measuring U.S. state legislative professionalism: An evaluation of five indices. State and Local Government Review, 26(2), 7078.
Mosteller, F., & Boruch, R. (Eds.). (2002). Evidence matters: Randomized trials in education research. Washington, DC: Brookings Institution Press.
Mutz, D. C., Brody, R. A., & Sniderman, P. M. (Eds.). (1996). Political persuasion and attitude change. Ann Arbor: University of Michigan Press.
Nathan, R. P. (2000). Social science in government: The role of policy researchers. Albany, NY: Rockefeller Institute Press.
Novick, D. (Ed.). (1965). Program budgeting: Program analysis and the federal budget. Cambridge, MA: Harvard University Press.
Orr, M., & Johnson, V. C. (Eds.). (2008). Power in the city: Clarence Stone and the politics of inequality. Lawrence: University Press of Kansas.
Pallas, A., & Jennings, J. (2009). Progress reports. In L. Haimson & A. Kjellberg (Eds.), New York City under Bloomberg and Klein: What parents, teachers, and policymakers need to know (pp. 99104). New York, NY: Lulu.
Patashnik, E. M. (2008). Reforms at risk: What happens after major policy changes are enacted. Princeton, NJ: Princeton University Press.
Prewitt, K. (2010). The U.S. decennial census: Politics and political science. Annual Review of Political Science, 13, 237254.
Ravitch, D. (2009). Student achievment in New York City: The NAEP results. In L. Haimson & A. Kjellberg (Eds.), New York City under Bloomberg and Klein: What parents, teachers, and policymakers need to know (pp. 2330). New York, NY: Lulu.
Ravitch, D. (2010). The death and life of the great American school system. New York, NY: Perseus.
Reenock, C. M., & Gerber, B. J. (2008). Political insulation, information exchange, and interest group access to the bureaucracy. Journal of Public Administration Research and Theory, 18(3), 415440.
Rochefort, D. A., & Cobb, R. W. (Eds.). (1994). The politics of problem definition: Shaping the. Lawrence: University Press of Kansas.
Rourke, F. (1969). Bureaucracy, politics and public policy. Boston, MA: Little, Brown.
Schneider, A., & Ingram, H. (1993). Social construction of target populations: Implications for politics and policy. American Political Science Review, 87(2), 334347.
Scott, J. (Ed.). (1994). Power: Critical concepts. New York, NY: Routledge.
Shah, P., & Marschall, M. (2005). Keeping policy churn off the agenda: Urban education and civic capacity. Policy Studies Journal, 33(2), 161180.
Sharkansky, I. (1968). Spending in the American states. Chicago, IL: Rand McNally.
Simon, H. A. (1985). Human nature in politics: The dialogue of psychology and political science. American Political Science Review, 79, 293304.
Soss, J., Schram, S., Vartanian, T., & OBrien, E. (2001). Setting the terms of relief: Explaining state policy choices in the devolution revolution. American Journal of Political Science, 45(2), 378395.
Spillane, J. P. (2002). Policy implementation and cognition: Reframing and refocusing implementation. Review of Educational Research, 72(3), 387431.
Stone, C., Henig, J. R., Jones, B. D., & Pierranunzi, C. (2001). Building civic capacity: Toward a new politics of urban school reform. Lawrence: University Press of Kansas.
Stone, C. N. (1989). Regime politics: Governing Atlanta, 19461988. Lawrence: University Press of Kansas.
Stone, D. (2002). Policy paradox: The art of political decision making. New York, NY: Norton.
Towne, L., & Shavelson, R. J. (2002). Scientific research in education. Washington, DC: National Academies Press.
Tucker, H. J. (1982). Incremental budgeting: Myth or model? Western Political Quarterly, 35(3), 327338.
Viteritti, J. P. (Ed.). (2009). When mayors take charge: School governance in the city. Washington, DC: Brookings Institution Press.
Walker, J. (1969). The diffusion of innovations among the American states. American Political Science Review, 63, 880899.
Walters, P. B., Lareau, A., & Ranis, S. H. (Eds.). (2009). Education research on trial: Policy reform and the call for scientific rigor. New York, NY: Routledge.
Weick, K. E. (1976). Educational organizations as loosely coupled systems. Administrative Science Quarterly, 21, 119.
Weiss, C. H. (1979). The many meanings of research utilization. Public Administration Review, 39(5), 426431.
Weiss, C. H. (1998). Evaluation: Methods for studying programs & policies (2nd ed.). Saddle River, NJ: Prentice Hall.
Wildavsky, A. (1966). The political economy of efficiency: Cost-benefit analysis, systems analysis, and program budgeting. Public Administration Review, 26(4), 292310.
Wildavsky, A. (1984). The politics of the budgetary process (4th ed.). Boston, MA: Little, Brown.
Wood, B. D. (1990). Does politics make a difference at the EEOC? American Journal of Political Science, 34(2), 503530.