The Rise of Intermediary Organizations in Knowledge Production, Advocacy, and Educational Policy
by Christopher A. Lubienski, Janelle Scott & Elizabeth H. DeBray - July 22, 2011
This commentary examines the rise of intermediary organizations that "broker" research for policymakers, and considers the implications for traditional forms of knowledge production.
Americans like to think that the policies used to address many of our social issues are based on reliable evidence of effectiveness, especially when substantial taxpayer resources are involved. Yet knowledge production can often be a highly politicized process in which scientific knowledge, research, and professional expertise are vulnerable to ideological interpretations. For instance, we expect that medical interventions or technological innovations are tested before they are made widely available. But debates over a response to climate change, or on whether to teach an alternative to evolution, show how questions of public policy around empirical issues are often informed by political forces (Oreskes & Conway, 2010).
Issues around the effectiveness of educational interventions in particular highlight the institutionalization of extra-governmental political forces in the policymaking process. Indeed, we are seeing new institutional forms emerging that are re-shaping the political economy that is, the institutional relationships of research production and use in education. Specifically, new intermediary organizations are increasingly determining the body of research made available for the policymaking process by brokering evidence. Here we discuss these new intermediaries, and note their power in the area of incentivist educational policies. Their emerging power raises questions about the future of traditional forms of research production.
The Sad Impact of Education Research
For quite some time, educational research has been the poor stepchild to other scholarly disciplines in influencing public policy. Its not without reason that some policymakers, legislative staff, and advocacy groups castigate, at best, the research produced by universities, and by colleges of education in particular, or simply ignore it.
At the same time, policymakers across the political spectrum have been calling for a clear emphasis on what works in considering policy proposals around schooling. The Bush administration established the What Works Clearinghouse to grade educational research, and its signature No Child Left Behind Act repeatedly calls for scientifically based research for different interventions. The Obama Administration has declared that the multi-billion dollar Race to the Top (RttT) discretionary funding will be dispersed based on what works (Washington Post, 2009). And venture philanthropists, influenced by the effective philanthropy movement, are searching for assurance that investments will pay off in terms of better outcomes for students.
How do we account for the void between the calls for research in education and the neglect of education research often by the same people? Perhaps more importantly, what forces are contributing to, or capitalizing on this chasm? The rise of intermediary organizations sheds some light on this issue.
Certainly, the history of research use (and neglect) by policymakers around education issues speaks to significant problems both real and imagined in education research. Compared to, say, the standing of the National Research Council, or other groups within the National Academies, the National Academy of Education and AERA appear to spend more time asking to be heard, rather than responding to queries from policymakers. Indeed, they have to overcome a reputation for educational research as being too abstract, or of poor quality.
And there is something to these labels. University-based education research too often suffers from a number of problems. It is often inefficient, pouring more money into sustaining higher educational institutions than into diagnosing and solving school problems. Research is produced in a climate often characterized by opinion more than evidence. Research agendas can be quite unresponsive to the actual needs of children, focusing instead only on the interests of researchers or the whims of funders. Much of the research is too abstract to be of value to anyone. And many researchers, driven by university promotion and tenure incentives to publish in peer-reviewed journals, have little interest in engaging in broader conversations with the policy, school, or media communities.
The Rise of the Intermediaries
Yet amidst the problems with some educational research, there exists robust research utilizing qualitative, quantitative, and mixed-methods that effectively addresses pertinent issues. Much of this work never reaches they eyes or captures the attention of policy makers, however. Increasingly, intermediary organizations are playing decisive roles in gathering, interpreting, and packaging particular research for policymakers.
These intermediary organizations (IOs) are established to fill a key function in brokering evidence in support of specific agendas. While these organizations usually do not themselves conduct research, they have become very effective at assembling and promoting evidence for use by policymakers tasks in which university-based researchers have been negligent. In the void left by traditional education researchers, we are seeing new forms of research organizations step into this environment: not simply traditional think tanks, but philanthropies, policy coalitions, and single- or multi-issue advocacy organizations with notable media savvy effectively geared toward shaping education policy.
The prolific expansion of these groups speaks to their perceived success (at least to funders), and they certainly bring some strategic advantages to the table. In contrast to traditional, university-based research, which tends to dwell in abstract, inaccessible language and findings qualified by methodological considerations, IOs are geared to deliver a streamlined, highly understandable message to policymakers. Less concerned about the prestige of the journal in which they publish, they instead deliver slick, highly produced materials, and undergird their efforts by garnering strategic support in op-ed pieces, blogs and podcasts, and even documentary films that draw on and support the efforts of their coalition allies. Likewise, they can be extremely responsive to perceived demand for specific types of information. Rather than going through onerous peer-review processes, they can market research (by anybody) quickly, often inserting their message to the news cycle by attaching it to the story of the day.
But there are also some drawbacks to the rise of intermediary organizations. Perhaps most importantly, by their nature, IOs make ideology, and not evidence, the over-riding consideration. Usually dependent on private funding, these organizations are based on specific agendas, which they must advance in order to secure funds. Moreover, the advantage of IOs in terms of responsiveness comes at the price of established quality control mechanisms associated with university research. Although a few academics have adopted elements of the streamlined approach pioneered by IOs, or have affiliated with them, the rush to advance research through policy windows when the opportunity arises means that IOs necessarily sacrifice time-intensive review processes. Overall, this lends weight to strategies that rely on sheer firepower of multiple (confederated) groups promoting a similar message (often around a specific moment in time), rather than publishing in obscure, if prestigious, academic journals.
The Case of Incentivist Policy
Education research is proceeding in a new environment characterized now by multiple institutional sources of information, and a producer-to-consumer relationship that is less of a pipeline and more of a quasi-market of vendors trying to promote goods. On the supply side, universities must compete with new organizations specifically created to advance particular agendas, and highly adept at (and well-resourced for) packaging and marketing a defined message to policymakers. On the demand side, policymakers appear to be embracing evidence that is concise, highly produced, timely and accessible and aligned with their policy positions.
A prime example of this new quasi-market in education policy is the policymaking around what can be called incentivist policies (e.g., Greene, et al., 2008; Stern, 2008). Such policies advance from the assumption that individuals and organizations respond positively to extrinsic rewards, and that polices can arrange incentives in order to encourage desired behaviors and outcomes (Moe, 2008). Popular examples such as charter schools, voucher programs, merit-pay plans, and cash incentives for students exemplify recent efforts by policymakers across the political spectrum to reconfigure school systems around incentives for individuals and organizations. But the evidentiary basis for these policies appears to have less to do with research coming from traditional, peer-reviewed sources, and instead to be much more grounded in the literature produced by intermediary groups specifically established to promote research supporting such measures.
Consider the case of vouchers, for instance. States are currently moving forward with plans to send students to private schools at public expense, based largely on evidence of their effectiveness in boosting achievement, as provided by groups such as the Foundation for Educational Choice, Stand for Children, Students First, the American Federation for Children, the Center for Education Reform, and the Foundation for Excellence in Education. While allowing families to choose private schools at public expense may or may not be a good idea on a number of different dimensions, empirical evidence on the effectiveness of these programs in raising student achievement is weak, at best. Many of the reports on these programs are funded by interest groups that promote the programs; and few, if any, research studies showing clear and compelling evidence of their effectiveness have ever been published in independent, peer-reviewed journals. Instead, many of the reports are produced by a small cadre of researchers-activists, who tend to cite each others studies, with little regard for peer-reviewed work, often in journals they themselves have created raising the possibility of an echo-chamber effect in advancing these agendas . We see similar patterns with regard to the other incentivist proposals such as merit-pay, charter schools, and pay-for-performance schemes. For instance, even as Washington and the states are moving forward with incentive-based policies for teachers, the new report of the National Academy of Sciences (2011) tracked a wide range of accountability-based incentives for teachers nationally since 2002, including merit pay, and concluded that the incentives often distorted the work of educators (Hout & Elliot, 2011).
This disconnect, then, between educational research and policy does not necessarily lend credence to policymakers claims of a need for research on educational issues. Indeed, many of these program have grown not only in lieu of, but often before or in spite of, any evidence on their effectiveness . In fact, while the notion that policymakers avoid education research because it is poor quality may have some validity, policymakers have also largely ignored many researchers outside of colleges of education (such as economists) when their findings dont support the incentivist agenda. This raises questions about policymakers tendency to admonish education research, which may instead simply be an excuse to ignore research if the findings run counter to ideological commitments. So claims about the need for research evidence may have less to do with the research, and more to do with the need to get evidence (or the right evidence) to policymakers.
But, inasmuch as policymakers actually use research either to arrive at, or lend empirical legitimacy to, a position these patterns also speak to the power of IOs in selling evidence on the effectiveness of these policies. Their success points to a new political economy of knowledge production and use in education. In this environment, there are advantages for organizations that can more effectively package and promote research, orchestrate a concerted effort to convey a consistent message through multiple media outlets, and place it in front of key people in the policy pipeline. Intermediary organizations have demonstrated a notable ability to succeed in this climate.
Despite the rise of intermediary organizations in educational policy, there is surprisingly little understanding about how such groups function in the overall policymaking process. In other words, while we know that many groups aspire to broker research to policymakers, there is little awareness of the mechanisms by which such brokering takes place, how policymakers interpret such research, and how these interpretations are applied to both local and federal-level policies. This field must understand how reform ideas may potentially bypass formal governmental entities that is, when claims about local successes may be spread up to the formal policy process in Congress thereby informing what we know about policy diffusion. In the past, studies of national policy looked to fairly stable models of issue networks and iron triangles to explain the adoption of federal policies, but these ideas, like those about the influence of university-based academics, need an update.
Greene, J. P., Carroll, T. W., Coulson, A. J., Enlow, R., Hirsch, E. D., Ladner, M., et al. (2008, January 24). Is school choice enough? http://www.city-journal.org/2008/forum0124.html.
Hout, M., & Elliott, S.W. (Eds.). (2011). Incentives and test-based accountability in public education. Washington, DC: National Academy of Sciences Press.
Lubienski, C., & Weitzel, P. (2010). Information use and epidemics in charter school policy. In C. Lubienki & P. Weitzel (Eds.), The charter school experiment: Expectations, evidence, and implications. (pp. 197-217). Cambridge, MA: Harvard Education Press.
Moe, T. M. (2008). Beyond the free market: The structure of school choice. Brigham Young University Law Review, 2008(1), 557-592.
Oreskes, N., & Conway, E. M. (2010). Merchants of doubt: How a handful of scientists obscured the truth on issues from tobacco smoke to global warming. New York: Bloomsbury Press.
Stern, S. (2008, Winter). School choice isn't enough. City Journal, 18, http://www.city-journal.org/2008/2018_2001_instructional_reform.html
Washington Post. (2009, July 30). Dollars for schools. Washington Post, editorial.