The OECD as Pivot of the Emerging Global Educational Accountability Regime: How Accountable are the Accountants?


by Heinz-Dieter Meyer - 2014

Background/Context: PISA has catapulted the OECD—an organization whose mission is the global growth of market economies—to a central role in international education policy making, rivaling and sometimes outdoing the various national governments in influence. While claiming scientific evidence as the basis for the accountability regime it promotes, the policy frameworks and principles it has adopted over the past two decades are rooted in ideas about the superiority of market mechanisms that are contested in many member nations. Yet, while exerting strong influence on national educational practices in pursuit of a narrow and contested policy agenda, it is not easily influenced through established democratic processes.

Purpose: In this paper, I discuss key OECD ideological and policy shifts that form the background for PISA. Secondly, I focus on the OECD’s governance mechanisms and the obstacles it presents to public scrutiny.

Research Design: Drawing on recent scholarship on the power of transnational policy networks to influence national policy through the soft power of epistemic communities, this paper analyzes policy documents and relevant research from OECD and PISA to identify ideological commitments and configurations of power that form the backdrop for PISA.

Conclusion: I propose that in a world of global policy making, the influence of organizations like the OECD highlights deficits in the constitution of the global democratic public sphere that would foster the accountability of these organizations.




OECD AND THE RISING INFLUENCE OF TRANSNATIONAL POLICY NETWORKS


During the past decade, the OECD-sponsored PISA assessment has become increasingly influential in reshaping educational policy and practice around the world. Although the OECD is an organization whose mission is to further the growth of market economies, it is fast becoming a powerful global educational authority, rivaling and sometimes outdoing in influence the various national governments in whose domain it operates (Breakspear 2012; Ertl 2006; Martens and Jakobi 2010; Martens and Niemann 2010). The rise of OECD and PISA illustrate the emergence of a new power in education policy and arguably signify a new era in public education. In this era the control of nation states over education is waning, as educational governance is increasingly subject to an emerging global governance regime, in which OECD and PISA act as a crucial lynchpin (Ball 2012; Jakobi and Martens 2010; Sellar and Lingard 2013; Mahon and McBride 2008; Rubenson 2008; Henry, Lingard, Rizvi, and Taylor 2001; Daun 2005). OECD exerts its influence through instruments of “soft governance” (Lawn 2006) which focus on providing the definitions, classifications, standards, and measures to assess public policies. By controlling these standards OECD can exert significant control on policy.


The watchword with which OECD has propelled PISA to global influence is accountability (Power 1999; Mahon and McBride 2008). By that OECD does not mean holding governments accountable for their steering role in public education, but holding schools—constricted and constrained though they are through myriads of laws and institutional arrangements (H.-D. Meyer and Rowan 2006)—accountable for how they affect student performance on standardized tests. With PISA, the OECD has developed an instrument which measures not what or how a given nation teaches its students but to what extent 15 year olds around the world have mastered the skills that the OECD deems important for young people to productively participate in the economy (Schleicher 2007; see also Labaree, in this issue). This also represents a significant narrowing and, I shall argue below, a distortion of educational objectives. It stands in contrast to sustained expert critique which discourages the use of single tests as a measure to gauge quality or make other broad pronouncements about the characteristics of large systems (Delandshere 2002; H.-D. Meyer and Benavot 2013; Prais 2003; Koretz 2008; Bracey 2004, 2009; Steiner-Khamsi 2003; Torrance 2006).


The new accountability regime pivoting on OECD and PISA represents a shift in the balance of power and control of public education—from democratically constituted national governments to an international policy organization that seems to many beyond the reach of democratic control. With some notable exceptions (Rizvi and Lingard 2010; Lingard 2011; Lingard and Rawolle 2011; Ball 2012) the consequences of this shift have, to date, gone largely unnoticed. In particular, the centralization of influence at a global policy hub like OECD might be more palatable if it was matched by a proportional increase in accountability on the part of the OECD. But the growth in OECD’s power has come as result of a rise of global transnational policy networks that make power elusive.


The ability of OECD’s PISA test to effectively reshape education policy around the globe is illustrative of a global process that has increased the power of markets and weakened the power of states (H. D. Meyer and Benavot 2013). Globalization processes, including the integration of national economies into a world economy and changes in technology and finance, have diminished the power of states to affect their own affairs. As a result, authority has shifted to non-state actors, including transnational corporations (TNCs) and international organizations like OECD, World Bank, and others (Held 1995; Strange 1996; Kamens 2012).


As market forces are inherently international, the pressure for accountability leaves the nation state increasingly dependent on global forces (Strange 1996). The gap left by the state’s withdrawal from many areas of social life, including education, is filled by international organizations (IOs) like the OECD, which are standing by with a well-defined package of policies and management methods (Bourdieu 1998a, 1998b). Using only soft power instruments, IOs are able to socialize states to accept new political goals and new social values (Finnemore 1996; Barnett and Finnemore 1999). IOs make rules and develop standards and measures (like PISA). In so doing, they create social classifications that frame and reframe our understanding of social practice. IOs define tasks, create new social actors (like auditors and consultants), form new interests for actors, and transfer new models of political organization around the world (Harvey 2005). Perhaps most importantly, IOs become the home base of “epistemic communities” (Haas 1992) which are held together through a shared expertise and a common set of beliefs in specific and often contested principles of policy and analysis. In the language of “world culture” or “world society” theory, the world’s educational practices become increasingly isomorphic with specific Western models supported by universal beliefs in human rights and economic growth and diffused through elite experts in global policy networks (J. W. Meyer 2009; Ramirez et al. 2006).


The accountability or audit explosion (Power 1999) is an example of this kind of recasting of public policy where institutions like public education are increasingly viewed as efficiency-oriented organizations, subject to the imperatives of calculability, predictability, control, and ruled by numbers and league tables (Power 1999; Apple 2005). If unchecked, the result is the instrumentalization of education for the purpose of economic productivity. As Bourdieu puts it: “Education is never taken account of as such at a time when it plays a determining role in the production of goods and services as in the production of the producers themselves” (Bourdieu 1998b).


Given that the OECD represents “an important but underemphasized node in the growing networks of transnational governance” (Mahon and McBride 2008: 21), and given its increasing influence in education policy, education researchers have an interest in understanding the principles and the process of the organization’s education policy making. What is OECD’s educational agenda? What are the guiding ideas that have given rise to PISA’s global assessment regime, and how do member nations control that agenda?


In what follows I will first highlight the profound changes in OECD’s education policy agenda. Secondly, I will discuss how OECD, as it imposes an increasingly densely knit assessment regime on the world in the name of accountability, is itself not easily held to account to the people in the member nations. I conclude with observations about the need for new kinds of public accountability mechanisms.


OECD’S IDEOLOGICAL TURN: FROM PUBLIC RESPONSIBILITY TO “NEW PUBLIC MANAGEMENT”


In this section I argue that PISA is part of a significant and fairly recent policy reversal inside the OECD. If in the early 1990s OECD education policy was still reflecting a skeptical and cautious view of the wisdom of standardized global assessments and an awareness of public education as a project of both economic and political and cultural participation, with PISA that balance has been abandoned in favor of a set of policies that place economic effectiveness above all.


The change in policy direction was part of a general shift in the mid-1990s whereby OECD embraced neoliberal free-market policies promoting an expansive use of markets and quasi-markets as a panacea to a wide range of public policy problems. Pioneered in the United States and the UK under the leadership of Ronald Reagan and Margret Thatcher, but widely contested in those countries and many member states, this shift to neoliberalism represented a decided ideological shift in OECD (Pal 2008; Mahon and McBride 2008).


This shift is most visible in the OECD’s adoption of “new public management” (NPM) which was announced in OECD’s Governance in Transition report (OECD 1995)—a key policy document that argued that effective public policy should make optimal use of market mechanisms (Lane 2000; Pal 2008). In Governance in Transition (1995) OECD proposed that traditional “governance structures and managerial responses are increasingly ineffectual” (Pal 2008: 62), but that the problem could be addressed by introducing devolution, accountability, competition, and choice as key principles of public policy.


The ideological basis for NPM are concepts loosely related to the principal-agent (PA) theorem widely popular in neoclassical economics (Pratt and Zeckhauser 1985; Lane 2000). In the principal-agent perspective, the employer-employee relationship is seen as one of mutual distrust and control. When “principals” (business owners or top managers) hire agents (workers) to assist them, the latter will often pursue their self-interest, using opportunity to slack off, deceive, and otherwise act against the interest of their principal or employer. Employers must use methods of monitoring and control to counteract the ubiquitous tendency of employees to act against the organization’s interest.


Organizational theorists recognize this idea as the “theory X” approach first advocated by Frederick Taylor’s scientific management, from where it found its way into the theory and practice of school administration (Callahan 1964; Tyack 1974). Many organization researchers agree that the theory represents, at the very least, a one-sided depiction of human motivation, ignoring the equally profound disposition of people to take pride in their work (McGregor 1960/2006). Treating people as if they were slackers often functions like a self-fulfilling prophecy. Workers who are treated as untrustworthy are likely to return the favor.


A cornerstone of the NPM-credo was to treat agencies in public administration as independent agents (“profit centers”) by making them compete against each other and farming out all services to private businesses that the latter can produce “more efficiently” than public agencies.


The OECD’s embrace of “new public management” made the idea that much more appealing to reformers in the member countries where agencies from garbage collection to tax collection, and everything from public schools to public pools were newly using market mechanisms to cut slack, act as “cost centers” and become more entrepreneurial. Often these new goals were accompanied by organizational restructuring like devolution and decentralization.


In the logic of the principal-agent theorem, public agents and agencies (from prisons to tax collectors, and from police men to school teachers) have an interest in lowering quality. This tendency can presumably be counteracted by making public agencies compete for resources, by creating price transparency through “pay for use” policies like toll roads, and by setting specific performance targets and holding agencies accountable for reaching them. Educators will readily recognize that this is also the logic behind Race to the Top, in particular its Annual Professional Performance Review (see Leonardatos and Zahedi, this issue) where performance targets are defined in terms of student test score improvements and tied to teacher salaries and promotion. PISA rankings have a similar effect as they reveal the majority of the world’s public education systems to fall short of “world class” quality defined by educational outliers like Finland, Hong Kong, Singapore, or Korea (H.-D. Meyer and Schiller 2013).


But while PISA and Race to the Top are still in their early phase of implementation, NPM has been around long enough for its flaws to be revealed in practice.

New Public Management: From Youthful Enthusiasm to Belated Sobriety


While the OECD launched into NPM with true-believer enthusiasm, only a decade later a noticeable cooling took place. After a decade of reforming public administration in the spirit of new public management, many member countries returned empty handed. NPM simply failed to deliver. In a review of NPM in its 2003 policy brief, OECD conceded that “mistakes were made” due to the OECD’s “single-minded devotion to efficiency” (Pal 2008: 69). Another problem was its insufficient attention “to culture or to diverse paths towards modernity.” OECD researchers realized that “the core public service is controlled more by culture than by rules” (69).


In embracing NPM the organization worked from the false assumption that there was a single generic cause for administrative inefficiency, namely “bureaucracy.” And, most importantly, the OECD now conceded NPM critics’ most important point: that values like efficiency and transparency do not “override all other public values” like “equity and responsibility” (67).


Thus it took the OECD the better part of 15 years to catch up with insights that NPM critics had offered from the start (Lane 2000) including the notion that competition, in and of itself, does not increase quality. Rather, it may induce a “race to the bottom” in order to attract the largest market segments. Also, private companies don’t necessarily deliver more cheaply and they certainly don’t deliver with the same values. A for-profit prison business is concerned with keeping the inmates imprisoned at the lowest cost. They are not concerned with lowering recidivism, encouraging education, etc.

-------------------------------------------

OECD AT A GLANCE


ORIGIN IN MARSHALL PLAN AND OEEC

In its current form, the OECD is the successor organization to the OEEC, the Organization for European Economic Cooperation, founded in 1948 to administer the Marshall Plan. Member nations used to submit their economic policies for review to U.S. government in exchange for financial assistance. Today, the OECD comprises 34 countries.

MISSION

The OECD is a forum for “countries committed to democracy and the market economy.” “The Organisation provides a setting where governments compare policy experiences, seek answers to common problems, identify good practice and coordinate domestic and international policies.”* In organizational terms, the OECD can be located somewhere between a giant think tank and a sprawling transnational network of experts, consultants, researchers, and government officials. OECD quite accurately describes itself as a “hub for the discussion of global policy issues, we interact with governments, international organisations, parliamentarians, business sector, trade unions, academics, media and NGOs, building an unprecedented network of knowledge based on a deep-rooted conviction about the benefits of multilateral co-operation.”**

The OECD participates as an independent organization in coordinated governance bodies including NATO, the European Union (EU), and the European Patent Organization. It entertains cooperative relations to many other international organizations, like World Bank and World Trade Organization (WTO).

STRUCTURE
The OECD Council, made up of member nations’ economics or finance ministers, is the OECD’s highest formal decision-making body. The council organizes annual meetings, among which the meeting of the ministers of economics is the most important.

Committees, one for each work area of the OECD, comprise subject-matter experts from member and nonmember governments. There are committees on economics, trade, science, employment, education, financial markets, among others, for a total of about 35 committees, and another 200 working groups and expert groups. About 40,000 senior officials from national administrations come to Paris each year to participate in their work.

The OECD Secretariat leads the organizations day-to-day operations. It is led by the Secretary-General (currently Ángel Gurría) and organized in directorates, which include about 2,500 staff. There are directorates for tax policy and administration, development cooperation, labor and employment, science, and technology. Significantly, education recently ascended to directorate status, attesting to its growing importance in OECD.

FUNDING

The secretariat is funded by member countries, whose contributions are based on their GNP. The largest contributor is the United States, providing one-fourth of the $510 million USD (followed by Japan, Germany, UK, and France). The council and committees are funded by a budget that amounts to another $500 million USD in direct member nations contributions.

OECD & PISA
Through National Project Managers, participating countries implement OECD/PISA at the national level subject to the agreed administration procedures. Each OECD country participating in PISA has a representative on the PISA Governing Board, appointed by the country’s education ministry. The Board determines the policy priorities for PISA. The current U.S. representative on the board is Daniel J. McGrath, Director of International Activities of the National Center for Education Statistics.


*About OECD. OECD. Retrieved 2010-01-16.

** http://www.oecd.org/careers/whatmakesustheoecd.htm



THE RISE OF PISA: FROM DOUBTS ABOUT STANDARDIZED ASSESSMENTS TO “MEASURING EVERYTHING”


In the area of education policy and PISA, however, OECD is still in the phase of youthful exuberance. In fact, PISA has become the poster-child for OECD’s “effectiveness” in promoting institutional change around the world. To get to this point, OECD had to abandon a cautious skepticism towards the use of standardized assessments that characterized the organization’s stance as recently as the 1990s.


A case in point are the views of Malcolm Skilbeck who directed the OECD’s educational work through much of the 1990s. In 1990 Skilbeck published a report on curriculum reform in which he took a rather dim view of international assessments:

International comparisons of performance are popular if frequently of dubious validity. The initiative generally comes largely from central government itself . . . where it is not a venture of the international testing industry. While such initiatives are paramount in the United States, it should not be forgotten that they are paralleled by an unprecedented outpouring of critiques. (Skilbeck 1990: 29)


Malcolm Skilbeck, an Australian with a Ph.D. from the University of London, was an academic with a background in education and curriculum studies. He had been a professor, dean, and president of Deakon University before ascending to Deputy Director of OECD’s Directorate for Education, Employment, Labour and Social Affairs, the most influential position in OECD’s education sector activities.


In the “Curriculum Reform” Skilbeck deplored “the treatment of school curriculum in strategic terms by governments seeking to restructure or firmly steer their economies” (1990: 76). He also expressed sympathy with the view of professional educators that “the balance between freedom and control is a delicate one and many professionals are not convinced that it is being maintained” (76). At the same time, Skilbeck noted an ongoing “tendency . . . to submerge long-term concerns for access, opportunity, equity, fairness in education to economic imperatives” (Skilbeck 1990: 79).


Skilbeck’s leadership of the OECD’s education activities clearly reflected an understanding of the complexity of public education and the inevitable conflict between educational and economic imperatives. The awareness of the need to balance administrative and educational objectives also pervaded OECD’s 1994 publication “The Curriculum Redefined.” In there, the American John Nisbet expressed his fear “that . . . the demand for accountability, for certification and selection, for hard evidence, will take precedence over the equally legitimate requirement that assessment should promote learning” (166). He points to the two sides of assessment and the “tension between assessment as an instrument of management or control, and assessment as a means to support learning” (Nisbet 1994: 167).


In other words: in the early 1990s, OECD’s position on assessments and public education was consistent with the views among prominent education scholars that cautioned against sacrificing educational quality and professional control to the imperatives of administrative and economic efficiency.

“The broadest possible picture of the global talent pool”


If these cautious voices were still influential as late as 1994, OECD’s 1995 Governance in Transition indicated a radical shift in power and policy. It prepared the ground for the formation of the PISA consortium in 1997, which published the results of the first survey in 2000. Spurred by the large public echo PISA received, OECD’s zeal to assess the role of public education in narrow terms of economic costs and benefits has, if anything, grown. The skeptical and cautious views of standardized assessments now gave way to a full-throttle embrace of education in terms of economically useful skills and “competences” and identifying accountability with “measurability.” Schools are openly treated as “sector of the economy.” For OECD’s PISA coordinator, Andreas Schleicher, a key problem is that nations don’t know what they get for the money spent on schools. “No sector of the economy [sic] could operate like that” (Spiegel 2007). Education is a performative, economically relevant activity that must be held to efficiency standards akin to economic enterprises. The “policy by numbers” approach explains OECDs ambitious agenda in which PISA—expansive as it is—is only a first step on a path to measuring everything affecting skills and productivity (Sellar and Lingard 2013).

Expanding the PISA Franchise


PISA has become so successful a policy enterprise, that, according to Sellar and Lingard (2013), it is now the poster child for OECD’s newly invigorated global presence. Education’s increasing importance is also reflected in its recent ascent to “Directorate” status within OECD’s core operations, with ambitious plans to expand its operations both geographically as well as functionally. An OECD official within the Education Directorate explained the need for this expansion:

We need to embrace a broader range of competencies . . . you need to build in interpersonal competencies, problem solving; intrapersonal competencies and motivation, self-concept and so on, and these are things we just need to do better and need to work hard on to broaden the horizon. (quoted in Sellar and Lingard 2013: 195)


One component of this expanded coverage is PIAAC (Programme for the International Assessment of Adult Competencies) which also aims to assess “personal traits” such as “grit” (persistence and self-discipline), social and cultural engagement, political efficacy, and social trust. With added information of this kind, one OECD official points out, “the potential for influencing the running of societies I think is really, really good” (Sellar and Lingard 2013: 196; emphasis added). The goal is to obtain “the broadest possible picture of the global talent pool.”


Another addition to the PISA brand is TALIS, the “Teaching and Learning International Survey.” This is the OECD’s program to assess teacher working conditions and learning environments in schools through teacher questionnaires. The first round was conducted in 2008 and the second round will be completed in 2013.


With its dramatically increasing fleet of surveys and assessments the OECD is clearly on its way to become the world’s most influential educational authority, as its rankings, reports, and analyses become the key data to orient policy making in the member nations. And where rankings and reports don’t speak loud enough, the OECD stands ready to assist with policy implementation, as it did in the case of Mexico (see below).


As OECD’s expanding educational activities increasingly influence “the running of societies,” the question arises: how do these global policy makers make themselves accountable?


ASYMMETRIES OF POWER AND OECD’S GOVERNANCE OF OPACITY


Both NPM and PISA represent a serious deviation from OECD’s previously more balanced and more cautious approach to monitoring educational trends. In particular, its adoption of a heavily market-oriented, neoliberal model of accountability would seem representative of the views and interests of only limited segments of the democratic publics of its member nations.


One reason why this change has to date been little noticed can be found in the nature of the OECD’s culture and decision making process, which relies on three key mechanisms: consensus, peer review, and peer pressure (Guilmette 2007) operating in the context of pronounced power asymmetries. The interaction of these three mechanisms guarantees a decision process that seems to obscure OECD decision making and insulate it against participation from member nations’ democratic publics, thus guaranteeing OECD’s “relative autonomy in constructing ideas” (Mahon and McBride 2008a: 15).


A key stipulation governing OECD policies is the consensus rule, which means that the organization is expected to speak with one voice. To the casual consumer of OECD reports this is noticeable, for example, in the fact that even long and complex documents involving not only data but analysis and interpretation are customarily not attributed to individual authors, but to the OECD as a whole.


Formally, the OECD is connected to its member nations by virtue of member nations’ senior finance or economic officials in the council, the organization’s highest ranking institution (see “what is the OECD?” insert). But this does not give the member nations real democratic power to counteract or reject OECD policies. The more powerful member nations, especially the three founders, the United States, the United Kingdom, and Canada (who together contribute the lion’s share of OECD’s financial resources) typically find a version of their proposal succeed. How is this achieved in a setting involving a multitude of nations which differ dramatically in history, culture, and values? The modus operandi of policy making in democratic publics is certainly not consensus, but debate. Where debate is not open and disagreement not facilitated, consensus can often only be achieved by employing pressure mechanisms. In the case of OECD, the relevant mechanisms are peer review and peer pressure, which operate against a backdrop of steep power asymmetries.


One of OECD’s most strictly enforced rules is that every member nation commits to opening its books to all other members. Key policy issues are researched based on data from all member nations. Moreover, nations that are found to deviate from agreed-upon policies are pressured to adjust. A typical case might be a nation that tries to gain economic advantages through protectionist measures. If the peer review process surfaces such measures, the country is pressured to adjust its policies.


This culture of consensus and peer review emerged during OECD’s early history as OEEC charged with administering the Marshall Plan. Through OEEC, the United States successfully fostered economic growth in Western Europe by offering financial support for post-war reconstruction to nations like Germany in exchange for economic policy concessions from those nations. During these decades of facilitating economic growth in member nations, the OECD became a “colossally successful challenge to Soviet and Chinese Communism” (Mahon and McBride 2008a: 7).



POWER ASYMMETRIES


In practice, consensus means that the mightier members have greater say. Peer pressure ensures that majorities are able to “bring around” the recalcitrant. Debate of more sensitive topics are “confidential.” Minutes are not always made public. Formal decisions are rare (Mahon and McBride 2008a: 18).


Porter and Webb explain: “states experience peer pressure differently depending on their power in the OECD. The largest member states—the United States, European countries such as Germany, Britain, and France, and Japan—have the greatest impact on the implicit criteria used as the basis for peer review of national policies” (Porter and Webb 2008: 51). As a result, ideas largely flow “from North to South . . . [and] from Anglo-Saxon group of the OECD member countries to all the rest” (51). “OECD’s internal discourse has often been viewed as dominated by Anglo-American-trained economists” (15).


OECD officials openly acknowledge that “size matters very much—political and military importance are substantial elements in determining which countries can lead or block the process” (OECD 2002, p. 5, quoted in Porter and Webb 2008: 51).


As the chief rules of consensus, peer review, and peer pressure operate against a backdrop of power asymmetries, consensus guarantees that the strongest member nations carry especial weight and become the center of gravitation in the OECD’s internal opinion formation. This explains why the Thatcher-Reagan agenda of neoliberal cutback of democratic state governance in favor of expanded market mechanisms took root in the OECD, despite strong reservations in many member nations.


THE ROLE OF RESEARCH


An important pillar of OECD’s global legitimacy is its claim that its policies are grounded in impartial research. And the breathtaking amount of the OECD’s research activity would seem to bear this out. Apart from a few major research universities, the OECD is probably one of the biggest producers of economic and social policy research anywhere. However, unlike research activities in universities, OECD’s policy research presents itself as beyond debate and contestation.


For contrast and comparison, one might look at the policy research published in major policy journals like Foreign Affairs. The research published there is often of the same caliber as that put out by the OECD, with the difference that it is always open for debate and always debated. It is also presented as the work of identifiable individuals, rather than packaged as a “report” issued in the name of an organization with a staff of several thousand.


As a result, non-OECD scholars face formidable obstacles if they want to partake in the OECD’s policy formation or challenge its results. On the few occasions where OECD research has been systematically reviewed by outsiders, the results have been disheartening. An example is the OECD’s 1977 McCracken report, which, according to Mahon and McBride (2008: 15) “marked the beginning of neoliberalism” within OECD. One of the report’s critics, the American economist Robert O. Keohane, asked “[t]o what extent are the policy recommendations of these OECD economists derived from the findings of economics per se—and to what extent do they stem, instead, from unexamined political or ideological assumptions” (Mahon and McBride 2008: 16). Keohane went on to say that the writers of the McCracken report were “quite content to select important variables in a casual, ad hoc fashion and to evaluate their significance impressionistically” (16), relying on doctrines “by no means uncontested within the economics profession.”

PISA’s Black-Boxing of Test Construction Issues


PISA is a case in point. Since PISA’s inception, serious doubts have been expressed about the test’s validity and reliability (Bracey 2004, 2009; Prais 2003; Hopmann et al. 2007) and about the underlying idea of “competency-based testing” as opposed to curriculum based testing (see, e.g., Larabee, this issue). It has been shown, for example, that slight changes in the assessment strategy (as evinced by the differences between TIMSSR and PISA) produce significantly different results for participating nations. For example, although both TIMSSR and PISA look at UK students born in 1984, PISA produced much higher scores for the UK (Prais 2003).


This makes decisions of test design and test evaluation potentially highly political, but they are often hidden in the statistical fine print that even trained statisticians find hard to penetrate. Thus, one researcher expressed “dismay that the paper on how Rasch transformations were applied to get from raw scores to scaled scores ‘is not easy reading even for professional mathematicians and makes no concessions to those who are not fully adept research-psychometricians’” (Bracey 2004, 478).


Daniel Koretz (2008), one of the foremost authorities on testing and test design gives an example concerning decisions about sampling of content: “for example, what percentage of mathematics test items should be allocated to algebra?” TIMSS allocated 25%, PISA only 11%. On what basis? Another design decision that may be more far reaching than it looks: what percentage of items should be multiple choice versus open answer? TIMSS had two-thirds open answer, PISA only one-third (Koretz 2008: 104).


As a result countries’ math performance has fluctuated appreciably depending whether it is measured by TIMSS or PISA. Tom Loveless (2013), of the Brookings Institution, pointed out that “[d]iscrepant results from PISA and TIMSS are revealing. When releasing the latest PISA results in 2010, Angel Gurría, Secretary-General of the OECD called New Zealand a ‘high flier.’ New Zealand is definitely not a high flyer on the TIMSS math tests, scoring only 486 in 4th grade and 488 in 8th grade--significantly below the international means of 500.”


PISA is based on a controversial constructivist math approach. The countries that have adopted this approach in their curriculum (e.g. New Zealand, Holland, UK, and Finland) do well on PISA’s math tests. But when nonconstructivist TIMSS is used as a benchmark results change. Loveless: “Finland’s 2011 math scores are statistically indistinguishable from the U.S. at both 4th and 8th grades. While U.S. scores have improved since 1999, Finland’s have declined.”


Based on the criticism from education researchers, it is easy to imagine a PISA-style test which, after a series of small changes in metrics and methodology, would produce significantly different rankings. In addition, different nations have clearly different values and educational traditions that would make them want to measure outcomes that PISA ignores (see, e.g., Münch, in this issue). To date, these qualms and concerns are largely sacrificed to the project of global comparability, using one narrow, idiosyncratic standard, to create a global educational quality ranking—jettisoning important doubts held by the organization only a few years earlier.


CONFLICTS OF INTEREST


THREE IN ONE: THE OECD AS DIAGNOSTICIAN, POLICY ADVISOR, AND LOCAL PRESSURE GROUP


In the past, the OECD has gained trust and influence due to a position of neutrality, limiting itself to the provision of data and analysis, and committed to abstain from interfering with the internal politics and practice of member governments. Increasingly, that wise commitment to neutrality and abstinence from interference is giving way to a blurring of boundaries between analysis and the OECD’s involvement in reforms on the ground. A case in point is OECD’s 2008 cooperation agreement with the Mexican government, the “Calidad Educativa/Quality Education” described by the OECD as “joint initiative of the OECD Directorate for Education and the Mexican Ministry of Education, Secretaría de Educación Pública - SEP, aiming to improve the quality of education in Mexico.”


This agreement has the OECD assist the Mexican government in education reform in areas such as school management and social participation, teacher selection and recruitment, teacher education and training, teacher incentives and stimuli, and evaluation (OECD Directorate for Education 2013).


In line with this agreement, OECD experts met in July 2010 with officials from Mexico’s ministry of education for “a very important meeting to start what we call ‘Knowledge transfer’ to the Mexican stakeholders who have to decide the best way to take our recommendations on board.” A few weeks later OECD and Mexican officials met again at the World Bank in Washington to discuss “co-financing” of reforms between the World Bank and the Mexican ministry. At this meeting “new guidelines” were discussed which “aim to align key elements from the OECD recommendations elaborated by the OECD Steering Group on School Management and Teacher Policy in Mexico.” At the meeting the OECD presented further recommendations, which was followed by an “exploration of pathways” of implementation (OECD 2011).


While OECD has been careful to frame its role with the Mexican education reform as advisory, it is clear that its influence is going beyond offering information and advice that the government can take or leave. Especially when OECD’s moral and framing power is combined with the World Bank’s financial power, the OECD’s ability to bend government policy to its will is huge. Acting simultaneously as diagnostician, policy advisor, and implementation-supervisor of education policies, it becomes a formidable force in shaping education practice—while effectively exempting itself from traditional democratic separation of powers.


PEARSON, MCKINSEY AND CO: PAVING THE WAY FOR FOR-PROFIT COMPANIES


On OECD’s website, the 2008 OECD-Mexico agreement is characteristically framed by a quote saying that "The quality of an education system cannot exceed the quality of its teachers.” The source of this quote is not an educator or scholar, but the international consulting firm McKinsey and Company.


OECD’s reference to McKinsey is illustrative of another worrisome OECD trend: its increasing reliance on for-profit companies that are assuming a strategic role in OECD’s PISA reform program. It is of course not new that private, for-profit companies are trying to penetrate public education (Burch 2006), but with the help of OECD they are finally making inroads on a global scale.


Two organizations which are particularly prominent here are McKinsey and the UK-based publishing corporation Pearson PLC, which also owns the Financial Times, Penguin Books, and 50% of the share capital of The Economist. McKinsey is the world’s leading management consultant firm, which for some time now has been expanding its portfolio into the education sector. And Pearson is, in its own description, “the world's leading learning company,” with interests in 70 countries, including primary education in Kenya and higher education in South Africa.


Both companies have become quite visible to educators in the United States and Europe: McKinsey as an advocate and provider of blueprints for efficiency oriented education reform (Mourshed, Chinezi, Barber 2010) and Pearson as a comprehensive provider of everything needed for implementation: textbooks, standardized tests, test assessment, and analysis as well as consultants, trainers, and web designers. The extent to which Pearson is involved with education reform in the United States is illustrated by the fact that in some states the line between Pearson and the education department is virtually undistinguishable. In the spring of 2011, the New York State department of education sent out reform directives to school administrators under a letterhead that placed the NYSED (New York State Education Department) and Pearson logo side by side at the top of the page. Administrators who called the letter’s phone number were surprised to be directly routed to Pearson staff (see Leonardatos and Zahedi, this issue).


There is also a high degree of personal interconnectedness among the three organizations. A key figure at both McKinsey and Pearson is Sir Michael Barber, a former school teacher who moved from a position as union representative to an influential post in public administration reform in Tony Blair’s government. From there he became “head of global education practice” in McKinsey consulting and most recently was appointed Chief Education Adviser at Pearson. Barber strongly believes in the essential role of for-profits: “innovation” he says, “will come from the private sector or public-private partnerships, rather than government” (quoted in Wilby 2011).


This dovetails with OECD’s announcement that for some PISA related assessments schools will need to “contract an approved provider of assessment and analysis services in order to have their results officially recognized” (quoted in Sellar and Lingard 2013: 200). There is little question that Pearson is best positioned to be such an approved provider. An example is the OECD’s video series Strong Performers and Successful Reformers in Education, which purports to analyze lessons from PISA and is made available by OECD at Pearson’s Web site: http://www.pearsonfoundation.org/oecd.


Nor is Pearson’s vision limited to the UK or the United States. Its vision is truly global and the developing world key parts of its market strategy. In his new role as chief education advisor to Pearson, Barber has set up a “US Education Delivery Unit” which is entering into consulting arrangements with education departments of the several states.


Likewise, OECD’s push to have PISA adopted in Africa and Latin America parallels Pearson’s most recent project of supporting the expansion of private education in developing countries:

Low-cost private education is an important, complementary element of education in developing countries and should be seen as an active partner with governments looking to ensure all children have access to a high quality education. We are convinced that affordable schools, operated on a for-profit basis, can make a big difference. (Pearson 2012)


Barber’s approach to education reform is an efficiency oriented strategy focusing on administrative streamlining, an approach he calls “deliverology” (US EDI 2011). The book he coauthored with McKinsey consultants Moffit and Kihn (Barber, Moffit, Kihn 2011) is replete with language recapping familiar mantras of managerialism around the idea that everything can be measured and what can be measured can be managed.


CONCLUSION


OECD has become a global education authority with the ability to reshape the education around the globe, its power often rivaling and surpassing that of national governments. As a major actor in transnational policy networks, it is able to develop and impose “consensual knowledge about how the world works” and to change “the thinking of the people” (Wolfe 2008: 41). But while it is effectively influencing educational practice of millions around the world in the name of a neoliberal concept of accountability, the twists and turns of its policy and assessment regime are largely beyond the influence of democratic publics. OECD’s PISA accountability regime thus poses not only an educational but also a political problem. The educational problem is one of narrowing and homogenizing educational practices worldwide. The political problem involves a profound challenge to democracy deriving “from the divergence that sometimes exists between the totality of those affected by a political decision and those who participated in making it” (Held 1995: ix).


The irony of OECD’s campaign for accountability in education is thus that it raises serious issues about the accountability of OECD as an institution of global governance itself. To redress the asymmetries between strong influence and weak democratic control will require profound advances in the organization of the global public sphere. Those changes are likely to require a broadening of the global educational discourse, in which the accountability narrative, which currently so clearly dominates the debate, is complemented and counterbalanced by narratives of local institutional learning, educational tradition, democratic participation, and cultural diversity.


It may be useful to remember here that OECD is by no means the only international organization with an educational agenda. Around the time when OECD started the PISA consortium in 1997, UNESCO (the United Nations’ Educational, Scientific, and Cultural Organization) published a report on culture, education, and development, noting, inter alia, that “[t]he cultural dimensions of human life are possibly more essential than growth. . . . Education, for example, promotes economic growth and is therefore of instrumental value, and at the same time is an essential part of cultural development, with intrinsic value. Hence we cannot reduce culture to a subsidiary position as a mere promoter of economic growth” (UNESCO 1996: 14).


References


Apple, M. W. (2005). “Education, Markets, and an Audit Culture.” Critical Quarterly 47 (1–2): 11–29.


Ball, S. J. (2012). Global Education Inc: New Policy Networks and the Neo-liberal Imaginary. London: Routledge.


Barber, M., A. Moffit, and P. Kihn.  (2011). Deliverology 101. Thousand Oaks, CA: Corwin


Barber, M., K. Donnelly, and S. Rizvi. (2012). Oceans of Innovation. The Atlantic, the Pacific, Global Leadership and the Future of Education. London: Institute for Public Policy Research.


Barnett, M. N. and M. Finnemore (1999) “The Politics of Power, and Pathologies of International Organizations.” International Organizations 53 (4): 699–732.


Bourdieu, P. (1998a)  Acts of Resistance against the New Myths of our Time. Cambridge: Polity Press,


Bourdieu, P. (1998b). “The Essence of Neoliberalism.” LeMonde Diplomatique, December, http://mondediplo.com/1998/12/08bourdieu.


Burch, P. E. (2006). “The New Educational Privatization: Educational Contracting and High Stakes Accountability.” Teachers College Record 108 (4): 2582–2610.


Bracey, G. W. (2004). “International Comparisons: Less Than Meets the Eye?” Phi Delta Kappan 85 (6): 477–478.


Bracey, G. W. (2009). “PISA: Not Leaning Hard on U.S. Economy.” Phi Delta Kappan 90 (6): 450–451.


Breakspear, S. (2012). “The Policy Impact of PISA.” OECD Education Working Paper 71. Paris: OECD.


Callahan, R. E., (1964). Education and the Cult of Efficiency, Chicago: Chicago University Press.


Daun, H. (2005). “Globalisation and the Governance of National Education Systems.” In International Handbook on Globalisation, Education and Policy Research, edited by J. Zajda, 93–107. Dordrecht: Springer.


Delandshere, G. (2002). “Assessment as Inquiry.” Teachers College Record, 104 (7): 1461–1484.


Ertl, H. (2006). “Educational Standards and the Changing Discourse on Education: the Reception and Consequences of the PISA study in Germany.” Oxford Review of Education 32 (5): 619–634.


Finnemore, M. (1996). National Interest in International Society. Ithaca, NY: Cornell University Press.


Guilmette, Jean-H. (2007). The Power of Peer Learning: Networks and Development Cooperation. New Delhi: Academic Foundation.


Haas, P. M. (1992). “Introduction: Epistemic Communities and International Policy Coordination,” International Organization 46 (1), 1–37.


Harvey, D. (2005). A Brief History of Neoliberalism. Oxford: Oxford University Press.


Held, D. (1995). Democracy and the Global Order. New York: Polity Press.


Henry, M., B. Lingard, F. Rizvi, and S. Taylor (2001) The OECD, Globalisation and Education Policy. Amsterdam: Pergamon.


Hopmann, S. T., G. Brinkek, and M. Retzel (2007). PISA According to PISA: Does PISA Keep What it Promises? Vienna: Lit Verlag.


Jakobi, A. P., and K. Martens.  (2010). “Introduction: The OECD as an actor in international politics.” In Mechanisms of OECD Governance: International Incentives for National Policy-Making?, edited by K. Martens and A. P. Jakobi, 163–179. Oxford: Oxford University Press.


Kamens, David (2012). Beyond the Nation State: The Reconstruction of Nationhood and Citizenship. London: Emerald Publishing.


Koretz, D. (2008). Measuring Up. What Educational Tests Really Tell Us. Cambridge, MA: Harvard University Press.


Lane, J.-E. (2000). New Public Management: An Introduction. London: Routledge.


Lawn, M. (2006). “Soft Governance and the Learning Spaces of Europe.” Comparative European Politics 4 (2-3): 272–288.


Lingard, B. (2011). “Policy as Numbers: Accounting for Educational Research.” The Australian Educational Researcher 38 (4): 355–382.


Lingard, B., and S. Rawolle (2011). “New Scalar Politics: Implications for Education Policy,” Comparative Education 47(4): 489–502.


Loveless, T. (2013). International Tests Are Not All the Same. Washington, DC: Brookings Institute.http://www.brookings.edu/blogs/brown-center-chalkboard/posts/2013/01/09-timss-pisa-loveless


Mahon, R. and S. McBride, eds. (2008). The OECD and Transnational Governance. Vancouver: University of British Columbia Press.


Mahon, R., and S. McBride (2008a).  “Introduction.” In The OECD and Transnational Governance, edited by R. Mahon and S. McBride, 3–24. Vancouver: University of British Columbia Press.


Martens, K., and A. J. Jakobi (2010). “Expanding and Intensifying Governance: The OECD in Education Policy.” In Mechanisms of OECD Governance: International Incentives for National Policy-making?, edited by K. Martens and A. P. Jakobi, 163–179. Oxford: Oxford University Press.


Martens, K., and D. Niemann (2010). “Governance by Comparison. How Ratings and Rankings Impact National Policy-Making in Education.” Transtate Working Papers 139. Bremen, Germany: University of Bremen. http://www.staatlichkeit.uni-bremen.de.


McGregor, D. (1960/2000). The Human Side of Enterprise. New York: McGraw-Hill.


Meyer, H.-D., and A. Benavot, eds. (2013). PISA, Power, and Policy. The Emergence of Global Educational Governance. Oxford, UK: Symposium.


Meyer, H.-D., and B. Rowan, eds. (2006) The New Institutionalism in Education. Albany, NY: SUNY Press.


Meyer, H.-D., and K. Schiller (2013). “Gauging the Role of Non-Educational Effects in Large-Scale Assessments: Socio-economics, Culture, and PISA Outcomes.” In PISA, Power, and Policy. The Emergence of Global Educational Governance, edited by H.-D. Meyer and A. Benavot, 207–224. Oxford, UK: Symposium.


Meyer, J. W. (2009). World Society: The Writings of John W. Meyer, edited by G. Drori and G. Krucken. Oxford: Oxford University Press.


Mourshed, M., C. Chinezi, and M. Barber. (2010). “How the World’s Most Improved School Systems Keep Getting Better.” McKinsey Report, November.


Nisbet, J. (1994). “Relating Pupil Assessment and Evaluation to Teaching and Learning.” In The Curriculum Redefined: Schooling for the 21st Century. OECD: CERI, Paris.


OECD. (1995) Governance in Transition - Public Management Reforms in OECD Countries. Paris: OECD.


OECD (2011). “OECD-Mexico Agreement to Improve the Quality of Education in Mexican Schools.” Update No. 5 (Final): February, http://www.oecd.org/edu/school/calidadeducativaqualityeducation.htm.


OECD (2013). Education Policy Implementation: Mexico, http://www.oecd.org/edu/calidadeducativa.


Osborne, D. E., and T. Gaebler. (1993)/ Reinventing Government - How the Entrepreneurial Spirit Is Transforming the Public Sector. New York: Penguin.


Pal, L. A. (2008). “Inversions without End: The OECD and Global Public Management Reform.” In The OECD and Transnational Governance, edited by R. Mahon and S. McBride, 60–76. Vancouver: University of  British Columbia Press.


Pearson. (2012). New Pearson Investment Fund to Enhance Education Opportunity among the World’s Poorest, http://www.pearson.com/news/2012/july/new-pearson-investment-fund-to-enhance-education-opportunity-amo.html.


Porter, T. and M. Webb (2008). “Role of OECD in the Orchestration of Global Knowledge Networks,” in The OECD and Transnational Governance, edited by R. Mahon and S. McBride, 43-59. Vancouver: University of British Columbia Press.


Power, M. (1999). The Audit Society: Rituals of Verification. 2nd edition. Oxford: Oxford University Press.


Prais, S. J. (2003). “Cautions on OECD’s Recent Educational Survey (PISA).” Oxford Review of Education 29 (2): 139–163.


Pratt J. W. and R. J. Zeckhauser (1985). Principals and Agents: The Structure of Business. Boston: Harvard Business School Press.


Ramirez, F. O., X. Luo, E. Schofer, and J. W. Meyer (2006). “Student Achievement and National Economic Growth.” American Journal of Education 113 (1): 1–31.


Rizvi, F. and B. Lingard (2010). Globalizing Education Policy. London: Routledge.


Rubenson, K. (2008) OECD Education Policies and World Hegemony. In The OECD and Transnational Governance, edited by R. Mahon and S. McBride, 242-259. Vancouver: University of British Columbia Press.


Schleicher, A. (2007). “Can Competencies Assessed by PISA be Considered the Fundamental School Knowledge 15-year-olds Should Possess?” Journal of Educational Change 8 (4): 349–357.


Sellar, S., and B. Lingard (2013). ”Who Succeeds at PISA and Why? The Role of International Benchmarking in Emerging Global Governance.” In PISA, Power, and Policy. The Emergence of Global Educational Governance, edited by Heinz-Dieter Meyer and Aaron Benavot, 185–224. Oxford, UK: Symposium.


Skilbeck, M. (1990). Curriculum Reform. An Overview of Trends. Paris: Centre for Educational Research and Innovation (CERI), OECD.


Spiegel. (2007). Doppelporträt: Mr. Pisa gegen Mr. Pisa, http://www.spiegel.de/schulspiegel/wissen/doppelportraet-mr-pisa-gegen-mr-pisa-a-521067.html


Steiner-Khamsi, G. (2003). “The Politics of League Tables.” Journal of Social Science 1: 1–6.


Strange, S. (1996) The Retreat of the State: The Diffusion of Power in the World Economy. Cambridge: Cambridge University Press.


Torrance, H. (2006). “Globalizing Empiricism: What, if Anything, Can be Learned from International Comparisons of Educational Achievement.” In Education, Globalization and Social Change, edited by H. Lauder, P. Brown, J.-A. Dillabough, and A. H. Halsey, 824–834. Oxford: Oxford University Press.


Tyack, D. (1974). The One Best System. Cambridge, MA: Harvard University Press.


UNESCO (1996). Our Creative Diversity: Report of the World Commission on Culture and Development. UNESCO: Paris.


US EDI (2011). Delivery Approach, http://www.deliveryinstitute.org/delivery-approach.


Walzer, M. (1995). Toward a Global Civil Society. Providence, RI: Berghahn.


Wilby, P. (2011). “Mad Professor Goes Global.” The Guardian. June 14.


Wolfe, R. (2008). “From Reconstructing Europe to Constructing Globalization: The OECD in Historical Perspective.” In The OECD and Transnational Governance, edited by R. Mahon and S. McBride, 25-42, Vancouver: University of British Columbia Press.




Cite This Article as: Teachers College Record Volume 116 Number 9, 2014, p. 1-20
https://www.tcrecord.org ID Number: 17543, Date Accessed: 12/4/2021 12:32:25 AM

Purchase Reprint Rights for this article or review