Making Failure Pay: For-Profit Tutoring, High-Stakes Testing, and Public Schools

No Child Left Behind (NCLB) has been the focus of dozens of recent books. As the most visible federal educational policy effort in this young century, scholars have rushed to explain the law’s strengths and its shortcomings, the theories and ideologies behind it, and its impact on the ground. Receiving comparatively little attention in the NCLB literature, however, has been the growth of the supplemental educational services (SES) sector. As the law stipulates, schools marked as “in need of improvement” for three consecutive years must use federal funds to contract with private tutoring companies—a tremendous boon to an industry that to date has swelled to include roughly 3,000 providers, 63 percent of whom operate on a for-profit basis. But while NCLB’s SES provision now nets tutoring companies roughly $200 million annually, its benefit for students remains unclear.

Jill P. Koyama’s book, Making Failure Pay: For-Profit Tutoring, High-Stakes Testing, and Public Schools, seeks to shine some light on the SES industry in the NCLB era. An ethnography based on Koyama’s three years of research in New York City, the book carefully details the interactions between a for-profit SES provider—United Education, a composite of several providers she studied—and 42 of its New York City partner schools.

The fact that the research base for the book is ethnographic is somewhat surprising. Isn’t this the territory of political scientists, historians, and data-crunchers? Koyama makes a lengthy, and theory-laden, argument for the importance of anthropological methods in policy study. But one might equally make an austerely pragmatic case for ethnographic policy research: the SES provisions of NCLB are unwieldy to regulate and nearly impossible to govern; as a result, processes of implementation and management are constantly being written on the fly by participants on the ground; who better than an anthropologist, then, to figure out what goes into that process and how the actions of particular individuals are shaped?

The book’s central thesis, driven by Koyama’s reliance on actor-network theory, is that actors “construct” their reality through their actions in order to “make sense” of changing circumstances. It’s a valid claim given the evidence. The actors in question—teachers, administrators, SES providers, parents—do, in fact, construct a new reality through their actions. Section 1116(e) of NCLB specifies aspects of SES like program duration, funding mechanisms, and reporting requirements; the rest, as is often the case with top-down policy efforts, is unwritten. Thus, in walking particular routes day-in and day-out, those affected by the law eventually blaze their own trail—the policy-in-use.  

The assertion, however, that actors are engaged in the process of sense-making, is one in need of clarification. The individuals in question, at least as detailed in Koyama’s study, are not engaging in discussion or debate about the nature of NCLB and the uses to which supplemental educational services might be put. Nor are they trying to resolve particular inconsistencies in the law. Rather, they are concerned with issues of compliance and opportunity, and the book is most interesting when it examines the intersection of two questions that actors persistently ask themselves. The first question—what does the SES provision of the law require?—is rooted in NCLB’s mandates, reporting requirements, and accountability mechanisms. The second—how can we use this to our advantage?—springs from the fact that the law’s inherently broad and abstract nature leaves local actors with plenty of wiggle room to pursue their own aims. The result is often adherence to the letter of the law, if not its spirit.

Sense-making is a particularly ironic phrase given the effect that system gaming has on undermining the law’s original intent. SES providers, for instance, frequently create deceptively low pre-test scores and inflated post-test results to prove their efficacy. Thus, rather than being used as pedagogical diagnostics, the tests are an act of performance, satisfying requirements while promoting self-interest. As Koyama writes, “even if all the students of one class correctly answered all the examination questions on, for instance, converting fractions, they would still be taught three lessons on converting fractions as prescribed in the standard United SES curriculum” (p. 149). Teachers, too, game the law, providing students with additional time to finish questions or correct answers on tests, or finding ways to manipulate NCLB scorecards. Even sympathetic educators seeking only to serve their students consistently find themselves considering the law’s loopholes and weaknesses as a way of pursuing their aims.

While some wiggle away from the law, others try to comply. But the story to that end is equally troubling. Rather than promoting substantive change, the law often results only in formalism. New buzz words are adopted and deployed at various levels of the system, though they are often disconnected from any substantive behavior. New roles are created, but they are staffed by the same (often ineffective) personnel. New forms are created, but the result, as one teacher put it, is only “heaping mounds of paper” (p. 113). Even new behaviors ring hollow, with one principal musing that, “it’s really about doing the tests.  We need to do the tests…so that kids and parents see we are serious about this testing stuff” (p. 121).

Koyama’s point about all of this is that attending to school failure “provokes a host of inadvertent, conspicuous, and abstruse consequences, or problems, not the least of which is more failure” (p. 7). And that is certainly true. Yet there is more to this story that should be drawn out and inspected under the microscope. Such analysis would not need to come at the expense of the rich ethnographic detail of the work (though if anything were to be cut, chapter two’s dissertation-like literature review could certainly use a trim). But its absence is a missed opportunity.

Ultimately, Koyama frames the book as a story about how failure is constructed rather than found. It’s an important point that should not be overlooked, especially given the American obsession with failing schools. But why is the language of failure so ubiquitous when polls show that Americans are generally happy with the schools their children attend, and when a body of research indicates that American schools may not, after all, be on a course of long decline? Koyama makes the case that all the gloom and doom is due to the fact that “many actors have been working hard at making it matter” (p. 157). That’s one take. But there is also another explanation that one might draw out of Making Failure Pay. Policy rhetoric, in order to initiate change in our diffuse system, is dependent on the language of crisis. Yet, particularly given the way local actors adapt and adjust reforms, results often fail to match promises. Rather than transforming society, grand undertakings produce incremental and often formalistic results. Call it the tragedy of American educational policy.