Democratic Evaluation and Democracy: Exploring the Reality
reviewed by Todd DeStigter — December 07, 2017
Title: Democratic Evaluation and Democracy: Exploring the Reality
Author(s): Donna Podems
Publisher: Information Age Publishing, Charlotte
ISBN: 1681237881, Pages: 256, Year: 2017
Search for book at Amazon.com
Anyone who has stood in line for two hours at the Department of Motor Vehicles will appreciate this volume's call for greater efficiency in government services. This books authors, however, are concerned with matters beyond mere convenience, using the case of South Africa to explore the role that evaluation can play in strengthening democracy. The contributors agree that in a complex, modern state, especially one struggling to overcome Apartheids legacy of segregation and inequality, evaluation plays an indispensable role in holding government accountable and in providing information to guide just policy-making. In making this argument, these authors establish a vocational solidarity with any teacher who sees her work as fostering democracy by preparing a knowledgeable and engaged citizenry.
The authors constitute a veritable "whos who" of expert evaluators from South Africas government and NGOs, and Donna Podems has thoughtfully arranged their essays to introduce readers to the field of evaluation before delving into the South African case. The first three chapters provide a conceptual framework for democratic evaluation by, among other things, establishing that democracy must go beyond suffrage to include people's meaningful participation in decisions that affect their lives, and by describing entry points for supporting policies that address the needs of marginalized people. These initial chapters also introduce an idea that will become a leitmotif throughout the book; namely, that a democratic society is advanced by evaluation practices that are themselves democratic in that they are conducted in the spirit of working not for people but with them (p. 38). The next seven chapters provide grounded examples of the possibilities and challenges of evaluation in an emerging democracy. These include descriptions of a civil society organization within South Africas evaluation network and the governments evolving efforts to adopt an evidence-based approach to policy-making. Subsequent chapters provide detailed explanations of evaluation practices in the nations Department of Human Settlements and education system, along with an account of a provincial agriculture departments efforts to accommodate national evaluation guidelines. The last three chapters question whether evaluation in South Africa is sufficiently effective and inclusive, explore conceptual connections among the preceding chapters, and suggest how the South African case can provide insights to guide democratic evaluation elsewhere.
Its not always obvious how American educators might benefit from granular descriptions of the administrative structure of organizations referred to with unwieldy initialisms like SAMEA and GWM&ES. What is considerably more obvious, however, is the value of teachers contemplating the implications of one of this books foundational concepts: that any evaluation practice is not a neutral technology, but a political activity that inevitably supports certain values and interests (p. 41). If readers accept this premise, that to evaluate anything is to exercise power, then the questions raised by this book emerge as ambitious ones about how to deploy such power responsibly and ethically, or, conversely, how to mitigate its potential for abuse.
The authors address these questions by saying that we should get better at evaluation, and that this can be accomplished in one of two ways. The first is to make evaluation more technologically sophisticated and to ensure that everyone complies with its findings. This strategy for improving evaluation is set forth most directly in Chapter Five, "Evaluation and Democratic Governance: The Public Management Perspective," by Fanie Cloete. Cloete insists that to govern efficiently and according to the "rule of law" requires "a systematic evaluation from an evidence-informed perspective (p. 81). Unfortunately, he laments, this approach is still emerging in South Africa because evaluators lack sufficient computer tools and rigorous research methods, and because an infrastructure of literally thousands of competent people would be needed to conduct systematic evaluations and enforce their findings (p. 98). Cloete thus suggests that while evaluation may be a human science, its a science nonetheless, and its got to be done right. Cloete also notes, however, that a further challenge arises when people fail to use better information to reach better decisions and outcomes. This utilization problem" (p. 89) is taken up in Benita Williams and Vanessa Schermans chapter describing South Africas national education measurement tool, the Annual National Assessment (ANA). Although the authors concede that the ANAs technical design and execution . . . has given cause for criticism (p. 140), their primary explanation for the ANAs troubled history is that not enough attention is paid to how the evaluation information is supposed to be used in service of the improvement of education quality at the local level (p. 140). Who isnt paying attention? That is to say, to whom can the ANAs utilization problem be attributed? Williams and Scherman claim that its important not to blame teachers for all that is wrong in the South African educational system [emphasis added] (p. 151). Still, the authors reluctance to blame individual teachers is matched only by their eagerness to blame teachers unions, which are repeatedly referred to in the context of efforts to subvert the ANAs apparently egalitarian aims (pp. 140141, 145, 153, 155). An audience of teachers could be forgiven for taking issue with being characterized as obstructionist for refusing to cede their professional knowledge and responsibilities to the remote authority of a standardized testing regime. That said, in the broader context of this volume, the significance of this chapter is that it echoes Cloetes view that the politics of evaluation can best be managed by applying technical expertise to improve evaluation practices.
A second, related, approach to improving evaluation is to democratize the evaluation process itself by including the perspectives and interests of all stakeholders. Carlisle J. Levine makes precisely this point in Chapter Three when she underscores the importance of building trust, competence, and communication so that all people involved in and affected by an evaluation can participate as equal partners. Similarly, in Chapter Twelve, Dugan Fraser and Patricia J. Rogers contend that South Africa has been slow in addressing its extreme poverty and violence in part because its evaluation policy frameworks lack inclusive dialogue, i.e. sustained engagement with people to discover their real interests, and deliberation, which the authors define as a process of forming judgments that is "grounded in . . . evidence and principles of valid argument" (p. 221). This second approach to making evaluation better, then, is a vision of overcoming ignorance and parochial ambition by taking everyones views into account and arriving at decisions that promote the general welfare.
Together, these two approaches to improving evaluation indicate the authors substantial agreement that the unruly contentiousness of politics is a problem to be solved through evaluation practices featuring technical expertise and/or inclusive conversation. This agreement, however, constitutes an unacknowledged paradox; namely that the authors, in regards to their work as evaluators, emphasize both the inevitability of politics and the need to overcome it.
If this sounds familiar to U.S. educators, its likely because it reflects the thought of one of this countrys best-known proponents of democracy, John Dewey. Deweys conception of the ideal political subject was one who participates in creating a cooperatively organized intelligence, which is achieved by applying the scientific method to the study of all human affairs and then deployed to guide rationally agreed-upon public policies (1927/1988; 1935/2008, p. 83). However, Deweys chief intellectual rival, Reinhold Niebuhr, was deeply skeptical of Deweys reliance on reasoned conversation to solve problems and had no patience for educators and social scientists who failed to recognize that reason and moral suasion would almost always be subordinated to humans predatory self-interest (1932/2013, p. xxx).
Such a conception of the public sphere as an inhospitable site of ceaseless antagonisms is the theoretical backdrop to Chapter Eleven, Terence Beneys "Paying for Troublemaking: Strengthening Democracy by Institutionalizing Multiple Centers of Evaluation, which is this volumes most provocative essay. Beneys point of departure is what he calls the "confrontational character of democracy (p. 193), and his central argument is that independent evaluations must be diffused among a variety of government and civil society institutions in order to impede the concentration of power. In Beneys treatment, evaluation is a site of manipulation and intrigue; a means, if unchecked, by which the ruling class can ignore problems, stymie public discourse, and limit imagined alternatives to the status quo. At the same time, however, Beney insists that dispersing the authority to conduct and act upon evaluations preserves evaluations potential to strengthen the hand of protest and direction action" (p. 192). Beney also infuses a bit of professional modesty into this volume by arguing that "a robust democratic dispensation is contingent upon factors well beyond the sway of evaluation" (p. 204), and the factors he has in mind are mainly economic ones. Thus, while his fellow contributors make the case that democracy can be fostered through proper evaluation, Beney reverses cause and effect by suggesting that, in the absence of broader social equality, the whole notion of democratic evaluation is a fantasy. Niebuhr wrote that in relations between groups, conflict is inevitable, and in this conflict power must be challenged by power (1932/2013, p. xxxi). Beneys chapter is an invigorating outlier in this book because he not only accepts that view but endorses it.
For teachers, the most immediate issue at stake in this book is whether the evaluations we do of our students can promote democracy. We might also use these essays to consider how the familiar discourses of educational standards, data-driven instruction, and accountability either foster or undermine democratic ideals. However, as discussed above, also at stake are our conceptions of democracy itself. Is democracy what Dewey called a mode of associated living (1916/1988, p. 93) characterized by collective inquiry and reasoned deliberation, and should it be? Or is it a site of relentless hostilities among people with incompatible interests?
But even beyond that, this volume invites readers to explore political and ethical dissonances with histories extending at least as far back as the 17th century when the Dutch first arrived on the southern cape of Africa. Those settlers brought with them not only technological advancements but also nascent conceptions of human rights, individual liberty, and the social contract. They brought, in other words, Enlightenment ideas that eventually formed the foundation of liberal democracies. But these new arrivals were not emissaries of freedom, they were colonizers, and evaluation was among the arts of modern statecraft they deployed to render the dark continent legible so that it could be organized, monitored, and ultimately controlled. Framed in this historical context, this book is situated at the nexus of modernity and brings into relief its contradictions. Precisely because our schools and classrooms are institutional embodiments of these contradictions, they demand the attention of anyone who presumes to teach for democracy.
Dewey, J. (1916/1988). Democracy and education. In Jo Ann Boydston (Ed.) John Dewey, the middle works, 18991924 (Vol. 9). Carbondale, IL: Southern Illinois University Press.
Dewey, J. (1927/1988). The public and its problems. In Jo Ann Boydston (Ed.) John Dewey, the later works, 19251927 (Vol. 2). Carbondale, IL: Southern Illinois University Press.
Dewey, J. (1935/2008). Liberalism and social action. Amherst, NY: Prometheus Books.
Niebuhr, R. (1932/2013). Moral man and immoral society: a study in ethics and politics. Louisville, KY: Westminster John Knox Press.