Home Articles Reader Opinion Editorial Book Reviews Discussion Writers Guide About TCRecord
transparent 13
Topics
Discussion
Announcements
 

The Future of Evaluation in Society: A Tribute to Michael Scriven


reviewed by Cara Jackson - August 22, 2014

coverTitle: The Future of Evaluation in Society: A Tribute to Michael Scriven
Author(s): Stewart I. Donaldson (Ed.)
Publisher: Information Age Publishing, Charlotte
ISBN: 1623964512, Pages: 200, Year: 2013
Search for book at Amazon.com


This tribute to Michael Scriven grew out of the 2011 Staufer Symposium at the Claremont Colleges and aims to address both his considerable influence on the field of evaluation and to delineate future directions for the field. Though it lacks an organizational framework to integrate the various chapters, the volume largely succeeds in providing a rich description of Scriven’s intellectual and philosophical contributions to evaluation as well as useful recommendations for evaluators.


Rodney Hopson opens with an adaptation of Gil Scott-Heron’s The Revolution Will Not Be Televised; The Revolution Will Not Be Evaluated is a creative ode to ideas put forth by Scriven. The volume’s editor, Stewart Donaldson, provides an overview of Scriven’s many notable contributions to the field of evaluation, and briefly describes each of the subsequent chapters, in which the authors both reflect on Scriven’s work and offer their own perspectives on evaluation’s future.


Scriven’s own chapter explores the history of evaluation, including the rejection of evaluation from scientific legitimacy and the reformulation of evaluation as useful and defensible. He defines key terms commonly used in evaluation, and distinguishes between formative and summative evaluation. Scriven organizes his historical overview of evaluation around various paradigm shifts. The first paradigm shift involved the acceptance of “the doctrine of value-free science” and subsequent ban of evaluation from scientific legitimacy by mainstream social sciences. Scriven notes the parallels between this period and recent efforts to denigrate qualitative as well as non-experimental quantitative approaches. In the second paradigm shift, educational researchers started thinking systematically about evaluation and came to the conclusion that objective evaluations are feasible, usable, and defensible. The third paradigm shift frames evaluation as a transdiscipline, one of several disciplines that provide important tools to other disciplines.  Scriven foresees fourth and fifth paradigm shifts that would move evaluation as a discipline into alpha status with ethics as the alpha value.


Deftly integrating personal anecdotes into a chapter that provides useful recommendations for evaluators, Michael Patton provides a top ten list of future directions for evaluation. The list includes the continued importance and difficulty of speaking truth to power, with examples of Scriven’s formidable leadership in this regard. Patton stipulates that utility ought take precedence over accuracy among evaluation’s standards of excellence, and in keeping with the theme of utility, advocates for better timing of evaluation findings and real-time feedback.


Chapter Four by Ernest House starts with an overview of conflicts of interest, using pharmaceutical studies and the great financial crisis as examples. In a section on strategies to prevent bias and conflicts of interest from tainting evaluation, House highlights Scriven’s three principles of organizational bias control and suggests how these might be applied in the aforementioned examples. House concludes that the pervasive conflicts of interest throughout society support Scriven’s conception of evaluation as a transdiscipline.


In a play on Scriven’s own focus on checklists, Daniel Stufflebeam provides readers a checklist of twelve key contributions Scriven made to the field of evaluation, including, of course, checklists. Stufflebeam provides numerous references for further reading for those interested in delving more deeply into Scriven’s work. In the second part of his chapter, Stufflebeam focuses on the utility and impact of evaluation, and ends with nine steps to increase the impact of evaluation.


Christina Christie’s Chapter Six focuses on Scriven’s contribution to the evaluation lexicon. She notes original terminology coined by Scriven, his Evaluation Thesaurus (1991), and his popularization of Cronbach’s notion of formative and summative evaluation. In Chapter Seven, Robert Stake takes the field of evaluation to task, arguing that evaluation is seldom done as it should be and that far more attention ought to be paid to the quality of evaluations currently being conducted. He foresees the future as being a replication of the present in the absence of identifying bad practice.


In Chapter Eight, Jennifer Green concurs with Scriven in emphasizing the importance of keeping the focus of evaluation on the needs of consumers. She identifies intended beneficiaries as evaluation’s most important audience, and notes that consumer needs do not always coincide with the program objectives.  Greene and Scriven’s views diverge around the micro role of evaluators and the nature of relationships between evaluators and stakeholders. Greene argues in favor of generating relational trust, contending that technical rationality cannot overcome the fallibility of human judgment, though her argument does not entirely address Scriven’s concern that friendly relationships might undermine the evaluators’ ability to be forthright about the less positive results of an evaluation.


Kirkhart’s Chapter Nine is an ode to the Key Evaluation Checklist, and describes its application to multicultural validity. She offers a “culture checklist” as a device to make the multicultural validity framework accessible, support the work of evaluators, and expand consciousness. The culture checklist includes items that must be checked to avoid invalidity in the evaluation.


Melvin Mark closes out with astute observations on the ties that bind the chapters, including the observation that each of the author’s visions of the future of evaluation are largely variations on themes from their own past writing. Some chapters are primarily a summarization of the author’s own work and how that work relates to Scriven’s ideas. Other chapters are more thoroughly researched and offer a broader perspective on the field of evaluation as a whole, or provide more immediately applicable recommendations for future evaluations.


The various contributions to this volume cover a lot of ground around Scriven’s considerable contributions to the field of evaluation. One of the volume’s strengths is the inclusion of divergent viewpoints; several authors comment on areas of dissent with Scriven. Despite the limited attention given to organizing the chapters around common threads among different authors, students and scholars should find book helpful as a starting point for identifying strategies to strengthen the quality of future evaluations. Many of the chapters would be appropriate for use in evaluation courses and as a working guide for evaluation practitioners.


Reference


Scriven, M. (1991). Evaluation thesaurus (4th ed.).  Newbury Park, CA: Sage.




Cite This Article as: Teachers College Record, Date Published: August 22, 2014
https://www.tcrecord.org ID Number: 17659, Date Accessed: 5/25/2022 1:06:00 PM

Purchase Reprint Rights for this article or review
 
Article Tools
Related Articles

Related Discussion
 
Post a Comment | Read All

About the Author
  • Cara Jackson
    Urban Teacher Center
    E-mail Author
    CARA JACKSON Is the Assistant Director of Accountability at the Urban Teacher Center. Her current work focuses on the design and implementation of teacher evaluation systems. She has co-authored articles on educator performance pay and schools’ efforts to engage parents, and recently completed her dissertation, in which she explored the relationship between teachers’ working conditions and teacher effectiveness.
 
Member Center
In Print
This Month's Issue

Submit
EMAIL

Twitter

RSS