The Untested Accusation: Principals, Research Knowledge, and Policy Making in Schools

Educational research has taken a beating in the last few decades due to scathing critiques from academic circles, the media, political leaders, and even educators themselves.  To quote Bruce Biddle and Lawrence Saha in their latest book The Untested Accusation: “These judgments suggested that educational research was far inferior to research in most other fields – that its methods were uniquely weak, its contributions were vacuous, and its impact on potential users was minimal to nonexistent” (p. 2).  In their opening chapters, the two authors openly suggest that the results of this ongoing attack have been nothing short of detrimental for the entire education community – slashed funding for research, the discouragement of capable scholars entering the field, and a crippling of its reputation in general.  However, when the authors each endeavoured to undertake a meta-analysis of empirical data showing the low impact of educational research, they found that no real evidence existed to reinforce or dispute these claims.  Inevitably, they asked themselves if this debate could be more over ideologically-charged hype, and less about empirical truths.  Obviously distressed over these seemingly unfounded slurs against the education profession, Biddle and Saha have fought back.  And they have done so on their own terms - not through aspersions or grandstanding, but through dogged research.  In doing so, they have shown educational research advocates a blueprint for challenging this widespread but “untested accusation.”

Originally submitted as a report to the U.S. Office of Educational Research and Improvement in 1993, this book evidences the roots of this genre throughout.  Rather than trying to overextend their study into the entire educational community, the authors limited themselves to interviewing 120 school principals in the United States and Australia.  As well, they began with clear-cut questions to be answered:  how do principals acquire, evaluate and think about knowledge derived from educational research, and how do they use that knowledge to influence policies and practices in their schools?  Their ensuing research design, set up to answer these questions, can be taken as a paragon of transparency.  In fact, I would recommend that any researcher undertaking a similar feat read this study as a prototype.  Chapter after chapter reveals the minute details of how they came to their methodology, samples, and analysis procedure.  Charts and tables abound to give readers further insight and validity to their conclusions.  Extensive appendices show the reader the interview schedules and questionnaires.  In doing so, they accomplish an excellent job of wringing all possible information out of the data provided to them.

So logical and clear are the authors’ method and analysis style, in fact, that when the conclusions are displayed there are no gasps of disbelief from the reader.  Rather, the final recommendations seem like natural extensions of the research itself.  In the end, Biddle and Saha can safely state that a majority of principals in both countries were regularly exposed to research knowledge (albeit secondary forms of research like professional journals), that most had a generally positive but discerning eye for this research, and that most concerned themselves with the more practical applications of research than in generating new theoretical frameworks (as would be expected).  Many interviewees even said that they use research in their decision making, and in influencing colleagues, parents and teachers.  

The authors appear very guarded in making generalizations from this data and, in fact, include findings that seem rather obvious (for example, “principals who read professional books more often favor the use of research knowledge…” [p. 234]).  However, this seems to be part of their subtle (but all the more powerful) critique of those who would allow their findings to overrun the data and shout out shrill, monolithic pronouncements (such as “principals don’t use research”).   Indeed, the greatest pleasure of this book for me was the easy, professional demeanor reflected in the authors’ writing style.  Rather than the “hurry up before it’s too late” tone that many educational tracts have used these days, this book’s straightforward manner gives all the more strength to its rather devastating attack.  This is not limited to the critics mentioned above (although they get their share).  Blame is also placed on the indiscriminate dissemination methods utilized by educational research bodies such as the Education Resources Information Center (ERIC).  In the end, the authors conclude that while a free-market dissemination system is perfectly acceptable, to avoid misrepresentation it must go hand-in-hand with better education for those who read and report on the data.  Although these are strong words, the reader feels more validated than affronted by these statements.  

While it can be said that most of this book’s recommendations are as true today as in the previous Bush era (when the authors performed the data collection), this should certainly not be seen as the “last word” on the subject - nor do I believe that the authors wish this to be.  While human nature may remain the same, other facets of educational life have changed dramatically.  For one thing, computers have given principals access to a great deal more information, creating a potential for an increased knowledge of educational research.  Nevertheless, as foreshadowed by this book, it may have the negative impact of multiplying their contact with un-refereed sources and potential propaganda.   Biddle and Saha also show that older principals (with 20 years experience) were less likely to be interested in research.  Would this phenomenon still hold true today or was this just a facet of administrators trained in the early 1970s?  Do the differences and similarities still exist between the United States and other western countries or have they shifted?  I offer these points as suggestions rather than criticisms of this book, however.  In fact, I believe that these are the exactly the questions that the two authors have spurred us on to answer.  Of course, many people would like to ignore this book completely.  To do otherwise would mean that they would finally have to remain silent or actually engage in the serious labour of empirical research to try to disprove Biddle and Saha’s work.  And it is persistence and dedication that make these two researchers deserving of our praise.  However, as I only represent a secondary source of information on this research study, I would urge the public not to take my word for it – read it yourself!