When Research Matters: How Scholarship Influences Educational Policy

As an educator invested in infusing the measurement and improvement of school climate as well as social-emotional-ethical-intellectual learning policy and practice guidelines into K-12 school life, I have always thought that research should shape policy. I have long known that little in life is simply rational, (I also work as a child and adult psychologist/psychoanalyst). This edited book, When Research Matters: How Scholarship Influences Educational Policy, “unpacks” and helps to reveal the myriad of factors that shape the complicated inter-relationship between educational research and policy. The contributors to this volume help illuminate how the needs of policy makers shape policy in ways that are not simply aligned to the systematic study of a given educational issue. They also focus on how the needs of researchers shape what and how they study given educational issues.

This is an important book for educators and policy makers who care about our K-12 schools and the research-policy nexus. This collection of papers grows out of a 2006 Brookings Institute conference. At the conference a number of the participants gave voice to a sense of frustration that research was being ignored by federal and state policy makers as well as advocacy groups and influential foundation officers. The editor, Frederick M. Hess, has brought together a group of knowledgeable contributors to delineate their understandings of how scholarship influences education policy. Many of the contributors have been active agents in national policy-practice debates and developments. As a result, the book is filled with meaningful and detailed examples of how research influences educational policy.

A number of the early chapters present a fascinating and helpful historical context beginning in the 1960s and progressing through to the present. Many of the chapters underscore that the rigor and quality of educational research has been increasing. There are many fascinating themes that are raised in this volume. Here I want to highlight two major questions the book addresses. What are some of the factors that limit the impact of research on policy? And, how have technology and the growth of think tanks and advocacy groups shaped how research influences policy?

One of the major points that many of the chapters underscore, is that the impact of research on policy is importantly a function of policy makers/advisors needing research findings that support their pre-existing policy positions. In other words, research findings most often influence policy makers who are pre-disposed to believe a given set of results.

The volume also underscores how timing matters. When a legislative proposal is being developed, policy advisors need research findings to “back” their perspective. Educational research on the other hand is necessarily a difficult and time-consuming process. This gap contributes to research not influencing policy.

The volume raises a series of overlapping questions about the nature of educational research and what “counts” as research. Recently, the Institute of Educational Science (IES) has been a loud and important advocate for “rigorous scientific standards.” Educational research that “counts” today is random assignment controlled experimental methodologies. Authors in this volume raise the question, “should rigor or relevance be the goal?” Many suggest that IES clearly favors the former. Yet, most would agree that not all good research is experimental or quantitative. In fact, some of the most important questions simply cannot be addressed with experimental methods. There are so many inter-related variables that color and shape learning.  The small (classroom) and large group (schools, districts and the larger educational community) dynamics that shape learning and teaching are inherently difficult for making the kind of causal statements which policy makers crave. In addition, students and teachers cannot be sent randomly to different schools or classrooms or intentionally given less education or a lower quality education.

There are major problems associated with IES’s decision to apply the medical model of research to education. The medical model relies on randomized field trails, in which therapies (e.g., pharmacological) are administered to patients under an explicit protocol. This is a very powerful research method but it is also very limited in its application. As Hess states in the book’s concluding chapter, “reforms relating to governance, management, compensation, deregulation, and other innovations intended to improve organizational effectiveness are rarely precise, do not take place in controlled circumstances, and are not administered to discrete subjects” (p. 241). Experimental research may be able to illuminate how such reforms work and how their impact is affected – to some extent – by context. However, it will not be able to tell policy makers whether a particular policy is “correct.” There are many essential aspects of teaching and learning that it will never illuminate. Here is just one example: educational research does not and cannot solely deal with facts. Facts can be studied, but education is inherently colored by values. One of the authors quotes Howard Gardner (2002, A23) saying, “How could we ever design an educational system that would please Jesse Helms, Jesse Jackson, and Jesse Ventura? We can’t conduct meaningful scientific research on education practices unless we articulate a value system with some specificity.” Educational researchers have to grapple with value systems that shape educational understandings, goals and methods.

Another important topic that this volume highlights is how current educational data sets contribute to and reinforce academics focusing on particular research questions and not others. The development of major federal data sets has a major impact on what is studied and hence what is discovered. Availability of data and the structure of the academic tenure system help to explain what questions researchers choose to address and the methods they utilize. When we look at student learning for example, everyone knows that teachers are not just promoting linguistic and mathematical learning. Whether teachers want to or not, they are always social, emotional, civic and intellectual teachers. The only question is whether they are intentionally and helpfully promoting the social, emotional, and civic skills and dispositions that provide the foundation for school – and life – success or not. Yet, we tend not to study these issues because they are not measured and hence “do not count.”

Another important point that many of the contributors to this volume address is how technology and the growth of think tanks and advocacy groups have changed the manner in which research is being disseminated. New technologies have allowed researchers to publish their work in new ways. Researchers who want to influence policy are increasingly involved with rapidly distributing results rather than adhering to a more traditional – and time consuming – process of peer review. For example, only three of the thirteen studies that the Editorial Projects in Education identified in 2006 as the most influential of the previous decade were released through referred scholarly journals.

This volume also highlights the importance of advocacy groups and journalists who communicate research findings to the public and policy makers. Hess calls these groups “intermediaries.” A number of the authors show how the intermediaries’ agendas cause certain research to be  highlighted and, therefore, to become influential.

In conclusion, research can and has helped to shape policy. However, research is used selectively. This volume highlights the incentives that shape research, research synthesizers and policy maker actions that in turn help them to advance their careers, ideas and preferred solutions. Researchers tend to focus on work that their peers find to be compelling contributions to knowledge. Synthesizers tend to develop and use research to promote their organizational missions. Policy makers often use research to promote their preexisting points of view. As an educator invested in shaping state-level policy, I believe this book helps to illuminate these dynamics. As a clinician, these findings are intriguing… but not surprising. Conscious and unrecognized social-emotional and intellectual needs always shape behavior. But another truth that the authors and editor make, explicitly and implicitly, is that this is democracy in action. More often than not, research has not provided simple answers for policy makers. Interestingly, this book shows how research has often changed the way in which we understand the questions policy makers are grappling with.

Reference

Gardner, H. (2002, December 20). You can’t test lessons the way you do meds. Newsday, A23.