Achieving Accountability in Higher Education: Balancing Public, Academic, and Market Demands

In recent years, numerous books and articles dealing with the effect of the new climate of accountability on K–12 education have appeared, examining topics ranging from high-stakes testing in elementary classrooms (Diamond & Spillane, 2004; Dorgan, 2004) to the effect of the No Child Left Behind Act (Meier & Wood, 2005; Peterson & West, 2004). Implicit in each of these works is a set of questions that frame all discussions of accountability: Who is accountable to whom, for what purposes, for whose benefit, by which means, and with what consequences (Behn, 2001; O’Day, 2002)?

Joseph C. Burke and his associates address these questions and more in Achieving Accountability in Higher Education: Balancing Public, Academic, and Market Demands. A timely, valuable, and extremely informative work, of potential interest to college and university faculty members, state and federal policymakers, and consumers, Achieving Accountability responds to the increasing demands being placed on higher education institutions to demonstrate evidence of student learning and improve performance. The book examines the underlying forces that have contributed to the current policy interest in accountability; presents a framework for conceiving higher education accountability within the context of academic, market, and state and federal concerns; provides expert analysis of the most widely known, most widely used, and most promising instruments available for assessing higher education quality and performance; and offers recommendations for improvement.

Burke has assembled a prestigious list of scholars to analyze the structures and accountability programs described in the book; indeed, the list reads like a Who’s Who of higher education research, policy, and governance. Peter Ewell of the National Center for Higher Education Management Systems brings his substantial experience to bear on the subject of assessment. George Kuh, author of the popular National Survey of Student Engagement (NSSE), discusses the use, both current and potential, of student and alumni surveys. William Massy, President of the Jackson Hole Group, argues that academic audits are the tool best suited to serving both institutional improvement and accountability agendas. Patrick Callan and Joni Finney, whose National Center for Public Policy and Higher Education publishes the biannual Measuring Up report card, discuss the states’ interest in raising the level of educational attainment of the workforce. Robert Zemsky, author (with Susan Shaman) of the College Results Instrument (CRI), points out that markets neither limit nor promote quality in higher education. William Zumeta, Richard Richardson and Thomas Smalling, Ralph Wolff, T. Dary Erwin, Burke, and J. Fredericks Volkwein and Stephen Grunig provide equally valuable discussions of accountability and private higher education; accountability and governance; accreditation; standardized testing; performance reporting, funding, and budgeting; and reputational rankings, respectively.

Burke succinctly describes the need for the book on the first page:

Accountability is the most advocated and least analyzed word in higher education. . . . Clearing up the confusion is critical, because the conflict over accountability is eroding what was once a national consensus—that higher education is a public good for all Americans and not just a private benefit for college graduates.

Tracing the current interest in accountability to a shift in public policy that took place during the 1990s, Burke and others contend that declining state revenues, public skepticism about the efficiency and effectiveness of higher education, and the increasing importance of a college degree in an information-based economy have made postsecondary accountability a focus that is “here to stay.” Zemsky’s likening of policymakers’ interest in college and university performance to the kind of accountability that Consumer Reports exacts from automobile manufacturers provides an apt analogy.

Accordingly, Burke argues that effective accountability systems should respond to the three imperatives that now govern higher education policy: state priorities, academic concerns, and market forces. Conceiving of these forces as the corners of an accountability triangle, Burke asserts that each of the accountability programs discussed in the book, as well as a fully functioning system of accountability, should gravitate toward the center of the triangle and try to avoid the dualistic thinking posed by the tensions between institutional autonomy and external accountability. The accountability triangle concept provides thematic continuity for the book while alerting the reader to the fact that accountability is contextual and that accountability pressures and concerns differ depending on one’s position on the triangle.

Indeed, the accountability triangle demonstrates graphically that the demand for accountability by a public and civic society increasingly concerned about “performance” is at odds with the principal reason academics and academic institutions typically undertake activities like, for example, assessment. In the current accountability climate, as Zumeta observes, institutional performance is associated with factors like costs per student, student retention and completion rates, postgraduate attainments, efficient management, effective price controls, and proof that students are learning. By contrast, Ewell defines higher education assessment as “a program of locally designed and operated evaluation research intended to determine the effects of a college or university on its students, centered on learning outcomes, and engaged in principally for the purpose of improving teaching and learning” (p. 105). Although both the performance and quality agendas driving accountability discourse include a focus on student learning, the former sees learning as an aspect of doing business well, whereas the latter regards evidence of learning outcomes as a means of internal institutional improvement.

Ewell’s remark that “the point is not so much that outcomes are visibly deficient as the fact that no one seems to know what they are” (p. 122) is telling. A theme repeated throughout the book is that accountability purposes cannot be served unless results of various accountability measures are made public. Accountability and publicity are linked: Public disclosure of accreditation actions, student and alumni survey results, and learning outcomes is necessary if policymakers and consumers are to become more informed, exercise influence, and make better decisions about investing their resources. Comparability is nearly as important. If institutions are to be “held accountable” for performance, external audiences need some basis for evaluating how well a college or university is doing in achieving its stated goals.

This presents a serious problem, though. Kuh, Volkwein and Grunig, and Zemsky all point out that although valid, reliable indicators of educational quality and student performance exist, including the College Student Experience Questionnaire, the College Student Survey, the NSSE, and the CRI, their use in quality assurance programs is limited. Kuh lists issues of validity and reliability, confounding purposes, the inability of institutions to translate results into action, fear of embarrassment, and the lack of participation by “medallion” or leading institutions as key factors. The latter point, although less significant for Kuh, Zemsky sees as a predictable outcome of the inability of markets to regulate quality:

This absence of medallion participation should surprise no one. Already enjoying superior market position, these institutions had nothing to gain and potentially a great deal to lose if their outcomes or levels of engagement were no better than those of institutions charging substantially lower prices. Even institutions that did participate insisted, as part of their formal agreement to administer either the CRI or NSSE, that they and they alone could make their results public. It was as if the producers of products or services tested by Consumer Reports could decide, after they knew how well they had scored in the test, whether or not to make the results public (p. 282).

Two related points flow from this one. First, absent institutionally generated comparable data, market forces generate their own: U.S. News & World Report rankings are the most obvious and influential example of this phenomenon. Second, as discussed in illuminating chapter by Volkwein and Grunig, and Zemsky, U.S. News rankings not only correlate positively with market position—that is, with institutional size and resources, which further correlate positively with faculty scholarship and student selectivity—but measure what Zemsky argues that sophisticated parents and students actually want when shopping for a college education: competitive advantage. Evidence that an institution is of high quality, as defined by learning outcomes or superior teaching, is not nearly as important to these consumers as selecting an institution that will provide students with a competitive advantage in terms of prestige and access to remunerative professions. Convincing college and university faculty and staff members who know this to undertake exhaustive analyses of student learning outcomes is a difficult task.

This may be one reason why Burke, in the book’s concluding chapter, recommends that regional accreditation is the approach best suited to connecting the various accountability initiatives discussed in the book. Regional accreditation, though fundamentally an academic enterprise, has in recent years been under increasing pressure from Congress to address public priorities and assure quality by requiring evidence of student learning. Burke argues that accreditation could combine elements of assessment, academic audits, student and alumni surveys, state-by-state report cards, state performance reports, and market responsiveness to create an integrated accountability system. Rather than relying on the academic audits discussed by Massy, which in some sense work precisely because they conceive institutional improvement as a joint academic/public policy goal, rather than a requirement, accreditation is required for participation in federal programs and for state operations. Incorporating the consultative philosophy underlying academic audits into a comprehensive regional accreditation process could improve quality processes while ensuring compliance. And unlike Callan and Finney’s state-by-state report card methodology, which intentionally refrains from comparing individual institutions, some dissemination of the results of accreditation reviews could increase consumer knowledge and benefit decision making while providing policymakers with a basis for allocating resources.

For all of this to work, as Richardson, Callan and Finney, and others observe, some set of individuals or groups must decide and communicate the outcomes for which higher education will be accountable (at least at the state level), how they will be measured, and what performance will be regarded as acceptable. They must do so recognizing that, as Ewell says, assessment must be locally driven to serve both accountability and improvement agendas. Relying on the insights offered in this book will go a long way toward achieving that goal: Achieving Accountability in Higher Education is the best, most comprehensive source book on higher education accountability currently on the market.

References

Behn, R. D. (2001). Rethinking democratic accountability. Washington, DC: Brookings Institution Press.

Diamond, J. B., & Spillane, J. (2004). High-stakes accountability in urban elementary schools: Challenging or reproducing inequality? Teachers College Record, 106(6), 1145–1176.

Dorgan, K. (2004). A year in the life of an elementary school: One school’s experiences in meeting new mathematics standards. Teachers College Record, 106(6), 1203–1228.

Meier, D., & Wood, G. (2005). Many children left behind: How the No Child Left Behind Act is damaging our children and our schools. Boston: Beacon Press.

O’Day, J. A. (2002). Complexity, accountability, and school improvement. Harvard Educational Review, 72(3), 293–329.

Peterson, P. E., & West, M. R. (2003). No child left behind? The politics and practice of school accountability. Washington, DC: Brookings Institution Press.