Assessment, Equity, and Opportunity to Learn

Assessment, Equity, and Opportunity to Learn is a collection of research essays on educational assessment. It is the result of a Spencer Foundation project to bring together scholars from a variety of disciplines to challenge and expand assessment. The team begins by asking what assessments tell us about learning and teaching. They agree that current assessments "have not been validated" as reports of state standards, or even if those state standards match what students need to become productive citizens (pp. viii-ix). The goal of assessment, they contend, should be to track students' progress and teachers' efforts in that progress so that we can contribute to what we know about teaching and learning.

The first chapter is essential for those new to assessment theory. As a nation, we have employed a test-based accountability to enhance opportunity to learn (OTL). We test students because we believe knowledge resides inside one's head. The theory in support of this perspective might be called psychometric, although like any theory there are many variants. Divergent from this is sociocultural theory, also called situated cognition. The SC perspective contends that knowledge is social, that learning relies on social interaction within a system and is dependent on the relationship that a person has within an environment. While the psychometric perspective is explained, the authors make it clear that the book is informed rather by the SC perspective.  

In chapter two, Diana C. Pullin and Edward H. Haertel take us through the historic movements, from IQ testing to NCLB. The chapter reveals how assessment testing affects OTL by determining resource distribution, by changing classroom content, and by influencing the teaching and learning process. We carry-on with testing as if it were purely objective, even as the weight of evidence shows us otherwise; and we test with the good intention of trying to improve OTL for all students. Sociological research, however, has changed the way we think about OTL. The conventional notion of OTL is contrasted against this perspective, which is developed through the rest of the volume.

Hugh Mehan, in chapter three, contrasts the view that the school system is meritocratic against the view that the school system reproduces social inequalities. The chapter includes a discussion of social and cultural capital as contributing to educational success. Children of the elite acquire the necessary cultural knowledge to reach success in school, and many profit from social connections.  However, peer groups, parents and students' perceptions contribute to the processes, revealing that the theories of merit or reproduction are overly deterministic. Social and cultural mediation, like the kind Mehan discusses, can improve OTL for the underprivileged.

There is no reason to deny that knowledge can exist in someone's head, but this is a dominant view that drives standardized testing. In order to improve assessment, says James Paul Gee in the next chapter, we need to realize that knowledge and learning often occur in our interactions with others and in our environments. Our ability to perceive, our use of "mediating devices" like tools and our interactions with people, including our facility with varieties of language, all help us learn (p. 88). Gee's essay is itself a mediating device. It is filled with concrete examples that aid the reader in understanding the contributions of the SC perspective.

For chapter five, Diana C. Pullin provides an overview of special education in the United States. Although there are current problems with identifying students with disabilities, and in providing them the opportunity to learn, the system does a number of things that might improve assessment and OTL for all students. Special education programs are individualized, teachers work collaboratively, parental involvement is required, and when done correctly, assessment is tied to instruction.

Carol Lee ties assessment to instruction while reconceptualizing both. She describes the Cultural Modeling Project that is a framework of classroom instruction linking subject matter to everyday experience. She provides examples in literature and science education that reveal how the use of "cultural data sets" (Rap music, African drum) will improve the opportunity to learn for underprivileged students (p. 144). The strategies discussed have positive implications for all students, as well as for professional development.

While Mehan and Gee explain the theory in their chapters, James G. Greeno and Melissa S. Gresalfi extend the SC perspective by describing learning as a "trajectory" (p. 171). The learning trajectory of an individual is based upon his or her level of participation, while the learning trajectory of a group involves changing group practices. This trajectory is also dependent on "affordances," which include resources, access to those resources, and the abilities of students (p. 172). In addition, participation is subject to "agency" (p. 179). When students determine an outcome by using an established procedure this is disciplinary agency. Conceptual agency, however, enables students to think, interact, and make choices about how best to solve a problem. Both are valuable, but disciplinary agency is more common even though conceptual agency better supports OTL because it nurtures participation (p. 180). The authors offer examples from mathematics classes.  

James Paul Gee returns in chapter eight to test the theory that the most effective understanding is situated, as opposed to verbal. A situated understanding enables the learner to apply knowledge to actual situations whereas a verbal understanding may only enable the learner to pass a test. The difference between these levels of understanding might best be represented in the adage, "Tell me and I forget, show me and I remember, involve me and I understand." Gee provides examples of situated understandings through the use of computers or computer games as part of a "well-designed learning system." (pp. 207, 217-218). He also explains how some commercial video games provide organized assessments so players can see how they are doing, even though in many cases, assessment is the game itself. This has implications for educational assessment, but it raises a question: "Why, then, would we need any assessment apart from the game itself?" (p. 215). His answer is that the player (Janie) "needs a formative or developmental assessment that can help her theorize her play and change it for the better, and this is what the game gives her." (p. 215) In a thirteen-point list, Gee outlines the assessment offered to players of the game Rise of Nations. He describes what Janie will see, but the reader wonders if Janie could tell us. If learning requires activity and experience, and if the best understanding of a concept is situated, why are "charts and graphs" considered effective assessment for Janie or for students (p. 216)? If we want students to learn and understand how they are doing, shouldn't we be assessing in the same situated, sociocultural manner as we are teaching?

Pamela A. Moss answers this last question in chapter nine. Her essay provides the language for the kinds of assessment that might, arguably, be most important. When we use the word "assessment," we are thinking of documentary assessment. And yet, it is largely the "inherent" and the "discursive" forms that get conveyed and understood most often by students and teachers alike. Through the work of anthropologists Jordan and Putz (2004), Moss defines inherent assessment "as happening informally and nonverbally in all social situations," and discursive assessment "as occurring when members of a social group talk about what they are doing in an evaluative way," (p. 225). She argues, "that the use of evidence to address questions or problems -- to support interpretations, decisions, and actions -- is an ongoing aspect of interaction (whether formally designated as assessment or not)" (p. 227). Building on Yrjo Engestrom's work (2001), Moss provides an effective summary of the kinds of questions and criteria we should be considering when assessing the quality of learning and OTL. She analyzes a fifth-grade mathematics classroom to reveal how an approach to assessment must be more flexible, and must realize that "not all assessments should or can be subjected to an explicit documentation -- in fact, most assessments are of this variety" (p. 254).   

Most of the book is informed by the SC perspective, and I was therefore interested to read Robert J. Mislevy, Professor of Statistics, who I believed might be a dissenting voice. However, he's not so much the contrarian as he is a unifier. Mislevy recognizes the value in the SC perspective, and he contends that his "evidence centered" assessment design (ECD) can be used effectively in integrated or SC designs (p. 260). New assessment designs make use of multiple perspectives, and these new designs -- along with advances in technology and methodology -- make it possible to "scale-up" or do large-scale assessments that create data we can use to improve OTL.

In chapter eleven, Pamela A. Moss, Brian J. Girard, and James G. Greeno create a table of questions for analyzing documentary assessments, then they analyze four examples of documentary assessment that range from an externally mandated framework to an internal or more local framework. Some interesting implications for externally mandated assessments are discussed, particularly the notion of what Jordan and Putz (2004) call "work arounds," which serve to improve data without improving learning. The authors recognize the importance of compiling evidence that can be compared on a larger scale, but they find the most effective ways of doing this in the assessment systems that are controlled locally.

Finally, Diana C. Pullin reviews the previous chapters and offers a summation of the authors' analogous ideas. The book is meant to challenge the paradigm of psychometrics in assessment in the hope that we can begin to realize that "measuring outcomes is important, but that the way achievement is assessed is also important. . . . [to insure that] all students, particularly those most at risk of educational failure, are beneficiaries of an effective opportunity to learn meaningful content" (p. 334). The book meets its objectives, and its success, among other things, is certainly the result of its multidisciplinary voices. This is the direction assessment is going, and instructors should include this book in any graduate seminar covering educational assessment.