Home Articles Reader Opinion Editorial Book Reviews Discussion Writers Guide About TCRecord
transparent 13

Improving Large-Scale Assessment in Education: Theory, Issues, and Practice

reviewed by Joan C. Fingon - June 07, 2013

coverTitle: Improving Large-Scale Assessment in Education: Theory, Issues, and Practice
Author(s): Marielle Simon, Kadriye Ercikan & Michel Rousseau (eds.)
Publisher: Routledge, New York
ISBN: 0415894573, Pages: 320, Year: 2012
Search for book at Amazon.com

The role of large-scale assessment (LSA) in public education has never been greater. Mandated state assessments have been increasing in quantity, content, and performance demands for students and schools worldwide. As school districts are continuing to be held accountable to “high stakes” assessments for all students, one might ask, what are the expected and unexpected consequences of LSA? What will the role of LSA be in the future?  Improving large-scale assessment in education: Theory, issues, and practice, edited and authored by Marielle Simon, Kadriye Ercikan, and Michel Rousseau covers much ground on the central issues and technical requirements in educational measurement as well as describing the practical uses and key components of successful planning, development, and implementation of LSAs.  This multi-authored volume contains an array of world class contributors from Canada, Europe, and the United States who describe a combination of conceptual, methodological, technical, and statistical factors that practitioners, researchers, and policy makers tackle within different LSA contexts. This book would be of interest to graduate level classes on educational assessment in education and psychology, LSA researchers, designers, users, and others interested in the advancement of LSA methods and approaches.

This volume leads off with a preliminary chapter describing large-scale assessment (LSA) as “standardized assessments conducted on a regional, national, or international scale involving large student populations” (p. 1). It presents a brief historical background of LSA and a description of the book’s overall organization. Next, this book is assembled into four parts or themes comprising three or four chapters per part based on their “conceptual, pragmatic, and psychometric nature” (p. xvi). Each chapter outlines key points with descriptions of the theory, issues, and practice related to a LSA component linked respectively to one of the four parts. A brief synopsis of the book’s four parts with selected chapters highlighted from each part follows.

Part I, “Assessment design, development, and delivery” (Chapters Two through Five) describes the procedures or nuts and bolts of planning and organizing all components of LSA before administration. It investigates conceptual frameworks and designs for measurement based on student cognitive-psychological and learning principles and development of background questionnaires to collect contextual data regarding LSAs. It also covers student motivation to increase participation in LSAs and computer-based testing as a viable delivery option for LSAs. One chapter of interest, Ruth Child and Orlena Broomes’, Role and design of background questionnaires in large scale assessments, (Chapter Three) emphasizes the collection of background information on all participants (students, teachers, and family) and the need for multiple measures of academic achievement when planning LSAs. It also points out the necessity to understand students’ attitudes, beliefs, and social identities regarding what they think they are learning in schools and the long term effects on student achievement. Examples of how to write effective questions and how to develop questionnaires that are more valid for serving large student populations are offered. Another chapter noted, authored by Christina Van Barneveld and colleagues, “Student motivation in large-scale assessments” (Chapter Four) focuses on the complexities of identifying observable behaviors and engaging with activities that we think represent student motivation and the use of self-report questionnaires and interviews as measures of student motivation. The authors stress the importance of efforts to examine “ways of motivating students, identifying unmotivated students, and critically assessing the statistical models we use to represent the complex relationships between students and their test items” (p. 60).   

Part II, “Assessing diverse populations” (Chapters Six through Eight) considers the implications of the recent broadened pool of students participating in large-scale assessments. Each chapter aims at the issues and challenges of academic assessment and accommodations for one of three student populations; English language learners (ELLs), students in minority and multiple language contexts, and students with cognitive disabilities. Overall, this section emphasizes the growing challenges due to the comparative nature of LSAs call for “sameness” of administering and score reporting and the interpretation that all scores for all students are assumed comparable. One chapter of mention, Gullermo Solano-Flores and Martha Gustafson’s “Academic assessment of English language learners: A critical, probabilistic, systematic view” (Chapter Six) describes the current limitations and practices in the assessment of the rapidly growing numbers of ELLs and the varied aspects and validity of measuring English proficiency. The authors present examples of test items that could be biased to ELLs and a guide for evaluating ELL large scale assessment systems. It also emphasizes the need to identify better ways at developing fair and valid assessment for ELLs worldwide and the ways we think about assessment systems. Also of interest is, “Score comparability of multiple language versions of assessments within jurisdictions,” (Chapter Seven), written by Kadriye Ercikan, Marion Simon, and Maria Elena Oliveri that describes the increasing global use of multiple language versions in tests and the current practice within Canada and challenges and solutions to ensuring score comparability.

Part III, “Scoring, score reporting, and use of scores” (Chapters Nine through Twelve) starts off explaining LSA scores and performance results of students and inferences based on these results. It also covers standard setting past, present, and future, developing and maintaining LSA on-line test and score reporting, and making value added inferences from LSAs.  Of special interest to teachers is “Making value-added inferences from large-scale assessments” (Chapter Twelve) by Derek Briggs which presents the on-going debate and current issues, challenges, and assumptions of how and what to measure regarding teacher effectiveness through utilizing value-added models (VAMs). The author suggests two different perspectives using the results of VAMs “as descriptive indicators to identify schools or teachers that yield greater or lesser gains than expected” (p. 6). Essentially, Briggs brings home the idea that one should not assume that test scores are the only or best measure of teacher effectiveness of student achievement and that the validity of such tests might themselves be suspect in creating value added models.  

Lastly, Part IV “Psychometric modeling and statistical analysis (Chapters Thirteen through Sixteen), focuses specifically on “complex psychometric and statistical issues researchers and practitioners encounter in LSAs” (p. 6). Collectively, these chapters explore existing and problematic issues such as sampling designs and estimation procedures used in LSA and atypical response patterns and item response theory. It also describes missing data common in mixed methods and quantitative research and measurement and statistical issues with longitudinal data.  One chapter of note, Michel Rosseau’s “Missing data: Issues and treatments” (Chapter Fifteen) focuses on data base consumption on which primary and secondary statistical analyses are conducted noting that, “little is known or being compensated surrounding the potential bias caused by missing data and the validity of results” (p. 260). Part IV differs somewhat from the other sections of this book because it describes more detailed and technical principles of psychometric modeling and statistical analysis that might be more suitable to readers with more experience and knowledge in these areas.

From my viewpoint, there are several distinct features of this book. First, it is presents recent research, exemplary practices, alternatives, and recommendations regarding dynamic issues related to LSAs in different contexts and in different countries. It is a how-to book offering training modules and practice-based cutting edge research. Another interesting feature of this book is that all chapters went through a rigorous double-blind peer review process performed by a variety of LSA international experts. Part II of this book particularly seems relevant and well-timed by highlighting issues related to the inclusion of student accommodations to meet the increasing demands for valid and reliable measures of the performance of ELLs and other marginalized students.

This book is also strategic, using a combination of researchers and measurement practitioners who collaboratively and collectively address the current challenges associated with LSA. For example, while advancements of research and technological alternatives are developed, current practice seems to continue to rely on traditional ways that typically measure products and performance which, in turn, “leads to standardization of the curriculum content, teaching, learning, and student assessment at the classroom level” (p.7).  Another challenge resides in how the future role of LSA may change due to new technological advancements assisting large scale assessment administration, scoring, and the reporting of results.  

In summary, this book delivers a variety of unique features and substantive components associated with LSA. As such, for these reasons and more, this book offers a refreshing, diverse, and international perspective on current practices and research, laying a solid foundation and direction for further discussion and exploration towards achieving long-term solutions for achieving success with LSAs.

Cite This Article as: Teachers College Record, Date Published: June 07, 2013
https://www.tcrecord.org ID Number: 17146, Date Accessed: 1/22/2022 9:19:03 PM

Purchase Reprint Rights for this article or review
Article Tools
Related Articles

Related Discussion
Post a Comment | Read All

About the Author
  • Joan Fingon
    California State University, Los Angeles
    E-mail Author
    JOAN C. FINGON is Professor of Reading and Education at California State University, Los Angeles (CSULA). She teaches curriculum theory in the E.d.D. Educational Leadership Program and reading assessment and diagnostic courses in the graduate reading program at CSULA. Her research interests include literacy assessment and practices in urban classrooms, student motivation in literacy learning, and program evaluation. Her most recent publication includes Learning from culturally and linguistically diverse learners: Using inquiry to inform practice (2012), which she co-edited and authored.
Member Center
In Print
This Month's Issue