Home Articles Reader Opinion Editorial Book Reviews Discussion Writers Guide About TCRecord
transparent 13

The Relationship Between Item Context Characteristics and Student Performance: The Case of the 2006 and 2009 PISA Science Items

by Maria Araceli Ruiz-Primo & Min Li - 2015

Background: A long-standing premise in test design is that contextualizing test items makes them concrete, less demanding, and more conducive to determining whether students can apply or transfer their knowledge.

Purpose: We assert that despite decades of study and experience, much remains to be learned about how to construct effective and fair test items with contexts. Too little is known about how item contexts can be appropriately constructed and used, and even less about the relationship between context characteristics and student performance. The exploratory study presented in this paper seeks to contribute to knowledge about test design and construction by focusing on this gap.

Research Design: We address two key questions: (a) What are the characteristics of contexts used in the PISA science items? and (b) What are the relationships between different context characteristics and student performance? We propose a profiling approach to capture information about six context dimensions: type of context, context role, complexity, resources, level of abstraction, and connectivity. To test the approach empirically we sampled a total of 52 science items from PISA 2006 and 2009. We describe the context characteristics of the items at two levels (named layers): general (testlet context) and specific (item context).

Conclusion: We provide empirical evidence about the relationships of these characteristics with student performance as measured by the international percentage of correct responses. We found that the dimension of context resources (e.g., pictures, drawings, photographs) for general contexts and level of abstractness for specific contexts are associated with student performance.

View Full Text in PDF Format

Cite This Article as: Teachers College Record Volume 117 Number 1, 2015, p. 1-36
https://www.tcrecord.org ID Number: 17728, Date Accessed: 10/23/2021 8:25:17 PM

Purchase Reprint Rights for this article or review
Article Tools
Related Articles

Related Discussion
Post a Comment | Read All

About the Author
  • Maria Araceli Ruiz-Primo
    University of Colorado Denver
    E-mail Author
    MARIA ARACELI RUIZ-PRIMO is Director of the Research Center at the School of Education and Human Development at the University of Colorado Denver. Her research focuses on the development of technically sound assessments and a methodology for instructionally sensitive assessments, student-engaged formative assessment, and inquiry-based science instruction.
  • Min Li
    University of Washington Seattle
    E-mail Author
    MIN LI is an associate professor at the University of Washington Seattle. Her research interests include the development of children & youth, quantitative research methods, and science & mathematics.
Member Center
In Print
This Month's Issue