Home Articles Reader Opinion Editorial Book Reviews Discussion Writers Guide About TCRecord
transparent 13
Topics
Discussion
Announcements

Complexity of Illustrations in PISA 2009 Science Items and Its Relationship to the Performance of Students from Shanghai-China, the United States, and Mexico


by Guillermo Solano-Flores & Chao Wang — 2015

Background:While illustrations are widely used in international test comparisons, very scant research has been conducted on their design and on their influence on student performance. It is not clear how the features of illustration act in combination supporting students’ access to the content of items or increasing their interpretation demands.

Purpose: The purpose of this study was to examine how the characteristics of illustrations used in science items are related to the performance of students in PISA (Programme for International Student Assessment)-2009.

Population: We used PISA-2009 data on the responses of 1571 to 1582, 1601 to 1617, and 11,662 to 11,746 students, respectively from Shanghai-China, the U.S., and Mexico.

Research Design: This was a secondary analysis study which compared the three PISA-2009 jurisdictions as to the magnitudes and directions of the correlations between the complexity of illustrations used in science items and the difficulty of those items. We used the information on item difficulty provided by the Organisation for Economic Co-operation and Development (OECD, 2010). The features of the illustrations were coded with a system that coded the presence or absence of about 100 types of illustration features. The complexity of each illustration was computed as the number of different types of features observed.

Findings: For the U.S. and Mexico, items with higher illustration complexities were more difficult than items with lower illustration complexities. For Shanghai-China, the correlation of number of illustration features and item difficulty was close to zero. This pattern of correlations is consistent with the three jurisdictions’ ranking in PISA-2009.

Conclusions: While this was an exploratory study that examined the performance of only three of the 65 PISA-2009 jurisdictions, the results speak to the importance of examining illustration complexity as a factor that shapes student performance in science tests and which should be addressed systematically in the design of science items.



To view the full-text for this article you must be signed-in with the appropropriate membership. Please review your options below:

Sign-in
Email:
Password:
Store a cookie on my computer that will allow me to skip this sign-in in the future.
Send me my password -- I can't remember it
 
Purchase this Article
Purchase Complexity of Illustrations in PISA 2009 Science Items and Its Relationship to the Performance of Students from Shanghai-China, the United States, and Mexico
Individual-Resource passes allow you to purchase access to resources one resource at a time. There are no recurring fees.
$12
Become a Member
Online Access
With this membership you receive online access to all of TCRecord's content. The introductory rate of $25 is available for a limited time.
$25
Print and Online Access
With this membership you receive the print journal and free online access to all of TCRecord's content.
$210


Cite This Article as: Teachers College Record Volume 117 Number 1, 2015, p. 1-18
http://www.tcrecord.org ID Number: 17725, Date Accessed: 10/23/2017 4:11:38 AM

Purchase Reprint Rights for this article or review
Article Tools

Related Media


Related Articles

Related Discussion
 
Post a Comment | Read All

About the Author
  • Guillermo Solano-Flores
    University of Colorado, Boulder
    E-mail Author
    GUILLERMO SOLANO-FLORES is Professor of Bilingual Education and English as a Second Language at the School of Education of the University of Colorado Boulder. He specializes in educational assessment and the linguistic and cultural issues that are relevant to both international test comparisons and the testing of cultural and linguistic minorities. His research is based on the use of multidisciplinary approaches that use psychometrics, sociolinguistics, semiotics, and cognitive science in combination.
  • Chao Wang
    University of Colorado, Boulder
    E-mail Author
    CHAO WANG is Research Associate at the School of Education of the University of Colorado Boulder. She specializes in the linguistic and sociocultural issues that are relevant to the academic success of culturally and linguistically diverse students. Her research interests include literacy, the testing of linguistically diverse populations, and the use of illustrations in science assessment.
Member Center
In Print
This Month's Issue

Submit
EMAIL

Twitter

RSS