ELL Assessment Accommodations in a Time of Changing Academic Standards and Assessment

INTRODUCTION

With the passing of the Every Student Succeeds Act (2015), the urgency of fair, valid, and reliable academic achievement assessment for English Language Learners (ELLs) was underscored as a central focus for the American education system. Within this legislation, assessment accommodations are frequently identified as a tool for allowing ELLs to access the content of tests and accurately demonstrate academic knowledge. As the educational landscape continues moving to newly-designed computer-based assessments, the promising trends, unforeseen tensions, and lingering questions related to the use of accommodations beg closer examination.

BACKGROUND: WHAT ARE TESTING ACCOMMODATIONS?

Federal and state governments mandate the use of testing accommodations to alleviate construct-irrelevant barriers associated with the assessment of ELLs, such as measuring English language proficiency instead of content area knowledge. Defined as changes to the test materials or environment that facilitate access to test content, accommodations ideally address individual characteristics that would seemingly otherwise interfere with students’ capabilities to fully demonstrate their knowledge, skills, and abilities related to the measured construct.

Accommodations can provide both direct and indirect linguistic support for ELLs. The top ten most frequently used accommodations required by states are: (a) allowing dual language dictionaries, (b) reading items aloud, (c) translating directions orally into native language, (d) clarifying/explaining directions in English, (e) repeating directions, (f) reading directions aloud, (g) clarifying/explaining directions in the native language, (h) simplifying directions, (i) allowing students to respond orally in English and describing responses, and (j) granting extended time (Willner, Rivera, & Acosta, 2008).

In order to serve their purpose, accommodations must be effective, valid, and feasible (Kieffer, Rivera, & Francis, 2012). That is to say, they must be effective in leading to improvised scores, valid in not modifying the measured content, and feasible in not imposing costly, complex, or burdensome implementation.

EFFECTIVENESS OF ACCOMMODATIONS

In spite of the general assumption that accommodations are an effective means of leveling the playing field for ELLs, the research on this matter has not been conclusive. After conducting a meta-analysis of existing research on accommodation effectiveness, Kieffer, Lesaux, Rivera, and Francis (2009) argued that accommodations are generally ineffective in improving ELLs’ performance on large-scale assessments. However, Pennock-Roman and Rivera (2011) reached the opposite conclusion in finding that when coupled with extra time, most accommodations improve ELLs’ test performance. The most effective accommodations are dual language dictionaries, bilingual glossaries, and English glossaries. The authors also point out that the read aloud accommodation, which they find to be mostly ineffective, is also the accommodation most frequently used across all U.S. states. Other researchers find that English proficiency accounts for significant variations in terms of accommodation effectiveness. This insightful finding has influenced the Partnership for Assessment of Readiness for Colleges and Careers (PARCC) policy on accessibility features, which are differentiated according to student language proficiency levels.

Kieffer et al. (2012) update their initial report and conclude that simplifying English, providing English dictionaries or glossaries, and permitting extra time “had statistically significant, small average effects on ELLs’ test performance” (p. 14). These researchers reiterated their earlier position that to eliminate ELL achievement gaps, all students must receive high-quality content instruction in the language of the test they are given. The updated report also included the following recommendations for test developers: (a) use simplified English in test design and eliminate irrelevant language demands for all students; (b) provide English dictionaries/glossaries to ELLs; (c) match the test language to the language of instruction; and (d) provide extended time or unlimited tests to all students.

Although it is unclear whether any of these recommendations have been addressed on updated assessments, the newly designed assessments are significantly different from the old ones. Current assessments are closely tied to academically rigorous standards, such as the Common Core State Standards, which are still used in several states across the country although they are under review for revocation by a number of state legislatures. Aligning assessments to these standards has led to an increase in overall expectations for testing, such as the inclusion of short answer written responses rather than solely multiple choice, matching, and true/false style questions. Furthermore, online assessments now allow for accommodation features never before available with traditional paper-and-pencil tests.

CHANGES TO ACCOMMODATIONS

All of the high-stakes assessments currently available on the market are based on the Universal Design for Learning framework (see http://www.cast.org/our-work/about-udl.html). These principles claim to eliminate the need for modified assessment (e.g., linguistically simplified assessments for ELLs) and offer accessibility features to all students, as technology supports are embedded within the assessment itself. For example, the color of the screen and print size can be adjusted, page content not immediately needed can be masked, and text-to-speech accommodation for directions and test items is now available through the use of headphones. In addition to universal accessibility tools, both PARCC and Smarter Balanced Assessment Consortium (SBAC), the two main testing consortia, continue using accommodations to include ELLs in standardized tests.

SBAC promises to provide accurate measures of achievement and growth for students with disabilities and English language learners (SBAC, 2016). They intend to meet these goals through universal accessibility tools (e.g., digital notepad and scratch paper, English glossary), designated supports identified by school personnel as necessary for individual students (e.g., translated pop-up glossary, stacked translations, text-to-speech), and accommodations available to students with an Individualized Education Plan (IEP) or 504 plan (e.g., Braille, closed captioning).

PARCC similarly reports using “universal design principles to make the new tests as accessible as possible to all students, including English learners and students with disabilities” (PARCC, 2016). Accessibility features available to students include answer masking, audio amplification, bookmarks, color contrast, blank scratch paper, highlighting tools, pop-up glossaries, spell check, and so on. PARCC also provides a list of accommodations available to ELLs and cross-references each accommodation with recommendations regarding the effectiveness of the accommodation based on student English Language Proficiency (ELP) level. For example, while students at all language proficiency levels might benefit from extended time, word-to-word dual language dictionaries (English/native language) might not be appropriate for a student who is just beginning to learn English.

TENSIONS AND PROMISES

The impact of assessment and accommodation changes for ELLs is still unfolding. For example, the emphasis on complex texts and analytical writing on PARCC assessments means teachers must figure out how to make grade-level texts, traditionally designed for more advanced English language learners, accessible to all learners. On the one hand, such changes might necessitate a positive conceptual shift in educating ELLs. On the other, students who are just beginning to learn English might be set up for failure.

Universal Design for Learning principles applied to testing accommodations for computer-based assessments show great promise, as they help provide accessibility to all learners. As with all changes that take place in education, assessment accommodations must be carefully considered and validated before implementation. It seems that the new assessments lend themselves to a more personalized, individual approach, allowing ELLs to demonstrate their competencies, and minimizing the negative impact of English language deficiencies. However, the effectiveness of accommodations, supports, and accessibility features should be further investigated as it might prove illusory. Do students know and actually use afforded accessibility features? Future research should demonstrate that new accommodations adequately address ELLs’ needs. At least some studies have found assigning inappropriate accommodations to ELLs is as ineffective as giving them no accommodations at all (Kopriva, Emick, Hipolito-Delgado, & Cameron, 2007). This proves that inappropriate accommodations can impede student performance greatly and have serious consequences for children, schools, districts, and states.

These potential problems are further complicated by the frequent mandate of a one-to-one relationship between accommodations administered in high-stakes tests and those used daily in the classroom. The controversial rhetoric of scaffolds, supports, accommodations, and accessibility features allows for multiple interpretations and is often confusing for educators administering the assessment. As a result, how accommodations are enacted at a local level across the country is both an area for future study and a topic that teacher educators must address in the classroom.

Finally, high-stakes testing often positions ELLs in deficit ways, where particular types of knowledge are foregrounded and multilingualism, multiculturalism, and attendant knowledge are shifted to the background. Both at an individual level in the classroom and across larger educational institutions, governments, and even legislation, we must continue to advocate for ELLs as we embed these new ways of teaching and assessment into our practice.

References

Every Student Succeeds Act, Pub. L. No. 114-95 (2015).

Kieffer, M., Lesaux, N., Rivera, M., & Francis, D. (2009). Accommodations for English Language Learners taking large-scale assessments: A meta-analysis on effectiveness and validity. Review of Educational Research, 79(3), 1168–1201.

Kieffer, M. J., Rivera, M., & Francis, D. J. (2012). Practical guidelines for the education of English language learners: Research-based recommendations for the use of accommodations in large-scale assessments. 2012 update. Portsmouth, NH: RMC Research Corporation, Center on Instruction.

Kopriva, R. J., Emick, J. E., Hipolito-Delgado, C. P., & Cameron, C. A. (2007). Do proper accommodation assignments made a difference? Examining the impact of improved decision making on scores for English language learners. Educational Measurement: Issues and Practice, 26(3), 11–20.

Partnership for Assessment of Readiness for College and Careers (2016). Accessibility: Universal design principles at work. Retrieved from http://www.parcconline.org/assessments/accessibility

Pennock-Roman, M., & Rivera, C. (2011). Mean effects of test accommodations for ELLs and non-ELLs: A meta-analysis of experimental studies. Educational Measurement: Issues and Practice, 30(3), 10–28.

Smarter Balanced Assessment Consortium (2016). Accessibility and accommodations. Retrieved from http://www.smarterbalanced.org/parents-students/support-for-under-represented-students/

Willner, L. S., Rivera, C. & Acosta, B. D. (2008). Descriptive study of state assessment policies for accommodating English language learners. Arlington, VA: The George Washington University Center for Equity and Excellence in Education.