|
|
|
|
Accommodation Practices for English Language Learners in States’ Mathematics Assessmentsby Mikyung Kim Wolf, Jenny C. Kao, Nichole M. Rivera & Sandy M. Chang - 2012 Background/Context: Testing accommodations have been widely utilized as a way of increasing the validity of content assessments for English language learner (ELL) students. However, concerns have also arisen regarding the appropriateness of accommodation use, including the accessibility and fairness of accommodations. While many states have developed ELL-specific accommodation policies and guidelines, little research has been available on how the accommodation policies are carried out in practice. Purpose/Objective/Research Question/Focus of Study: The present study investigated two states’ accommodation policies, specifically for the states’ respective large-scale Grade 8 math assessments, and conducted a case study to examine teachers’ understanding of the policies and uses. The study aimed to identify issues to consider for an appropriate use of accommodations and provide useful information for policymakers to improve their accommodation policies. Research Design: The study utilized a qualitative method employing teacher surveys and interviews. The survey and interview instruments were developed based on previous literature and experts’ feedback. The interview transcripts were coded by two researchers using a systematic coding scheme. Descriptive statistics of the responses were computed to observe trends across and within districts and schools. The results were also compared between states or between ELL and math teachers when applicable. Conclusions/Recommendations: Despite the limitation of the small sample in this study, the findings of the study offer practical implications for policymakers and educators in the use of accommodations for ELL students. The study found considerable variation reported by teachers with regard to the perception of accommodation decision makers, selection criteria, and the types of accommodations allowed in each state’s math assessment. This variation raised serious concerns regarding the adequacy of the accommodation uses and the comparability of accommodated test results across schools. In this paper, we discussed a number of possible reasons for teachers’ reported difficulty in keeping up with state policies: (a) lack of clear guidelines in making accommodation decisions and implementing accommodations in a standardized way, (b) lack of or limited opportunities in receiving information and communicating about accommodations among decision makers and teachers, and (c) limited resources and logistical difficulties. To support an appropriate use of accommodations for teachers, we recommend that states make efforts to provide comprehensive, operationalized guidelines for ELL accommodations, monitor the use of the guidelines, and hold regular professional meetings for ELL and content teachers. Policies pertaining to the assessment of English language learner (ELL) students have gained increasing attention as a result of federal mandates to include ELL students into states large-scale, standards-based content assessments for accountability purposes. One critical issue in assessing ELL students relates to the validity of inferences made about the students knowledge and skills based on content assessments. That is, students lack of English proficiency may interfere with demonstrating what the students know and can do in a content assessment. To make assessments valid and accessible to ELL students, a growing number of states have established or modified testing accommodation policies specific to ELL students, distinguishing from ones made for students with disabilities (Rivera & Collum, 2006; Shafer Willner, Rivera, & Acosta, 2008; Wolf, Kao, Griffin, et al., 2008; Wolf, Kao, Herman, et al., 2008). Accommodations, which are changes to a test or testing environment, are intended to help ELL students reduce any language barriers the students may have when taking a test. However, a widespread concern has developed regarding the appropriate use of accommodations, which may affect the validity of accommodated test results. Recognizing the important role that accommodations play in determining test validity, researchers have examined the state of current accommodation policies and practices and identified common validity issues and challenges in the use of accommodations for ELL students. For example, Rivera, Collum, Shafer Willner, and Sia (2006) examined accommodation policies for all 50 states and the District of Columbia in 20002001 and found substantial variations in accommodation policies and practices. The authors analysis revealed that most states did not focus on the unique linguistic needs of ELL students, making little distinction between ELL students and students with disabilities regarding test accommodations. This implies a potential validity threat if accommodations were not properly used for ELL students in practice. Further, one plausible reason for the previously mixed results on the effects of accommodations might be related to the policy and practice of accommodation usage. Thus, examining the surrounding policies and practices of ELL students is essential in the validation process for ELL assessment. A recent study of the accommodation assignment criteria conducted by Kopriva, Emick, Hipolito-Delgado, and Cameron (2007) provided evidence about the importance of examining accommodation practices to inform the validity of accommodations. The researchers discussed the lack of existing guidance for decision makers in assigning specific accommodations for ELL students with different characteristics. Through a series of experimental studies, the researchers found that ELL students who were assigned accommodations based on specific criteria to meet their individual needs performed better compared to ELL students who were assigned accommodations not guided by any consistent decision rules. There was no significant difference between the group with no accommodations and the group with accommodations with inconsistent rules. This study underscores the importance of identifying and utilizing appropriate accommodation selection criteria. Research shows that teachers may be unaware of the best way to meet the needs of ELLs in the classroom (Yoon, 2008). This lack of knowledge may adversely impact teachers attitude and beliefs, with negative outcomes, especially for minority students such as ELLs (Flores & Howard, 2008). Despite the importance, research examining teacher knowledge and the actual practice of ELL accommodations for states accountability assessments is limited. The latest policy review studies (Shafer Willner et al., 2008; Wolf, Kao, Griffin, et al., 2008) noted that states have made progress in establishing accommodation policies for ELL students. In a 20062007 policy review, Wolf, Kao, Griffin, et al. found that most states listed allowable accommodations for ELL students separately from students with disabilities. However, a gap often exists between policies created at the state level and the practices performed at the local district and school levels. Investigating the actual use of accommodations will offer insight into the validity issues to consider when interpreting accommodated test results. It will also provide valuable information for policymakers and practitioners to consider in developing and implementing accommodation guidelines for ELL students. PURPOSES OF THE STUDY The present study examined two states accommodation policies for the states large-scale, standards-based mathematics assessments in Grade 8 and how the policies were carried out in practice. The study also investigated teachers perspectives on the usage and helpfulness of accommodations for ELL assessment through surveys and interviews. The study aimed to identify issues to consider for an appropriate use of accommodations and provide useful information for policymakers to improve their accommodation policies. Specifically, we addressed the following research questions: (a) How varied are the state and school ELL accommodation policies within a given state for the states large-scale, standards-based mathematics assessment? (b) How do teachers use the accommodations for ELL students in their states standards-based mathematics assessment? (c) What are teachers perceptions of the helpfulness of accommodations for ELL students? To investigate our research questions, we explored the following topics related to the two states policies and practices of accommodations for ELL students: accommodation decision makers, accommodation selection criteria, permitted and used accommodations for the states large-scale mathematics assessment, accommodations during instruction, policy communication channel, recording practice of the accommodation data, and teachers perceptions of the helpfulness of specific accommodations. Each topic is more fully described in its respective section. ELL ACCOMMODATION POLICIES IN TWO STATES OF THE STUDY The two states that participated in this study are henceforth referred to as State X and State Y to preserve anonymity. Table 1 illustrates states policies related to ELL students accommodations, based on information gathered from the states public websites, including the states ELL accommodation guideline documents. Note that both states allowed local districts to consider their specific needs to make their own decisions in the use of accommodations. Table 1. Policies on Accommodations for ELL Students in State Standards-Based Assessments in Mathematics for the 20072008 School Year by State
Note. The following practices are considered best practices for administering a test for all students in State Y, and are not considered to be accommodations and do not require documentation: teacher faces test taker, test administered with minimal distractions, person familiar to test taker (e.g., ELL teacher) administers the test, and test administered in familiar room. METHOD PARTICIPANTS The targeted sample included ELL specialists or teachers as well as math teachers who taught Grade 8 ELL students during the 20072008 school year. A total of 165 middle school teachers from two states voluntarily participated in this study. In State X, a total of 64 teachers from 19 schools in two large urban districts participated in the study. Thirty-four teachers completed the survey, and 42 teachers participated in the interview. Some of the teachers completed the interview and survey, while others completed only one component. In State Y, a total of 101 teachers from 54 schools in 18 mid-size urban and suburban districts participated in the study. All 101 teachers completed the survey. Of those, 35 teachers also participated in the interview. Table 2 summarizes the number of teachers who participated in the two components of this study by state and by teacher assignment or type (i.e., math teacher vs. ELL teacher). Although the two participating states do not have the largest proportion of ELL students in the country, the states students public school enrollment consisted of between 10% and 20% of their total student populations, consistent with average ELL population distributions nationwide. The proportion of Grade 8 ELL students of the participating schools ranged from 7.6% to 41.0% for State X and from 1% to 53.1% for State Y. Table 2. Number of Participants by State, Teacher, and Participation Type
Note. *Not the same 18 schools. A total of 19 schools from State X participated. One school did not participate in interviews, and one did not participate in surveys. INSTRUMENTS Teacher Survey The teacher survey scales (i.e., constructs and item scales) were constructed based upon previous research and surveys regarding the use of accommodations and instructional strategies for ELL students (see Abedi, Courtney, Leon, Kao, & Azzam, 2006; Martinez, Bailey, Kerr, Huang, & Beauregard, 2009). The survey included five subscales: instructional strategies, classroom assessment practices, accommodation practices during instruction, experience with accommodations in state testing, and teacher perception of the helpfulness of accommodations. Background questions (e.g., license status, teaching experience, degree) were also included. Some items were asked on a nominal scale; others were asked on a Likert-type scale. The majority of the items were previously examined to ensure the reliability and validity of the scales. The draft of the survey was also reviewed by three former and current teachers as well as two external researchers for the appropriateness of the survey content and the clarity of the items. Teacher Interview Protocol The teacher interview protocol was developed to gather in-depth information about schools accommodation practices and teachers perceptions of the use of accommodations. The questions primarily focused on four topics: knowledge of accommodation policies, mechanism to record accommodation data, previous experience with accommodation usage for state testing administration, as well as accommodation usage during instruction. The interview was designed to be completed in approximately 30 to 45 minutes. (See Appendix for sample survey and interview questions.) PROCEDURE AND ANALYSIS Surveys were collected via mail, fax, e-mail, or online. Teacher interviews were conducted one-on-one, either face-to-face or via telephone. All interviews were audio recorded, and then later transcribed. For reliability of the survey instrument, internal consistency using Cronbachs alpha was examined for the items on Likert-type scales. Acceptable levels of reliability were obtained; .86 for the items on instructional strategies, .59 for the items on classroom assessment practices, .76 for the items on accommodation practices during instruction, and .81 for the items on teacher perception on the helpfulness of accommodations. Descriptive statistics including the frequencies and means of the responses were computed to observe trends across and within districts and schools. The results were also compared between states or between ELL and math teachers when applicable. For interview data, a coding scheme was developed and applied by trained researchers to transcripts using Atlas.ti.1 More than 28% of the transcripts were coded by two researchers with 80.1% of exact agreement on average. Where there were disagreements between the two raters, consensus was reached through discussion. Once acceptable reliability was obtained, the remaining transcripts were coded by one researcher each. RESULTS Survey results for the first two research questions are presented according to subtopics, across and within states as well as by teacher type (i.e., math vs. ELL teacher). Interview results are also reported where applicable to provide more in-depth information. ACCOMMODATION DECISION MAKERS The policies for States X and Y were similar with respect to accommodation decision makers. Both stated that accommodation decisions should be made by a team of people, including teachers most familiar with students English proficiency (in State X) or content area teachers (in State Y). To investigate the personnel involved in making accommodation decisions for each individual ELL student in schools, teachers were asked who determined the selection of accommodations for ELL students at their school for the Grade 8 state math assessment. Teachers were given the following options and asked to choose all that applied or specify other personnel or source, if applicable: ELL teacher, math teacher, principal, parents, students, dont know, and other. Table 3. Decision Makers for Selecting Accommodations by State: Frequency (Percentage)
Note. ELL & Math team included both math and ELL teachers. ELL & other team and Math & other team included some personnel and ELL teacher, some personnel and math teacher, respectively. Other included special education teacher, testing coordinator, assistant principal, district personnel, parent, and student. As shown in Table 3, many teachers in both states responded that the decision was made by a team, typically including an ELL teacher. In State X, however, an ELL teacher was often reported as the sole decision maker. In State Y, a math teacher was frequently reported as being involved in the accommodation decision. A considerable percentage of the respondents reported dont know to the question about decision makers. The majority of teachers who reported dont know were math teachers, across both states. While a few of these math teachers reported that less than 5% of the students in their math classes were ELLs, most math teachers in the sample reported that 15% to 95% of their students were ELLs. ACCOMMODATION SELECTION CRITERIA Both states guidelines suggested that accommodations should meet individual students needs. To examine the specific criteria or rules on which schools relied in selecting accommodations, teachers were asked about the sources or criteria used to make this decision for the Grade 8 state math assessment. Teachers were given the following options and asked to choose all that applied or to specify other criteria, if applicable: English language proficiency (ELP) level, student IEP (Individualized Education Plan), state standardized test scores, no specific criteria (e.g., blanket accommodation in which all ELL students receive the same accommodations), dont know, and other. Table 4 presents the frequency and percentage of the criteria chosen by the teachers. Results indicate a variety of options were utilized, ranging from a single source to varied combinations of multiple sources. Table 4 also shows that 31.3% of the sample math teachers selected dont know compared to 9.4% of the sample ELL teachers. Table 4. Accommodation Selection Criteria by State and by Teacher Type: Frequency (Percentage)
Note. Other criteria included instructional uses, teacher recommendations, teacher observations, performance on other assessments, other types of educational plans, or based on what students received the prior year. During the interview, teachers mentioned logistic difficulties and lack of resources as the rationale for providing blanket accommodations, as in the following excerpts: State X ELL teacher: They all get the same accommodations because its just easier logistically. Because say, if I gave to all my students the same accommodations, they can all test with me. Whereas if I gave half my students [a] bilingual dictionary to use, and the other half no dictionary to use, or no re-reading the directions, they couldntwed have to find another testing room for them . . . So my main criteria is, well, if they have me for English, they get to test with me, and they get all the same accommodations. State Y ELL teacher: I think its frustrating because were such a highly impacted district. In our situation, [ELL students are] the majority of our school, and we dont have a lot of extra places to take kids for smaller groupings for different accommodations. And, in addition, the staff is limited for breaking kids up into groups and giving them their accommodations. PERMITTED AND USED ACCOMMODATIONS As summarized earlier, both states specified a list of allowable accommodations for their state math assessments. To examine the extent to which this state policy was enacted at the school level, teachers were given a list of accommodations (see Appendix) and asked to identify the types of accommodations permitted as well as the types of accommodations actually used for the state math assessment at Grade 8. The results are summarized for State X and State Y in Table 5 and Table 6, respectively. For both states, results indicate differing knowledge about permitted accommodations among the teachers. For instance, although State Xs policy restricted the use of reading aloud test items in students native language for the state math assessment, a sizeable number of State X sample teachers reported that reading aloud test items in the students native language was permitted and even used in practice. Similarly, nearly a quarter of State Y sample teachers reported that reading aloud test items in students native language was not permitted, while it was permitted according to the states policy documents. The results also indicate that this different knowledge may stem from district-level policies compared to the state policy. More than 40% of the sample teachers in both states chose not sure about whether a dictionary was allowed or not for the math assessment. Table 5. State X Permitted and Used Accommodations for the State Math Assessment: Frequency (Percentage)
Note. In State X, the language of dictionary (i.e., bilingual, English) was not specified in the survey. Table 6. State Y Permitted and Used Accommodations for the State Math Assessment: Frequency (Percentage)
Regarding accommodations that were actually used in practice, extended time and directions being read aloud in English were frequently reported in both states. Test administration by an ELL teacher was the most frequently used accommodation as reported by the majority of State X sample teachers, while test administration in a small group was the most frequently used accommodation reported by State Y sample teachers. Again, the types of accommodations reported used varied depending on the teachers and schools within each state. Teacher interview data revealed that this variation occurred across districts and even within the same district. For example, about half of the sample teachers in one district of State X reported that they read aloud the test in English while other teachers in the same district reported not using this accommodation. This within-district variation was also found in State Y. In one district of State Y, a little more than 40% of the sample teachers reported that they read aloud the test in English. ACCOMMODATIONS DURING INSTRUCTION Both previous research and the states policies emphasized that the accommodations selected for state testing should be ones in which students are familiar with through instructional practice. To examine the instructional uses of accommodations, teachers were asked to indicate how frequently a specific accommodation was provided to ELL students during classroom tests and instructional tasks, on a 6-point scale with 1 meaning never used and 6 meaning used every day. Table 7. Accommodations Provided to ELL Students for Grade 8 Math Classroom Tests by State: Mean and Standard Deviation (SD)
Note. Rating scale, 1 = never to 6 = every day. Language of dictionary was not specified for State X. Table 8. Accommodations Provided to ELL Students to Complete Tasks During Grade 8 Math Class by State: Mean and Standard Deviation (SD)
Note. Rating scale, 1 = never to 6 = every day. Language of dictionary was unspecified for State X. Table 7 shows the means and standard deviations of the scale of how often teachers provided accommodations to ELL students during math classroom assessments. Teachers in both states reported extended time most frequently, followed by read aloud of directions. Similarly, results for accommodations provided to ELL students during math class to complete tasks or problems, as shown in Table 8, indicated extended time followed by read aloud of problems as the most frequently reported in both states. Figure 1. Percentage of teachers reporting never to a given accommodation for ELL students during Grade 8 math classroom assessments.
Note. Language of dictionary was unspecified for State X. Teachers often mentioned that they never gave accommodations during classroom assessments. Figure 1 demonstrates accommodations reported as never having been provided by teachers during classroom assessments. Higher percentages of never were reported for some accommodation types, such as use of dictionary and glossary. Roughly three quarters of both State X and State Y reported never providing a dictionary for classroom math assessments. RECORDING OF ACCOMMODATION USE FOR STATE TESTING A series of questions regarding the recording practices included teachers familiarity with accommodation data records, recording procedures, personnel in charge of recording and maintaining the accommodation, and data accessibility to teachers. Overall, ELL teachers in both states reported being familiar with accommodation recording practices, with 73.7% and 93.8% of ELL teachers in State X and Y, respectively, reporting being at least somewhat familiar with the recording policies. More State Y teachers overall reported familiarity (74.3%), while just a little over half of State X teachers reported familiarity (51.7%). One of the recording procedure questions was related to the personnel responsible for keeping records of the accommodations given to individual ELL students during the state math assessment. While an ELL teacher or the schools ELL department was predominantly reported as responsible for the accommodation records in State X, many math teachers in State Y reported that they were involved in the record-keeping procedures. As described in the State Y policy, math teachers were required to document an individualized accommodation plan for ELL students for their math instruction, by a specific date before the state math assessment. Subsequently, math teachers in State Y may have had a tendency to consider themselves the record-keepers for the state math assessment. However, when prompted, they were often not aware of what happened with the documentation plans after the plans left their hands. Both states sample teachers were unclear about procedures regarding the maintenance and accessibility of accommodation data. While about half of the ELL teachers in State X referred to their states official accommodations record form, some were uncertain whether those paper forms were later transferred to any electronic database. The teachers responses can imply either a nonsystemic method of maintaining accommodation records or limited knowledge about the process. COMMUNICATION CHANNEL The interview also asked teachers how they were informed about state and school accommodation policies with the assumption that the means and quality of communication could have an influence on the use of accommodations. The interview questions specifically focused on the clarity with which information on accommodation policies was distributed to teachers, and the ways in which accommodation policy information was delivered to the teachers. Many teachers expressed reservations about whether information on accommodation policies was made clear to teachers. A math teacher in State Y commented on the lack of clarity. A State Y Math teacher observed, Last year, I feel like it was a hodgepodge. I dont think it was really clear . . . I think it wasnt clear what options were possible, how to make sure those people got them. Regarding the ways teachers received accommodation information, many teachers in both states mentioned that they had meetings before testing with a school-based or itinerant district testing coordinator and typically received a testing manual including accommodation information. However, whether a discussion specifically focused on accommodation policies for ELL students occurred was unclear. An ELL teacher in State Y remarked that the 20072008 school year was the first year the states Department of Education distributed a separate accommodations manual specific to ELL students, separate from students with disabilities: State Y ELL teacher: They [The State Department of Education] put out an accommodations manual for ELL students. So in the past it had kind of been part of the regular accommodations manual. And last year was the first year that they did it separately. And its extensive documents. Its sixty something pages, and theres charts for every conceivable situation of an ELL student in their previous instruction, whether it was in English or native language and whether they were literate in their native language or not. State Y math teachers also consistently commented that they received the accommodation information in the beginning of the school year to begin planning and using the accommodations. The teachers attributed this practice to the State Y policy requirement of documenting the accommodation plan by a specific date before the testing as well as for content instruction. PERCEPTIONS OF THE HELPFULNESS OF ACCOMMODATIONS The third research question explored teachers perceptions of the helpfulness or usefulness of accommodations. In the survey, teachers were given a list of accommodations and asked to rank each one on a scale of 1 to 4, with 1 meaning not very helpful and 4 meaning extremely helpful. To examine whether ELL and math teachers possessed different perceptions of the helpfulness of accommodations, the responses were grouped by teacher type, as shown in Table 9. Table 9. Teachers Perception on the Helpfulness of Accommodations for ELL Students by Teacher Type: Mean (SD)
Note. Dictionary cannot be compared across states, since State X teachers did not have the option of a bilingual dictionary. Score scale of 1 = not very helpful to 4 = extremely helpful. Results by teacher type indicated that math teachers reported extended time as the most helpful (M = 2.96) and that ELL teachers reported test administered by an ELL teacher as the most helpful (M = 3.43). Math teachers and ELL teachers both reported English dictionary, glossary, and electronic translator accommodations as relatively low in helpfulness. Teacher responses from interview data regarding accommodation helpfulness were categorized as positive, negative, or mixed in which common themes emerged within the categories. While positive opinions about accommodation helpfulness were related to reducing ELL students language barriers, negative opinions were concerned with logistical issues and students lack of familiarity. Teachers with mixed opinions considered ELL students background when discussing the helpfulness of accommodations. For instance, an ELL teacher mentioned students language proficiency levels as factors that affect the helpfulness of a dictionary accommodation for students, as indicated in the following excerpt: State Y ELL teacher: It depends on their proficiency level because if they feel that they dont understand anything, then theyre going to look up every word, and [using a dictionary accommodation is] a daunting task. I think most students will just start guessing . . . Its pretty overwhelming, so I think, their background understanding of the dictionary, practice with the dictionary, and then if its appropriate for their fluency level. To illustrate the common themes on teachers perceptions about accommodation helpfulness, a summary of teachers general opinions on the helpfulness of read aloud and dictionary or glossary, as examples, is presented in Figure 2. Figure 2. Summary of teacher perceptions of the helpfulness of read aloud and dictionary/glossary accommodations.
DISCUSSION Before the findings of the study are discussed, the nature of this study and its limitations should be noted. Although the study was concerned with statewide policies and their implementation in practice, as a case study, the sample size is limited, and thus, generalization of the findings should be made with caution. This study investigated how state accommodation policies were communicated to schools and implemented in practice, to identify important validity issues to consider in the use of accommodations for ELL students. The states in our study have made great, initial efforts to provide a systematic, principled way of using accommodations for ELL students, separate from students with disabilities. However, the wide range of variation found between the policies and practices within this modest sample suggests that, while progress has been made in policies for ELL accommodations, additional work must be done in ensuring these policies are appropriately communicated and implemented. We discuss some key findings and their implications in relation to the three research questions (RQs) posited in this study.
RQ1. HOW VARIED ARE THE STATE AND SCHOOL POLICIES WITHIN A GIVEN STATE ON THE USE OF ACCOMMODATIONS FOR ELL STUDENTS IN THE STATES LARGE-SCALE, STANDARDS-BASED MATHEMATICS ASSESSMENT? We found that teachers reported considerable variation about perception of accommodation decision makers, selection criteria, and the types of accommodations allowed in each states math assessment. This variation raised a question about how adequately accommodations were provided for ELL students across schools. This finding also raised a question about the comparability of accommodated test results across schools. The interview data suggested that the variation in the use of accommodations may be due to limited guidelines, limited communication of guidelines, or limited resources. For instance, while both states strongly recommended that accommodations should be selected based upon students needs, what sources teachers should utilize to identify the students different needs were not specified in the state guidelines. Previous studies reviewing the states policies indicated that various criteria were used in assigning accommodations across states, including students language proficiency, academic performance, instructional service, parental input, and teacher observation, to name a few (Rivera et al., 2006; Wolf, Kao, Griffin, et al., 2008). The present study found that this variation also occurred within the state, and often within the same district. This result was partly due to local districts and schools determining their own accommodation rules for their own needs. Yet the lack of detailed, operationalized guidelines raises a concern as some teachers, even those who were involved as decision makers, reported they had little knowledge of any systematic criteria used in making accommodation decisions. The state recommendation of selecting accommodations on an individual-need basis was not followed in some schools. Some teachers reported that a blanket accommodation administration, or an accommodation given to all qualifying ELL students regardless of actual need, was inevitable regardless of the status of ELL students, citing logistical challenges and strained resources as reasons for the difficulty of providing tailored accommodations for each student. For instance, several teachers commented on the difficulty of acquiring bilingual dictionaries in every language for their diverse students. Thus, while states listed a number of allowable accommodations, at the school level, a smaller set of accommodations were permitted and feasible. While the results suggested that the accommodation policies and practices varied among the schools in this study, states policies still had a clear impact on the accommodation uses in schools, in general. For example, one of the accommodation policy stipulations in State Y included a strict requirement for documenting individual accommodation plans for instruction and assessment with specific dates before the state testing. As a result, a recurring trend was observed: The sample math teachers in State Y tended to report more familiarity with accommodation policies for ELL students and more involvement in accommodation uses and data recording procedures, compared to the sample State X math teachers. The absence of this requirement in State X seemed to lead to a greater communication gap between ELL and math teachers on the accommodation uses for the state math assessment, which can subsequently result in a gap between the instructional uses and testing uses of accommodations. Another example was found in the different patterns of accommodation decision makers. State X documents stated that decision makers should include the teacher or school administrator most familiar with students English language acquisition while State Y documents stated that decision makers should include the teacher primarily responsible for delivery of instruction in the content area being assessed. Our finding demonstrated that, in State X sample schools, ELL teachers were often reported as the sole decision maker whereas in State Y sample schools, ELL and math teachers were reported as decision makers. RQ2. HOW DO TEACHERS USE THE ACCOMMODATIONS FOR ELL STUDENTS IN THEIR STATES STANDARDS-BASED MATHEMATICS ASSESSMENT? While some general patterns emerged regarding the most frequently used accommodations, wide variation was also noted across schools in terms of the actual provision of accommodations. The most frequently used accommodations reported by the teachers of this study in both states were extended time and directions read aloud. Extended time and read-aloud directions seemed to be relatively easily provided to all students without physically separating any specific ELL student or requiring additional resources. This result suggests that the ease of implementation, that is, feasibility and practicality, likely played a key role in selecting accommodations. The use of other types of accommodations varied widely depending on schools and teachers. For example, results indicated that read aloud of the test was reportedly used by half of the schools in one district, posing the question why other schools in the same district did not provide the accommodation. On a related note, we found that teachers reported different permitted accommodations for the state math assessment even within the same district and within the same school. Surprisingly, a number of teachers reported that they were not sure whether certain types of accommodations were allowable for the state math assessment such as reading aloud the test in students native language, glossary, and dictionary. Teachers varied knowledge and familiarity with the policies seemed to have an effect on limiting the types of accommodations that the teachers actually provided during the state math assessment. This alarming finding emphasizes the need to exercise caution when examining accommodated test scores and aggregating test data. The varied practice of the accommodation uses across schools is, to some extent, attributed to a lack of regular, systematic communication channels and training meetings for the use of accommodations. While some variation is inevitable due to local decisions, the decisions should be made in a systematic way. To prevent variation resulting from an unsystematic, random use of accommodations, ELL and content teachers must be provided opportunities to engage in discussing the policies and practices of accommodations for ELL students. RQ3. WHAT ARE TEACHERS PERCEPTIONS OF THE HELPFULNESS OF ACCOMMODATIONS FOR ELL STUDENTS? Teachers perceptions of the helpfulness of accommodations for ELL students were mixed. For example, ELL teachers generally rated test administered by an ELL teacher most helpful, while math teachers rated extended time most helpful. Many teachers recognized that different individual students had different needs for accommodations, and thus, evaluating the general helpfulness of accommodations was different. Teachers also pointed out the students different language proficiency levels and skills were factors in the accommodations effectiveness. These opinions raise an important issuethe backgrounds of students and the contexts should be taken into consideration when evaluating effective types of accommodations. RECOMMENDATIONS AND CONCLUSIONS We discussed a number of possible reasons for teachers reported difficulty in keeping up with state policies: (a) lack of clear guidelines in making accommodation decisions and implementing accommodations in a standardized way, (b) lack of or limited opportunities to receive information and communicate on accommodations among decision makers and teachers, and (c) limited resources and logistical difficulties. Despite the studys limitations, we believe that the findings suggest some initial, relatively simple steps that state policymakers and practitioners could implement to support the valid use of accommodations in their schools: (a) States should provide clear, operationalized guidelines (such as in a comprehensive manual) on the use of accommodations, including the definition, selection criteria, allowable or prohibited accommodations, and details of the implementation procedures. (b) States or districts should monitor the use of guidelines regularly to ensure the adequate application of accommodation policies in practice. (c) Regular professional development meetings should be held to communicate the policies and intended use of accommodations across the state, districts, and schools, as well as between ELL and content teachers. (d) The types of accommodations provided to each individual student during the states large-scale assessment should be accurately recorded and accessible to teachers as well as administrators. Establishing systematic recording practices for accommodations is essential for accurate reporting of results. Analyses of such data also can be an important source for evaluating the validity of accommodations as well as for monitoring, communicating, and improving accommodation policies and practices. The intent of the study was to learn from the existing policies and practices to help policymakers in improving accommodation guidelines and to inform practitioners of the current status of accommodation uses to consider important accommodation issues for ELL students. The study findings highlight the significance of examining practices when evaluating the validity of accommodation uses. Implications for these findings draw attention to the comparability issue of accommodated and nonaccommodated test scores among ELL students, within and across districts and states. Further, this finding brings up equity and accessibility issues in that some students were excluded from receiving accommodations for reasons unrelated to the students needs. We hope the present paper can be used as a resource to initiate dialogue among policymakers and practitioners. Author Note The study was conducted while Mikyung Kim Wolf and Jenny C. Kao were at CRESST/UCLA. This study was supported by the research grant (R305A050004) awarded by the U.S. Department of Educations Institute of Education Sciences (IES). The findings and opinions expressed herein are those of the authors and do not necessarily reflect the positions or policies of the IES. We would like to thank Joan Herman and Noelle Griffin for their invaluable feedback. We also thank Patina Bachman, Julie Nollner, Alice Hsu, Yuichiro Otani, Jean Jho, and Stella Tsang Li for their helpful research assistance. Finally, our sincere thanks go to all the teachers who participated in this study, and all the state and district representatives who provided enormous support and help with our study. Note 1. ATLAS.ti Scientific Software Development GmbH, Nassauische Str. 58, D-10717 Berlin Germany. References Abedi, J., Courtney, M., Leon, S., Kao, J., & Azzam, T. (2006). English language learners and math achievement: A study of opportunity to learn and language accommodation (CSE Tech. Rep. No. 702). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing. Flores, B. B., & Smith, H. L. (2008). Teachers' characteristics and attitudinal beliefs about linguistic and cultural diversity. Bilingual Research Journal, 31(1), 323358. Kopriva, R. J., Emick, J. E., Hipolito-Delgado, C. P., & Cameron, C. A. (2007). Do proper accommodation assignments make a difference? Examining the impact of improved decision making on scores for English language learners. Educational Measurement: Issues and Practice, 26(3), 1120. Martinez, J. F., Bailey, A. L., Kerr, D., Huang, B. H., & Beauregard, S. (2009, April). Measuring opportunity to learn and academic language exposure for English language learners in elementary science classrooms. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA. Rivera, C., & Collum, E. (Eds.). (2006). State assessment policy and practice for English language learners: A national perspective. Mahwah, NJ: Erlbaum. Rivera, C., Collum, E., Shafer Willner, L., & Sia, J. K., Jr. (2006). An analysis of state assessment policies regarding the accommodation of English language learners. In C. Rivera & E. Collum (Eds.), State assessment policy and practice for English language learners: A national perspective (pp. 1173). Mahwah, NJ: Erlbaum. Shafer Willner, L., Rivera, C., & Acosta, B. (2008). Descriptive study: State assessment policies for accommodating English language learners. Arlington, VA: The George Washington University Center for Equity and Excellence in Education. Wolf, M. K., Kao, J., Griffin, N., Herman, J. L., Bachman, P., Chang, S. M., & Farnsworth, T. (2008). Issues in assessing English language learners: English language proficiency measures and accommodation uses--Practice review (CRESST Rep. No. 732). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing. Wolf, M. K., Kao, J., Herman, J., Bachman, L. F., Bailey, A., Bachman, P. L., . . . Chang, S. M. (2008). Issues in assessing English language learners: English language proficiency measures and accommodation uses--Literature review (CRESST Rep. No. 731). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing. Yoon, B. (2008). Uninvited guests: The influence of teachers roles and pedagogies on the positioning of English language learners in the regular classroom. American Educational Research Journal, 45(2), 495522. APPENDIX SAMPLE TEACHER SURVEY AND INTERVIEW QUESTIONS
Policy Communication Channel " Do you feel that state and district policies on accommodations for ELLs are made clear to you? " How do you get information on accommodation policies? Recording Practice of the Accommodation Data " Are you familiar with how students accommodations are kept on file? " How are students accommodations recorded? " Who records them? Or fills out the forms? " How often are these forms accessed or updated?
|
|
|
|
   |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||
![[39_16302.htm_g/00002.jpg]](/Html/39_16302.htm_g/00002.jpg)
![[39_16302.htm_g/00004.jpg]](/Html/39_16302.htm_g/00004.jpg)
![[39_16302.htm_g/00006.jpg]](/Html/39_16302.htm_g/00006.jpg)
![[39_16302.htm_g/00008.jpg]](/Html/39_16302.htm_g/00008.jpg)
![[39_16302.htm_g/00010.jpg]](/Html/39_16302.htm_g/00010.jpg)
Executive Summary
Full Text in PDF Format
Email this article
Print this article
Post a Comment
