Standards Gaps: Unintended Consequences of Local Standards-Based Reform
by Judith Haymore Sandholtz, Rodney T. Ogawa & Samantha Paredes Scribner - 2004
In response to the establishment of standards by states and professional organizations, many local school districts have adopted a standards-based curriculum. The expressed purpose of standards is to improve student academic performance by providing teachers with a common sequence of targets at which to aim instruction. In this study, we examine unintended consequences of a school district's standards-based reform effort. Though the district intended to enhance student achievement and equalize educational opportunities for students, it instead caused the evolution of what can be called standards gaps, which resulted in differentiated curriculum and instruction along lines of students' academic ability.
Curriculum standards are sweeping across the education landscape. They are being developed and adopted at all levels of the American educational system (Marzano & Kendall, 1997; Ravitch, 1996). At the national level, the push for academic standards has bipartisan support, and federal officials strongly encourage the adoption of standards. In 1992, the National Council on Education Standards and Testing, a bipartisan group established by the Bush administration, issued a report supporting national standards and testing (Ravitch, 1995). The Clinton administration subsequently supported education standards through Goals 2000, which codified national goals and provided funds for states to develop standards and assessments (Ravitch, 1995). Federal legislation also prompted, and in some cases required, standards-based reforms (National Research Council, 1999). Albert Shanker, president of the American Federation of Teachers during the 1990s, similarly advocated higher academic standards and encouraged governors to create a national system of standards and assessment, calling for higher stakes and more powerful incentives that would make people take education seriously (Ravitch, 2000, p. 431). In addition, influential professional organizations, including the National Council of Teachers of Mathematics, the National Council of Teachers of English and the National Academy of Science, developed and promoted model standards for specific disciplines (Tucker & Codding, 2001).
Responding to the national emphasis on standards, most states launched some form of standards-based reform. Using funds provided through Goals 2000, states set out to raise the bar for schools through systemic reforms (Massell, 1994). While states have developed varying versions of systemic reform, what remains constant is a focus on curricular and instructional standards (Fuhrman & Massell, 1992). Most states, including California, developed and adopted standards across content areas. Without oversight, however, states developed many different versions of standards, which vary widely in rigor and implementation (Tucker & Codding, 2001).
In reaction to developments at the state and national levels, most local school districts adopted their states standards (Berger, 2000). Some districts, however, opted to create their own standards in core academic subjects, locally pursuing the overall goal of raising achievement for all students through the implementation of a standards-based curriculum and rigorous assessment (Gandal & Vranek, 2001). Though the act of adopting or developing standards is quite common across states and districts, there is considerable variation in what is included in the standards and how, or even if, they are implemented.
The widespread enthusiasm for standards springs from the belief that they can contribute to improving and equalizing student achievement. Academic standards are intended to create more intellectually demanding content and pedagogy, thereby improving the quality of education for all students, and to establish uniform goals for schools, thus producing greater equality in students academic achievement (Cohen, 1996; ODay & Smith, 1993; Rowan, 1996). Proponents claim that standards offer teachers a coherent guide for their instructional practice. By specifying what knowledge or skills students must demonstrate, standards point toward the instructional practices that teachers should employ (Cohen, 1996; Darling-Hammond, 1997; Rowan, 1996). Thus, as tools of reform, standards place a renewed focus on student learning and set uniformly high expectations for the academic performance of all students.
Standards-based reform, it is argued, will produce improved student achievement and equality of educational opportunity (Berger, 2000; Buttram & Water, 1997; Sirotnik & Kimball, 1999; Sutton & Krueger, 1997). By holding all students accountable for meeting the same set of standards, the reform aims to address the wide disparity in achievement among students. With a focus on equity, standards-based reform intends to depart from a system of differentiated curriculum that ultimately leads to inequities (Thompson, 2001).
PURPOSE OF THE STUDY
The present study stems from a larger project that examined the interplay between a district-wide standards-based curriculum program and a teacher-centered project to align mathematics standards across a high school, middle school, and elementary school. In conducting the larger study, we discovered that the districts implementation of its standards-based curriculum took an unanticipated turn, one that seemed to move away from the expressed purpose of setting uniform academic expectations and thus providing equality of educational opportunity. We learned that the districts standards were set lower than the states and that the district specified different levels of standards: minimum, essential, and accelerated. We were thus led to focus attention on what we call standards gaps. By this term, we refer to ways in which the districts standards and criterion-referenced test fall short of providing the uniform, rigorous, and comprehensive system that proponents of standards-based reform claim can enhance the overall achievement of students and equalize educational opportunity.
RELATED RESEARCH ON ACADEMIC TRACKING
Our discovery that the school district had developed multiple levels of curriculum standards led us to wonder if the districts standards possibly embodied something akin to academic tracking. That is, students would be held to different standards based on their perceived academic abilities. Given this turn, we investigated the literature on academic tracking, which proved useful in helping us to study how the district implemented its standards-based curriculum.
THE NATURE OF TRACKING
The general term tracking is often used to refer to two related practices: academic tracking and ability grouping. Academic tracking is the practice of dividing students into separate classes according to achievement. Students take different sequences of courses depending on their academic track: college-preparatory, general, or vocational. The term ability grouping refers to a specific kind of tracking, which involves dividing students into different classes within a given subject area. Typical divisions include advanced, average, and remedial. Though academic tracking and ability grouping are more extensive in secondary schools, the practices also exist at the elementary level. For example, elementary schools may have separate classes for gifted and talented programs and different reading groups within a given class.
Similar to the aims of standards-based reform, the professed goals of academic tracking and ability grouping are improved student achievement and equalized educational opportunity for all students. The reasoning is that, by grouping students with similar abilities or prior achievement, teachers will be better able to adjust instruction according to the specific needs of students. If teachers can meet the needs of all students, overall student achievement should be enhanced. Because tracking enables schools to provide educational treatments matched to particular groups of students, it is believed to promote higher achievement for all students under conditions of equal educational opportunities (Oakes, 1986, p. 13). With larger and increasingly diverse student populations, academic tracking and ability grouping are seen as offering teachers efficient ways to manage and address student differences and thus meet the individual needs of more students (Feldhusen, 1989).
THE IMPACT OF TRACKING
In examining the effects of tracking, certain researchers have found that, in contrast to the intended consequences, tracking did not increase overall student achievement in schools (Gamoran, 1992; Slavin, 1987, 1990). Moreover, rather than promoting equality of educational opportunity, tracking fostered inequity (Gamoran, 1986, 1992). The research suggests that tracking can affect key educational domains, including curriculum, instruction, and assessment.
Rather than exposing all students to a common curriculum, tracking leads to a differentiated curriculum. Teachers tailor their curriculum to the perceived needs and abilities of the students assigned to them, leading to important discrepancies in curriculum content across tracks (Oakes, 1986). Students in high-track English classes, for example, study classic and modern fiction and analyze writing, while low-track students focus on basic reading skills and memorization; similarly, high-track mathematics classes emphasize conceptual understanding, and low-track classes focus on computational skills and math facts (Oakes, 1985, 1986).
Instruction differs across tracks as well. Page (1991) reports that students in lower tracks receive instruction that is fragmented and emphasizes worksheets and recitation. In contrast, instruction in high-track classes covers more academic material, offers students more opportunities to think critically, and examines concepts in greater depth (Hallinan, 1987; Oakes, 1985; Slavin, 1990). Teachers of high-track classes invest more time in preparing to teach (Rosenbaum, 1976), and they offer clearer instruction, spend more class time on instructional activities, and include more variation in learning tasks (Oakes, 1985). In sum, the type and quality of instruction favors students in the high-track classes.
The assessment of student performance necessarily mirrors differences in curriculum and instruction. When teachers focus their curriculum and instruction on analysis, critical thinking, and problem solving, they tend to build such activities into their assessment practices. Similarly, when teachers emphasize memorization and basic skills, they tend to develop tests built around these activities. Students in high-track classes study and are assessed on content that is required for college and thereby stand to increase their scores on college entrance exams (Oakes, 1986). A national study that followed more than 20,000 students from Grades 1012 reported that students in the high academic track gained significantly more on tests of math, science, reading, vocabulary, writing, and civics, compared to similar students in general and vocational tracks (Gamoran, 1992, p. 12). The same trend occurs with ability grouping; students in high-ability groups show greater achievement gains than students in low-ability groups (Oakes, Gamoran, & Page, 1992).
Researchers have documented the net effects of tracking as largely deleterious. On a systemic level, tracking contributes to mediocre schooling for most students. Tracking exaggerates, rather than reduces, initial differences in student ability and performance. Rather than equalizing educational opportunity, academic tracking and ability grouping tend to produce inequitable outcomes that fail to increase overall student achievement.
The overall aims of both tracking and standards-based reform are to promote student achievement and to equalize educational opportunities. Given our discovery of the gap between the district and state standards and the differing levels of district standards, we questioned whether there might be unintended consequences similar to those that arise from the practices of academic tracking and ability grouping. Guided by research on tracking, we sought to answer the following questions in this study: In what ways are the districts standards differentiated by academic ability? To what extent and in what ways are standards gaps reflected in key educational domains including assessment, curriculum, and instruction?
The research project was conducted in a medium-sized school district located in a diverse, working-class community about 55 miles east of Los Angeles, California. The district includes a total of 22 schools: 2 comprehensive high schools, 1 continuation high school, 3 middle schools, and 16 elementary schools. Enrollment is approximately 19,000. The district serves high proportions of students from low-income households and ethnolinguistically diverse backgrounds. Five schools are designated as Title 1 schools. Fifty-two percent of the districts students qualify for free or reduced lunch, and 24% are designated as limited English proficient or non-English proficient. Fifty-seven percent of the districts students are Hispanic, 35% are White, 5% are African American, and 3% are Asian and Other.
Student performance on standardized achievement tests is typically below state and national norms. For example, in 1998, the districts percentile rankings across all subjects and grade levels fell below the 50th percentile, with the majority of rankings in the 30s. Although enhancing student achievement has long been a district goal, the district felt increased pressure in the late 1990s due to the states emphasis on standardized tests and the ranking of all schools based on student performance on these tests. In 1997, district administrators began developing a standards-based curriculum and a criterion-referenced testing system to assess student attainment of grade-level standards.
We employed a case study design, covering a 4-year period. Data collection drew from four sources: documents, interviews, observations, and surveys. The use of multiple types and sources of data provided the basis for triangulation during data analysis (Patton, 1987). The research team, which included the projects principal investigators and graduate student researchers, gathered and analyzed the data. Relevant documents included, for example, curriculum standards, position papers, curriculum guides, agendas, minutes of meetings, handouts from meetings and workshops, memos and correspondence, newsletters, district student report cards, and materials collected by teachers at workshops and conferences.
The research team conducted formal interviews with three groups of participants. First, we interviewed district administrators who were involved in developing and implementing the standards-based reform program: the superintendent, assistant superintendent of instruction, director of staff development, and director of curriculum. Second, we interviewed school principals to learn, from their perspectives, about the development and implementation of the standards-based program and the effects on curriculum, instruction, and assessment. Finally, we interviewed elementary, middle school, and high school teachers involved in the mathematics standards-alignment project. Each interview was audiotaped and transcribed.
Observations included activities related to the reforms and classroom teaching. The research team compiled field notes recording observed events such as project team meetings, sessions focused on reviewing and aligning standards, professional development workshops, department and grade-level meetings, and other activities related to the reforms. Members of the team observed in classrooms and collected observation summaries from teachers who visited other teachers classrooms as part of the mathematics standards-alignment project.
To verify emergent patterns in the case study, we administered a survey to all teachers in the mathematics standards-alignment project schools. The survey asked teachers to indicate their level of agreement (on a 4-point scale, ranging from strongly agree to strongly disagree) with statements about how the district standards influenced their classroom practices and to respond to open-ended questions about their curriculum and instruction.
Data analysis centered on case study analytic techniques, including pattern matching and explanation building (Yin, 1984). Data analysis occurred in three phases. First, upon discovering that the district standards were lower than state standards, we initially focused our analysis on identifying other gaps and variations in the district standards. In addition, we looked for related gaps in the districts testing system that was developed in connection with the standards. We employed a form of pattern matching (Trochim, 1989) by comparing data to the different types of gaps we identified.
Second, after identifying the gaps, we sought to ascertain the rationale─ from both administrators’ and teachers’ perspectives─ for embedding these gaps and variations into the standards-based reform effort. Here we engaged in explanation building (Yin, 1984) by noting excerpts from interviews in which administrators and teachers explained why gaps occurred in the districts standards and criterion-referenced test.
The third phase of our analysis centered on determining how the gaps affected the key educational domains of curriculum, instruction, and assessment. In examining the impact in these domains, we looked for differences between elementary and secondary levels as well as differences related to academic abilities of students. Throughout these analyses, we continually looked across the four data sources for disconfirming and corroborating evidence in the process of identifying patterns and explanations.
In conducting a larger study of the school districts standards-based reform effort, we examined the development process and learned that it began with a review of standards from California, other states, and national organizations (see Ogawa, Sandholtz, Martinez-Flores, & Scribner, 2003, for a discussion of the process). We discovered that the district ended up adopting standards that were lower than state standards and differentiated its own standards by academic levels, thereby creating what we are calling standards gaps. Upon closer examination, we discovered an additional gap in the districts assessment system, which is tied to the standards-based curriculum.
The three gaps in the districts standards and assessment system influenced three key educational domains: student assessment, curriculum, and instruction. In this section, we first examine the gaps and the reasoning behind the districts approaches. Then we explore how the gaps influenced assessment of student performance, curriculum, and instructional practices. Finally, we discuss differences between elementary and secondary levels and student performance on standardized tests.
The districts curriculum standards and associated assessment system are characterized by three kinds of gaps, two related to standards and one to the assessment system. In all cases, district and school administrators and, to a lesser extent, teachers explained the rationale for consciously building the gaps into the districts standards and criterion-referenced test.
We uncovered two kinds of gaps in the districts curriculum standards: a lag behind standards adopted by state and national professional organizations and standards differentiation by academic level.
Lag behind other standards. The districts standards lagged behind the states standards and other standards used as models, such as those of professional organizations. In some instances, district standards trailed state standards by more than a grade level. District administrators, school principals, and teachers readily acknowledged the difference between the state and district standards. In explaining the gap, district administrators described the need to develop standards based on local conditions. One administrator noted that, while state and national standards provide models, we best know our community. The assistant superintendent was even more specific: I wanted it to be a realistic curriculum for the kids in our district. He explained that most of the standards developed by professional organizations and states were bloated and quite world class. . . . Im not sure that we have a world class society, world class support from the state, world class communities to instantly implement these quite bloated standards. In a district position paper, he wrote, while some argue that adopting state standards raise [sic] expectations, others point out that unrealistically high standards lead to inappropriate frustration and failure rates. He suggested that the state standards are too substantive, lack clarity and relevance, and are skewed toward higher order skills. For example, he wrote, Many state standards are ambiguous, and most would argue with their breadth and their imbalanced emphasis on the highest levels of critical thinking at most grades. Concluding that adopting the state standards would lead to low implementation and not so exemplary student achievement, he proposed that state standards may be a good long-term goal (for stretching us to our potential over a period of 1015 years) but inappropriate for this more immediate time and place.
School administrators offered similar explanations and often echoed the words of district administrators. One principal said that the district used the state standards as a framework but adapted them and made them more specific for our needs. Another principal suggested that the district took a pragmatic approach to meeting the world class standards and that students will approach the state standards over time. The notion is that when students achieve mastery of district standards, the district will gradually elevate them until they match state standards.
Teachers, putting a slightly different spin on their explanations, proposed that the districts rationale for creating its own standards was to address the low expectations of the local student population. One teacher suggested, They [district administrators] thought the state standards were too high for the group of kids in our district; then they created something that they thought would be easier to meet. Another proposed that the feeling in our district was that we couldnt make standards that our students would not achieve. Some teachers questioned the strategy of gradually shifting to state standards. As one teacher said, If the content we are teaching is so far behind state standards, I think that we have wasted a lot of time and money with our own standards . . . which I think will eventually become the state standards.
Standards differentiation. Over 4 years of implementing its standards-based curriculum, the district categorized standards by academic level. The three levels, in ascending order, are (1) minimal, (2) essential, and (3) accelerated. The district also considered standards for English language learners and designated some standards special education until the superintendent insisted that special education cannot have different standards. Table 1 provides examples of the differing levels of standards.
Table 1. Examples of levels of standards
When asked about the reasons for creating these levels, the assistant superintendent said, I think we started with a philosophy and a theory, a belief that one size doesnt fit all. Kids vary significantly. To have one-size-fits-all world class standards would do a disservice to many children. He further explained that the state law was pushing for retention unless kids met standards. So we had to determine what standards needed to be met. . . . We felt it would be important to try to focus on the variations in kids and create differentiated standards. The assistant superintendent suggested that the districts differentiated standards should result in curriculum and instruction that accommodates the varying needs of students, including both high and low achievers. In a district position paper, he wrote, ‘‘In sum, the use of minimum and accelerated standards is smart curriculum development, helping teachers address a major task they face daily─ meeting students’ individual needs through pitching standards at an appropriate level of difficulty.
An Assessment Gap
The district developed an overall assessment plan which included both state and district tests. The district administers its own test four times a year: benchmark tests during the first three quarters of the academic year and a year-end criterion-referenced test. The districts tests are based on its curriculum standards. Each spring, students also take the state-mandated, standardized achievement test. Although state testing begins in the second grade, the district benchmark and year-end tests are administered to all students, including those in kindergarten and first grade.
The gap we identified in the assessment plan relates to the content of the districts tests. Although the district adopted standards in mathematics, language arts, science and social studies, the criterion-referenced test covers only mathematics and language arts. The plan is to develop criterion-referenced tests for social studies and science in the coming years, but only at the secondary level. In explaining this decision, the assistant superintendent said, At the elementary level, we are just trying to focus on the basics which to us is [sic] language arts, reading, writing, and mathematics.
IMPACT OF STANDARDS AND ASSESSMENT GAPS
We found that, like tracking, the gaps embedded in the districts standards and associated testing program affected student assessment, curriculum and instruction in ways that compromised the expressed purposes of improving academic performance and equalizing educational opportunity.
The districts criterion-referenced test duplicates the gap between the content of the state and district standards. The test was specifically developed to align with the district standards and, consequently, addresses the content of the district, not the state, standards. The test is based on the districts minimum and essential standards, but student retention is based on the minimum standards. Consequently, in practice, the minimum standards function as the grade-level standards. After the scores reach the 80% and 90% range at most of the district schools, the district eliminates the minimum standards, which occurred for Grades K2 by the 4th year of implementation. The assistant superintendent predicts that this process of eliminating minimum standards will be lengthy and may not occur at the secondary levels: It is real clear that, even in our own standards, we have many standards at the intermediate and secondary grade levels that, unless I am wrong, what we are going to have to do is probably diminish the essential standards back a bit, so we can get all the kids to master the standards.
After administering the criterion-referenced test, the district annually reports the performance of students. The district breaks out test results by teacher and by individual standards, providing a system by which teachers can be held accountable for the performance of students in their classes. Administrators view this feature of the test reporting system as an important way to monitor implementation of the standards. A school administrator pointed out that the criterion-referenced test results provide information on how well each teacher is teaching to the standards and can be broken out by individual standards. Hence, administrators are in a position to tell which teachers are covering which particular standard well or poorly. Another principal talked about two ways to determine whether teachers are teaching to the standards. One is to observe them and the other is to monitor the CRT [criterion-referenced test] scores. The assistant superintendent, in a lengthy memo to teachers, indicated that the student assessment program has only a weak and indirect link to teacher evaluation. Rather, he emphasized that the student assessments provide information that teachers should use to steadily improve the focus of their instruction toward improved levels of student success.
Curriculum and Instruction
Together with the districts standards, the components of the overall assessment plan and the way in which test results are used have influenced teachers practice by limiting the curriculum and shaping instructional practice. In this section, we describe the impact of the standards and assessment gaps on both curriculum and instruction because the two domains are so intertwined. Specifically, we found that the gaps affected curriculum and instruction in three main ways: curriculum restriction, instruction aimed at different standards, and limited instructional strategies.
Curriculum restriction. In response to the districts benchmark and criterion-referenced tests, which include only mathematics and language arts, teachers are restricting the curriculum to certain subjects, and within those subjects to content that reflects the districts standards. Elementary teachers tend to teach primarily mathematics and language arts, limiting or eliminating instruction in other subjects. Teachers note that science, social studies, and the arts receive little attention. On surveys, elementary teachers reported spending 70100% of their instructional time teaching language arts and mathematics. Nearly a third of the teachers reported spending no time at all on science and social studies. As one elementary teacher put it, There is no time for anything else, and I think that is my biggest concern. I mean music, art, social studies, and science are all important because they help the childs growth. . . . It has affected the scope of the curriculum. Its very limiting. Teachers attribute this change directly to the fact that the districts criterion-referenced test focuses only on mathematics and language arts. In describing the change in curriculum, an elementary teacher explained as follows:
We go by the standards. We are being told that your students have to meet these standards; they have to, have to. So we started getting rid of things that had nothing to do with the standards. Thats when we got rid of social studies and science. . . . The standards are it; and if it doesnt fit the standards, then toss it, because we have to stick to the standards.
Another elementary teacher pointed out that, at her school, teachers are required to label each lesson with a standard, and any that arent labeled, like a fun activity or review, are scrutinized by administration. In addition, teachers indicate that the school administration has designated a certain amount of time to be spent on language arts and mathematics each day; for example, in the primary grades, the daily guideline is 3.5 hours for language arts and 1.5 hours for mathematics. The weekly plan also includes 100 minutes for physical education, an average of 20 minutes per day. The elementary level report cards reflect the emphasis on language arts and mathematics. The district revised the report cards after launching the standards-based reform to include more information in language arts and mathematics. The report cards include the following as graded subjects: reading, writing, oral communication, mathematics, science, and social studies. The previous report cards also included physical education and visual and performing arts as graded subjects; but the revised version asks teachers to record only student participation in these areas. Some teachers report marking N/A (not addressed) for areas, such as social studies, that they have eliminated from the curriculum.
At the secondary level, a pattern of curriculum restriction is less evident. Given the graduation requirements and the organization of secondary schools, subject areas such as science and social studies remain key components of the overall curriculum. In many of the elective subjects, such as art and music, district standards have not yet been adopted. In the core subjects, the response of teachers is mixed. Many note little, if any, change in their curriculum because they continue to rely on textbooks or previously established curricular frameworks. Others report a push to cover more topics, and, consequently, they focus on breadth rather than depth. When asked about changes in curriculum and instruction since the district adopted standards, teachers wrote comments such as
No time for in-depth, extra, or supplemental work. Pushing to cover all the material.
Has become increasingly difficult to cover everything─so much to cover.
Great breadth of instruction with less depth.
I am not able to teach in depth in the core classes because I am trying to expose students to as many standards as possible.
One teacher, explaining the conflict in more detail, said the following:
We would like to do these extra things where students could really get an in-depth concept of what a fraction is or whatever [the topic] is, bu we feel so pressured by the amount of stuff that we have to teach, and these standards-based tests are kind of a pressure. And it is limiting your creativity in that way, because you would like to expand and take a couple of days on a concept and make sure the students really understand.
Some high school teachers suggest that their curriculum is shaped more by the upcoming California High School Exit Exam than the district criterion-referenced test. In an English department, the teachers met and determined the most important standards for helping students to pass the exit exam. In mathematics, teachers tend to follow the textbook, more than district standards, to prepare students for subsequent courses. In one department, teachers revised an Algebra 1 course and eliminated a chapter on graphing inequalities since it wasnt an essential district standard. Later, when the Algebra 2 teachers realized the topic hadnt been covered in Algebra 1, the curriculum adaptation became a problem, and the teachers agreed to follow the textbook topics.
In interviews, school administrators at all levels confirmed the emphasis on teaching to the district standards and acknowledged the expectation that teachers adjust their curricula. They pointed out that district administrators proposed a 70% to 30% guideline for classroom instruction:
The curriculum standards by rule of thumb should take about 70% of the classroom time. . . . That leaves the individual classroom teacher about 30% of the time for latitude and creativity to expand in the areas that they are most interested in or that their kids seem to be the most interested in.
It [the curriculum standards] really permeates every single thing we do. Field trips have to address standards. . . . I believe the assistant superintendent has asked the teachers to spend 70% of the time teaching to the standards and the other 30% doing twists and flairs to things they particularly enjoy.
One principal said that he tells teachers that it is imperative that you understand that part of this process means this is what you need to teach. If you need to drop some of the other things from your instruction in order to teach these, then you need to do that. The assistant superintendent agreed that the focus is definitely on mathematics and language arts and we minimize [science and social studies] as a district at the elementary level. He explained, We are not buying into that you can do everything. We are saying, What are the basics? We want kids to be able to read, write, and do arithmetic really well. An elementary principal, acknowledging the teachers concern about not having a well-rounded curriculum but fully supporting the districts focus on language arts and mathematics, suggested that teachers can incorporate science and social studies into language arts through reading and writing. He proposed, It doesnt have to be science separately from reading.
The differing language used by teachers and administrators underscores their differing perspectives about the restricted curriculum. What teachers describe as vital to a well-rounded curriculum and student growth, administrators refer to as areas of enrichment or special interest.
Instruction aimed at different standards. As a result of the district standards, teachers emphasize different standards in their instruction depending on the academic abilities of students in the class. At the elementary level, where students in each class have a broad range of academic abilities, teachers often feel compelled to aim their curriculum and instruction at the lower standards because those standards are emphasized in the districts criterion-referenced test. Since test results are reported in terms of how each class performed on each standard and student retention decisions are based on the minimum standards, teachers are motivated to ensure that all students attain minimum standards, despite wide variation in their academic abilities. Rather than raising the academic expectations for students, the districts standards in effect prompt teachers to teach toward a lower level. The following teachers comments illustrate this contradiction:
I dont think the standards stress higher level thinking skills. The standards require only the bare minimum, and they just want what will make them [the students] pass the test, what the test requires. In some ways, it is lowering the expectations.
The CRTs [criterion-referenced tests] tell us what standards our whole class knows, and they only test the minimum. . . .It has made instruction so targeted toward just minimum standards. Basically what we teach all day is the minimum standards and trying to get the students up to par.
One teacher we observed feels the test pressure so much that she frequently mentions the state and district tests during instruction, telling students, This is something that is really heavily tested. She explained, I am not teaching to the test. I am teaching to the standards. But I am telling my students, This is what you are tested on. . . . You need to do the best you can on this test.
At the secondary level, teachers tend to emphasize different standards, depending on the academic ability and level of students in their classes. That is, teachers teach accelerated, or the highest, standards to students in their most advanced courses and minimum, or the lowest, standards to students in remedial courses. In middle school mathematics, for example, a teacher described the difference in focus this way: We emphasize minimum and essential standards in general math classes. Higher level skills receive more emphasis in CPM [college preparatory math] and algebra classes.
That teachers emphasize different standards according to the academic ability of students is a natural, and perhaps not unintended, consequence of the districts development of differentiated standards. In a district position paper, the assistant superintendent suggested that minimum standards help define priorities and can guide us in various ways, including our purchase of materials, our instructional focus, and our retention decisions. He further explained that accelerated standards assist teachers in helping students, who quickly master essential standards, progress to higher levels. The expectation is that providing curriculum at an appropriate level of difficulty will lead to student achievement. The districts differentiated standards clearly reinforce tracking based on student ability in secondary schools.
Limited instructional strategies. Teachers, particularly those in elementary grades and lower level secondary courses, employ instructional strategies that emphasize basic skills corresponding to district standards in mathematics and language arts. On survey questions, more than 90% of elementary teachers indicated that they spend the majority of their instructional time working on improving students basic skills, in contrast to 40% who indicated working on improving students’ higher order thinking skills. Elementary teachers, for example, described their classroom instruction as ‘‘focused only on the standards,’’ ‘‘regimented,’’ and ‘‘standards-based─ a lot of basics, a lot of drill and kill, a lot of practice, a lot of routinization. Other elementary teachers said they did not have time to do as much hands-on learning and have had to let go of thematic units that cover a quarter of the year.
In describing how their curricula and instruction have changed since the district adopted standards, high school language arts teachers responded:
Less literature, less discussion, less interaction with students about everyday problems. More skills preparation, more in-depth on basic elements, more review until students get it.
I have had to let go of many enrichment activities and focus on skills, skills, skills.
Teachers explain that such targeted and concentrated activity is necessary to maximize the likelihood that students will attain standards that are covered in the districts year-end, criterion-referenced test. As a middle school teacher wrote, I am specifically addressing standards hoping to better prepare students for testing success. An elementary teacher suggested,
Those [teaching strategies] are all gone, just because everyone is at different levels. You have little Jimmy over here who has met half of the standards, and you have Johnny who only knows one standard, and you need to work with Johnny quickly because he will be taking the tests soon. There is no time for manipulatives or group work. Your scores are being seen by everyone in the district.
One teacher said she was considering rewarding the achievers with more creative activities as an incentive to students who still have not achieved the basics that we need to pound away on.
Instead of exposing all students to more engaging and stimulating teaching strategies, teachers often end up limiting their instructional repertoire, emphasizing drill and practice, and focusing on basic skills rather than conceptual understanding. One teacher described her instruction as including less open-ended concept-based activities and more drill and worksheets. [I] rush through curriculum faster to meet testing deadlines. Another noted, We are going back to basics. We are really taking out the critical thinking. [Our teaching] is: this is how you do it, this is what you do. A high school mathematics teacher suggested that the testing system has a great deal of influence over how we teach. He went on to point out how standardized testing emphasizes the correct answer rather than conceptual understanding:
I will often give students partial credit on unit tests if they understand the process, say of solving equations, but make a simple error between positive and negative numbers. But on a standardized, multiple-choice test, there is no way to make an allowance for conceptual knowledge. . . . Testing does not care about the process or conceptual knowledge. All it cares about is the correct answer. I feel conflict between trying to help my students be independent workers and the need to teach test-taking skills.
A middle-school mathematics teacher expressed similar concerns about students who are mechanically going through steps, but dont understand what they are doing. She said,
because of the pressure to teach the standards and have your students successful on the standards, which is measured by a standardized test, you are focusing on algorithms as opposed to thought processes, how to think through it, problem solving. . . . We are creating a group [of students] who cant think.
In observing other mathematics classrooms, teachers noted the dominance of direct teaching and the emphasis on procedures. In written observation notes, one teacher commented on how the teacher followed the textbook that involved a lot of rote, procedural methods and added, It looked pretty boring, really. She described the student work on problems as Now repeat after me 1,000 times so youll remember. Rather than noting a contrast with her own instructional approaches, she concluded, I realized I teach very rotely most of the time.
Differences between elementary and secondary levels. Our findings suggest that unintended consequences related to curriculum and instruction are more prevalent at the elementary level. These unintended consequences appear correlated to higher levels of implementation of the standards-based reform at the elementary level. District and school administrators consistently indicate that elementary teachers have adopted the districts standards more readily than secondary teachers. As a principal put it, the implementation at the middle school and high school levels has been less uniform. The assistant superintendent, responding to an interview question about how the reform had progressed over four years, distinguished between the elementary and secondary levels, using letter grades to assess the progress.
[At the elementary level] I would give it a pretty solid B to B+. I think the secondary level is probably more at the C level. It is a real mixed bag there. It is kind of like changing a university─ a lot of independent contractors. Professors tend to have total academic freedom, and I think secondary people feel the same bent. They want to do their own thing, and they haven’t quite caught on to the standards movement.
A high school administrator stated that while some of his teachers have really taken it [implementation of district standards] to heart, we are a long way from having done a good job at the high school level. He suggested the reason was due to differences in school size, departmentalization, teacher isolation, and a different mind set between high school teachers and elementary teachers. High school teachers are far more independent, self-employed people than elementary teachers. In the view of administrators, the independence of secondary teachers inhibits their implementation of the district standards, which in turn limits student performance on the district tests.
STUDENT PERFORMANCE ON STANDARDIZED TESTS
Administrators tend to gauge teachers implementation of the district standards by student performance on the district benchmark and year-end tests. The assistant superintendent projected that a 90% implementation rate among elementary teachers would show real positive gains for our achievement. Figures 14 summarize student scores on state and district tests in mathematics and language arts over a 3-year period. Figures 1 and 2
present the districts percentile rankings on the SAT-9, the standardized achievement test used by the state during the period of this study. Figures 3 and 4 summarize student scores on the districts year-end criterion referenced test. As described previously, the districts test covers only mathematics and language arts.
The figures illustrate a clear pattern of higher scores on the district test than the state test. Except for four instances at the elementary level, the rankings on the state test are below the 50th percentile across Grades 2 through 11 for all 3 years. On the district test, scores in the early elementary grades (K3) are predominantly in the 7090% range, while scores at the secondary level are in the 5065% range. The general pattern of higher
scores in the early elementary grades is similar on both the district and state tests. On the state test, the rankings show gradual increases in the elementary grades, with the greatest gains in mathematics in Grades 2 through 4. At the secondary level, the rankings generally remain flat with some slight decreases. On the district test, there tends to be more variation but a general pattern of higher scores and greater gains in the elementary grades than the secondary grades.
By emphasizing scores on the district test, administrators are able to report higher scores and to create a sense of greater student achievement than is evident on the state test. Administrators point to rising scores in the early grades as evidence that instruction at those grade levels is aligned with district standards and that improvement on district tests will lead to higher achievement on state tests. Yet the overall gap between scores on the state and district tests remains striking.
Though student achievement was not the focus of this study, it seems clear that the gap between scores on the district and state standardized tests reflects the gap between district and state standards. The emphasis on district standards appears to have had little effect on most students overall performance on state tests; rankings remain predominantly below the 50th percentile. Our data on student performance are limited and inconclusive, but the overall trends indicate a substantial and continuous difference between scores on district and state tests. Rather than achieving its overall aim of improving students academic performance, the reform instead may be masking the consequences for student achievement.
States have adopted curriculum standards to equalize educational opportunities and enhance the academic performance of students. Analysts have noted, however, that the decisions and actions of local school districts and schools are the critical link between standards and the instructional practice of teachers (Berger, 2000; Sirotnik & Kimball, 1999). By analyzing a school districts development and implementation of a standards-based curriculum that was prompted in part by the states adoption of curriculum standards, we uncovered unintended consequences that actually work against equality of educational opportunity.
We found that the district, in developing its own standards and assessment system, incorporated what we have termed standards gaps. The districts standards lag behind state standards and other standards, such as those of professional organizations, used as models in the development process. The district designated three levels of standards, ranging from minimum to essential to accelerated. In addition, the districts assessment system tests students only in mathematics and language arts. We found that these gaps influenced how students are assessed, the content of what is taught, and the instructional strategies that teachers employ. The districts criterion-referenced test emphasizes minimum and essential standards, which fall below state standards; and student retention is based on the minimum standards. Consequently, teachers, particularly at the elementary level, focus almost solely on mathematics and language arts, eliminating other subjects including science, social studies, and art. Teachers teach to different standards depending on the academic levels of their courses and students. In addition, they tend to use instructional strategies that target the lower standards, relying largely on approaches that emphasize basic skills.
These findings mirror the results of research on the impact of tracking which, like standards, is purported to improve student achievement and equalize educational opportunity. Instead, tracking leads to a differentiated curriculum and the use of different instructional strategies depending on the perceived academic ability of students, which results in limited schooling for most students and thus exacerbates initial differences in student ability and performance. The similarities between the findings of the present study and research on tracking lead us to two related conclusions.
First, the adoption of statewide or national standards to set uniform expectations for the academic performance of all students offers a limited strategy of educational reform and improvement. Such standards confront educators who work in schools and districts that historically have performed below state and national norms with what must seem another occasion for failure. Left to implement standards without additional resources and under local conditions, administrators and teachers in this study explained their desire to adopt realistic standards that more likely could be attained by their students, avoiding failure and frustration. Thus, the district adopted standards that fall below state standards and developed multiple levels of standards, using a local criterion-referenced test to assess the attainment of lower level standards. As we explain later, in adopting this strategy, the district enacted a rationale and practice that has long been employed by American schools.
Second, at least in the case of this one school district, the same assumptions that underlie tracking also motivated the use of standards. These assumptions hold that schools can improve the overall academic performance of students and enhance equality of educational opportunity by organizing to enhance the ability of teachers to meet the needs of students with various levels of academic ability and achievement. Both strategies also assume, of course, that students can be meaningfully placed into separate categories of academic ability using indicators that include test scores and letter grades in previous classes.
Tracking and standards differ in the mechanisms they employ to differentiate students along the lines of academic ability. Tracking and standards, in fact, emphasize opposite ends of the instructional process. Whereas tracking organizes inputs in the form of student ability into homogeneous groups, standards establish expected levels of output, or student attainment, for students of different levels of ability. The end result is the same. Students who are considered to have high ability receive richer curriculum and instruction than those who are considered to have low ability. Moreover, in the present study, the school district set its standards below those of the state to match the ability of its students.
As a consequence, the districts standards-based curriculum has undermined the expressed purposes of improving academic performance and providing equal educational opportunities. Rather than holding all students to uniform standards and providing educational opportunities to attain those standards, all students are held to standards lower than those adopted by the state. Moreover, students, particularly at the secondary level, receive different content and instruction depending on their academic ability, belying the intention of providing equality of educational opportunity and attainment.
What purpose, then, is served by the district adopting a standards-based curriculum that is characterized by the gaps we have described? One plausible answer is that the purpose served by the locally developed standards-based curriculum and assessment system is a symbolic one (Ogawa et al., 2003). By developing standards that fall below the states standards and then testing for attainment of lower grade-level standards, the school district increases the likelihood that high proportions of students will reach standards at their respective grade levels. Administrators explained that the districts standards would be raised over time, as increasing numbers of students attained existing standards, until eventually they would match the standards set by the state. Many teachers, however, held that the district did not want to adopt standards that its students could not meet, suggesting that the local standards were in keeping with the low expectations that many administrators and even teachers held for the local student population. Indeed, an administrator explained that unrealistic standards can produce only frustration and failure, not advances in student performance. Hence the district reports positive gains on its criterion-referenced test, while its schools and most of its students underperform on the states achievement test. But the district has demonstrated that it has adopted standards and that increasing proportions of students are meeting them, ignoring the gaps that undermine the stated purposes of enhancing academic achievement and equalizing educational opportunity.
While we recognize the limitations of a single case study for deriving implications for policy and practice, our findings raise issues that we believe warrant the attention of policy makers, practitioners, and researchers. The findings remind state policy makers that curriculum standards alone will not produce curriculum and teaching that will close the achievement gap between schools that serve wealthy communities and schools that serve low-income and minority communities. Policies must attend to the maintenance of high expectations for all students and the provision of resources─ financial, human, and the like─ to equalize educational opportunity.
Our findings also suggest that local policy makers and practitioners need to consider unintended consequences of standards and assessment on curriculum and instruction provided to students in historically low-performing communities. Aiming instruction at lower standards, while perhaps well-intentioned, can both lower expectations and produce uninspired teaching.
Similarly, this study reinforces the need for researchers to identify and examine the significance of unintended consequences of reform. As in other cases of reform, contradictions may arise between the intent of policies and what actually happens in classrooms (McNeil, 2000). Through detailed studies of local implementation that extend over time, we can begin to uncover how widespread these unintended consequences are.
We thank Marilyn Martinez-Flores, who was a member of the research team on the larger project, and the teachers and administrators who participated in this study. The research on which this article is based was supported by grants from the School University Partnership Program of the University of California, Riversides ALPHA Center and the Spencer Foundation. A version of this article was presented at the 2002 meeting of the American Educational Research Association, New Orleans.
Berger, J. (2000). Does top-down, standards-based reform work? A review of the status of statewide standards-based reform. NASSP Bulletin, 84, 5765.
Buttram, J. L., & Waters, T. (1997). Improving Americas schools through standards-based education. NASSP Bulletin, 81, 16.
Cohen, D. K. (1996). Standards-based school reform: Policy, practice, and performance. In H. F. Ladd (Ed.), Holding schools accountable: Performance-based reform in education (pp. 99127). Washington, D.C.: Brookings Institution.
Darling-Hammond, L. (1997). The right to learn: A blueprint for creating schools that work. San Francisco: Jossey-Bass.
Feldhusen, J. F. (1989). Synthesis of research on gifted youth. Educational Leadership, 46(6), 611.
Fuhrman, S., & Massell, D. (1992). Issues and strategies in systemic reform. New Brunswick, NJ: Consortium for Policy Research in Education.
Gamoran, A. (1986). Instructional and institutional effects of ability grouping. Sociology of Education, 59, 185198.
Gamoran, A. (1992). Synthesis of research: Is ability grouping equitable? Educational Leadership, 50(2), 1117.
Gandal, M., & Vranek, J. (2001). Standards: Here today, here tomorrow. Educational Leadership, 59(1), 713.
Hallinan, M. T. (1987). Ability grouping and student learning. In M. T. Hallinan (Ed.), The social organization of schools: New conceptualizations of the learning process (pp. 4169). New York: Plenum.
Marzano, R. J., & Kendall, J. S. (1997). National and state standards: the problems and the promise. NASSP Bulletin, 81, 2641.
Massell, D. (1994). Achieving consensus: Setting the agenda for state curriculum reform. In R. Elmore & S. Fuhrman (Eds.), The governance of curriculum: 1994 Yearbook of the Association for Supervision and Curriculum Development (pp. 1229). Alexandria, VA: Association for Supervision and Curriculum Development.
McNeil, L. M. (2000). Contradictions of school reform: Educational costs of standardized testing. New York: Routledge.
National Research Council. (1999). Testing, teaching and learning: A guide for states and school districts. Washington DC: National Academy Press.
Oakes, J. (1985). Keeping track: How schools structure inequality. New Haven, CT: Yale University Press.
Oakes, J. (1986). Keeping track (Part 1). Phi Delta Kappan, 68(1), 1217.
Oakes, J., Gamoran, A., & Page, R. N. (1992). Curriculum differentiation, opportunities, outcomes, and meanings. In P. W. Jackson (Ed.), Handbook of research on curriculum (pp. 570 608). Washington DC: American Educational Research Association.
ODay, M. S., & Smith, M. S. (1993). Systemic reform and educational opportunity. In S. H. Fuhrman (Ed.), Designing coherent education policy: Improving the system (pp. 250312). San Francisco: Jossey-Bass.
Ogawa, R., Sandholtz, J. H., Martinez-Flores, M., & Scribner, S. (2003). The substantive and symbolic consequences of a districts standards-based curriculum. American Educational Research Journal, 40(1), 147176.
Page, R. N. (1991). Lower track classrooms: A curricular and cultural perspective. New York: Teachers College Press.
Patton, M. Q. (1987). How to use qualitative methods in evaluation. Newbury Park: Sage.
Ravitch, D. (1995). Introduction. In D. Ravitch (Ed.), Debating the future of American education: Do we need national standards and assessments? (pp. 18). Washington DC: Brookings Institution.
Ravitch, D. (1996). The case for national standards and assessment. The Clearing House, 69, 134136.
Ravitch, D. (2000). Left back: A century of failed school reforms. New York: Simon & Schuster.
Rosenbaum, J. E. (1976). Making inequality: The hidden curriculum of high school tracking. New York: Wiley.
Rowan, B. (1996). Standards as incentives for instructional reform. In S. H. Fuhrman & J. A. ODay (Eds.), Rewards and reform: Creating educational incentives that work (pp. 195225). San Francisco: Jossey-Bass.
Sirotnik, K. A., & Kimball, K. (1999). Standards for standards-based accountability systems. Phi Delta Kappan, 81, 209214.
Slavin, R. E. (1987). Ability grouping and student achievement in elementary schools: A best-evidence synthesis. Review of Educational Research, 57(3), 293336.
Slavin, R. E. (1990). Achievement effects of ability grouping in secondary schools: A best-evidence synthesis. Review of Educational Research, 60(3), 471499.
Sutton, J. T., & Krueger, A. (1997). Do we need NEW national standards in mathematics and science? NASSP Bulletin, 81, 4855.
Thompson, S. (2001). The authentic standards movement and its evil twin. Phi Delta Kappan, 82(5), 358365.
Trochim, W. (1989). Outcome pattern matching and program theory. Evaluation and Program Planning, 12, 355366.
Tucker, M. S., & Codding, J. B. (2001). Setting high standards for everyone. In The Jossey-Bass reader on school reform (pp. 470482). San Francisco: Jossey-Bass.
Yin, R. K. (1984). Case study research: Design and methods ((2nd ed).). Thousand Oaks, CA: Sage.