An Uncertain Lever: Exploring the Influence of State-Level Testing in New York State on Teaching Social Studies
by S.G. Grant - 2001
In this paper, the author presents case studies of two high school social studies teachers and the influence of state-level testing on their teaching practices. Based on classroom observations of a unit on the U.S. civil rights movement and teacher interviews before and after that unit, he examines the role the eleventh-grade New York State Regents test in U.S. History and Government plays in each teacherís instructional planning, delivery, and assessment. His analysis suggests that while the state test figures into each teacherís instruction, it does so by interacting with a range of other factors, especially the teachersí views of subject matter and learners. From these findings, he argues that state tests may be an uncertain lever and that, while reformers continue pinning their hopes on new tests, faith in tests as a means of instructional change may be hard to sustain.
Let me begin with a brief story. A third year high school social studies teacher sat in my office one day. Visibly upset, she said that her students had performed poorly on the New York State Regents examination and that the school board was delaying for a year its decision on her tenure to see how the next group of students performed. I had not seen this woman teach (she came to me as a student in a masters-level social studies course), but she seemed thoughtful, concerned about her students, and committed to ambitious teaching—and scared. She explained that she had had students take practice test questions throughout the year and that she had spent the last six weeks of the year reviewing for the test. “What am I supposed to do?” she asked. Clueless, I asked, “What do you want to do?” “I don’t know,” she said, “but I’m not going to do all that stuff again.” If, as we are so often told, tests drive teaching, then where is the state Regents test driving this teacher?
Whether truth, myth, or some of both, it’s the rare educator who has not heard the assertion “tests drive ____” (fill in the blank with the object of your choice—instruction, curriculum, change). Controversy around the role of state-level testing in schooling has percolated for more than thirty years, and it shows no signs of abating. Some argue that the influence of state-level assessments is overblown (Cohen & Barnes, 1993; Firestone, Mayrowetz, & Fairman, 1998). Others argue that the influence is weak, but still vibrant (Corbett & Wilson, 1991; Koretz, 1995). And still others argue that tests have a real and definite influence (or at least, should have) on classroom teaching and learning (Feltovich, Spiro, & Coulson, 1993; Frederickson, 1984; Heubert & Hauser, 1999; Popham, 1998; Shanker, 1995; Smith & O’Day, 1991).
The talk about testing seems endless. Cohen and Barnes (1993) point out, however, that most of that talk is based on interviews and self-report; little reflects observations of teachers’ actual instructional practice. It would be silly to discount interview and self-report data, but the lack of observational data begs the questions of if and how tests influence teaching and learning.
In this paper, I use case studies of two
The two teachers,
Strait was part of a research study sample of urban and suburban teachers who were identified by district curriculum coordinators as taking an innovative approach to teaching social studies. After observing a number of her classes and interviewing her several times over the course of a year, I became interested in also studying her colleague, Blair, who she described as “a total opposite from me.” The notion that two teachers with so many surface similarities (i.e., academic background, type of students, school context, and state-level test) could construct radically different instructional practices intrigued me, and so I approached Blair and secured his permission to study his practice over the next year.
Data collection consisted of observations and interviews. I observed and interviewed each teacher on a number of occasions. For this paper, however, I highlight each teacher’s unit on the
I interviewed each teacher twice around their units. The first interview consisted of questions related to the teachers’ knowledge and interpretation of the state social studies framework and if and how their classroom practices have changed over time. The second interview focused on the civil rights unit. Here, I asked how the teachers decided what to teach, what they hoped students would learn, and how, if at all, their teaching of this unit was different from the previous year’s. Questions for both interviews came from semi-structured protocols; all interviews were audio-taped and transcribed.
My analysis reflects the interpretative tradition within qualitative research (Bogdan & Biklen, 1982; Connelly & Clandinin, 1990; Erickson, 1992).To analyze these data, I coded field notes and interview transcripts with two principal interests in mind. One interest was pedagogical. Here, I focused on each teacher’s instructional planning, delivery, and assessment. My second interest was in influences on teachers’ practices. In an earlier paper (Grant, 1997a), I found that teachers’ views of subject matter and learners figured prominently in their pedagogical decisions. I return to those factors here in my analysis of how testing influences teachers’ instructional practices.
ON TESTS AND TEACHING
Controversial as they are, standardized tests nevertheless draw attention. The professional literature is replete with debates about tests as a means of accountability, as measures of performance, and as levers of change (Corbett &
Advocates of tests as a vehicle for driving educational change tend to cite general good effects rather than specifics. Some (Feltovich, Spiro, & Coulson, 1993; Finn, 1995; Popham, 1998; Shanker, 1995) argue that good tests will inevitably drive good instruction. With no more specificity, Popham, Cruse, Rankin, Sandifer, and Williams (1985) suggest that what is being tested is important and that good results equal good education. Systemic reformers (Fuhrman, 1993; Smith & O’Day, 1991) include testing as part of an overall strategy aimed at fundamental school change. Others (English, 1980; Glatthorn, 1987; Heubert & Hauser, 1999) simply argue that because standardized tests are a reality in most school districts, they should be used as a fundamental part of curriculum planning.
Critics of testing tend to be a bit more direct in their assessment of the impact of testing on teaching. Madaus (1988) claims, among other things, that teachers will teach to the test, that they will adjust their instruction to follow the form of the questions asked (e.g., multiple choice, essay), and that tests transfer control over the curriculum to whomever controls the test. LeMahieu (1984), Koretz (1995), and Romberg, Zarinnia, and Williams (1989) are a bit more tentative, but they too conclude that teachers might tailor their curricula to the content covered on the test. Recent empirical work supports some of these claims. Corbett and Wilson (1991), for example, argue that testing, especially minimum-competency testing, has a pernicious effect on teachers in that it causes them to narrow their sense of educational purposes and to focus on activities designed to raise test scores whether or not they think those activities are good for students (see also Haladyna, Nolen, & Haas, 1991; Noble & Smith, 1994; Smith, 1991). They conclude that squeezing teachers in this fashion encourages them to rebel against reform measures good and bad. “Statewide testing programs do control activity at the local level, but the subsequent activity is not reform” (Corbett & Wilson, 1991, p. 1).
Other researchers are less sure that a direct relationship exists between standardized testing and teachers’ classroom practices. Freeman et al. (1980), Grant (2000), Kelleghan, Madaus, and Airasian (1982), and Salmon-Cox (1981) found little direct impact of standardized testing on teachers’ daily instruction. Pellegrino (1992) makes the point bluntly: “[Standardized] test information is of little practical assistance to teachers and other individuals involved directly in the instructional process” (p. 277).
Two issues surface from this brief review. One is that we simply do not know very much about the relationship between teachers and tests. The relationship between students and tests has been much explored, but a similar volume of work inquiring into if, how, and in what ways teachers are influenced by standardized tests does not exist. The second point, not surprisingly, is that the research around teachers and tests shows no clear pattern of influence. Tests seem to matter, but how and to what extent is unclear.
REGENTS TESTING IN
The Regents test is an established tradition in
Several other features of the Regents testing program mark it as unique among state testing efforts. First, the Regents exams are more than assessments of basic skills. Regents tests are generally viewed as academically challenging and as strong measures of student performance. The social studies tests, in particular, are content driven, with a combination of multiple-choice and essay questions that reflect the state-recommended courses of study. The Regents test in U. S. History and Government, for example, consists of two parts. The first is a set of 48 multiple-choice questions worth 55 points. These questions ask students, for example, to define terms (e.g., nativism), recall portions of the U.S. Constitution (e.g., congressional powers), and interpret political cartoons and graphs. The second part involves writing three essays. In the first section, students select one of two essay prompts. In the second section, they must write on two of five prompts. The three essays are worth 45 points. Content of the essay questions varies, but a typical question asks students to explain a concept (e.g., nationalism) and then to describe how that concept played out in a number of historical settings. (See below for examples of essay questions.) A second feature of the state tests is their high stakes: Students must pass all the exams to receive a Regents diploma. Students may opt to take the easier Regents Competency Test (RCT) and receive a school diploma. A Regents diploma, however, carries greater status. One last feature of the Regents assessment program is that they are graded by the classroom teachers who administer them. Tests are not scored blindly, nor are teachers’ grades systematically reviewed.
Regents tests are no less high stakes for teachers than they are for students. Since the mid1990s, state policymakers have introduced a number of curriculum reforms (e.g., new state standards for social studies), yet it is concern about the state tests which surfaces most regularly in teachers’ talk (Grant, 1997b, 2000). This makes sense for two reasons. First, the curriculum documents produced thus far offer teachers little assistance in making concrete instructional decisions (Grant, 1997c). Second, the messages teachers receive often promote the view that tests are intended to drive change (Grant, 1997b). For example, during sessions devoted to new state social studies standards, one representative from the
TEACHERS AND TESTS
George Blair and
Before focusing on tests, however, let’s look at these two teachers’ instructional practices. George Blair and
GEORGE BLAIR: FRAMING A MASTER NARRATIVE
George Blair does not teach a civil rights unit as such. Instead, he addresses civil rights issues and events as they occur in the chronological order his textbook presents. Rather than march through endless names, dates, and places, however, Blair constructs a master narrative which frames each curriculum unit. The textbook chapter under study was Dwight Eisenhower’s presidency. In Blair’s hands, it became a story of how Eisenhower faced and managed dilemmas in foreign and domestic policy. Here is a glimpse of that framing narrative:
Eisenhower was conservative....But it will blow up in his face....He made several appointments to the Supreme Court, but one at least is very liberal . . . and (emphatically) that shocks the hell out of Eisenhower.... Remember there was tremendous pressure....Very serious things happen and early on in Eisenhower’s presidency....He’shit in the face with the Brown decision....Eisenhower disagrees, but he has to enforce it and he does....and there is a serious confrontation in the South....
Eisenhower also confronts the Soviets....We hate the
With that setup, Blair begins a lecture on
Now we move on to some more interesting things. . . . I remember a lot of this....This is the beginning of the serious civil rights....Now you remember Plessy versus
The issue is going to come up again.... Several decisions will be made [around] equal rights for the black population.... In 1953, Eisenhower appoints a new chief justice . . . and it was not a popular choice....Earl Warren was not a great jurist....he was a politician in
After that decision, the South refuses to integrate schools....
(Quietly) One of the young ladies recently published a book . . . and she talks about the threats on her life....The threats to her life were unbelievable.... [She talked about how] the black community took the kids away every summer and put them with black families around the country.... This woman lived with a doctor in
She told stories of kids kicking her and pushing her down stairs.... (Softly) And when I read this, tears came to my eyes . . . man’s inhumanity to man . . . .
Here, Blair pauses, walks over to the overhead projector, puts up the next set of outline notes, and then walks back to the right-hand corner of the room. He continues in a more matter of fact tone, but his voice gradually grows louder and his tone more insistent:
I don’t think we need to spend a lot of time on Rosa Parks....Civil rights just gets going and going and going....Rosa Parks was just a plain, simple lady.... She refused to give her seat up to a white man. . . .When the buses were busy, blacks had to move to the back of the bus....Rosa Parks refuses and when push comes to shove, she’s arrested....The ultimate threat to blacks was “Don’t you know your place?” (Sadly) I know you don’t identify with this and I’m glad you can’t.... [There were so many] gutsy folks . . . moving toward civil rights . . . and I hope some day we’ll have true civil rights....
(Loudly) What happens? A Baptist minister by the name of Martin Luther King comes to
(Softly) As I say these things, things go through my mind . . . very big things. This was a very painful time for both the black and white population.... This was not trite, it was earth shattering. . . . I know I’m going through these things quickly, but they are not trite....
In this presentation of civil rights, we see several elements of George Blair’s narrative instructional style. His story is faithful to the facts—the important people (Rosa Parks, Martin Luther King, Thurgood Marshall), policies (Brown v. Board of Education), and events (the confrontation at
It is a solo performance though. In fact, it is difficult to know what sense students make of this story. For while Blair talks, the students copy notes displayed on the overhead projector. Few look up and there were no student questions or comments during this segment. Nevertheless, it is difficult to imagine them being completely disengaged from the story told.
Three features define Blair’s narrative approach. One is the notion of weaving together the
George Blair’s approach to teaching civil rights is to take it as it comes in each textbook chapter. In this unit on the Eisenhower administration, Blair’s instruction consists of displaying an outline of the textbook chapter on an overhead projector and lecturing while students copy the overhead notes. The actual time spent on civil rights issues in this unit is about 20 minutes, and the unit ends with a chapter test. Later in the year, Blair picks up the theme of civil rights during a unit on the Kennedy and Johnson presidencies. His instructional approach of combining dramatic stories with outline notes on the overhead remains consistent.
GEORGE BLAIR: TEACHING AND TESTS
From the above, it appears George Blair gives little explicit attention to the Regents test. In point of fact, the only explicit reference to the test surfaces as a seemingly offhand remark. During his explanation of the Cold War, Blair describes Harry Truman’s dismissal of General Douglas MacArthur and the appointment of General Omar Bradley as commander of the UN forces in
That said, one might argue that Blair’s instruction implicitly mirrors the state test. After all, the content covered reflects attention to the kind of big ticket names, places, and events that are commonly assessed on standardized tests. In that context, Blair’s dismissal of Omar Bradley makes sense; in effect, Blair is telling his charges that, while Bradley’s name and contribution probably would not appear on the test, there is a good chance the other people and actions he mentions may. Blair’s instruction may also be seen as aligned with the state test. Blair uses a narrative approach to deliver his lessons, and that approach would seem well-suited to cover the material which routinely appears on the state test. That approach allows Blair to weave together the relevant names, dates, and places. It also gives him an opportunity to craft one or more storylines as a means of helping students remember the information. Finally, Blair’s approach to assessment also suggests attention to the Regents examination. The state test features both multiple-choice and essay questions but gives more weight to the multiple-choice section (55 points to 45 points). Blair’s tests do not always include an essay question, but they do always feature objective-style questions, and of those, multiple-choice questions dominate. In these several ways, then, one might argue that George Blair’s instructional practice, if not explicitly attentive to the Regents test, is at least implicitly so.
In stark contrast to George Blair’s brief treatment of civil rights,
In summary form, Strait’s civil rights unit maps out this way:
Day 1: Videotape from the Southern Poverty Law Center entitled, The Shadow of Hate, which describes majority discrimination against several religious (Quakers, Baptists, Jews), racial (Native, African, and Asian Americans), and ethnic groups (Irish-Catholics, Mexican Americans). Following the videotape, Strait solicits written reactions from the class.
Day 2: Small group activity where students discuss and list their reactions to the videotape on large pieces of chart paper. Strait later displays each chart on the back wall. At the end of the period, she distributes a feature article on school desegregation from Time magazine (April 18, 1995).
Day 3: Based on the previous night’s reading, Strait gives students an ungraded “quiz” which asks them to categorize nine statements as either an instance of civil rights or civil liberties. Strait then reviews part one of the notes she prepares and distributes for each unit.
Day 4: Videotape from the Southern
Day 5: Roundtable discussion of the four questions posed the previous day. Strait then introduces an activity that takes up the rest of this class and all the next days. The assignment calls for students to imagine that they are living in the early 1950s and that a white skating rink owner refuses to admit minority customers. In small groups, students are to create a strategy for winning access to the rink by listing their reasons, methods, and arguments on a
Day 6: Simulation where Strait portrays the skating rink operator and responds as students, in their groups, make their cases for admitting minority customers.
Day 8: Review of the practice essays. Strait then rearranges student desks into a large circle and leads a class reading from a handout entitled, Forty Lives for Freedom, a list she prepared of individuals who lost their lives to the cause of civil rights. Each student reads aloud one person’s name and the circumstances of his or her death. Strait then distributes and reviews a handout entitled, Hate Crimes—Summer, 1991, a list of 13 crimes committed between June 4–
This freestanding unit has several distinct features. First, Strait constructs a unit which spans time, circumstance, and peoples. She emphasizes African American experiences, but more, it seems, as a case in point than as the definitive civil rights group experience. Second, Strait employs a wide variety of activities in an instructional tour de force. Multiple opportunities arise—reading, writing, discussing, viewing, role playing—for students to engage ideas and emotions. Third, Strait expands the role of teacher. She plays the traditional roles of knowledge-giver (when she reviews unit notes) and knowledge-evaluator (when she scores the end-of-unit quizzes). Strait plays less traditional roles when she organizes small groups as a means of eliciting reactions to a videotape and when she organizes the skating rink activity. Strait is not an “invisible” teacher who directs class from the sidelines (Wineburg & Wilson, 1991), but she pushes beyond the traditional parameters of the role. Fourth, Strait promotes an expansive view of the subject matter. She gives attention to the major actors and events of the civil rights movement, while she also gives attention to lesser known people (the Forty Lives for Freedom list) and events (the Hate Crimes list). The two videotapes and the skating rink simulation seem particularly well suited for bringing the civil rights movement down to a recognizable level. Strait covers much of the standard political and economic history curriculum, yet her unit delves far into the ordinary lives that represent social history. Finally, Strait’s assignments work toward cognitive, affective, and social ends; students work both individually and in groups on assignments that push both their knowledge of the civil rights movement and their empathy for historical actors. Strait gave a quiz at the end of the unit, but the majority of the students’ grades came from the array of assignments.
The Regents test plays no obvious role in the first six days of Strait’s civil rights unit. Instead, she uses a variety of instructional approaches and resources to provide a vivid set of classroom experiences. That changes at the end of Day 7. With 10 minutes left in the class period, Strait distributed copies of two essay questions taken from recent Regents exams:
Essay #1: During various time periods in
Native American Indians (1790–1890)
Japanese Americans (1900–1970)
African Americans (1945–1970)
a. Select three of the groups listed. For each one selected, discuss a specific example of how the group was excluded from full participation in American society during the time period indicated.
b. For one of the groups you selected in answer to a, discuss a specific action taken by the Federal Government or an organization during or after the time period indicated to help this group achieve full participation in American society.
Essay #2: Since 1865, agents of change have acted to advance the cause of civil rights and civil liberties in the
Agents of Change
A nongovernmental group
For each of the agents of change listed:
a. Explain one action taken by that agent of change to help advance civil rights or civil liberties in the
b. Describe the historical circumstances that led to that action.
Strait instructs students to work with a partner on one or the other essay and “to outline the question . . . as you would do on the Regents.” She adds, “I think you’ll have enough information with the videos and notes to handle either of these essays.” About half the students turn to the task, the others chatter about a range of off-task topics. After a few minutes, Strait directs class attention to the second essay. She asks for examples of “non governmental groups” (that had acted as agents of change). Students volunteer a range of answers: the KKK, the NAACP, the Nation of Islam. Strait acknowledges these responses as she circulates around the room, prodding students to work and answering their individual questions. To one pair, she explains how the test is structured (i.e., multiple-choice and essay questions). After looking at the work of another pair, she explains, “See how much you know)” Then she talks with them about how little the test questions change from year to year.
The next day Strait spends approximately 30 minutes reviewing class efforts on the practice essays. She begins by asking if students had any difficulty in outlining the essay questions. There is a good deal of chatter and she gets no particular response beyond one girl who says, “no.” Strait responds, “I didn’t think so.” She continues, “In June, you should make a little outline. . . . I don’t take BS on essays. . . . I might have done so before, but I don’t now.”
With that introduction, Strait reviews the information students might use in each section of each essay. She explains, “You always have to know what the problem is and one solution.... If you can’t identify a piece of legislation, talk about it....But if you don’t know, don’t use it because I have to take off a point for incorrect information.”
What follows are snippets of the talk around each section of the essay questions. The first question asks students to choose three of five groups and to (a) discuss a specific example of how the group was excluded from American society and (b) discuss a specific action taken by a government agency or nongovernmental agency to help the group achieve full participation. Unless indicated, the responses for each section below are the students’:
Native American Indians (1790–1890)—(a) Trail of Tears, Manifest Destiny, culture taken away; (b) (no student response; Strait volunteers the Dawes Act)
Japanese Americans (1900–1970)—(a) put in camps, thought to be spies; (b) federal government offer of $20,000 tax free (Strait disqualifies this answer because it does not fit the time frame stated in the question. There are no other student responses. Strait mentions the Civil Rights Act of 1964 with the reminder that it did not apply to African Americans alone. She adds, “You can always use that piece of legislation for any of the groups.”)
Women (1940–1990)—(a) women’s suffrage (Strait disqualifies this response as it does not fit within the time period specified. She cautions, “You have to stay in the time period....You don’t have to memorize dates, but you have to be aware of time periods.”), hiring, pay (Strait accepts these responses, then adds, “Think of other groups as having the similar struggles with blacks.... I don’t think anyone could pick blacks and not write anything about this time period.”)
Strait then turns to the second essay, which asks for one action by government, a nongovernmental group, and an individual who helped advance civil rights, and the historical circumstances that led to the action. Strait categorizes this question as “very easy.” A boy calls out, “No way)” Strait laughs and explains, “You could talk about Brown or any of the civil rights legislation.... there are no time [period] constraints.” The boy responds, “Just something positive?” Strait nods and lists some nongovernmental groups—NAACP, CORE—and explains, “Then you just have to describe the historical circumstances. That’s a cinch for 15 points.” Students are generally quiet during this time, but few seem attentive or engaged; I see few actively listening or taking notes.
Several points seem interesting about these vignettes. First, Strait talks explicitly and at some length about the Regents test structure (i.e., the types of questions and points allotted) and content (i.e., distinctions between events and implications). Second, she also talks directly about test-taking strategies (e.g., using the Civil Rights Act of 1964 as an all purpose answer and remembering to consider question parameters such as explicit time frames). Finally, she gives students an opportunity to practice outlining their responses and comparing those responses with her assessment of the tasks. Thus, in contrast to Blair’s apparent disregard for the Regents, Strait devotes the equivalent of almost a full class period—40 minutes over two days—to test preparation and practice.
Comparing the two teachers, then, we see variation in both the explicit and implicit attention each gives to the Regents test. Blair gives little explicit attention to the test, yet one can argue that his content selection, teaching, and assessment mirror test-based influences. By contrast, Strait’s instructional practices seem less directly reflective of the test, yet she purposely leads her students to think about and practice test-taking strategies.
AN UNCERTAIN LEVER: THE NYS REGENTS TEST AS AN INFLUENCE ON TEACHERS’ PRACTICE
What are we to make of such varied approaches to teaching and testing? If tests drive curriculum, then why do these teachers’ content decisions vary so much? If tests drive instruction, then why do these teachers construct their teaching practices in such different ways? If tests drive teachers to focus on student test performance, then why do these two teachers give such different attention to the state exam? Finally, what might all this mean for testing as a lever of instructional change?
Tests do matter. George Blair and
TESTING AND TEACHERS’ PRACTICES
The Regents test is not irrelevant in George Blair and
The Influence of Testing on George Blair’s Practice
I argue above that, while George Blair’s instruction offers little direct indication of being influenced by the state Regents test, a case can be made for a strong implicit influence. Understanding the relationship between Blair’s teaching and the test, however, proves to be complicated. For factors such as Blair’s sense of the subject matter and his view of learners figure into his teaching at least as strongly as does the state test. The Regents exam matters, but it holds no privileged position.
Blair supports the Regents testing program. In response to a question about the nature of school reform in
At first glance, this emphasis on the Regents test seems odd given Blair’s lack of explicit attention to the exam in his daily practice. But perhaps not. Blair routinely stages a two-week review session just before the test administration in early June. He discounts, however, the importance of test preparation. “I don’t spend a lot of time reading Regents questions or putting them together for the kids or anything like that,” he said. So while Blair believes the test is important, he asserts that explicit preparation for the test counts for little. He is not alone in that view. There is little research on the effects of test preparation on students’ performance, but as Haney, Madaus, and Lyons (1993) conclude, there may be some reason to believe that “overemphasis on test preparation in schools has a negative impact on teaching and learning” (p. 179).
While Blair may not give much direct classroom attention to the Regents test, he is nevertheless sensitive to the exam. Scoring his students’ test papers provides an opportunity to see what kinds of questions and content are represented. “When you do the grading,” he said, “you know generally what they hit and what they don’t hit.... It’s experience, that’s all it is.” Blair translates that experience into a range of instructional decisions, some quite obvious. Recall his comment that remembering Omar Bradley’s name is not important because “it never shows up on the Regents.” During an interview, he made a similar comment about military actions during the Civil War:
I never deal with militarism in the Civil War, never, because it’s never on the Regents, so I don’t deal with it. . . . I spend about 25 minutes at the end of class, of one class, on a couple of battles, the western battles, the eastern battles, the naval blockade, the naval engagement, the Monitor and the Merrimac. It’s all done in 25 minutes total. It’s finished and it’s because it’s not on the Regents. And I tell them there’s a chapter in the book on the military action, read it between now and the Regents. But there’s seven essays, and if they get a military history question, I’m sure I’ve given them enough information that they could skip that and get some other question anyway.
These examples suggest a link between the state test and Blair’s content decisions. And as noted earlier, an argument can also be mounted which posits that Blair’s instruction reflects a strong, if implicit, connection to the Regents exam.
Other evidence, however, suggests the picture is more complex. For while Blair gives the test its due, he does not follow the presumed dictates of the test in a slavish fashion. In fact, it appears that factors such as Blair’s view of the subject matter and of his students figure as prominently in his instructional decisions as the test does.
One example of the influence of subject matter on Blair’s teaching centers on the Federal period in
I find it very difficult to cut things out of American History. I’m sorry, I just find it very difficult. The new course that came out in ’89 . . . they removed the Federal period of American history. How the hell do you leave out the Federal period of American history? How the hell do you do this? Where’s
Continuing, Blair makes the case that such curricular decisions by the state inherently disrupt the study of
There’s something inside of me that says you can’t do that. So I teach a survey course and I break my ass getting through it cause I have to do an eightweek Constitution unit and that takes eight weeks out of the course. Like somehow we get there. Would I change that? No. I’m not going to change that.
So strongly does Blair feel about presenting a coherent view of
One might argue that Blair’s defense of content coverage mirrors the state test emphasis on key people, places, and events. Nevertheless, Blair’s stubborn commitment to teaching “the whole course” and his specific commitment to teaching the Federal period suggest that he holds a strong view of the subject matter, and that that view, should it conflict with state curricula and testing dictates, may well prevail. Arguably, Blair’s classroom life would be eased and his students’ test scores increased if, for example, he let go of content like the untested Federal period. He refuses to do so, largely it seems, because of his conception of and responsibility to the subject matter he teaches.
If Blair’s view of the subject matter undercuts the influence of the Regents exam on his teaching, so too does his view of his students. Two beliefs define Blair’s sense of students. One is that they need a survey course in
Seventh and eighth they get this two-year program. They’re 12 years old. I mean they don’t know up from down anyway. When they come here they don’t have the faintest idea . . . You want a kid to remember a couple of things from seventh and eighth grade to eleventh grade? You’ve got to be kidding me. They went through three years of puberty. Do you know what they’re thinking about? They’re not thinking about the Wilmot Proviso. And to say there’s carryover, that’s absolutely asinine.
Blair’s conviction that students arrive with a fragmented knowledge of history coincides with another belief—that his students arrive with little ability to think or reason. Blair holds no truck with teaching methods that promote critical or higher level thinking. He dismisses simulations and other experiential activities as games, doubting that they have any real or lasting effect. He explains, “I’d say I don’t think the kid can make the generalization from this game to an historical event or general ideas of life or the f low of society or the f low of history, or the flow of psychology, the f low of people, the f low of culture.” Similarly, Blair discounts the importance of small group work, student presentations, and nontraditional assessments. “Does learning go on?” he asks, “Is the maturity level there for these students to truly understand what they’re doing?” Blair assigns a research paper, but reports, “I don’t expect much.”
Blair believes he faces a simple reality: He must cover the entire history of the
One last point involves test results as a measure of success. Test scores matter to Westwood school administrators and parents. They also matter to George Blair, who seems to hold his students’ test performance as an indication of both his and their achievement. Again, however, the picture is more complicated, as other measures of success compete with test scores. One of those measures is Blair’s commitment to covering the course material. Another is his sense that students need both “humanity and understanding.” Blair worries, for example, about the state of race relations in
The Influence of Testing on
Although there is much that is different in the classroom practices of
Perhaps because she is relatively new to teaching, Strait seems more concerned with and worried about tests and test scores than Blair does. And unlike her colleague, Strait sees little value in the Regents exam. She chafes at the content constraints she feels the state test imposes, and she is daunted by the breadth of content covered. “For the Regents exam, knowing what the exams are like, I see so much content there,” she said, “The curriculum [of the test] is just too broad. There’s just too much there.” The breadth of content means Strait must constantly negotiate the tension between her sense of what is important and the test makers’. She explains, “Sometimes I spend a lot of time with something I think that’s interesting....I just think it’s part of what they need to know as viable human beings in this society.” Strait pauses, and then adds, “But the Regents doesn’t test anything on it.” At times this tension does not seem to matter; she simply goes ahead with what she thinks is appropriate. Other times she defers to the test. “It’s sometimes difficult,” she said, “because there’s a lot of things I want to do and [given the test] I just skip over [them].” Still other times, Strait is able to accommodate both impulses—hers and the test makers’. The skating rink simulation is a good example. This activity balances ideas and experiences that help students empathize with historical actors with a potential Regents test question (e.g., the portions of the practice essays which ask students to talk about actions groups might have taken to protest discrimination). As her teaching experience grows, Strait is increasingly able to create these balances. She begrudges the planning time this takes, but she knows that test scores matter in this upscale suburban environment, and so she continually walks a line between her goals and the test makers’.
If Strait’s content decisions are influenced by the Regents test, so too are her instructional decisions. Were there no test, Strait might still have her students writing practice essays. Nevertheless, she clearly communicates the influence of the test in the content of the questions and in her strategic review of the students’ responses. Strait’s students may develop as skilled and thoughtful essayists, but with her guidance they will also know how to respond to an essay prompt on the upcoming state test.
And yet, if the state test influences Strait’s content and instructional decisions, that influence is far from pure or direct. An argument can be made that the test drives Strait’s practice no more so than her attention to the subject matter and to learners.
Strait comes to the civil rights unit with two goals in mind. The first is to help students understand that “[racism] is really still happening today. I think it’s pushed to the back of the memory of people so that they don’t seem to see that this is still a really awful problem,” she said, “I see it and I have always seen it as the biggest problem in America today....how the races get along.” Although she believes students are more tolerant today, that toleration seems limited to religious differences; racial differences are another matter. “It seems like, on the surface, superficially, that they might be more tolerant in terms of religion,” she said, “...you[get] more heated, close-minded debates about the race issue as opposed to the religion issue.”
Strait’s second goal is broader in that it reflects the view that history is more than politics and economics and more than a record of the victorious:
What I try to convey is that
As Strait explains, “I’m not necessarily trying to tell everyone that you’ve got to love everybody, but you have to accept them and you can’t, you shouldn’t just hate them because of the differences.”
These goals suggest that Strait’s instruction is sensitive to more than the state Regents test. Two examples underscore the power of Strait’s view of subject matter as an influence on her teaching practice. The first is the amount of time she devotes to the civil rights movement. If she considered only the dictates of the Regents test, planning an eight day unit makes little sense, for the civil rights movement simply does not figure prominently on the exam. True, there is occasionally an essay question around civil rights (such as those Strait introduced in class), but there is no guarantee of one. Moreover, there are generally no more than a couple multiple-choice questions devoted to civil rights. Given that Blair covers a good chunk of the civil rights movement in about 20 minutes, Strait’s very different unit reflects her sense of what subject matter is important.
This sense of the subject matter also seems behind Strait’s emphasis on social history. Regents test questions tend to focus on political and economic events. Strait does not discount this material as, for example, her lecture notes cover the major civil rights legislation. Nevertheless, much of her unit focuses on social history or the “lives of ordinary people in all their richness” (Appleby, Hunt, & Jacob, 1994, p. 84). The two videotapes, the roundtable discussion, and the skating rink simulation, and the class readings of Forty Lives for Freedom and Hate Crimes—Summer, 1991 all emphasize the roles and experiences of people unlikely to ever appear on a standardized state test. Make no mistake, Strait’s students have access to the people, places, and events that make up standard accounts of the
If Strait’s view of the subject matter influences her classroom decisions, so too does her view of learners. The principal evidence for this point comes from the range of
The range of instructional activity evident in Strait’s civil rights unit owes something to her sense that students need to be ready to take the state Regents test. Scores on that test do matter, after all, and Strait would never suggest that they are an unimportant measure of success. But just as important, at least for the civil rights unit, are the developing sensitivities and sensibilities of her students.
Much rides on Strait’s students’ performance on the Regents test. She wants them to do well and, to that end, she pushes her students academically. At the same time, Strait wants students to feel something of what discrimination is like. “I would like students to be more empathetic with other people,” she said. “Try and just imagine if you were in their shoes. Even if they can never be your best friend, you know, try and understand others.” The range of instructional activity evident in Strait’s civil rights unit owes much to her sense that students need to master a considerable amount of subject matter knowledge and to experience something of the passion of the times. Knowledge without empathy, she asserts, is meaningless.
The analysis above suggests that the direct influence of state-level testing through the Regents exam in U.S. History and Government is thin. George Blair and
It is likely that more factors than the teachers’ views of subject matter and learners interact with the state test to influence their practices. After all, considerations of personal biography (e.g., individual knowledge, beliefs, and experiences), local organizational structures (e.g., school norms and expectations, time, and resources), and the state and district policy climate (e.g., the state social studies curriculum developed in the late 1980s and the more recent state social studies standards) may also figure into the content and instructional decisions teachers like Strait and Blair make (Grant, 1996, 1998b). Thus, my emphasis on the views each teacher holds of subject matter and learners is meant to be illustrative rather than definitive. In any case, the point is the same: While state tests influence Blair’s and Strait’s practices, they are not the only influence and, in fact, may not even be the principal influence. In short, I see little direct, deep, and consistent influence of tests on these teachers’ classroom practices. The pervading sense that tests drive content, instruction, and the like seems alternately overstated, ill informed, or misplaced. If tests are an influence on practice, and more importantly, if they are intended as a means of changing teachers’ practices, then they may be an uncertain lever at best.
CONCLUSION: TESTS AS AN UNCERTAIN LEVER OF CHANGE
Politicians and reformers are likely to continue pinning their hopes on new tests. The findings of this study suggest, however, that faith in tests as a lever of change may be increasingly hard to sustain. The current Regents exam is a relatively straightforward, content-based test. The message it sends seems simple and direct—teach the facts. If George Blair and
The author would like to thank Sandra Cimbricz, Terrie Epstein, and Bruce VanSledright for their thoughtful readings of an earlier version of this paper.
Corbett and Wilson (1991) point out, however, that Madaus’ claims are based on limited data: “anecdotes, testimony from public hearings, historical accounts, and an occasional international study” (p. 26).
The Regents testing program is currently being revised in light of new curriculum standards developed in the late 1990s. For a review of the changes in the standards and the tests, see Grant (2000).
Interestingly enough, however, an extensive ERIC search failed to produce a single empirical study which examined the relationship between students’ test performance and any other social value (e.g., performance in college).
The RCT program was initiated in the late 1970s as a minimum competency test for non-college bound students. The first tests in social studies (Global Studies and U.S. History 422 and Government) were administered in the late 1980s. The RCT follows the same multiple-choice and essay format of the Regents exam with the exception that the RCT has 50 multiple-choice questions and students write on two of four essay prompts. Current plans call for the RCT to be phased out after 2000.
Statewide approximately 60% of students take the Regents examinations, which are administered in January, June, and August. Tests are scored by the teacher of each class and then forwarded to the State Education Department. In addition to deciding whether or not a student receives a Regents diploma, test scores also figure into each student’s course grade. At Westwood, 73% of the students received a Regents diploma (during the year in which this study was done) compared with 63% in the district, 53% in similar schools, and 40% in all N. Y. state schools.
Interestingly enough, a second reader is required on the essay portion of the RCT, but not on the Regents exam.
Of the 230 students enrolled in U.S. History during the year this study was done, 205 took the Regents test. Of that number, 80% passed the test (with a cutoff score of 65) while 42% achieved “mastery” (a score of 85 or better). Those percentages suggest Westwood students fare well compared to students in similar schools (65% passing; 25% mastery) and those in all N. Y. state schools (49% passing; 16% mastery).
The absence of student participation is characteristic of George Blair’s classroom. I have observed Blair several times and have heard only one substantive question.
Statements included, “...a 9yearold girl is not allowed to play on the school basketball team” and “. . . you are arrested for burning the
This set of notes consisted of definitions of “civil rights” and “civil liberties” and a chronology of the civil rights movement with special attention to Harry Truman’s actions and the Brown versus the Board of Education decision. Also attached were additional readings on the Brown decision, equal opportunity, and the Swann versus Charlotte Mecklenburg Board of Education decision in 1971, which allowed the use of forced busing to end patterns of discrimination.
These notes included sections on the early philosophy of the civil rights movement, early leaders and activists, civil rights presidents, later philosophies, more “radical” leaders, assassinations, other civil rights movements (e.g., Women, Native Americans, Mexican Americans, and Handicapped0Disabled Persons), and Supreme Court cases.
For example, Rev. George Lee—killed for leading voter registration drive (Belzoni, Mississippi, 1955); Willie Edwards—killed by the Klan (Montgomery, Alabama, 1957); Paul Guihard—European reporter killed during the Ole Miss riot (Oxford, Mississippi, 1962); and Virgil Lamar Ware—youth killed during wave of racist violence (Birmingham, Alabama, 1963).
For example, Albuquerque, New Mexico ( June 4, 1991)—A cross was burned on the lawn of a racially mixed family; Woodbridge, New Jersey ( June 28, 1991)—Thirteen people were arrested for assaulting and harassing Asian Indians; Fullerton, California ( July 7, 1991)—A Chinese American teenager was beaten unconscious by Skinheads. Strait adapted the Forty Lives and Hate Crimes lists from materials she received from the Southern Poverty Law Center.
Strait had to be absent from school the next day. She prepared a 12 question, multiple-choice quiz to be administered that day. The questions ranged from definitional (e.g., Which action is the best example of civil disobedience?) to generalization (e.g., Which is the most valid conclusion to be drawn from the study of the civil rights movement in the U. S. since 1954?) to interpretive (e.g., students were presented with the quotation, “We will match your capacity to inflict suffering with our capacity to endure suffering....We will not hate you, but we cannot obey your unjust laws,” and asked who was the most likely author—Booker T. Washington.
While this idea was floated briefly at the end of former Education Commissioner Thomas Sobol’s tenure, it has been firmly rejected by current Commissioner Richard Mills.
18 Although the state syllabus gives substantial attention to the constitutional period, the chronology of
Appleby, J., Hunt, L., & Jacob, M. (1994). Telling the truth about history.
Barton, K. (1997). “I just kinda know.” Elementary students’ ideas about historical evidence. Theory and Research in Social Education, 25 (4), 407–430.
Bogdan, R., & Biklen, S. (1982). Qualitative research for education: An introduction to theory and methods.
Brophy, J. (1992).
Cohen, D., & Barnes, C. (1993). Pedagogy and policy. In D. Cohen, M. McLaughlin, & J. Talbert (Eds.), Teaching for understanding: Challenges for policy and practice (pp. 207–239).
Connelly, F. M., & Clandinin, D. J. (1990). Stories of experience and narrative inquiry. Educational Researcher, 19 (4), 2–14.
Corbett, H. D., & Wilson, B. (1991). Testing, reform, and rebellion.
Editors. (1994). Introduction. Symposium: Equity and educational assessment. Harvard Educational Review, 64, 1–3.
English, F. (1980). Improving curriculum management in the schools.
Erickson, F. (1992). Why the clinical trial method doesn’t work as a metaphor or educational research: A response to Shrag. Educational Researcher, 21 (5), 9–11.
Evans, R. (1990). Teacher conceptions of history revisited: Ideology, curriculum, and student belief. Theory and Research in Social Education, 28 (2), 101–138.
Feltovich, P., Spiro, R., & Coulson, R. (1993). Learning, teaching, and testing for complex conceptual understanding. In N. Frederickson, R. Mislevy, & I. Bejarq (Eds.), Test theory for a new generation of tests (pp. 181–217).
Finn, C. (1995). Who’s afraid of the big, bad test? In D. Ravitch (Ed.), Debating the future of American education: Do we need national standards and assessments? (pp. 120–144).
Firestone, W., Mayrowetz, D, & Fairman, J. (1998). Performance-based assessment and instructional change: The effects of testing in
Frederickson, N. (1984). The real test bias. American Psychologist, 39, 193–202.
Freeman, D., Kuhs, T., Porter, A., Knappen, L., Floden, R., Schmidt, W., & Schwille, J. (1980). The fourth grade mathematics curriculum as inferred from textbooks and tests.
Fuhrman, S., Clune, W., & Elmore, R. (1988). Research on educational reforms: Lessons on the implementation of policy. Teachers College Record, 90, 237–258. Glatthorn, A. (1987). Curriculum leadership.
Grant, S. G. (1996). Locating authority over content and pedagogy: Crosscurrent influences on teachers’ thinking and practice. Theory and Research in Social Education, 24 (3), 237–272.
Grant, S. G. (1997a). On subject-tspecific pedagogy: Two teachers; two approaches. Paper presented at the National Council for the Social Studies,
Grant, S. G. (1997b). Opportunities lost: Teachers learning about the
Grant, S. G. (1997c). A policy at odds with itself: The tension between constructivist and traditional views in the
Grant, S. G. (1998). Reforming reading, writing, and mathematics: Teachers’ responses and the prospects for systemic reform.
Grant, S. G. (2000). Teachers and tests: Exploring teachers’ perceptions of changes in the
Haladyna, T., Nolen, S., & Haas, N. (1991). Raising standardized achievement test scores and the origins of test score pollution. Educational Researcher, 20 (5), 2–7.
Haney, W., Madaus, G., & Lyons, R. (1993). The fractured marketplace for standardized testing.
Heubert, J., & Hauser, R. (Eds.) (1999). High stakes: Testing for tracking, promotion, and graduation.
Kelleghan, T., Madaus, G., & Airasian, P. (1982). The effects of standardized testing.
Koretz, D. (1988). Arriving in
Koretz, D. (1995). Sometimes a cigar is only a cigar, and often a test is only a test. In D. Ravitch (Ed.), Debating the future of American education: Do we need national standards and assessments? (pp. 154–166).
LeMahieu, P. (1984). The effects on achievement and instructional content of a program of student monitoring through frequent testing. Educational Evaluation and Policy Analysis, 6 (2), 175–187.
Levstik, L. (1993). I wanted to be there: The impact of narrative on children’s historical thinking. In M. Tummell & R. Amnon (Eds.), The story of ourselves (pp. 65–77).
Levstik, L. (1995). Narrative constructions: Cultural frames for history. The Social Studies, 86, 113–116.
Madaus, G. (1988). The influence of testing on the curriculum. In L. Tanner (Ed.), Critical issues in curriculum: 87th yearbook of the NSSE, Part 1.
Natriello, G., & Pallas, A. (1998). The development and
impact of high stakes testing. Available online: http://www.columbia.edu/;gjn6/histake.html
New York State Education Department. (1996). Learning standards for social studies.
Noble, A., & Smith, M. (1994). Old and new beliefs about measurement-driven reform: “Build it and they will come.” Educational Policy, 8 (2), 111–136.
Pellegrino, J. (1992). Commentary: Understanding what we measure and measuring what we understand. In B. Gifford & M. O’Connor (Eds.), Changing assessments: Alternative views of aptitude, achievement, and instruction (pp. 275–294).
Popham, J., Cruse, K., Rankin, S., Sandifer, P., & Williams, P. (1985). Measurement-driven instruction: It’s on the road. Phi Delta Kappan, 66 (9), 628–634.
Popham, W. J. (1998). Farewell, curriculum: Confessions of an assessment convert. Phi Delta Kappan, 380–438.
Ravitch, D. (Ed.). (1995). Debating the future of American education: Do we need national standards and assessments?
Resnick, D., & Resnick, L. (1985). Standards, curriculum, and performance: A historical and comparative perspective. Educational Researcher, 14 (4), 5–20.
Romberg, T., Zarinnia, A, & Williams, S. (1989). The influence of mandated testing on mathematics instruction: Grade 8 teachers’ perceptions.
SalmonCox, L. (1981). Teachers and standardized achievement tests: What’s really happening? Phi Delta Kappan, 62, 631–634.
Shanker, A. (1995). The case for high stakes and real consequences. In D. Ravitch, (Ed.), Debating the future of American education: Do we need national standards and assessments? (pp. 145– 153).
Smith, M., & O’Day, J. (1991). Systemic school reform. In S. Fuhrman & B. Malen (Eds.), The politics of curriculum and testing (pp. 233–267).
Smith, M. L. (1991). Put to the test: The effects of external testing on teachers. Educational Researcher, 20 (5), 8–11.
Stiggins, R., & Conklin, N. (1992). In teachers’ hands: Investigating the practices of classroom assessment.
VanSledright, B., & Brophy, J. (1992). Storytelling, imagination, and fanciful elaboration in children’s historical reconstructions. American Educational Research Journal, 29 (4), 837–859.
VanSledright, B., & Brophy, J. (1995). “Storytellers,” “scientists,” and “reformers” in the teaching of
Wineburg, S., &
S. G. GRANT is associate professor of social studies education, State University of New York at