The Development and Impact of High Stakes Testing - Part 1 - Recent Developments

by TC Record - November 15, 2000

(Editor’s Note: This is a transcript of the lecture delivered. The text is not revised.)

Gary Natriello:

What I want to do is talk about the development and impact of high-stakes testing. And I'm going to be drawing on some work that I've done with Aaron Palace, who some of you may know, who's a new faculty member here in sociology and education. But some work that we did, looking at this for the Harvard Civil Rights Project, in a paper that's going to come out probably beginning of 2001.

And what I'm going to do is basically try to cover three big parts of this topic. Number one, I'm going to start by giving you a recap of recent developments in the imposition of high-stakes testing; that is, the growth of testing as a graduation requirement for youngsters who want to finish high school. Is there anyone in this room who actually had a high-stakes test to get out of high school? I think we're all too old; at least I know I was. So we've all made it; so, you know, the question is, should we be worried about the rest, or should we throw up new barriers?

And I'm going to argue that these barriers are getting thrown up; that testing is getting used on a pretty widespread basis, and it's getting attached as a graduation requirement for people getting a high school diploma. Second thing I want to do is to look at some of the performance patterns that we see on the kinds of tests that are being used and are likely to be used for high school graduation, and particularly to look at the racial-ethnic differences on the pass rates on these tests, and the performance tests; and to ask us to think a little bit about what those performance patterns mean, both in the way they motivate, potentially, policy makers and in the way they motivate kids and parents.

And then, finally, at the end, I'm going to raise the question about there being multiple ways of looking at this phenomena and interpreting this phenomena; and arguing that, the sense that you make of it depends as much upon your perspective going into it as it does upon anything else. So let's start with the growth of widespread testing in K-12 education, which we talked about a little bit last week.

And for those of you who recall that, we talked about the fact that testing in general is now pervading K-12 schools; that it's a very popular policy remedy; and that it's popular, I want to argue tonight, because it addresses a number of pressing issues that policy makers are interested in addressing. First of all, these kinds of programs of formal testing allow government, that is, state government; that is, the level of government with legal authority for schooling, they allow that level of government to exercise control over the activities of local schools and school districts where the schooling actually occurs. So that these formal state testing programs are increasingly an integral part of the monitoring process used by states to ensure that educational activities that are appropriate are occurring; and that, along with funding and regulation, these testing programs are one of the key methods of actually influencing activity at the local level. Last week, we talked a little bit about how that influence takes place and what impact it has, and the fact that people are actually noticing it. And it's particularly interesting in the US context, because we have this incredibly decentralized system, and in that kind of system, testing is one of the few devices that actually allows one level to influence another level.

Second goal that testing allows policy makers to address is, of course, related to that one; and that is that it really has the potential to influence the behavior of all the major actors in this system; not just the levels, but the individual actors as well. So, for example, testing can be interpreted as causing students to pay greater attention to the demands of the educational system, and to devote greater effort in meeting those demands. Now, some of you will probably have experiences which contradict that statement, and may have lots of examples of students who don't appear to be influenced at all, and their behavior doesn't appear to be influenced at all by formal testing. But, in general, I think the argument's been made that one of the reasons you want to do testing is you want to get the attention of students, you want to influence their behavior, and it's a way to address what's perceived to be low student effort on schooling tasks.

Second thing testing does, or a second way in which testing can be viewed in terms of influencing behavior, is that it's a mechanism to influence the behavior of professional educators, teachers and administrators. And it does this by presumably exposing the results of their performance to public examination in a kind of comparative framework. So, again, as people talked about last week, in the newspaper, on the Internet, for everyone to see, is the set of outcomes associated with the performance of a particular faculty and a particular school; and not only is that performance visible for everyone to see, but the scale is absolutely the same from one school to another, from one end of the state to the other, so all kinds of comparisons can be made.

And finally, testing is viewed by policy makers as a way to influence the way the public perceives public schools. And particularly when testing is attached to the award of a high school diploma, the idea is that it may be a way to guarantee to the public some basic level of accomplishment for high school graduates.

And so, in a sense, it's a quality assurance measure. And, as you recall, I think way back, a couple of weeks ago, when I talked about standards, or when, I guess, we let you take a look at the standards lecture, as I recall; we didn't actually talk about it here, in that I made the point that one of the perceived problems was that the public didn't have much confidence in the end product of K-12 schooling. And there was some concern about, what is it that a high school diploma means. And in the 80s, particularly, there was a lot of discussion about the fact that the high school diploma didn't count for anything anymore, and that employers couldn't be assured of what it means. So one of the motivations of attaching this kind of testing requirement to high school completion is that it's an attempt to try to shore up the meaning of the high school diploma, and to offer some kind of minimal guarantee, and maybe not so minimal in the end, of what it is that it means to actually have that diploma.

Now, another goal or another appeal of the widespread testing for the policy community, and I think I alluded to this earlier as well, is the inherent efficiency of testing as a monitoring process. And that is that testing really offers a very favorable ratio of the information gathered to the expense incurred. It's pretty cheap. Of all the things you can do to find out about how the system is working, testing is a pretty efficient way to do that. And then, finally, there's sort of this additional dimension of testing; that is, you can think of testing as, at the same time, a monitoring device and, in some ways of thinking about it, as a school reform initiative. So it's both a way to sort of assess the outcomes of schooling, or the changes in the performance of schooling, as well as being portrayed as a new initiative. And, in fact, if you watched the presidential debate a week or so ago, you found both candidates competing on their testing programs. So what are you going to do for education, and what's your great new program? Well, I'm going to test them; I've got lots of testing. And then some of other candidates said, well, you don't have enough testing; I've got more testing than you have! Do you have testing at these grades? No, I have testing at every grade! And you're not using this kind of, oh, I'm using a better test! So no one was questioning the rationale for testing; the competition was over how much testing there would be, and then a little bit about, which I think got lost, unfortunately, what you'd actually do when you got those test results. But it has a kind of currency and a kind of appeal.

And I want to argue at the outset, as we begin to look at some of these data I'm going to show you in a couple of minutes, that there are really two perspectives that I think are important to bring to thinking about the testing issue; and I'm going to start out with this bureaucratic rationale. And that is that, as I've mentioned the last three of four goals that I've gone through, what all of these goals suggest is that there are compelling reasons why policy makers, interested in exercising some kind of rational strategy to improve schooling, might look favorably on widespread testing. All right? It's very rational, it's very straightforward; we want to get the attention of the kids, we want to direct the educators, we want to hold the level at which schooling is actually enacted accountable to the level that it has legal responsibility. All of those things are sensible, rational approaches to try to gain control of this bureaucratic system. So there is a bureaucratic rationale, which is characteristic of the testing movement. And when you hear the proponents of testing talk, and if you listen carefully to what they're saying, that's throughout their entire discussion.

And that was true of the presidential debate: I mean, there were all kinds of things about testing that people were arguing, the candidates were arguing, are going to lead to improvements in the system. There are guarantees. There's better performance, a better chance for all kids; we could have uniformity; we're not going to deal with great inequities in the system; we're going to have the same kind of outcomes; we're going to standardize it. So, you know, it's the old rational model that people have been advocating for a long time; and indeed, it's the sort of model that dominates modern society.

But I want to suggest, there is an alternative rationale that you could think about as motivating all of this interest in testing, and that's a social and political rationale. And here I want to argue that, in additional to the reasons that I just talked about for the rapid expansion of formal testing tied to sanctions, such as the granting of a high school diploma, that there might be social forces that lead to the adoption of such systems. And here I'm going to say a little bit about this at the outset, and then come back to it at the end.

But if you think about the rise of testing technology in the early part of the 20th century, and the way in which that technology was used to respond and control the threat that was perceived to be posed by massive immigration and the possible displacement of advantaged classes in American society, I think there's a historic setting in which testing was used. It is at least worth considering in the contemporary period whether the current boom in testing is more than coincidentally arising in the wake of both the civil rights movement, which liberated millions of American blacks from strong social restrictions, or whether it may also be related to the massive movement of new immigrants into the US over the last 20 years. That is, there are large social and politics events that are taking place within US society that may, in fact, be influencing the growth of this testing movement. So it's not just the bureaucratic rationale which has its own set of arguments, but I'm going to suggest there is an equally plausible social and political rationale; and that it's possible to have these dual interpretations, that, yes, in fact, you can think of testing as driving and improvement of the system, but you can also think of testing as maintaining the existing order of privilege and advantage by clearly labeling some individuals and some groups in the society as having superior performance over others. And that it's at least reasonable to raise both of those as interpretations of why it is that we have this kind of growth in testing at the present time. And, just to give you a preview, I want to suggest, by the time we get to the end, that the data don't really allow us to distinguish one of these as a superior explanation to the other; that, in fact, depending upon the perspective you start with, each of them are equally plausible, which is what makes it an interesting policy issue, policy discussion.

I want to shift now from this initial discussion to talk a little bit about what we think we know about the impact of high-stakes testing. And let me start by saying a little bit about what each side in this battle has to say about it, because, in fact, there are definitive sides. There's clearly a set of folks who believe that high-stakes testing is an important new policy initiative and one that should be pursued vigorously, and there's another group of individuals who feel that it's really a disastrous policy, and that it should be avoided at all costs, and that we should step back from it where we've adopted it. So let's start with the proponents. The proponents of high-stakes testing, and this is testing that links performance on the test to some meaningful decision; and in this case, the typical meaningful decision is whether you get to graduate from high school. And keep in mind that graduating from high school is associated with a whole variety of other rights and privileges, such as the right to get a civil service job, and the right to work in certain settings.

And so it's not just graduating from high school. It's graduating from high school and all the other things that that allows you to have. And, in fact, even entering the armed forces is now tied to whether you can get a high school diploma. So it's a whole set of opportunities that are associated with that. Now, the proponents of high-stakes testing argue that such tests will alert parents and the public to the performance of individual students and to the system overall. And they suggest, as I mentioned earlier, that these tests are going to cause students to devote greater effort to their schooling, and that they're also going to orient schools to the goals for education adopted at the state level. There are going to be serious consequences. If you're not attending to schooling, if you're not able to pass the test, you're not going to graduate, and it's going to be known that you're not going to graduate. It's going to be clear to your community that the proportion of kids graduating is going to be very low or relatively low compared to what it otherwise might've been.

And, indeed, there is some evidence that formal state testing programs do influence the activities of teachers and school administrators, and they direct their attention to a curriculum linked to the tests. So we can find evidence that, when you go into schools and you look at how people are behaving, and look at what they're talking about, you look at the activities that they're engaged in, in fact, there's pretty clear evidence that those activities are influenced by the fact that there's a state testing program, and the fact that that testing program increasingly has meaningful consequences at least for the kids, if not also for the educators and the schools in which they are working. Now, the opponents of high-stakes testing policies argue that such policies, and, indeed, the entire standards movement, is based on a set of faulty assumptions regarding human motivation. And these individuals argue, for example, that rigid standards and narrow accountability and tangible sanctions may reduce the motivation of teachers and students. And they note further that students who are focused on tests and sanctions may lose intrinsic interest; that they may learn only superficially, and that they may fail to develop a desire for learning, particularly a lifelong desire for learning. And, in addition, the opponents observe that the reforms that include standards, accountability and sanctions may raise test scores; so they may show some progress over time, but they may impede our progress toward creating a population of lifelong learners who can adapt to changing needs and conditions.

Now, in the report that they edited for the National Research Council, Professor Hubert and his colleague, Professor Hauser, note that the thing that's interesting about these two sides is that the proponents and opponents of graduation testing can argue about the issue and argue about it vigorously precisely because we have relatively little research that addresses the real consequences of such testing.

So, you know, for all of those who stand up to tell you that it's very clear, that it's incontrovertible, that we know that it's going to drive improvement, that it can't be denied that things will be a certain way, at least the latest thinking from the National Research Council was that, in fact, the evidence is not clear. There's not that much of it; it's not been done over a wide variety of venues, over a long period of time. We don't really understand the long-term consequences. So that what makes this an interesting policy issue and a debatable issue is precisely the relative lack of overwhelmingly convincing evidence.

Cite This Article as: Teachers College Record, Date Published: November 15, 2000 https://www.tcrecord.org ID Number: 10642, Date Accessed: 10/23/2021 1:11:13 PM Purchase Reprint Rights for this article or review