Home Articles Reader Opinion Editorial Book Reviews Discussion Writers Guide About TCRecord
transparent 13
Topics
Discussion
Announcements
 

The Development and Impact of High Stakes Testing - Part 2 - The New Regents Examinations in New York State


by TC Record - November 15, 2000

(Editorís Note: This is a transcript of the lecture delivered. The text is not revised.)

Gary Natriello:

Now, what I'd like to do, and I will try to take you through this to give you a sort of up-close sense of how this is playing out, at least in one state, is focus on the case of New York, and the high-stakes testing requirements that are part-way in effect, but are going into effect in New York. And before I do that, I want to ask those of you who are probably even more up-to-date on this than I am, because you're living with the consequences, to feel free to jump in when you've got more recent information than I've got. Because I've been tracking probably through the end of last year. There are a lot of things that have happened in the spring; there's more data that's come out, and some of you are probably living with this, so feel free to jump in and add.

Let me just, for those of you who aren't from New York and who haven't lived through it, give you some background, which also may be some background for those of you who've lived through the New York situation. Because one of the things we've found, when we asked people about the New York reform, is that everyone has a different view of it; and that even if you're in schools where the testing reforms are being rolled out, depending upon where you are in the school, you either have a very clear image of what's going to happen, or you have a very distant image and a very fuzzy image. And in fact, we've also talked to people who want to distance themselves as much as they can from whatever image is coming down. So when we've interviewed people, I mean, I just talked about how there's evidence that people are paying attention to it, but there's also evidence, from the field work we've done, that a number of individuals are saying, gee, I'm close to retirement, or I'm in an area where the testing is not going to hit so squarely; let me see if I can back away from this and let someone else worry about this.

So it's not clear that everyone throughout the entire system has completely picked up on all the requirements. Now, as most of you who have been in New York's secondary schools at least, I think, are probably aware, the State Board of Regents has adopted new, more demanding standards for high school graduation. Now, these standards, the adoption of these standards, did not occur in isolation, I think, as Professor Sobol indicated a couple of weeks ago, but rather they were part of a broader movement of educational reform at the state level in New York. They were guided by a set of policies which attempt to develop and articulate and raise standards for schools and students. And the attempt is to promote adherence to this larger set of standards by developing a set of more comprehensive assessments, and assessments that are going to be simultaneously more challenging.

So the articulated goal is to develop a comprehensive system of learning goals and standards, curriculum frameworks, new assessments and support strategies, which sounds, say it one more time? I'm not even sure I said it right the first time, but let me try one more time. The goal was to develop a comprehensive system of learning goals and standards, so you have these end states that you're trying to get to; a set of curriculum frameworks; that is, a sense of how the curriculum should be organized; and guidance so that it could be aligned with those standards; a set of new assessments, so that there would be some new measures that would be, again, related to the goals in the curriculum frameworks; and then, finally, a set of support strategies. You know, things that would be done within the system to enable schools and teachers and administrators and kids to actually achieve the new standards. Now, there's a lot of discussion about the assessments, and there's a lot of discussion about the high stakes associated with the assessment; and there's been some discussion about the support strategies.

But again, one of the things that I think the policy-making community has done is, it's gravitated toward the assessments as a relatively low-cost strategy of intervening with the system; and what's lagged behind, in the same way we talked about a couple of weeks ago, the opportunity to learn standard, not really quite getting the attention that the outcome standards get. In the New York case, you see these support strategies lagging behind the imposition of the assessments themselves. And one of the things that I'm going to come back to at the end, when I sort of try to raise the issue of whether this is a rational, bureaucratic approach or whether it's a social and political approach, is that I'm going to argue as if it was a truly rational, bureaucratic approach. That is, if you're really to improve the, driving improvement of the system, you'd want to build those support strategies in a very powerful and robust way. And in fact, you might want to build those support strategies in before you put in the assessments. That's not, as it turns out, what's happening. The assessments are arriving in advance of the support strategies.

Now, you can argue that that's also a policy strategy or a political strategy, that first, what you want to do is call attention to the problem by having these comprehensive assessments that are going to deny high school diplomas to a large number of kids; and in fact, that will then call attention to the need for these support strategies to come on line. But if you were really sort of thinking about this as the rational process of driving improvement, you probably wouldn't expose any organization to that kind of inconsistency. So, you know, if you think about it, if you were in any kind of business, and you were trying to change the way you were going to do business, and you wanted to be successful -- suppose you were trying to launch a new product. So let's think about -- you know, people don't like this analogy if you're an educator -- so let's think about this as a product launch, right? And our new product is this new, improved Regents-certified high school graduate. You know, you remember our old product, and you remember the 1986 version. You know, it had less chrome and it didn't have the fancy brakes, and it didn't have all the safety features. Well, the new, improved Regents version, let's think about this as a whole new product.

Well, if we were going to launch a new product, and we were sort of rational business people launching a new product, we probably would not say, let's throw the product out there, watch it fail and then we'll come in and clean it up and save it, right? Because we'd probably want to make a better first impression. So we would probably think about all the things that we'd need to do to actually develop this new product. We'd say, you know, we want you now to think about us as offering a much higher-quality product than we've ever offered before; you remember our Corolla, now we've got the Lexus. We don't want you to think about the Corolla anymore, we want you to think about the Lexus. We've got these new higher standards that, you know, it's going to be soundproof; it's going to feel like you're in a big car; it's going to be secure. And one of the things we know, and we moved to that new product, you know, we're going to triple the price; a couple of little things like that are going to go into it. Well, we probably wouldn't kick out a kind of warmed-over Corolla and say, here it is, you know, and guess what? The way we've done this is, we're putting them all through the same line, but we're going to test them at the end. And only those that measure the new Lexus standard get the new Lexus stamp, and about 30 percent of them rolling off the line just aren't going to work. But the rest will be great, because we'll test them and you know.

What we'd probably do is ramp up a whole new production process, with new resources and new engineering and new design, because when you see it for the first time, we want you to say, wow, this is a great new product and every one of them works. But if you think about the way we've enacted the policy, we've enacted the policy in the other way. We've basically said, we're going to set these new high standards, we're going to impose them on everyone, we're going to make them meaningful; and oh, by the way, downstream we're going to have some support processes; we're going to have some support systems come on line. But we're going to lag the introduction of the real support processes after the new testing program. And that's what, I think, opens the policy community up to this alternative interpretation that, in fact, it may really not be about rational, bureaucratically-oriented improvement; it may be about social and political advantage, or at least it allows that to be an alternative and plausible interpretation.

Let me say a little bit about what's happened in New York. And again, jump in if you've got some additional information. Those of you who've been in New York for years realize that New York has had a high school graduation set of state exams for a long time. The Regents exams are not new, although there are new forms of them. And, in fact, the state has operated a kind of dual system, and the immediately previous requirements for graduation were developed in the 1980s. And under those arrangements, if you were a student and wanted a local high school diploma, you had to pass something called the Regents Competency Tests, which were relatively low-demand assessments; and you had to complete a certain corresponding number of course units of study. Only the students who wanted a Regents diploma had been required to pass a minimum of eight or more of the more demanding Regents examinations and the course units associated with them.

So we had basically this sort of dual-track system. And, as a result of the dual-track assessment policy, we had two different curricular tracks through high school, with local diploma students involved, for the most part, in less challenging classes, preparing for less challenging examinations; and students who were pursuing the Regents diploma taking more demanding Regents courses, preparing for more demanding examinations at the end. Now, the new graduation requirements still maintained a distinction between the local diploma and the Regents diploma, but all students now, no matter where you are in the system, have the opportunity to earn a Regents-endorsed diploma. And at the time we were looking at it, it may have changed a little bit, you would do this by earning a passing score of 65 on the new Regents examinations. So everyone's going to get the same set of examinations and presumably has the same opportunity to get this Regents diploma. And you can see, as you think about the old system and compare it to the new system, there are lots of reasons why we might want to argue for the new system: on equity grounds; on the grounds that everyone ought to have access to the same curriculum; that everyone ought to have a chance at the same high-quality exams, demonstrating their competence to employers; that the old dual-track system had lots of disadvantages to it. So it's not as if the old system was terrific, and the new system is somehow flawed in a way that the old system was somehow terrific. Now, the students taking the Regents examination might still receive a local diploma if they met a somewhat lower standard on the same examination.

Student:

(Inaudible)

Gary Natriello:

They've abolished the lower standard for the local diploma?

Student:

(Inaudible)

Gary Natriello:

So they don't have the lower local diploma score anymore?

Student:

No, (Inaudible)

Gary Natriello:

Or they just lowered the Regents score?

Student:

(Inaudible)

Gary Natriello:

Which was going to be the local diploma score on the original plan.

Student:

Right. Okay. But -

Gary Natriello:

So they just made one, and it's still hard, right, for certain kids.

Student:

What about kids with learning disabilities?

Gary Natriello:

There are a couple of different strategies. Sometimes you can take them in an alternate format. Sometimes, in some states, but I think not in New York, but in some states, you can actually take an alternate exam. You can no longer just exempt students, as far as I know, in New York State. And I think, increasingly, states are moving away from the exemption process. Now, it remains to be seen how that's going to play out. But you remember, we were talking a week or so ago about how do you interpret these changes in state achievement reports over time? And one of the problems in following these achievement tests over time was that different states had different exemption policies, and by changing the number of kids who were exempted, you could raise or lower the mean scores of the kids who were passing. So there was an interest in trying to get control over how many exemptions and how many excuses. There are a number of states that have kind of a local bypass, where people at the local school can make individual decisions for kids, and sort of an alternate way to the diploma by, in some states, it was the principal and the guidance counselor signing off on an alternate demonstration of graduation competency. I gather in New York State that's not going to be an option. And in general, there's an attempt to move away from that. Now, it still remains to be seen, and I've not followed the recent legal work on this area, but it still remains to be seen whether these things could be challenged under federal law or other law, in terms of certain kind of claims that could be made about denial of rights. It's a serious issue, and I know that people on the federal level have been thinking about it, and have been looking for examples on which a case might, in fact, be brought.

Now, that could all change at any time once we have an election, so who knows who's going to have what kind of motivation on either side, for whatever reason, to either bring those kinds of cases or not bring those kinds of cases. But the thing, I think, to focus on, which I think is coming through both in the examples and in the policy, is that the new policy really calls for a single set of standards to which all students are going to work. And presumably, there will then be put in place a set of curricula and a set of support services that will allow students to work toward those standards.

Now, keep in mind, when we went a couple of years ago and looked at how schools differed in their capacity, and we'll get to this issue in a minute, to actually respond to these new standards, there's dramatic differences across the state, from district to district, in how inherently prepared schools might be to adopt to these new standards. So we have, within the state, some schools where the vast majority of students have always taken the Regents examination; about 80, 90 percent of the kids moving in that direction. And, of course, what that means is that the school has oriented its program for a long time toward preparing kids to make that kind of achievement. And then we have other schools in the same state system where very small proportions of kids have been prepared to take the Regents examination. And, correspondingly, we then have curricula in those schools and staff in those schools that, in fact, have not been oriented to preparing kids for that kind of work. So when you think about the challenge that's going to confront these different kinds of schools serving different kinds of communities, some are going to be very well-positioned to make the move.

And in fact, when we interviewed certain administrators and teachers and schools about this new requirement, some people in the schools where large proportions of kids had always taken it said, this is no big change for us. We've basically taught the Regents curriculum or a version of the Regents curriculum for the last ten or 15 years. There are other schools that had started more recently to do that; it was part of this movement to kind of get away from the old curriculum tracks. But we also found other schools where they basically said, you know, we're talking about a very serious change here, and it's not just the change in the assessments that we're going to ask the kids to take; we have to have -- you know, people were pretty willing to talk about this off the record -- we have to recruit a whole new faculty. First of all, we have to find people who can teach the Regents curriculum. And we have so many people, as you recall from Professor Zumwalt's lecture two weeks ago, who are teaching in areas in which they don't have preparation themselves. So all of those folks who were teaching in math or science or English, but had no major in math or science and English, may have been able to wing it when they were teaching low-level standards to kids who were going to pass the Regents Competency Test; but all of a sudden now, they're being asked to teach those subjects to a very high level so that kids can pass these new, more demanding tests.

Which is not to say, in any way, that that's not a good thing to move the system toward; it's only to say that, for some people, that movement is much more substantial than for others. And it's not just a movement of getting the kid to make that change. It's a movement of getting the faculties to make that change, and getting the curriculum rewritten and getting it reorganized, and getting materials purchased and a whole variety of other things. And in some schools, it has to do with actually having the physical facilities to offer that kind of curriculum. For example, if you look at the science standards, the science standards imply a whole set of things about being able to do laboratory science. And yet, we know there are large numbers of schools that don't have working laboratory facilities. It would be very difficult to actually do laboratory science in the absence of those kinds of physical facilities.

And the thing to keep in mind, as we think about these things is, an alternative, rational approach could've been to actually do an initial assessment of whether we were actually equipped to begin to offer this more demanding kind of work; to think about what it would take to equip us, both in terms of physical infrastructure, as well as personnel infrastructure to get the people in place, and then to begin to phase in the curriculum and the assessments in a kind of progressive way. So, for example, one of the things that did not happen that might've happened, if we were thinking about this rationally, if we were going to produce our new product, right, we would probably think about putting the personnel in place who actually would produce this new product in a particular way, so that we'd get the product we want. We wouldn't just announce that we're going to have this new product. And you'd think about that in a very procedured and orderly way. How long would it take to get the new personnel in place? What kind of changes would that imply for, in this case, teacher preparation? And how many years lead time do we actually need to have those new people coming into the system in place, prepared to teach a new standard? How long will it take us to prepare the first cohort of kids who are going to go through the system to actually achieve these new standards? And that's another thing that the policy does -- yes?

Student:

The state left it to local school districts. Like the state isn't thinking about these things, because they're saying, your school has to figure it out. We're just telling you what you need to do. So it's kind of like, we're giving the state too much credit. Like of course they didn't think about these things, because kind of just said, this is what you have to do. And I feel like a lot of the stuff in my school district is just kind of, you know, you figure it out.

Gary Natriello:

I think that's the consensus that we've heard from other people in other local districts. I don't know if other people have contrasting experiences. But, yeah, the state has announced these kinds of end-product exams and basically done a little bit with the curriculum frameworks, but hasn't done a lot more than that. Hasn't, for example, addressed the facilities issue; hasn't addressed the personnel issue; you know, although now, as Professor Zumwalt said, is beginning to put in place some movement toward addressing the personnel issue. But again, presumably, if you wanted to think about this in a rational way, you'd think, before I say that I'm going to produce the Lexus, let me bring in craftsmen who can actually create a Lexus, so that Corollas aren't coming out the other end. You know, presuming that there's some different strategies involved in doing it. Yeah?

Student:

I was going to say that this isn't strange behavior for state governments or federal governments. I mean, this is the same thing when they mandate all children in foster care within one year need to have a termination of parental rights hearing. And states aren't equipped to do that; courts aren't equipped to do that. I mean, it seems like there's a middle ground between it being rational or not rational, and therefore the social model. I mean, kind of a middle ground of just really poor governmental bureaucratic implementation. (Laughs)

Gary Natriello:

So you think it's not so much social and political as sort of flawed bureaucracy. They don't quite get it.

Student:

Yeah. And collaboration and something to do with the American political environment, and the way we do things incrementally and fragmented and on multiple levels.

Gary Natriello:

Which I think would suggest, and which I think is what some of the proponents suggest, that, you know, in ten or 20 years, we'll get it. I mean, the alternative approach is, look, we're going to announce these assessments; we're going to drive everyone to it, and everyone you're going to get it. And there's really just this gap of ten or so many years, where kids are just going to be ripped to bits, and you know, we'll eventually get to this better system.

Student:

(Inaudible) the government to do it (Inaudible)

Gary Natriello:

Right.

Student:

(Inaudible) so rational (Inaudible).

Gary Natriello:

Right. Right. Well, they're very rational at doing certain things to appeal to certain kind of markets. And, as we'll get to in a couple of weeks, the question is whether anyone's interested in picking up the low ends of the market where it's more difficult to do business. But, yeah, and that's an argument that policy makers would make; that if I had to wait for everything to be in place, I'd never get any movement. So, instead of starting with my support services, which cost a lot of money and, in fact, are not all that popular, I've got to start somewhere, so let me put in place these outcome standards.

Student:

I mean, I don't personally agree. (Inaudible) because that's like the way.

Gary Natriello:

Yeah, well, my own sense of it is that it's difficult to definitively know which of these two things, you know, which of these two interpretations, is more plausible. And the only point I'm trying to sort of raise as we go through this is that I'm not fully accepting this sort of rational model; that, in fact, these folks really think they're going to drive the kind of improvement they're talking about, because there are just too many pieces of the puzzle that just aren't there. And even the folks who are proposing it can't be that naïve; I don't think. You know, maybe I'm missing something here. But, you know, you'd think that you'd at least have to, I mean, think carefully about the standards. We're talking about asking adolescents to achieve a standards in mathematics when we refuse to provide them with a teacher who could probably pass the test him- or herself.

All right? And these assessments are great, by the way. You go look at the Regents exams: they're neat. I'd love all kids to be able to meet these standards, write those essays, do those problems, do the science that's on there. But it's very ambitious kind of teaching that has to get done. And I can tell you, it's not going to get done by someone who doesn't know mathematics. It's just not going to happen. And it's not going to happen with faculties that have systematically not taught mathematics for 20 years. It's not going to happen overnight. The other thing that's not going to happen is, it's not going to happen for a group of kids who've spent eight or ten years in a system oriented one way, and get into the secondary level and, in a year or two, are told, oh, by the way, you've got to meet these new standards; or recent immigrants who come to the country and are struggling with the language issue, and within -- what's the period of transition? -- it's a very short transition period in New York -- that you have to then be ready to take the Regents exam.



Cite This Article as: Teachers College Record, Date Published: November 15, 2000
https://www.tcrecord.org ID Number: 10643, Date Accessed: 11/29/2021 9:50:43 PM

Purchase Reprint Rights for this article or review
 
 
Member Center
In Print
This Month's Issue

Submit
EMAIL

Twitter

RSS