State Assessment Becomes Political Spectacle--Part VI: Raising Standards in the Third Act
by Mary Lee Smith, Walter Heinecke & Audrey Noble - September 13, 2000
...Continued from Part V: Cast Changes in the Second Act of Assessment
THE ACADEMIC SUMMIT: Blitzkrieg Standard-Setting
The Academic Summit took place in a Scottsdale resort, in October, 1995. The nine Design Teams, one team for each of the nine content areas for which standards were to be developed, had had two prior meetings to become acquainted with each other and with the task before them. The schedule would be tight, but the summit planners believed it would be possible to form a team, write a three-page list of clear, measurable standards, present them to the other teams, get reactions during public hearings in December, and write a final draft to present to the State Board in time for its January, 1996 meeting. The Board would then approve the standards and issue a request for proposals to test publishers. The winning bidder would then construct pilot assessments to be administered in March of 1996. That was the plan.
Although officially managed by the ADE Deputy Superintendent, the Summit organization bore the marks of two groups: a set of consultants from Doyle Associates (a conservative think tank) and trainers and facilitators from Keegan's corporate partners. The influence of Dennis Doyle was apparent in that the new standards were to be written in terms of "Levels", rather than grades. Competency tests rather than time would be the means by which pupils would progress through the system. The influence of the corporate partners was felt in choice of facilitators, in language and concepts more appropriate to the corporate world than education ("Design Teams," clear and measurable standards, market incentives, performance equated with product, and the like), and in the prominent place of workplace skills and technology. A Board member who also represented the interests of the corporate community offered this influential perspective:
"I have always been concerned that seat time should not be a graduation requirement. Ever since I was in high school, seat time was all you really needed to get a diploma. And we all agreed that we wanted a diploma to mean something, to have some stakes to it, some risks to it, perhaps even get to the point ultimately where there could be a guarantee to the business community that if our students have a diploma that they can count on them having certain skills."
Each of the nine Design Teams had nine members plus one or more facilitators. The participants included parents, teachers, students, and laypersons who had been appointed by ADE from a list of self-nominations. By looking at the list of participants, it was difficult to figure out how these persons found themselves attached to these particular teams. That is, a math teacher was just as likely to turn up on the social studies team as on the math team. And curriculum specialists were conspicuously absent. Some of this seemed to be intentional and some the result of fortuitous circumstance. An official with ADE explained that many of the participants had nominated themselves based on Keegan's informal comments during a fact-finding trip. Later, when the Summit was announced more generally and officially, people (i.e., curriculum specialists) who asked to participate were told that the teams were already full. But the Department also made clear that loading the teams with nonspecialists would have the effect of reducing educational jargon and making the standards clear and measurable. During a Board meeting, one member said about curriculum specialists, "We don't want to know what they know. We deliberately cut them out of the process." Another Board member said in an interview:
We had teachers involved, but we did not have curriculum coordinators involved, nor did we have the Department of Education employees involved. We felt those latter two, while many could contribute, just as many, if not all, would have a stake in maintaining the status quo, or in making their job easier. And that was not our intent. We don't care really whether their job is easier. We want it to be just as challenging for them as it is for everybody else.
A Department observer commented on the composition of the teams this way
It's just another slap in the face of the current system, another way to say that , "We distrust what you're doing. We distrust your knowledge. You haven't done a good job."
Whether intentional or accidental, ADE did not pursue a strategy of representation on the teams, except to include laypersons on each. Among the missing were not only curriculum and assessment specialists, but members of ethnic groups as well.
In the language arts team, tensions surfaced. Although the summit directives attempted to guide participants away from constructivism toward the simple, clear, and measurable, resistance by the few professionals was evident in their attempts to introduce constructivist principles (some said jargon) from the standards of the National Council of Teachers of English. Again resisting directives, participants repeatedly tried to deal with issues of equity and quality education for second-language learners. Despite their pleas, the Superintendent resolved that, "Without question, the standards you are creating are for proficiency in the English language. Assessment of the standard will be in the English language." (1/25/95).
Keegan's memo marks the function of standards as symbols as it continued, "I know you are all well aware of the frighteningly low rate of reading proficiency as measured in the elementary years. And you know well that writing skills are seen to be in decline, especially by potential employers. Should the standards document you are completing suggest anything short of the highest degree of rigor and proficiency in the English language, I believe the public will question the sincerity of our effort."
On the language arts Design Team, 20 planned hours became, by some estimates, approximately 200 hours of meeting time, spread over nearly a calendar year, plus countless hours spent in reading, writing, and reflection. The language arts standards would not be accepted by the Board until the summer of 1996, and even then, only parts of the standards submitted by the Design Teams (reading and writing) were approved. The reality was that standards weren't complete by late 1997, when conservative groups were holding up science standards over inclusion of evolution and creationism, or by late 1998, when social studies standards were still bogged down in controversy.
By January, 1996, ADE had conceded that its schedule would not hold up. The ambitious plan to complete drafts of the standards during that three-day session proved impossible, and the Design Teams continued to meet thereafter. The drafts were passed back and forth from teams to ADE. In a letter from Keegan to schools (1/18/96), she announced that the drafts would not be submitted as planned to the Board and that the spring pilot assessment would therefore not be conducted that year.
According to an informant, even considering the diversity of backgrounds of the language arts team, the members came to common understandings of the issues and their own agenda. They took their "work seriously and produced draft standards with integrity, focus, and balance," (though this would not be realized until several months after the Summit). They incorporated constructivist principles in several ways, one of which was (defying orders) by deliberately writing standards that could not easily be tested with multiple choice tests. In addition, they attempted to inject issues of global literature, cultural comparisons, reading for pleasure, and self-assessment of student as writer, "but we knew from the beginning that that would be a tough battle." The team believed that it had incorporated constructivism by designating four varieties of standards within language arts, that is, listening/speaking and visual representation as well as reading and writing. Many of the constructivist ideas such as content integration, projects, thematics and problem-solving were in those latter parts of the draft language arts standards.
REVISING DRAFT STANDARDS
During the third week of December, after the nine draft standards documents had been distributed, 11 public hearings were conducted. ADE staff ran the meetings, some of which were attended by the Superintendent and Board of Education members as well as the Design Team participants. The audience for most of these meetings consisted primarily of organized constituencies: members of Parent-Teacher Associations, teacher associations, university content specialists, and conservative political groups, plus a few parents and teachers on their own. The meetings were all of a type - ADE presentation on the history of the standard-setting process, a question-and-answer session, and then a series of controlled, three-minute speeches from members of the audience who had signed a list to speak. The members of the podium listened politely but took no notes, suggesting that the comments would not enter into the department's actions. Occasionally, tempers grew short, as contenders over such issues as Outcomes-Based Education, bilingual education, and phonics instruction, argued against each other or against the state. There was no reasoned debate, the contenders failing to share each others' basic assumptions. Instead each speaker ran out his or her 3-minute opportunity in the spotlight, a series of monologues. Observers would have been hard pressed to recognize democratic participation. This was performance, complete with claques, stage whispers, and catcalls.
Following the public meetings, further reactions to the draft standards came to ADE by mail and fax. No one could say, however, just how extensive was the distribution of the design drafts or how representative were the comments that were sent back. If state teacher organizations had an official response, it was not reported in the newspapers. ADE later claimed that extensive teacher input had been sought and received. Many were dubious. A national representative of the American Federation of Teachers compared the drafts with the AFT standards for standards, and found them deficient and the process "absolutely flawed from the beginning."
ADE appointed review teams for each set of draft standards, adding people to the original Design Teams. Two members of the State Board appointed themselves and some of their friends to the review teams and assumed dominant roles, pressing the Board's and Keegan's agenda. This was a critical event, as a team member explained:
The subcommittee of the Board was supposed to work with the design team to try to come up with common ground of what we could live with. This is where you see a tightening of examples and leaving out the 'fluffy stuff' and the multicultural stuff and all that. And this is where you see all the concessions about how much emphasis to give to phonics and whether phonics gets its own bullet. There was this constant refining and rewriting and paring down. "Is this tight enough? Is this clear enough? Can this be tested? Is this a one-answer thing?" That kind of hammering pressure gradually shifted. It wasn't consciously decided about how much constructivism we could safely put in there. We knew who the players were and where the pressure points were. It was a subtle -- and sometimes about as subtle as throwing a brick -- but it was a constant movement in that direction, a constant struggle between those who had a commitment to the standards as first written and those who had the authority to turn them into approved items and then the test.... This is where the shift from process and constructivist theory to basic skills emphasis took place. We were very conscious about putting some of those broader, more integrated more holistic aspects of language development --those went into the other two areas. So you lost the vision of language arts for the future, which we worked very hard on incorporating. We had had a broader view. There are two other strands that we thought were important. We chose what to put in reading and writing based on the assumption that the other two parts [listening/speaking and visual representation] would also be there.
The idea of having kids making assumptions! Even the idea of personal experience narrative. "You shouldn't be having kids write about personal, private things. You shouldn't have kids speculate. You shouldn't have kids theorizing." It's like, "when you've mastered your skills , and when you've been educated, and when you've got to that point when you've matured, then you can start considering possibilities and rationale, but you don't do that until you're finished being educated." It's that common thread. "We don't like kids making comparisons about literature, and what other literature in the world would we want them to look at anyway? What other culture would we want them to study besides this one? Why would you put that in there?"
Revisions were made and passed back and forth from review teams to ADE and the Summit facilitators over the next several months. The evolution of the language arts drafts reflected the tensions already evident at the Summit: the Board and ADE emphasizing the simple, brief, measurable, and ambitious; and the Team leaning toward the complex, process-oriented, holistic, integrated, and influenced by the national standards. Between drafts three and four, the Superintendent wrote a long memo to the review team, recommending additional components to the standards, as well as clarifications, elaborations, and rewording. To illustrate these substantive changes, she wrote in a memo, "We must develop a sample reading list for each of the five levels to give ... a sense of the quality and complexity of text students are expected to read and master.... we may want to add a requirement that students read a certain number of books per year (e.g., 20-30) from an identified number of writers and genres." Angered, the team ignored most of the recommendations. One change she suggested did work its way into (or more properly, out of) the writing standards. Originally, one standard at the readiness level read, "Perceiving themselves as writers." Her response, "(important, but how do you measure?)", led to deletion of that standard and the substitution of "spells simple words," and "writes the 26 letters of the alphabet." ADE's insistence on measurability also resulted in the deletion of standards related to developing students as "life-long readers."
The Board had two interests that opposed the Team's vision of integrated language arts. First there was the pressure of time and the need to show that something had been accomplished. The standard-setting process had already taken up many more months than had been planned. The second interest was to undo the old ASAP reform agenda and move the policy toward a more traditional pedagogical orientation. The Board therefore moved to fracture language arts into separate areas: reading, writing, listening/speaking, and visual representation. According to an informant, this decision proved to be critical, for the progressive elements were in the connections, and after the Board got through, what was left was disaggregated skills. The Team was frustrated in its attempts to preserve the old ASAP, professional autonomy, progressive pedagogy and sensitivity to ethnic and language diversity.