National Assessment: Pro and Con

by Galen Saylor - 1970

Although National Assessment is now well under way, seemingly all opposition has melted, and the bandwagon effect of getting "on board" is evident, it may, nonetheless, still be appropriate for the uncommitted to consider the contributions this project may make to educational evaluation and its shortcomings.

Although National Assessment is now well under way, seemingly all opposition has melted, and the bandwagon effect of getting "on board" is evident, it may, nonetheless, still be appropriate for the uncommitted to consider the contributions this project may make to educational evaluation and its shortcomings. Obviously, the thoughtful student of the subject will hardly share the wild acclaim accorded the project by an unsigned editorial in a recent issue of the Educational Researcher: "A grand dream which turned into a phenomenal task, which in time will add immeasurably to the effectiveness of education in the United States—that's the cur-, rent capsule view of the gigantic effort known as National Assessment."¹

The status of national assessment is this:² On July 1, 1969 the Educational Commission of the States took charge of the entire project, including the assets of the predecessor organization, Committee on Assessing the Progress of Education, that had been established in 1968 to conduct the assessment program, after the original planning group, The Exploratory Committee on Assessing the Progress of Education, established in 1964, had developed a detailed plan for assessment and had sponsored the preparation of the tests and exercises.

The Commission (ECS) has employed James A. Hazlett, formerly superintendent of schools at Kansas City, Mo., to head the project. In the meantime cape had contracted with Research Training Institute of Raleigh, NC, to conduct the first phase of testing. During the spring of 1969 approximately 32,000 randomly selected seventeen-year olds in hundreds of schools were administered the test exercises in citizenship, science, and writing. During the summer 20,000 adults, ages 26-35, and 2,000 seventeen-year olds not in school were also tested.

The newsletter from NEAP states that, because of Commissioner James Alien's interest in the improvement of reading, the cycle of testing has been revised so that reading has moved up.³ The revised schedule is as follows:

Cycle 1

Cycle 2

But obviously the future of national assessment will be determined by two factors: The reaction of everyone concerned—the public, the school people, the Congress, the press, the taxpayers, and ECS itself—to the results of the assessment, and hence to the project itself; and the availability of funds. The budget of about $2i million for the 1968-1969 fiscal year was funded by foundations and an appropriation of $1 million by Congress through the U.S. Office of Education. Where the estimated $4 million needed for fiscal 1969-1970 will come from has not been revealed, but obviously federal funds loom large in the planning. School administrators and researchers who have had their grants cut significantly this year may protest vigorously a diversion of a couple of million or more to assessment.

Pro Assessment

The principal argument advanced in favor of the program of national assessment is that it would provide all concerned with the education of American children and youth much more knowledge than they now possess about the status of education and schooling in this country.

Ralph W. Tyler states that "the purpose of the project on assessing the progress of education is to provide the intelligent lay public with census-like data on the educational levels of important sectors of our population in order to furnish a dependable background of information about our educational attainments, the progress we are making and the problems that we still face in achieving our educational aspirations."⁴ The Carnegie Corporation says that "it is presumably better to know more rather than less about anything, particularly the way we are educating our children."³

Dr. Tyler has often said that "we do not have sound and adequate information on educational results. Because dependable data are not available, personal views, distorted reports, and journalistic impressions are the sources of public opinion, and schools are frequently attacked and frequently defended without having the necessary evidence to support either claim. This situation will be corrected only by careful, consistent efforts to obtain valid data to provide sound evidence about the progress of American education."⁶ He and other proponents of a national assessment suggest that it could result in an index of Gross Educational Product comparable to the index designated as Gross National Product. Such an index, developed from an adequate assessment program, would presumably help us as a nation in determining educational policy and making decisions about educational programs. The conversion of the results of the assessment into an index of Gross Educational Product, GEP, say its supporters, would -- in the same manner in which the GNP aids economists -- serve educators, boards of education, legislative bodies, and Congress in policy-making for the schools.

The Uses of Evaluation

Some basic, yet comprehensive evaluation of educational outcomes is always necessary. Two basic types of program evaluation are essential: the evaluation which determines the extent to which a particular educational program or set of policies is achieving adequate objectives for which it was planned and designed; and external evaluation of the total educational program to determine whether the objectives themselves are valid and appropriate, and whether the schooling being provided is the kind needed by and significant for young people in today's world and the world of the future. It should be acknowledged that an assessment of educational progress like that proposed by the advocates of a national assessment could, if properly conceived and carried out, provide some important data for such an external evaluation.

But the primary purpose of any evaluation of a social agency or institution is to aid in decision-making. This is the prime contribution claimed by those who advocate a national assessment. Decisions, they say, could be based on better knowledge of the present situation. Tyler, for example, has stated that one of the fundamental purposes of evaluation is "to provide the public with dependable information to help in the understanding of educational problems and needs and to guide in efforts to develop sound policy regarding education."⁷ He goes on to say that such information is essential. Without it, he believes, "we scatter our efforts too widely and fail to achieve our goals." Francis Keppel, former United States Commissioner of Education, has said that the national assessment program "might contribute a more accurate guide than we currently possess for allocation of public and private funds, where they are needed, what they achieve, and decisions affecting education."⁸

An example of the kind of survey the assessment proponents had in mind can be found in the 1966 report entitled "Equality of Educational Opportunity." This was the famous "Coleman Report," based on tests and surveys made under the auspices of the Office of Education. The results were, for a time, widely used as a basis for making extensive proposals about the desegregation of schools, wherever de facto or legal segregation existed. Such national surveys obviously provide a powerful argument for recommending changes in basic structures, programs, policies, and the educational plans of the American school system; and a national assessment program would be no exception. Anything, of course, that contributes to the improvement of our educational programs contributes to the well-being of our young people and advances the social, political, and economic life of the nation. If, by means of a national assessment program, we could increase our citizens' knowledge and understanding of the schools, we would be increasing public interest in education and, perhaps, contributing to public realization of the need for greater federal support. In fact, many of the proponents, particularly Francis Keppel (when he was commissioner), maintained that continuing increases in federal support of education might well depend on the Congress receiving objective data showing that the money was being wisely used in advancing young people's well-being.

Contra Assessment

The basic purpose of all evaluation is to promote better programs of education. As the committee for the 1967 ASCD Yearbook⁹ insightfully pointed out: "The purpose of evaluation is to provide feedback and guidance to the whole educational process at every level." The primary question in evaluation, according to the committee, is how the results can be used "as a positive force towards better teaching, better learning, better balanced curriculum." Evaluation, they wrote, "controls the next step... all of our decisions are conditioned by perceptions of how we are doing in terms of what we had hoped to do."

It is precisely on this ground that the proposed national assessment falls short. It would provide very inadequate, limited evaluations of our educational product; therefore, the feedback received from this huge undertaking would be grossly misleading. It would, very likely, constitute a serious barrier to the development of better curricula and instructional programs in the schools. This is because evaluations of any aspect of a school's program play an important role in determining what is done in that program. If we "teach for tests," and if the kinds of evaluations made and the data obtained from the tests used are limited in scope, validity, and the type of objectives for which accomplishment is measured, the educational program will be distorted and miseducative. The proponents of a national assessment justify their program by arguing that "it's better to know more rather than less about anything." But inadequate knowledge about matters requiring basic planning and decision-making can be and usually is detrimental for sound decision-making. If the knowledge we gain through a given evaluation is biased or inadequate, we might well make better educational decisions on the basis of our experience -- serve educators, boards of education, legislative bodies, and Congress in policy-making for the schools.

The Uses of Evaluation

Some basic, yet comprehensive evaluation of educational outcomes is always necessary. Two basic types of program evaluation are essential: the evaluation which determines the extent to which a particular educational program or set of policies is achieving adequate objectives for which it was planned and designed; and external evaluation of the total educational program to determine whether the objectives themselves are valid and appropriate, and whether the schooling being provided is the kind needed by and significant for young people in today's world and the world of the future. It should be acknowledged that an assessment of educational progress like that proposed by the advocates of a national assessment could, if properly conceived and carried out, provide some important data for such an external evaluation.

But the primary purpose of any evaluation of a social agency or institution is to aid in decision-making. This is the prime contribution claimed by those who advocate a national assessment. Decisions, they say, could be based on better knowledge of the present situation. Tyler, for example, has stated that one of the fundamental purposes of evaluation is "to provide the public with dependable information to help in the understanding of educational problems and needs and to guide in efforts to develop sound policy regarding education."⁷ He goes on to say that such information is essential. Without it, he believes, "we scatter our efforts too widely and fail to achieve our goals." Francis Keppel, former United States Commissioner of Education, has said that the national assessment program "might contribute a more accurate guide than we currently possess for allocation of public and private funds, where they are needed, what they achieve, and decisions affecting education."⁸

An example of the kind of survey the assessment proponents had in mind can be found in the 1966 report entitled "Equality of Educational Opportunity." This was the famous "Coleman Report," based on tests and surveys made under the auspices of the Office of Education. The results were, for a time, widely used as a basis for making extensive proposals about the desegregation of schools, wherever de facto or legal segregation existed. Such national surveys obviously provide a powerful argument for recommending changes in basic structures, programs, policies, and the educational plans of the American school system; and a national assessment program would be no exception. Anything, of course, that contributes to the improvement of our educational programs contributes to the well-being of our young people and advances the social, political, and economic life of the nation. If, by means of a national assessment program, we could increase our citizens' knowledge and understanding of the schools, we would be increasing public interest in education and, perhaps, contributing to public realization of the need for greater federal support. In fact, many of the proponents, particularly Francis Keppel (when he was commissioner), maintained that continuing increases in federal support of education might well depend on the Congress receiving objective data showing that the money was being wisely used in advancing young people's well-being.

Contra Assessment

The basic purpose of all evaluation is to promote better programs of education. As the committee for the 1967 ASCD Yearbook⁹ insightfully pointed out: "The purpose of evaluation is to provide feedback and guidance to the whole educational process at every level." The primary question in evaluation, according to the committee, is how the results can be used "as a positive force towards better teaching, better learning, better balanced curriculum." Evaluation, they wrote, "controls the next step ... all of our decisions are conditioned by perceptions of how we are doing in terms of what we had hoped to do."

It is precisely on this ground that the proposed national assessment falls short. It would provide very inadequate, limited evaluations of our educational product; therefore, the feedback received from this huge undertaking would be grossly misleading. It would, very likely, constitute a serious barrier to the development of better curricula and instructional programs in the schools. This is because evaluations of any aspect of a school's program play an important role in determining what is done in that program. If we "teach for tests," and if the kinds of evaluations made and the data obtained from the tests used are limited in scope, validity, and the type of objectives for which accomplishment is measured, the educational program will be distorted and miseducative. The proponents of a national assessment justify their program by arguing that "it's better to know more rather than less about anything." But inadequate knowledge about matters requiring basic planning and decision-making can be and usually is detrimental for sound decision-making. If the knowledge we gain through a given evaluation is biased or inadequate, we might well make better educational decisions on the basis of our experience and the knowledge we already possess. Inadequate knowledge almost invariably results in inadequate planning, if not planning that is erroneous and inept.

It should not be forgotten that we do have a great deal of knowledge about educational achievement in the schools. Our pupils are the most tested human beings in the world; and we know a vast amount about the nature of their educational attainments. Many of our existing programs have been developed on the basis of an intelligent use of such evidence. Moreover, our educational system, for all its faults, has advanced to a preeminent place in the world today. Who is it that needs this huge national assessment to make sound educational decisions? Not we educators. I seriously question whether the assessment program can aid us in any significant way in our educational planning, largely because the knowledge to be acquired through this project is so likely to be misleading.

My grounds for stating this are that only a limited portion of the educational enterprise is to be evaluated by the national assessment; and the feedback to be obtained will apply only to a small part of the total program of education offered in schools. Educators and citizens, nevertheless, will be urged to accept these limited results as a basis for changing educational programs. Otherwise, why spend so much money?

The assessment, as it is now described, will not present evidence on the development of motivational forces; nor will it provide information on self-image and self-actualization; nor will it add to our knowledge respecting character development, moral education, health, and physical status. The feedback will have to do solely with knowledge and the incidental aspects of behavior that may be revealed through paper and pencil tests or interviews. This means that significant information will be lacking on some of the most important objectives of education in today's schools; and I feel that this results in an extreme distortion of the educational enterprise.

Again, I must point out that the nature of the evaluation made of any enterprise becomes the source for feedback used in planning, developing, and carrying on programs for the achievement of the objectives sought. The national assessment certainly would not constitute more than a small part of the total evaluative processes engaged in by educators. My point, however, is that the influence such an assessment would have on educational planning would divert the schools from their efforts to attain other significant educational goals we accept as basic to adequate education for boys and girls.

Inadequacies of Evaluation

According to present plans, the results of the national assessment will merely tell us what percentage of a particular unit of the 192 samples of school children (or, in some instances, adults) can achieve on a particular test item. Dr. Tyler, in providing an illustration, reports that the assessment will enable us to learn that "91 % of 13-year-old boys of higher socioeconomic status in the large cities of the Northeast region of the United States knew \ of the following important ingredients in a person's diet," or that "68% of this same sample population could answer this question correctly: Which of the following areas of scientific inquiry has been completely investigated and is thoroughly understood? a. Electricity b. Weather c. Gravity d. Heredity e. None of the above." Or, he goes on, that "57% indicated that they did not believe in any of the following list of superstitions."¹⁰ Now I ask you what we would do with such information if we were to receive it? How important or significant do we find it to be?

Another proposed assessment exercise is the following: "Prepare written directions which tell a friend how to get to your home from the nearest turnpike exit."¹¹ This exercise is to be given to adults in an effort to measure their ability to write. Suppose, after we administer such a test, we discover that only 47% of the adult population can write a satisfactory set of directions. What is to be done next? I recognize that these are only sample test items from a large body of test items to be given to the four age-group populations included in the sample. But regardless how extensive or comprehensive the test items given to each population in any one of the seven subject fields, what will we have when the evaluation is complete and the reports have been made?

Now that the sets of objectives for which test exercises were prepared in science, writing, and citizenship have been published,¹² controversy has already broken out long before the results have been published. Henry De Zutter, writing a special feature for the Chicago Daily News, stated that the "40-page booklet (the citizenship objectives) might be subtitled 'a credo for the middle American.' It is certain to be attacked by the left, the right or the just plain different."¹³ We can foresee some long, but bitter debates in Congress and state legislatures, and even school board meetings about the percentage of 9-, 13-, 17-year olds and adults who '.'are loyal to country, to friends, and to other groups whose values they share."¹⁴ But much more seriously, what will happen to the school program following these debates?

The Exploratory Committee has evaded the question of norms and standards when it comes to determining what is desirable in the way of educational attainment. Presumably, norms would be defined by all those concerned about education, acting through boards of education, state legislatures, Congress, and other decision-making agencies. The program of national assessment, it appears, is intended to provide information on the status of education alone, not on its quality. Educators and citizens would still have to decide whether the results were good or bad. How, then, would the assessment results help in the decision-making process? In spite of pious denials, there is a hidden factor: the intent that (later on) state legislatures, boards of education, or Congress will prescribe criteria governing the accomplishment to be expected in the schools. If this is not the case, what is the sense of gathering all this information?

Individual Development

The principal deficiency in the national assessment program seems to me to lie in the fact that it ignores the planning of educational programs for specific pupils. The most important objective of American education is to develop each child to the fullest extent possible, in socially approved directions. National assessment, as I see it, will provide little assistance to those engaged in planning with this end in view. The assessment committee makes clear that it will not provide test scores for individual children, schools, or communities. Every child who participates will simply be a statistic in an elaborate sampling procedure. Tested on a very limited set of measures, his score will become a statistic in a massive analysis of educational achievement. What help does an undertaking of this size offer to a teacher in a particular school trying to teach a particular group of active children of many kinds and descriptions? What good would it do the staff of a school system, wrestling with major problems of educational management, to know that 29% of adults tested painted a picture? What good would it do the state educational agency to know that 57% of Midwestern 17-year-olds can interpret a paragraph by Ralph Waldo Emerson?

Just at this point in educational history when school systems, research centers, and educational agencies, often with federal support, are making valiant efforts to introduce a much greater degree of individualization in the instructional program, the nation is saddled with a national assessment that gives us a limited index of educational status in mass terms, with no evidence on the development of an individual child.

Inadequacies of Aim-Definition

We all agree that any adequate assessment must be made in terms of objectives and aims educators are presently seeking to attain. I think that the lists of objective formulated by the test agencies and refined by committees of scholars, educators, and parents are grossly inadequate. Moreover, I believe—like many thoughtful educators and citizens—that a program of national assessment will lead to increased nationalization of educational programs; and I do not approve of this. I want local control over educational programs and am personally opposed to all efforts to nationalize education. Can you imagine the debate likely to occur about the level of knowledge or lack of knowledge on the part of 13-year olds respecting the powers of Congress, or about the extent to which 17-year olds read newspapers? It is apparent that all sorts of pressure groups will seek to bring about "reforms" to serve their particular ends.

It should be clear that I am vigorously opposed to national assessment as advocated by the Tyler Committee for the reasons spelled out above. Assessment, I believe, would force the schools to emphasize objectives having to do primarily with the acquisition of knowledge and skills in a limited part of the total educational enterprise. Having considerable experience in a nation with a highly centralized and nationalized educational system, I can assure you I want no part of nationalization in this country; and I am fearful that the plans for national assessment may lead in this direction. ;•

I would strongly urge that, instead of this far-flung national assessment project, we begin developing in our state and local school systems some comprehensive programs of evaluation. It is from such evaluations that we can gather evidence of help to local boards of education, administrative staffs, and teachers interested in undertaking reforms, modifying existing programs, and developing the kinds of new programs that would assure the children and youth of the community an improved education. I would advocate, therefore, that Congress make large sums of money available to the states for assisting local districts in undertaking expert evaluations conducted by specialists in the field. Our present methods of evaluation are often inadequate, invalid, or inconsequential; but this is not to say that we need a program of national assessment. If we use available resources to improve evaluative programs at the local level, we can use the information derived in revising school programs and improving instruction.

Just as the federal government has made funds available for the expansion of guidance and counseling programs, so it might set about helping local school systems and state education departments improve and develop their own assessment programs.¹⁵ A local school system should, it seems to me, have a well-staffed bureau of evaluation with an adequate number of staff members available to consult, advise, and work with individual teachers in a comprehensive and significant evaluation of their own educational program. This would obviate, I believe, the presumed "necessity" of a national assessment program.

References

1. "Education Commission of the States Absorbs CAPE," Educational Researcher, Vol. 20, No. 8,1969.
2. An earlier paper had reported the status of the project to 1967, "National Assessment: Current Status," in Robert R. Leeper, Ed. Humanizing Education: The Person in the Process. Washington, D.C.: Association for Supervision and Curriculum Development, 1967, pp. 104-114.
3. "Assessment Cycle Changed; Reading Moved Up," National Assessment of Educational Progress, Vol. 2, No. 7, December 1969.
4. Ralph W. Tyler, "The Current Status of the Project on Assessing the Progress," Educational Horizons, Vol. 45, No. 4, Summer 1967. (A more lengthy explanation of the purposes of the Project may be found in Tyler's "The Purposes of Assessment," in Wal-cort B. Beatty, Ed. Improving Educational Assessment and an Inventory of Measures of Affective Behavior, ASCD, 1969.)
5. "The Gross National Product: How Much Are Students Learning?" Carnegie Quarterly, Vol. 14, No. 2, Spring 1966, p. 2.
6. Ralph W. Tyler, "Assessing the Progress of Education," Phi Delta Kappan, Vol. 47, September 1965, p. 14.
7. Ralph W. Tyler, "The Objectives and Plans for National Assessment of Educational Progress," Journal of Educational Measurement, Vol. 3, No. 1, Spring 1966, p. 1.
8. Francis Keppel, "National Educational Assessment: We Badly Need It," National Educational Assessment: Pro and Con. Washington: American Association of School Administrators of National Education Association, 1966, p. 6.
9. Fred T. Wilhelms, Chairman and Editor. Evaluation as feedback and Guide. ASCD 1967 Yearbook. Washington, D.C.: ASCD, 1967.
10. Ralph W. Tyler, "Assessing the Progress of Education in Success," The Science Teacher, September 1966, p. 14.
11. Jack C. Merwin and Ralph W. Tyler, "What the Assessment of Education Will Ask," Nation's Schools, Vol. 78, November 1966, p. 79.
12. Committee on Assessing the Progress of Education, Science Objectives (33 pp.); Writing Objectives (19 pp.); Citizenship Objectives (57 pp.), Ann Arbor, Michigan, 1969.
13. Published in the Lincoln (Nebr.) Evening Journal, October 29, 1969.
14. Citizenship Objectives, op. cit., p. 12.
15. For one of the most insightful and helpful statements on the nature and character of the kinds of evaluation needed in education today, see Daniel L. Stuffelbeam, "Evaluation as Enlightenment for Decision Making," Improving Educational Assessment and An Inventory of Measures of Affective Behavior", op. cit., pp. 41-73.

Cite This Article as: Teachers College Record Volume 71 Number 4, 1970, p. 588-597 https://www.tcrecord.org ID Number: 1779, Date Accessed: 10/26/2021 6:19:10 PM Purchase Reprint Rights for this article or review