Are There Levels of Thinking?

This article was prepared at the National Center on Effective Secondary Schools, School of Education, University of Wisconsin-Madison, which is supported in part by a grant from the Office of Educational Research and Improvement (Grant No. OERI-G008690007). The opinions, findings, and conclusions or recommendations expressed in this article do not necessarily reflect the views of this agency or of the U.S. Department of Education.

Much attention has been focused in recent years on teaching “higher-order thinking” or “complex thinking.” I shall argue that attempts to identify criteria that mark out higher-order thinking and distinguish it from lower-order thinking are still far from satisfactory.

That we have an intuitive sense of a hierarchy of thinking with lower-order or more basic thinking at the bottom and higher-order or more complex thinking above is not to be doubted. The history of scientific progress provides us with an image of ever-more difficult problems finding solution. This image can be misleading for our purposes here, however. The fact that more difficult problems can be solved does not mean that the thinking is more difficult. Thanks to the fruits of previous inquiry, subsequent thinkers have greater resources at their disposal. Individual development presents the same picture. Multiplying 6 x 7 is extremely difficult for an eight-year-old, yet presents virtually no problem to most adults. Is that because the adult is a more powerful thinker? Not really, because most adults do not think this out at all; they have simply memorized the multiplication tables. The intuition that thinking itself varies from simple to complex is surprisingly difficult to formulate.

Let us look at a list of characteristics of higher-order thinking assembled by Lauren Resnick, a leading applied cognitive psychologist. Higher-order thinking (1) is nonalgorithmic, (2) is complex, (3) yields multiple solutions, (4) involves nuanced judgment, (5) involves the application of multiple criteria, (6) involves uncertainty, (7) requires self-regularization of the thinking process, (8) involves imposing meaning (finding structure in apparent disorder), and (9) is effortful.1 Some of these criteria suggest that higher-order thinking can be identified with certain types of tasks or problems. After all it is problems rather than thinking itself that can be characterized as yielding multiple or single solutions, requiring single or multiple criteria. Some of the criteria, especially 3 to 6 and 8, seem to fit ill-structured problems much better than well-structured problems.

What reason is there to think such problems require thinking of a higher order? Why does getting dressed in the morning or planning a dinner party, which seem to fit most of the criteria, require a higher order of thinking than winning a chess match against a formidable opponent? Why should driving a car, which fits all the criteria, require a higher order of thinking than solving an equation in mathematics, which fits only a few? Resnick does not claim that her list of criteria provides either necessary or jointly sufficient conditions of higher-order thinking; but since many easy problems fit most of the criteria and many hard problems fit only a few, it is not clear to me what features of thinking her criteria actually pick out.

One might deny that driving a car, which many of us do virtually without thinking, fits any of the criteria. True for us, but not for a novice. On the other hand, a task that appears to demand mere recall might actually require considerable effortful thinking. The other day I was trying to recall a student’s name. I remembered that her first name was also that of a popular actress whom I had recently seen on the screen. I tried to reconstruct the list of movies I had seen recently (by trying to remember the people I had gone to them with) and who starred in them. Before very long I identified the actress and then the name of my student came to me.

This brings out the point I made earlier, namely that whether a task requires higher-order thinking depends on the resources of the thinker. Consider whether higher-order thinking is algorithmic or not. Is there a recipe that will allow one to solve the problem mechanically? Whether there is an algorithm is one question; whether the thinker possesses the algorithm or uses it in the appropriate situation is an entirely different matter. We have algorithms for finding square roots and doing long division that our ancestors did not have. Many problems most of us find mundane and routine actually stumped the best and the brightest for centuries. Identification of level of thinking then, seems inescapably relative to the resources of the thinker, as Resnick clearly acknowledges. This point is recognized (and then forgotten) by Benjamin Bloom in his well-known discussion of educational objectives: “Before the reader can classify a particular test exercise, he must know, or at least make some assumptions about, the learning situations which have preceded the test .“2

Perhaps we can arrange problems in order of difficulty in the following way: If, in order to solve problem B one must first solve problem A, then one can say that B is more challenging than A. Such thinking lies behind Bloom’s taxonomy of educational objectives in the cognitive domain. The six major classes—knowledge, comprehension, application, analysis, synthesis, and evaluation—are alleged to “represent something of a hierarchical order of the different classes of objectives. As we have defined them, the objectives in one class are likely to make use of and be built on the behaviors found in the preceding classes in this list.“3

Let us, for the moment, accept the idea of such a hierarchy. If we are to make use of it in evaluating tasks, we must still have a way of identifying where on the hierarchy a particular task falls. How are we to do this?

Bloom, despite the acknowledgment just quoted, appears to believe that we can determine the correct classification by the formulation of the task alone. He offers a number of sample questions for readers to classify, and then supplies the correct answer for them to check their own decisions. Consider these two examples:

1. Which of the following actions would probably be least effective in correcting the undesirable features of group political pressures?

A-Overhaul the national patent system.

B-Concentrate economic power of the nation.

C-Give the public more information about the origin and extent of political lobbying.

D-Use a congressional investigating committee.

E-Encourage all groups to subordinate their interests to the national interest.4

2. Which has been a result of this country's policy of maintaining a high protective tariff? (1) Higher prices for domestic goods (2) lower prices for foreign goods (3) increased foreign trade (4) higher prices for farm

products sold in foreign markets.5

Now so far as I can see, there is no way of determining the kind of mental operations involved in answering these questions without some assumption about the knowledge available to the student. Having little knowledge of either tariffs or pressure groups, I rank these as roughly on a par. In each case, I must comprehend the question, including, of course, the concepts used in its formulation, and compare the proposed alternatives against inferences I would make based on the meager knowledge at my disposal.

Bloom himself ranks question 1 at the highest level (evaluation) and 2 at the lowest level (knowledge), but this classification is warranted only if we assume that a teacher or text did not identify the least effective means of correcting undesirable features of pressure groups and did so identify the results of high tariffs. It is true that question 1 has the verbal form ‘What is the least effective” while 2 asks ‘What is a result,” but such verbal clues are certainly not enough to determine the level of difficulty or thinking involved in a problem—as the following two examples, of my own creation, illustrate:

3. Which is the least effective means of drying clothes?

A. Hanging them out in the sun.

B. Drying them for one hour in an electric dryer.

C. Spinning them at high speed in the washer.

D. Pouring milk on them.

4. What is a result of women joining the work force in increasing numbers?

A. A high divorce rate.

B. The feminization of poverty.

C. The decline of the double-standard for sexual behavior.

D. An increasing number of teen-age suicides.

In question 4, whether any of the suggested answers is a result of the movement of women into the work force is, I would suppose, an enormously complex question that admits of no ready answer. As this question illustrates, it is sometimes very difficult to determine the effects of a given cause, and as question 3 shows, it is sometimes self-evident what is an effective or ineffective means of achieving some result. The form of wording, and this is the key point here, does not enable one to make any determination of the level of challenge posed. One cannot ensure that teachers challenge students to a high level of thinking merely by insisting that their questions contain certain semantic or syntactic features.

Discussion of Bloom’s sample questions suggests that his idea of a hierarchy of cognitive tasks is itself suspect. If evaluation is defined as “the making of judgments about the value, for some purpose, of ideas, artworks, solutions, methods, materials, etc.,“6 then question 4 is correctly identified as falling under the evaluation category. There is still no basis for saying that answering the evaluation question in some way requires answering questions of each of the other types or that it presupposes the ability to answer a question on the lower level dealing with the same subject—drying clothes. In fact, as far as I am concerned, question 4 is clearly the more encompassing. The ability to answer question 4 (which would be classified at the lowest level, under 1.22, knowledge of trends and sequences) requires comprehension, interpretation and evaluation of evidence, analysis of relationships, synthesis, and so on.

The concepts in some disciplines are often hierarchical, to be sure, in the sense that more sophisticated notions depend on simpler ones. This does enable us to rank tasks in terms of the organization of the discipline. To take a simple example from geometry, being able to calculate the volume of a cylinder presupposes being able to calculate the area of a circle or a rectangle (depending on how one represents the cylinder). If there are cognitive counterparts to cylinder, circle, and rectangle that cut across disciplines such that performing cognitive task A presupposes the ability to perform task B, however, we have not yet identified them.

Does all this mean that when we examine tasks assigned by teachers we have no way of telling whether students are being intellectually challenged? Not really. Given that we know what students are and are not familiar with, what materials they have at their disposal when they perform the task, and what kind of response is expected of them, we can estimate whether particular problems will prove intellectually challenging. Within a given domain, we can also make fairly confident judgments about whether a task would be more or less intellectually taxing than another for the average student.

Three problems remain however: (1) We have no way of “grading” the thinking required by a task in the absence of assumptions concerning the tools and resources the thinker has available for it; (2) the verbal formulation of the task provides no basis for identifying the cognitive operations that must be involved in performing it; and (3) we have only crudest means of assessing the relative cognitive demands of tasks in different domains.

Consider an analogy with territorial exploration.7 Within a particular kind of terrain—rocks, for example—we do “grade”’ the degree of difficulty of different routes, assuming a climber of average ability and experience. Although a particular route may look very hard or very easy to the untutored, there is a difference between appearance and reality. Consider two routes up a cliff, one graded as slightly more difficult than the other (for the average climber) but neither of extreme difficulty, and consider two climbers-a world-class expert and a complete novice. Will the expert be challenged more by the tougher route than the novice by the slightly easier one? Almost certainly not. Nor will the “climbing operations” performed by the two climbers be identical even if both successfully scale the cliff, for the expert will almost certainly be able to employ “moves” not in the novice’s repertoire. Finally, although we can rate the level of difficulty of rock climbs relative to each other with some confidence, we cannot reliably rate them relative to routes through deserts or tropical forests.

Where does this brief analysis leave us? When applied cognitive psychologists and curriculum workers write about higher-order thinking, they wish to designate something that has scientific status, that goes beyond parent or teacher talk of “challenging” or “intellectually demanding” work. Efforts so far to render this idea of levels of thinking precise and operational are, as far as I can see, unsuccessful. Perhaps a first step to more fruitful research and development is recognition of this failure.