Innovative Combinations' Test: A Tool for Measuring the Melioration Skill

INTRODUCTION

Most teaching processes are geared toward teaching students how to find “the right answer.” Most students will, by the time they finish their undergraduate studies, pass thousands of quizzes and examinations which test their knowledge and skills. This situation is not natural for an adult who must cope with two kinds of problems in his daily life—well-defined and ill-defined problems (Newell & Simon, 1972). A well-defined problem has a clear path to a solution—a pre-set algorithm attuned to the desired solution located in the problem arena. An ill-defined problem, on the other hand, has no clear solution. An ill-defined problem demands a sophisticated search in the realm of human information—a search which includes a multitude of alternative pathways. This set of problems requires a search conducted according to a heuristic search strategy. These include informal, intuitive, and speculative strategies, which leads a person to the point where he finds the solution on his own. The melioration skill can be categorized as a heuristic strategy for the solving of ill-defined problems.

The melioration skill was most recently defined (Passig, 2007) as a skill of the future, which is rarely, if ever, taught in school. It was suggested as the seventh category in the taxonomy of future cognitive skills (Passig, 2000; 2001). This taxonomy has futurized the taxonomy of Bloom (1956), and was based on studies which surveyed future developments in society, economy, and the sciences. We have found that each of the six cognitive categories which Bloom formulated might have a new aspect which reflects an alumnus’ future needs. On top of Bloom's six categories, the futurized taxonomy has added a seventh category, which was named melioration. Three behavioral terms have been identified to be included in the melioration skill (Passig, 2007):

•

Adaptation – The person creates an innovative product via the process of making a personal cognitive connection between two disparate fields;

•

Connotation – The person connects items of personal significance of which he is aware, and is able, based on those items, to carry out an adaptation between two disparate fields;

•

The Courage to Forget – The person is prepared to divest himself of conceptions, theories, or of situations which rigidify new thinking.

Two kinds of melioration of ideas were identified: melioration of ideas which lead to improved sophisticated tools, and melioration of tools that give birth to new ideas which influence mankind.

The melioration of an idea can be demonstrated through the concept of Emotional Intelligence (Goleman, 1995). This term brings about a better understanding of human intelligence. Two unrelated concepts—intelligence and emotion—produce a new definition for a concept which provides a better explanation of a social, organizational, and managerial situation in a complex world.

The melioration of tools can be demonstrated through the concept of molecular-computers, as discussed by Regis (1995). According to Regis, the purpose of computers is to work with bytes of information. Since a byte of information has no size as such, we aspire to make the computer, which deals with it, as small as possible. On the other hand, the smallest component part of anything known to man is the molecule. Biology has introduced us to complex, organized systems of molecules. Regis predicts that the day is not far away when we will develop a molecular-sized computer which will become a tool at the disposal of our sciences. This molecular computer will certainly be different from the brain in that it will be mechanical and not biological. It will also be much smaller, more efficient, and capable as any small machine, when compared with the human mind, which is relatively massive, and which loses and/or adds cells every day. The unrelated concepts of tools—computers and molecules—could merge to develop a new kind of tool that ultimately would enable humans to further develop ideas.

Initiating this line of thought, Eric Drexler (1992), then a student at MIT, and now the Head of the Foresight Institute for the Study of Nanotechnology in California, had the idea of taking biological components and metaphorically “convincing” them to do things for us, as they do in nature. His idea was to put nature's components together in a new way in order to get a different overall arrangement. What he is suggesting is not merely a laboratory arrangement of known biological structures, but an arrangement not seen in any natural serial structures of atoms and molecules.

The innovation in Drexler's idea is that these tools could push the cells to produce something of a completely new kind, as opposed to creating merely artificial versions of things that already exist. The idea of arranging building blocks of nature in a different manner is an example of the fusion of mechanical engineering and molecular biology to create a new science which meliorates both realms of science. This provides us with a new, meliorated tool with which we could work in ways that were unimaginable before.

Melioration is a skill which has just been defined, and for which, as yet, measuring tools have not been developed. One tool which may be close to fitting that role was found in a survey of the literature. It is called RAT–Remote Associates Test (Mednick & Mednick, 1967). The Mednicks maintain that the process of creative thinking involves making new associative connections between different elements, and that the further one element is from the other, the more creative the process. They believe that the creative person has a proclivity for putting disparate elements together into new and more efficient combinations. In order to measure interpersonal differences, they proposed a test which included groups of three words. The person being tested is asked to find a fourth word, which has a connection to the first three. Following are some examples: Cheese, blood, music; board, magic, death; rock, time, steel; athlete, net, rabbit; mouse, sharp, blue.

We found that the Remote-Associates-Test was not appropriate for measuring melioration, principally because it wasn’t suited for the target population we wanted to test—children in the sixth and seventh grades. Also, we felt that the linguistic level was too high for the young age we aimed to study. Reflecting these difficulties, we engaged in developing a simpler test, which we called the Innovative Combinations' Test (ICT). In order to develop the tool and test its validity, we had to develop a training program and test its reliability to measure the skill. The following is a description of the training program and the validating process.

AGE GROUP

We have involved, in the process of validating the ICT tool, 60 school children that did not differ in demographic variants (Table 1). All were from a large urban area with a median socio-economic background. The participants were sixth and seventh grade pupils aged 11 and 12 years. This age group was chosen for several cognitive reasons. Problem-solving is a complex skill, defined as a higher order thinking skill. It develops in the Formal Operational Stage, according to Piaget’s (1950) model of cognitive development. Thus, Vygotsky (1989) maintains, learning and development are mutual processes. Learning makes development possible, and in so doing, development advances learning. This Zone of Proximal Development, according to Vygotsky, includes the cognitive functions necessary for the maturing process. With the help of learning, these budding phenomena turn into cognitive skills. The best kind of learning, therefore, is directed toward these incipient functions, moves on to development, and leads on from there.

Table 1. Demographic variants among participants.

Sixth and seventh graders are in that zone of development, which is closest to the Stage of Formal Operations. We hypothesized that participation in a training program, described in more detail below, would enable the participants to enter this developmental stage from a firm base.

Initially, we have tested the experiment and control groups in their abilities to solve ill-defined problems as well as their abilities to combine information, in order to verify that a common ground exists in these abilities before the intervention can take place (Table 2). We have conducted two tests: the Torrance (1974) sub-test “Unusual-uses” and the “Innovative Combinations' Test” (ICT).

Table 2. Initial abilities to solve ill-defined problems and information combination.

TRAINING PROGRAM

According to Sternberg (1996), intelligence is not permanent, and can be changed and improved by interventions of various kinds. The training program which we developed for the purpose of validating our measuring tool was named “Thinking-Different.” This training program was developed with an online interface named WebQuest. It is a computerized tool which can be used for developing a learning environment of the sort which enables the student to browse Internet sites, and research a range of subjects through challenging tasks. Dodge (2001) found that WebQuest encourages the development of higher-order-thinking skills in a problem solving environment. Dodge also delineated five guiding principles for constructing a high quality WebQuest environment abbreviated as F.O.C.U.S: Find great sites, Orchestrate your learners and resources; Challenge your learners to think, Use the medium, and Scaffold high expectations.

The training program’s goal was to engage the participants in creating combinations of a variety of pieces of information and to generate new ideas in solving ill-defined problems. It challenged the participants to collect written, visual, and audio resources from the Internet. The resources could have been linked to form a web of connections. The participants collected materials and formed links according to specific tasks. For example, they were asked to surf virtual galleries displaying art works on a broad range of subjects quite removed from each other. While looking at the different pictures, the students were asked to identify special elements and connect them in order to solve a problem that was introduced to them.

The interface's shell of the training program contained a component that was always available to the student if needed. In these sections, the student could have found a thorough explanation about a variety of topics:

1.

Introduction – The goals and expectations of the training program;

2.

Task – A general list of the training program units;

3.

Process – Directions on how to complete the various study tasks, and to the virtual teacher for assistance in the process;

4.

Evaluation – Directions on how to evaluate the progress with some indicators;

5.

Summary – A summary of the learning processes involved in this training program;

6.

Problem reservoir – A collection of authentic ill-defined problems which can be selected and engage in their solution;

7.

Teacher’s guide – Presents theoretical information for the teacher in the fields of pedagogy and technology concerning WebQuest;

8.

The Virtual Teacher – Presents the students with advice, as well as audio and visual reinforcement.

Five units were developed in the training program. The students could collaborate to solve the problems introduced in the unit or engage in self-regulated learning as they saw fit. The following is a brief description of each unit.

Unit #1 – Aimed at developing the technique of a random word. This technique challenged the student to find a random word that might help in generating a new combination of resources and solve with it the problem at hand. For example, when a student selected a problem whose solution required accessing a site named Animator for Hire, he found virtual drawers from which he could draw random words presented as illustrations. He was then directed to choose a random word that might illuminate a new way to solve the problem.

Unit #2 – Aimed at developing the technique of escape, and to teach a process of formulating strange, illogical sentences of escape. For example, when a student selected a problem, whose solution required accessing virtual galleries which display the artworks of the artists Dali and Magritte, he was asked to escape for awhile from the problem and use the artwork as a spring-board for the problem’s solution. Viewing a sequence of artwork from different artists aimed at teaching how strange combinations of resources could work as a stepping-stone to solve ill-defined problems.

Unit #3 – Aimed at developing the technique of combining materials, and teaching the student the process of locating random combinations of words through which he or she could come up with ideas to solve an ill-defined problem. For example, when a student selected a problem, whose solution required accessing “Brainstorming software online” (www.brainstorming.co.uk/onlinetools/websoftwarerandomwebsite.html)—a tool which presents random words, he was asked to combine random words whose significance could produce a relevant solution to the problem at hand.

Unit #4 – Aimed at developing the technique of considering a variety of ideas, and to teach the student the process of dismantling and reorganizing the problem via concept-mapping. For example, in order to solve a problem, the students were asked to access sites such as optical illusions and music, and adopt ideas with a concept-mapping tool called Inspiration.

Unit #5 – Aimed at developing a subject matter course on water crisis, and to teach the student the problems and solutions surrounding water conservation.

These five units were packaged into one CD titled, "Thinking Different," that was distributed to the students who took part in the process of validating the tool for measuring the Melioration skill described herein. The units addressed a variety of learning styles that were exercised through experimental learning involving multiple intelligence.

The major limitation we have found in the course of this study with WebQuest was that teachers are deterred from using information technologies to teach high-order skills. They tend to believe that these skills are better taught by great masters.

THE INNOVATIVE COMBINATIONS TEST

The Innovative Combinations' Test (ICT) aimed at examining the student’s ability to meliorate ideas. We hypothesized that after taking the course, "Thinking Different," the student would be able to better solve ill-defined problems. We therefore developed two versions of the test: one version before the course and the second after the student took the course. The same tasks with minor nuances appeared in both versions.

The Innovative Combinations' tool was designed to measure the student's ability to solve real-life problems. Therefore, the starting point on each item of the test was a familiar daily life situation such as: a bus trip, a shopping situation in a supermarket, a telephone conversation, etc.

Real-life situations are situations to which technology has a great impact. For each situation, a problem was introduced, where two random basic assumptions were presented. These basic assumptions stem from different fields of knowledge far removed from each other.

It was assumed that a person possessing the melioration skill would be able to search through the realm of human knowledge and create strange and far removed combinations of information chunks that would suggest a viable way to solve a problem. In designing the measuring tool, we created an artificial situation which brought the realm of human knowledge close to the student through two basic assumptions: (1) In real life, we more or less rely on scraps of information for the creation of combinations which can help us solve our problems; (2) in our daily-life we choose for ourselves the bits of information from which we create the combinations we need to solve our problems.

Ten life situations were chosen in both versions of the two tests, presented in Table 3.

Table 3. The Innovative Combinations Test – Two Versions

Companies throughout the world are working on improving appliances and services which help human beings in their daily lives.

The following are 10 familiar activities which we perform in our daily lives. How will we perform these activities in the future?  

Next to each activity there are pairs of ideas from widely different areas. Your task is to combine the ideas and to come up with new ideas which will improve the activity.

Scoring the items of the tool was also based on the behavioral objectives of the melioration skill (Passig, 2000; 2001; 2007). The skill is characterized by three main behaviors:

1.

Consonance: Creating an innovative idea by making a personal cognitive connection between two areas which appear to be distant from each other;

2.

Connotation: Creating an innovative idea by bringing up front a personal association that attaches to a particular piece of information;

3.

Courage to disregard: Creating an innovative idea by exercising courage to forget prior associations.

Every relevant idea which was drafted by a student was scored in a scale of 0-4 (see examples in Table 4).

•

0 – If no combination was created, or if the one created wasn’t relevant to solve the problem;

•

1 - If a combination of bits of information was created which reflects a personal connotation, but which has no correlation with the offered assumptions;

•

2 - If a combination was created which reflects a correlation between the offered assumptions and a personal connotation;

•

3 - If a combination was created which reflects the courage to forget, but which has no correlation with the offered assumptions;

•

4 - If a combination was created which reflects a correlation between the offered assumptions, the personal connotation, and the courage to forget.

Table 4 describes a sample of real-life situations with a pair of assumptions that were attached to them and some graded ideas based on the grading scale.

Table 4. Examples of real-life situations, basic assumptions, and examples of students’ combinations

The student’s final grade in the Innovative Combinations' Test was the sum of points collected in all 10 items. This sum reflects the level of his ability to meliorate ideas (40 points would be considered 100%).

VALIDATING THE INNOVATIVE COMBINATIONS' TEST (ICT)

In order to validate the test, we built two parallel versions which served as the pre- and post-training ability to meliorate ideas. Both versions were shown to three experts—professors in leading universities specializing in cognitive studies, with all three experts affirming that the test indeed examines the ability to make combinations of disparate ideas, i.e., melioration.

The reliability of the test was examined via the parallel forms method. First, we sampled 54 sixth graders in a pilot study where they received two mixed versions of the test with a rotated order of the items.

In the pilot study, a t test was applied to the data and no statistically significant differences were found between the two versions (see Table 5).

Table 5. Comparison between two versions of the innovative combinations test in the pilot study (n=54)

In addition to the t test, a Pierson Correlation was calculated for the total grade for version A and B. Spearman Correlations were calculated on each item in version A vis-à-vis version B (see Table 6). The analyses showed that there is a significant positive correlation between the total grade on version A and that on version B (r{52}=.86, p<.001). We also found that there was a significantly positive correlation between all the items in version A for their parallel items in version B, except for Item #1, which had a correlation which was close to significant (p = 0.58). The results of the Spearman Correlations for each item are recorded in Table 6.

*      p<.05

**   p<.01

*** p<.001

Table 6. Spearman correlations between the items in both versions of the Innovative Combinations Test in the pilot study (n=54)

In order to determine the internal consistency of each version, Cronbach’s α was calculated for both versions of the test. We found a reliability of α=.81 and α= .77 for versions A and B, respectively (see Tables 7-8).

Table 7. Reliability of the Innovative combinations test in the pilot study – Version A

Table 8. Reliability of the Innovative Ccombinations test in the pilot study – Version B

In this study, version A was administered before the training program and version B after the training program, since no significant difference was found between the versions in our pilot study. The internal consistency test was administered to 60 other students. In version A, we found that Item # 5 was not consistent with the rest of the items. We therefore decided to remove it from the analysis of the results. After removing Item # 5, a result of Cronbach's α=.81 was calculated. In version B, we found a reliability of Cronbach’s α=.79 (Tables 9-10).

Table 9. Reliability of the Innovative Combinations' Test in the actual research – Version A

Table 10. Reliability of the Innovative combinations test in the actual research–Version B

SUMMARY

The melioration skill that reflects the ability to solve ill-defined problems makes it possible to enlarge the student’s toolkit for high-order thinking skills. This skill opens the opportunity to employ the language of innovation in learning processes. With it, students would be able to cope successfully with complex real-life problems and generate solutions with resources at hand.

This study was conducted in order to validate the tool among middle-school students. However, we found that the teachers of this group’s age are having difficulties in teaching a skill with no conventional subject matter attached to it. This study, therefore, implies that in-service and pre-service professional training needs to educate teachers to be aware of the cognitive prospectus of their students and provide them with tools to evaluate not merely the knowledge they acquire but their cognitive status too.

The Innovative Combinations' Test (ICT), in this initial form, aimed at providing the teacher with a tool with which one could examine progress in the students' ability to meliorate information. The importance of this measuring tool stems from the need to prepare students for tomorrow’s world. It will be a world which will demand flexible, innovative thinking based on the fragmented human drifting and chaotic info-sphere. We hope that this initial version of the ICT would open the path to curriculum planners and educators to better design learning experiences that would reflect the genuine needs of our alumni.

References

Bloom, B. S. (1956). Taxonomy of educational objectives: The classification of educational goals. New York: David McKay.

Dodge, B. J. (2001). FOCUS: Five rules for writing a great WebQuest. Learning & Leading with Technology. 28(8), 6-9.

Drexler, E. (1992). Nanosystems: Molecular machinery, manufacturing, and computation. Hoboken, NJ: John Wiley & Sons.

Goleman, D. (1995). Emotional intelligence. New York: Bantam Books.

Mednick, S. A., & Mednick, M. T. (1967). Examiner's manual, Remote Associates Test. Boston: Houghton Mifflin.

Newell, A., & Simon, H. (1972). Human problem solving. Englewood, NJ: Prentice Hall.

Passig, D. (2000). A taxonomy of future thinking and learning skills. School of Education, Bar-Ilan University. Ramat-Gan, Israel (in Hebrew). http://www.passig.com/pic/TaxonomyBookHebrewJun e2000Final.htm

Passig, D. (2001). A taxonomy of ICT mediated future thinking skills. In H. Taylor, & P. Hogenbirk (Eds.), Information and communication technologies in education: The school of the future, (pp. 103-112). Boston: Kluwer Academic Publishers.

Passig, D. (2007, January) Melioration as a higher thinking skill of future intelligence. Teachers College Record, 109(1). Accepted 18.4.2005.

Piaget, J. (1950). The psychology of intelligence. London: Routledge & Kegan Paul.

Regis, E. (1995). Nano: The emerging science of nanotechnology. New York: Little Brown.

Sternberg, R. J. (1996). Successful 3 intelligence: How practical and creative intelligence determine success in life. New York: Plume.

Torrance, E. P. (1974). Torrance Test of Creative Thinking: Direction manual and scoring guide – Verbal test, booklet A. Princeton, NJ: Personal Press, Inc.

Vygotsky, L. S. (1989). Thought and language. Cambridge, MA: The MIT Press.