|Read a Post for Getting at Student Understanding—The Key to Teachers’ Use of Test Data|
|Reply to this Post|
* Tests Measure Question Interpretation
|Posted By: Jay Powell on July 4, 2014|
|Since 1958, I have been exploring students' selection of "wrong" answers on both selected and supplied answer tests. For selected answers four papers are important (Powell, 1968 (EPM), Powell, 1977 (AJER), Powell & Shklov, 1992 (EPM) and Das et al. 2013 (JSM)). |
In the first paper, I showed that selection reasoning predicted answers. In the second one I showed that the wrong answers form a developmental sequence based upon Piaget's descriptions of reasoning stages. In the third paper, we showed that answer changes in repeated measurements supported the developmental sequence found in the second paper. Finally, the fourth paper was an independents study from our data that confirmed the findings of the former three.
That is, answers are selected based upon the ways students interpret questions. These selections are diagnostic of the nature of the students'' understanding. This assumption means that students can understand the question, less than the question and more than the question. Thus understanding level is more important than whether the answer is "right," which can be achieved from memorization without understanding.
The first implication is that ignoring the "wrong" answers destroys the integrity of these data.
The second implication is that total-correct scores are invalid measures of knowledge.
The third implication is that the interpretation of communications should be taught, not the content of communications. This is a revolutionary approach to teaching that transcends and transforms the teaching act into exploratory mind-games from information transmission.
The third paper (Powell & Shklov, 1992) provides an explanation of how tests should be scored using selection-pattern-analysis, as does my book, "Making Peasants into Kings (2010)."
We cannot improve educational effectiveness using an invalid feedback system. Bravo to these researchers, who have independently shown that our current approach to quality control leaves much to be desired.