|Read a Post for Postmodern Test Theory|
|Reply to this Post|
Considering all options
|Posted By: Jay Powell on July 1, 2016|
|In 1992, Norm Shklov and I published a paper in Educational and Psychological Measurement:|
Powell, J. C. & Shklov N. (1992). Obtaining information about learners’ thinking strategies from wrong answers on multiple-choice tests. Educational and Psychological Measurement, 52, 847-865.
This paper reports the results of a test-retest study designed to show how students at differing age levels change their answer selections in accordance with their maturing interpretation of the items they are answering. Mislevie's discussion of cognition in test taking as related to postmodern assessment did not cite this paper.
I suspect his conclusions would have been richer and less obtuse had he considered that answering reflects students' thinking as a major part of what they know.
Item designers have long had an intuitive sense of this possibility when designing optional choices.
The algorithm Norm and I developed bypasses linear dependency and extracts response dynamics reasonably well.
It's nearly a quarter century of neglecting the possibility that "zeroing out" the unaccptable answers may be a misapplication of the general linear model to nonlinear data.
This work has been replicated, cross-validated, independently supported and found to apply internationally. Someone has been asleep at the switch, causing the educational train to derail.