The Harrowing, Narrowing Effects of Data
by Kendall K. Paine - October 30, 2015
The use of data has produced a narrowing effect in education. It has caused schools to narrow the content we are teaching, focusing on key learning targets (e.g. Common Core State Standards). At the same time, it has caused us to narrow the students we are teaching. Since schools are evaluated by proficiency percentages, educators are using data to create categories of “green,” “yellow,” and “red” students, and diverting resources disproportionately toward “yellow” students as a means of boosting overall percentages. This commentary discusses the consequences of this phenomenon, particularly on student equity and on teacher morale. It ends by urging school systems to use data in a way that tracks growth rather than performance, in an effort to mitigate the triaging effect.
It will not come as a surprise that, in this era of accountability, schools are saturated in data. But what might be less apparent to outsiders is that educators are using this data to refine and narrow their focus in unprecedented ways.
As with all trends, this effect is nuanced. On the positive side, schools are streamlining instruction and focusing on core, critical standards. Given that we are living in an age in which classrooms have access to seemingly infinite amounts of information, this clear focus is imperativeamong other benefits, it guards against the tendency to become distracted by a constant stream of peripheral content. Data systems, in their effort to track progress on specific learning targets, have played a critical role in this narrowing process (Sloan, 2006). While these controls might initially confine and constrict teachers practices, they ultimately develop teachers expertise in key content areas, thereby enhancing the professionalism of the teaching force (Apple, 2000). Inevitably, teachers experience this streamlining in various wayssome with enthusiasm, and some with resentment. But the overall impact of all of this data on the quality of education that is being uniformly delivered throughout the United States is long overdue, and overwhelmingly positive.
The picture of the narrowing effect becomes more controversial when we consider not what educators are concentrating their efforts on, but for whom. While data systems are forcing teachers to track and target specific standards, they are also in some cases causing teachers to target specific students at the exclusion of others. In the process of tracking specific standards, schools often implement systems that label and stratify students according to their performance on key indicators. The mechanisms employed are varieddata walls, performance continuums, coded spreadsheetsbut almost invariably, students are categorized into some version of green, yellow, or red subsets (Booher-Jennings, 2005). Green students are those who consistently perform at or above grade level, yellow students are approaching grade-level performance, and red students are significantly below the prescribed academic standard for that indicator.
The intentions behind this process are benign, even noble. In theory, by categorizing students for each standard, schools can allocate resources according to need. In practice, however, the allocation of resources happens quite differently. Because high-stakes tests, and therefore school data systems, measure proficiency as a percentage, resources are disproportionately diverted toward the yellow students as a means of increasing the schools aggregate score (Booher-Jennings, 2005). By reallocating scarce resources in this way, schools can create the impression of improvement by augmenting their overall passing rates, even though an entire subset of studentsin fact, those who are most at-riskexperience very little movement toward their academic goals, if any movement at all. In effect, these data systems only serve to further entrench students at either end of the spectrum, while tipping middle-of-the-road students toward the higher end of the scale. And it is all happening because educators are, perhaps unwittingly, narrowing their focus to a specific subset of students.
It is our collective responsibilityas educators, policy-makers, researchers, parents, and concerned citizensto monitor the impact of education policy at the school level. We should track our progress toward its intended goals and guard against unintended consequences (Watanabe, 2007). While we can celebrate the ways the accountability movement has focused instruction on central concepts and standards, we also need to examine this phenomenon of educational triage (Booher-Jennings, 2005) that is taking place within our schools. It could be argued that there is perhaps nothing wrong with teaching some content at the expense of other content, but the same cannot be said of neglecting students. Certain content may be extraneous; no student is extraneous. Only by recognizing this phenomenon and understanding its consequenceson students and educators alikecan we avoid replicating policies that inflict similar dilemmas.
In the case of educational triage, the implications for students are self-evident, and certainly alarming. Clearly, there are significant equity issues at stake that provide sufficient reason to reevaluate our data usage in schools. Less obvious, though equally alarming, are the unintended effects of this phenomenon on educators. While it might be tempting to blame teachers for the ways they are using data to stratify students and unevenly allocate resources, research indicates that teachers engage in this process almost unknowingly, in many cases forsaking their ideals and identities to do so with corresponding unknown harm on their psyches. Watanabe (2007) frames the underlying mechanics of this phenomenon as a collision of paradigms. On one side, there are the constructivist ideals with which many educators enter the profession; on the other side is the essentialism that undergirds high-stakes accountability programs.
In this context, the essentialist nature of data systems compels teachers to shed their constructivism, and instead narrow their focusnot only on core standards, but also on pivotal, cusp students (e.g., students in the yellow subset of students). The ideal role of the teacher as champion of student best interests is cast aside, and the moral-ethical perspective of educators [becomes] supplanted by an instrumental concern for moving a designated number of accountable children above a particular bar (Booher-Jennings, 2005, p. 260). In short, by engaging with data systems that measure performance as an aggregate percentage, educators are being forced to adopt practices that contradict long-held beliefs about teaching in order to boost proficiency ratings (Marsh, Farrell, & Bertrand, 2014). Now that researchers have identified this tendency, we must actively guard against itfor the sake of our teachers and of our students.
The potential for data use to effect change is vastindeed, we are already seeing how data systems have streamlined our focus toward those standards that are considered most critical for students development. But in the long run, these advantages can only yield positive results if such focused instruction is delivered equitably to all studentsand by a vital, sustainable teaching force. Under our current practices, that vision is simply not a reality. Because schoolsand by extension teachers are measured by concrete percentages of proficiency, certain cohorts of students are being completely sidelined, and teachers notions of quality instruction are being turned inside out.
Rather than staying this ill-fated course, it would serve both groups to embed a wider spectrum of growth measurements into all of our data-collection mechanisms. While still focusing on a narrowed set of learning targets, we need to track students progress over time, as opposed to dichotomizing student performance as either passing or failing. This will hold educators accountable to all subsets of students, thereby simultaneously giving them more freedom to teach in a way that more closely approximates their constructivist ideas of education. To be certain, this is a lofty and complicated endeavor. Among other challenges, it means fundamentally shifting from a performance orientation to one of mastery (Marsh et al., 2014). Attempts to measure progress over time, as opposed to a blunt percentage, will inevitably undergo repeated cycles of trial-and-error before we achieve a more successful system. But if the alternative means neglecting the education of our most at-risk students, and demoralizing our teachers in the process, we simply cannot afford not to. This is one of the greatest educational imperatives of our time, and we have to engage it now.
Apple, M. (2000). Official knowledge: Democratic education in a conservative age (2nd. ed.). New York, NY: Routledge.
Booher-Jennings, J. (2005). Below the bubble: Educational triage and the Texas
accountability system. American Educational Research Journal, 42(2), 231268. doi: 10.3102/00028312042002231
Marsh, J., Farrell, C., & Bertrand, M. (2014). Trickle-down accountability: How middle school teachers engage students in data use. Educational Policy, 28(3), 138. doi: 10.1177/0895904814531653
Sloan, K. (2006). Teacher identity and agency in school worlds: Beyond the all-good/all-bad discourse on accountability-explicit curriculum policies. Curriculum Inquiry, 36(2), 119152. doi: 10.1111/j.1467-873X.2006.00350.x
Watanabe, M. (2007). Displaced teacher and state priorities in a high-stakes accountability context. Educational Policy, 21(2), 311368. doi:10.1177/0895904805284114