Toward the Relational Management of Educational Measurement Data

by Gregory K. W. K. Chung - 2014

Background:Historically, significant advances in scientific understanding have followed advances in measurement and observation. As the resolving power of an instrument increased, so have gains in the understanding of the phenomena being observed. Modern interactive systems are potentially the new “microscopes” when they are instrumented to record fine-grained observations of what students do in an online task. Advances in the conceptualization, design, and analyses of such interaction data enable the discovery of learning patterns and can power new applications. One application, personalization, is one of 14 engineering Grand Challenges identified by the National Academy of Engineering (2008).

Purpose: This article examines three levels of data available in online systems that can be used to understand student performance. Empirical research is reviewed to examine three fundamental questions: To what extent does students’ online behavior (a) relate to their cognitive processing, and to what extent can student behavior (b) be used to model their problem solving process and (c) be used diagnostically to reveal understandings and misconceptions?

Participants: The reviewed studies involved participants from college and K12 settings.

Research Design: The reviewed studies all focused on learning processes and outcomes. Nearly all studies had high frequency process-tracing data such as concurrent think-alouds, moment-to-moment telemetry, or both. Participants interacted with an online task on an academic subject. The task typically spanned one or a few class periods, and the studies collectively examined relations among students’ online behavior, cognitive processes (via think-alouds), and external measures of learning.

Data Collection and Analysis: In the reviewed studies, students’ online behavior was captured by instrumenting the system to capture and log interaction events. The more sophisticated approaches used a telemetry design based on the presumed cognitive processing occurring in the system.

Findings: In general, measures derived from students’ online behavior can be used (a) to decide when to intervene to influence learning processes (e.g., increased help seeking) and outcomes (e.g., improved course grades) and (b) as proxy measures of cognitive processing, understanding, and misconceptions.

Conclusions: In the coming years, multiple levels of data will be fused to better understand the student, including static data such as demographics, low-frequency data such as interactions within a learning management system, and high frequency data such as moment-to-moment interactions in a digital app. As education enters the era of big data and transmedia-based learning, data of and for an individual will power new applications to realize the promise of personalized instruction.

To view the full-text for this article you must be signed-in with the appropriate membership. Please review your options below:

Store a cookie on my computer that will allow me to skip this sign-in in the future.
Send me my password -- I can't remember it
Purchase this Article
Purchase Toward the Relational Management of Educational Measurement Data
Individual-Resource passes allow you to purchase access to resources one resource at a time. There are no recurring fees.
Become a Member
Online Access
With this membership you receive online access to all of TCRecord's content. The introductory rate of $25 is available for a limited time.
Print and Online Access
With this membership you receive the print journal and free online access to all of TCRecord's content.

Cite This Article as: Teachers College Record Volume 116 Number 11, 2014, p. 1-16 ID Number: 17650, Date Accessed: 6/16/2021 8:02:21 PM

Purchase Reprint Rights for this article or review