Home Articles Reader Opinion Editorial Book Reviews Discussion Writers Guide About TCRecord
transparent 13
Topics
Discussion
Announcements
 

Student Off-Task Behavior in Computer-Based Learning in the Philippines: Comparison to Prior Research in the USA


by Ma. Mercedes T. Rodrigo, Ryan S. J. D. Baker & Lisa Rossi - 2013

Background: Off-task behavior can be defined as any behavior that does not involve the learning task or material, or where learning from the material is not the primary goal. One suggested path for understanding how to address off-task behavior is to study classrooms where off-task behavior is less common, particularly in Asia, in order to understand why off-task behavior is less common in those settings.

Purpose: In this paper, we discuss three studies that examine the differences in the off-task behavior of students using intelligent tutoring software in the Philippines and the United States. We investigate whether Filipino students using intelligent tutors exhibit significantly less off-task behavior than their American counterparts, in line with the previously observed patterns of off-task behavior in traditional classrooms in East Asia and Southeast Asia. Using identical intelligent tutors in both countries allows us to control for confounds due to differences in curricula.

Research Design: In the first study, approximately matched cohorts of students in classrooms in the United States and Philippines used the same educational software (the Scatterplot Tutor) for an equal amount of time and according to an identical research protocol. In the second and third studies, students in the Philippines used different intelligent tutoring systems for a fixed amount of time.

Data Collection and Analysis: As students used the software, researchers used a quantitative field observation protocol to record their behaviors. From the quantitative field observations, we computed an approximate percentage of time that each student was off-task and then calculated the average time off-task for each group under study. An additional behavior, gaming the system (intentional misuse of educational software) was also coded.

Findings: In the first study, students in the United States were off-task, on average, seven times more than students in the Philippines. Interestingly, however, the proportion of gaming the system was higher in the Philippines than in the United States. In the second and third studies, the incidence of off-task behavior was comparable to the frequency of off-task behavior in the Philippines in the first study, and substantially lower than the frequency of off-task behavior in the United States in the first study or in prior research on off-task behavior in the United States.

Conclusions / Recommendations: The results of the first study suggest that the previously observed differences in off-task behavior between the United States and Asia cannot be attributed simply to differences in curricula. Understanding the relative roles of cultural factors and past experiences with adaptive educational technology in student off-task behavior is an important area of future work.


INTRODUCTION


Off-task behavior has been viewed as a major problem in North American and European schools for over a century (Currie, 1884, cited in Berliner, 1990), and it remains a concern today (Blatchford, Bassett, & Brown, 2005; Public Agenda, 2004). Off-task behavior can be defined as any behavior that does not involve the learning task or material, or where learning from the material is not the primary goal (Karweit & Slavin, 1982). Off-task behavior can take a number of forms, including off-task conversation with other students (or the teacher), interacting with materials other than the learning materials (such as a magazine), and sleeping in class (see Allday & Pakruar, 2007). Over the last several decades, there has been considerable evidence that off-task behavior is less common in East and Southeast Asia than in the United States.


However, comparisons have been confounded along several dimensions, including differences in curricula and research methods.  In this paper, we examine the differences in the off-task behavior exhibited by students in the Philippines and in the United States, using the same educational software and study protocol.   We use field observation methods to estimate the incidence of off-task behavior in each setting, as well as another disengaged behavior common in intelligent tutoring systems: gaming the system (Baker, Corbett, Koedinger, & Wagner, 2004). We also observe students in the Philippines using additional types of educational software, to validate that the pattern of off-task behavior seen in that first study is not characteristic of only one piece of educational software.


Understanding whether the national difference in off-task behavior previously studied in traditional classrooms will hold in educational software (which interacts with students in the same fashion in every country), will help us to narrow down which factors explain those previous observations.


OFF-TASK BEHAVIOR: THEORETICAL PERSPECTIVES AND PAST RESULTS


A common belief about off-task behavior is that every moment spent off-task, however it is used, is time not spent learning.  This view was first expressed as an explicit theoretical hypothesis in Carroll’s (1963) Time-on-Task hypothesis, within the Model of School Learning. This hypothesis is also found in later models of school learning, such as Bloom’s (1976) Theory of Human Characteristics and School Learning, and Fisher et al.’s (1980) Academic Learning Time construct.


The relationship between off-task behavior and learning has not been universally negative (e.g., Fredrick, Walberg, & Rasher, 1979; Karweit & Slavin, 1981; Kreijns, 2004), but the preponderance of evidence appears to suggest that off-task behavior is associated with poorer learning in a variety of instructional settings (Baker, Corbett, Koedinger, & Wagner, 2004; Caldwell, Huitt, & Graeber, 1982; Cobb, 1972; Fredrick & Walberg, 1980; Goodman, 1990; Karweit & Slavin, 1981, 1982; Lahaderne, 1968; Lee, Kelly, & Nyre, 1999; McKinney, Mason, Perkerson, & Clifford, 1975; Rossmiller, 1986, cited in Berliner, 1990). In particular, the few results where off-task behavior was not associated with poorer learning appear to be primarily in collaborative learning settings, where off-task behavior may improve classroom social relationships, resulting in improved participation and more effective collaboration (Kreijns, 2004).


The concern with off-task behavior, and the underlying disengagement that it reflects, has continued even as classroom pedagogy changes. Increasingly, classroom learning involves not just learning from teachers, classmates, and books, but also from working with adaptive educational technologies such as intelligent tutoring systems (Koedinger & Corbett, 2006). Though some research has suggested that this type of technology increases engagement compared to traditional curricular approaches (Schofield, 1995), off-task behavior persists in classrooms using adaptive learning technologies (see Baker, Corbett, Koedinger, & Wagner, 2004; Cetintas, Si, Xin, & Hord, 2009; Rowe, McQuiggan, Robison, & Lester, 2009). In addition, new forms of disengagement emerge in students using adaptive educational technologies, perhaps most notably “gaming the system,” in which the student attempts to succeed in an educational environment by exploiting the properties of the system rather than by learning the material and using that knowledge to answer correctly (Baker, Corbett, Koedinger, & Wagner, 2004). Examples of gaming the system include repeatedly and systematically guessing, and rapidly requesting additional hints until the software gives the student the answer.


In accordance with the evidence that off-task behavior is associated with poorer learning, and the belief that modifying pedagogical approaches can improve engagement (Bozack, Vega, McCaslin, & Good, 2008; Kelly & Turner, 2009), a number of projects in the last 20 years have attempted to decrease students’ off-task behavior (Abramowitz, O’Leary, & Rosen, 1987; Brooks, Horner, Todd, & Tofflemoyer, 2003; Clare, Jenson, Kehle, & Bray, 2000; Dalton, Martella, & Marchand-Martella, 1999; Moore, Sweeny, & Butterfield, 1993; Reid & Harris, 1993). Approaches that focus on self-modeling and self-monitoring have been particularly successful (Brooks et al., 2003; Clare et al., 2000; Dalton et al., 1999). Many successful interventions for off-task behavior, however, are relatively difficult to scale across entire classrooms, and hence the vision of universal interventions (cf. Kerr & Nelson, 1998; Walker, et al., 1996) for off-task behavior that are as effective as individualized classroom management has still not been realized.


OFF-TASK BEHAVIOR IN EAST ASIA AND SOUTHEST ASIA


One path that has been suggested for understanding how to address off-task behavior in the United States is to study classrooms where off-task behavior is less common, in order to understand why off-task behavior is less common in those settings (Winter, 1991). In specific, researchers have noted that classrooms in East Asia and Southeast Asia appear to have less off-task behavior than classrooms in the United States. While teachers in these regions often report off-task behavior as a problem in their classrooms (Weisz, Chaiyasit, Weiss, Eastman, & Jackson, 1995; Winter, 1991), students engage in significantly less off-task behavior in Asian classrooms than in Western classrooms. Weisz et al. (1995) report an average of half as much off-task behavior in classrooms in Thailand as in classrooms in the United States, using similar field observation methods. Multiple reports have suggested that off-task behavior is quite rare in Japanese classrooms as well (Abiko & George, 1986; Bennett, 1987). These findings have led researchers to suggest that off-task behavior may be a key component of differences in educational outcomes between East/Southeast Asia and the United States, and that bringing off-task behavior in U.S. classrooms into line with East and Southeast Asian classrooms may help to close this gap (see Chase & Mueller, 1989).


It is not clear, however, that instructional strategies from Asian classrooms will be successful in reducing off-task behavior and improving learning in American classrooms. Importantly, it remains unclear why students go off-task to such different degrees in Asian and Western classrooms. Thus far, a primary hypothesis for this difference is that cultural factors, including the cultural characteristics of both students and teachers, explain the difference in incidence of off-task behavior (Abiko & George, 1986; Weisz et al., 1995; Winter, 1991). Though there are many cultural differences between East and Southeast Asian cultures, some characteristics are relatively common across cultures in this region. One key cultural difference between Western students and East and Southeast Asian students is the individualist/collectivist dimension. Americans in particular typically describe themselves as individualistic (Hofstede, Hofstede, &, Minkov, 2010; Inglehart, 2009; Welzel, 2006). Compared with East and Southeast Asian students, Western students prefer to choose their own goals and to have more exciting, varied experiences. Southeast Asian students, on the other hand, describe themselves as obedient and tend to value collectivistic traits such as honoring their elders (Grimm, Church, Katigbak, & Reyes, 1999; Helmke & Vo, 1999; Hofstede et al., 2010; Inglehart, 2009; Welzel, 2006).  Lynch (1984) argues that behaving in a socially acceptable fashion is more important in the Philippines than in the United States, and Filipino students tend to blend their authority figures’ wishes, expectations, goals, and standards into their own personal motivations (Bernardo, 2008). Similar patterns have been observed in Chinese learners as well (Tweed  & Lehman, 2002), compared with learners in Western countries. In cultures that value individualism, people’s life satisfaction is seen as being a result of the experience of positive emotions (Suh, Diener, Oishi &, Triandis, 1998). In collectivist cultures, life satisfaction stems equally from respecting social norms and experiencing positive emotions (Suh et al., 1998).  Another key cultural difference is the value given to studying and academic effort. Chinese and Japanese parents tend to place much greater emphasis on studying and academic effort than American parents do (Stevenson & Lee, 1990; Stevenson, Chen, & Lee, 1993), and tend to rate their children as lower-performing (Stevenson et al., 1993), causing them to emphasize effort to a greater degree. For this reason, Stevenson, Chen, and Lee (1993) argue that, “the achievement gap . . . is unlikely to diminish until, among other things, there are marked changes in the attitudes and beliefs of American parents and students about education.” Similar patterns were found between Vietnamese and German students, where Vietnamese students reported putting significantly more value and effort into academic activities than German students (Helmke & Vo, 1999).   


However, it is difficult to be certain how large a role cultural differences play in driving student behavior, because curricula also vary widely between countries (Ginsburg, Leinwand, Anstrom, &, Pollock, 2005; Stigler, Gonzales, Kawanaka, Knoll, & Serrano, 1997). Differences in curricula can have substantial impacts on student engagement in learning (e.g., Schofield, 1995), suggesting that differences in curricula may pose an alternate explanation for why off-task behavior is less common in East and Southeast Asia.


Determining whether cultural or curricular factors are a better explanation of the difference in off-task behavior between countries will be an important step in understanding the roots of disengagement worldwide. If curricular factors better explain the difference, it would suggest that more research should go into determining which curricular differences reduce off-task behavior. These differences could then be adapted relatively smoothly by classrooms in countries where off-task behavior is more common. By contrast, if differences are culturally driven, the challenge will be greater. Cultural factors are fairly stable over time and difficult to intentionally manipulate (Hofstede et al., 2010), though systemic school reform projects have been successful at changing classroom culture in specific schools (see Ross, McDonald, Alberg, &, McSparrin-Gallagher, 2007). Hence, a finding that differences in off-task behavior are culturally driven would indicate that it will be challenging (though not necessarily impossible) to use practices from Asian classrooms to influence off-task behavior in American classrooms. An understanding of which hypothesis better explains international differences in off-task behavior has the potential to guide future research on how to influence off-task behavior in countries where it is prevalent.


However, the interpretability of research on international differences in off-task behavior has generally been limited by two confounding factors. First, research on off-task behavior in East and Southeast Asian settings has often been conducted using different protocols (and by different research groups) than research in Western classrooms. It has been shown that even small differences in research protocol can lead to substantial differences in observed off-task behavior (Karweit & Slavin, 1981, 1982). Second, most research has confounded culture with curriculum and teaching practices. In the cross-cultural comparisons listed above, students using different curriculum have been compared. Even if the same textbook is utilized in two countries, teachers may utilize it differently. Within this paper, we present studies designed to address these limitations.


FACILITATING CROSS-CULTURAL COMPARISON WITH INTELLIGENT TUTORING SYSTEMS


Recent developments have increased the feasibility of studying off-task behavior in different countries in a less confounded fashion. Specifically, adaptive learning technologies such as intelligent tutoring systems are now becoming increasingly incorporated into classrooms worldwide. Intelligent tutors are a type of educational technology that pose problems for students to solve and provides hints and feedback tailored to their individual learning. The most widely used intelligent tutoring system in the United States is the Cognitive Tutor software for mathematics, which is now used by 6% of high school students nationwide each year (Koedinger & Corbett, 2006). Intelligent tutoring software behaves in a predictable fashion no matter which student is using the software, reducing potential curricular confounds between countries. While some researchers have proposed that intelligent tutors should interact differently with learners in different countries (cf. Blanchard & Ogan, 2010), an intelligent tutor not designed specifically to do so will produce the same baseline experience for students in two different countries, with differences driven solely by the actions of the student himself. As such, these tutors may serve as a useful tool for studying differences between learners in different countries.


The use of intelligent tutors as a tool for international research was pioneered by Nicaud, Bittar, Chaachoua, Inamdar, and Maffei (2006), who presented a comparison of tutor use in India and France, finding that students tend to work at a similar rate in India and France, but persevere to a greater degree in India. Another study using tutors as a tool for international research was conducted by Baker, D'Mello, Rodrigo, and Graesser (2010), who studied the dynamics of academic emotions of students using intelligent tutors in the Philippines and United States, finding similar patterns between countries. Ogan et al. (2012), however, have found that collaborative patterns among students using intelligent tutoring systems differ considerably between Latin America and the United States. Hence, adaptive educational technologies appear to be a reasonable context in which to study international differences.


Furthermore, off-task behavior is sufficiently common in these types of educational technologies to make it feasible to study the behavior. In a study involving observations of 70 middle school students in two Pennsylvania schools, Baker, Corbett, Koedinger, and Wagner (2004) found students to be off-task 15% of the time while using Cognitive Tutor software for mathematics. In a study involving observations of 179 eighth-grade students in North Carolina, Rowe et al. (2009) found students to be off-task 15% of the time while using an intelligent tutor for science. In a longitudinal study involving 18 observation sessions of 12 elementary school students in Indiana, Cetintas et al. (2009) found students to be off-task 33% of the time while using an intelligent tutor for mathematics. Overall, these studies find a proportion of off-task behavior similar to the prevalence seen in classrooms using more traditional instructional methods (Lee et al., 1999; Lloyd & Loper, 1986).


Since intelligent tutoring systems control for curricula without reducing off-task behavior, they provide an opportunity to study the differences in off-task behavior between countries. So far, however, there has been no research comparing off-task behavior within intelligent tutoring systems between Western countries like the United States, and countries in East and Southeast Asia. To our knowledge, the only published study of student off-task behavior in these regions within intelligent tutoring systems involved university students (Gobel, 2008), a population that differs considerably from the high school, middle school, and elementary school students primarily studied in the United States.


RESEARCH FOCUS


In this paper, we examine the differences in the off-task behavior of students using intelligent tutoring software in the Philippines and the United States.  We investigate whether Filipino students using intelligent tutors exhibit significantly less off-task behavior than American counterparts, in line with the previously observed patterns of off-task behavior in traditional classrooms in East Asia and Southeast Asia.  To answer this question, we observed students in both the United States and the Philippines, using the exact same educational software and study protocol. The protocol is focused on estimating the incidence of off-task behavior, as well as another disengaged behavior common in intelligent tutoring systems: gaming the system (Baker, Corbett, Koedinger, & Wagner, 2004). We also observed students in the Philippines using additional types of educational software, to validate that the pattern of off-task behavior seen in that first study was not characteristic of only one piece of educational software.


Understanding whether the national difference in off-task behavior previously studied in traditional classrooms will hold in educational software will help us to narrow down which factors explain those previous observations. If off-task behavior in educational software is comparable in the United States and the Philippines, it will suggest that differences in classroom practice and/or curricular design are more likely to explain the differences in off-task behavior. By contrast, if off-task behavior is more common in the United States even in educational software, it will suggest that cultural factors are the more likely explanation.


STUDY ONE


In our first study, we analyze the frequency of off-task behavior among roughly similar populations of students in the United States and the Philippines, using the exact same educational software and the same study protocol.


POPULATION OF RESEARCH (STUDY 1)


This study compares two groups. The first group consisted of 53 students from two public schools in the suburbs of Pittsburgh, PA. The second group consisted of 60 students from a large public school in an urban area of Quezon City (a part of Manila), the Philippines.


There is no easy matching of schools between the Philippines and the United States, as the overall structure of how cities and metro regions are organized and populated differs greatly between the United States and Philippines, but efforts were made to ensure that the American population and Filipino population represented comparable demographics within their respective countries. The participating schools in both countries consisted predominantly of students from the local ethnic majority (i.e., Filipino students in the Philippines, white students in the United States). In both countries, students were drawn from a population that was neither unusually wealthy nor experiencing unusual degrees of poverty. Students in the United States were in mainstream mathematics classes (neither gifted nor special needs), and students in the Philippines were in both “science” and mainstream mathematics classes (neither of these tracks were gifted nor special needs, however the science track is considered to be more academically rigorous than the mainstream track). Both populations had an approximately equal number of males and females. In both countries, student ages ranged from approximately 12 to 14.


One difference between the populations was in terms of prior use of adaptive educational technology. The schools in the United States regularly use intelligent tutoring systems and related types of educational software, whereas the schools in the Philippines do not typically use these technologies. Rather than a confound, we consider this an inherent attribute of learning in the two settings in the early part of the 20th century—intelligent tutors and other types of adaptive educational technologies are now a routine part of education in many American schools, but remain rare in public schools in the Philippines (Rodrigo, 2004). The uptake of educational technology is driven both by economic and cultural factors, and as such is part of the “natural” or ecologically valid conditions of learning today, in different settings.  In other words, if a school in the Philippines in the year 2012 routinely used intelligent tutoring software for instruction, it would be sufficiently atypical of schools in the Philippines to raise questions about the validity of conclusions for the general population of Filipino students.


SYSTEM USED (STUDY 1): COGNITIVE TUTOR


In both the United States and the Philippines, students in Study 1 used a short Cognitive Tutor unit on scatterplot generation and interpretation (Baker, Corbett, Koedinger, & Wagner, 2004). Within Cognitive Tutors, each student works individually with the computer software to complete mathematics problems. A Cognitive Tutor breaks down each mathematics problem into the steps necessary to solve it, making the student’s thinking visible. As a student works through a problem, a running cognitive model assesses whether the student’s answers map to correct understanding or to a known misconception (Anderson, Corbett, Koedinger, & Pelletier, 1995), and errors are flagged. Cognitive Tutors give tailored feedback when a student’s answer is indicative of a known misconception. In addition, Cognitive Tutors offer multi-level on-demand hints to students. When a student requests a hint (by clicking a button), the software first gives a conceptual hint. The student can then request further hints, which become more and more specific until the student is given the answer. The hints are context-sensitive and tailored to the exact problem step the student is working on. In addition, the problems a student receives are individualized based on automated assessments of how well the student understands the mathematical skills and concepts taught in the lesson (a discussion of the computational approach used to produce these assessments is given in Corbett & Anderson, 1995).


STUDY METHOD


In both the United States and Philippines, students used the tutor software for 80 minutes. However, due to the different time scheduling practices of the schools, students in the United States used the software during two sessions on separate days whereas students in the Philippines used the software during a single session. In both the United States and Philippines, students had not explicitly covered scatterplots in class prior to the study. Before using the software, students viewed conceptual instruction, delivered via a PowerPoint presentation with voiceover and some simple animations.


We repeatedly collected data on each student’s pattern of behavior during tutor usage, using the quantitative field observation method from (Baker, Corbett, Koedinger, & Wagner, 2004). In this method, each observation lasted twenty seconds, and the coder repeatedly observed each student in a specific order determined before the class began.  During each observation, the observers stood diagonally behind or in front of the student being observed and avoided looking at the student directly (cf. Baker, Corbett, Koedinger, & Wagner, 2004), in order to make it less clear when an observation was occurring and to minimize the risk of observer effect.  Any behavior by a student other than the student currently being observed was not coded. Each observation lasted for 20 seconds—if a student was inactive for the entire 20 seconds, the student was coded as being inactive. If two distinct behaviors were seen during a single observation, only the first observed behavior was coded.


Behavior categories were coded using the following coding scheme (cf. Baker, Corbett, Koedinger, & Wagner, 2004):


1.

On-task – working within the tutor.

2.

On-task conversation – talking to the teacher or another student about the subject material or tutoring software.

3.

Off-task conversation – talking about any other subject.

4.

Off-task solitary behavior – any behavior that did not involve the tutoring software or another individual (such as reading a magazine or surfing the web).

5.

Inactivity – instead of interacting with other students or the software, the student instead stares into space or puts his/her head down on the desk.

6.

Gaming the System – systematic and rapid submission of incorrect answers or use of help, with several such actions taking place during the 20-second observation period. Gaming has been found in the United States to be associated with significantly poorer learning (Baker, Corbett, Koedinger, & Wagner, 2004), is associated with boredom (Baker et al., 2010), and with negative attitudes towards computers and mathematics (Baker et al., 2008).


For the purposes of this study, coding categories 3-5 were considered off-task.


The observation team in the Philippines was led by the first author, while the observation team in the United States was led by the second author. In each setting, there were three observers, only two of whom recorded observations during any given study session. The first and second authors have collaborated repeatedly on the use of quantitative field observation methods (see Rodrigo, et al., 2007), and significant effort was made to use identical observation procedures and coding schemes. The primary difference was that students used school computer labs in larger groups in the United States than in the Philippines—a standard practice in the schools due to the number of computers available. As a result, fewer observations were coded per student in the United States than in the Philippines (similar numbers of students used the software for the same amount of time in both countries, but there was significantly more total coder time in the Philippines due to the smaller groups using the software at any given time). In the United States, 490 field observations were conducted, for an average of 9.2 per student. In the Philippines, 1251 field observations were conducted, for an average of 20.9 per student.


Both teams conducting observations in the current study have previously achieved inter-rater reliability between 0.70 and 0.85 when coding student behavior with this method (Baker, 2007; Rodrigo et al., 2007; Baker & de Carvalho, 2008; Rodrigo et al., 2008; Baker et al., 2011), and observations of this type have been used to develop automated detectors of student off-task behavior which can predict student post-test scores (Baker, 2007; Cetintas et al., 2009). (We do not use off-task behavior detectors in this paper because they have not yet been validated to be accurate across different cultures.)



RESULTS


Table 1. Study 1 average incidence of off-task behavior

 

% Off-Task

% Gaming the System

Cognitive Tutor: United States

19.7% (17.8%)

5.3% (9.9%)

Philippines

2.7% (5.2%)

10.7% (15.3%)

Notes. Parentheses indicate standard deviation.


From the quantitative field observations, we computed an approximate percentage of time that each student was off-task and then calculated the average time off-task for each group under study. The incidence of off-task behavior was highly different between the students in the United States and the Philippines, as shown in Table 1.


Students in the United States were off-task an average of 19.7% of the time (SD = 17.8%), which is within the typical range reported in traditional classrooms in the United States, and similar to the proportions observed in prior research conducted with intelligent tutoring systems in the United States. By contrast, students in the Philippines were off-task an average of 2.7% of the time (SD = 5.2%). The difference between off-task behavior in the two countries was large—a 3.26 SD difference. This difference was statistically significant, t(111) = 7.02, p < 0.0001.


Interestingly, however, the proportion of gaming the system was higher in the Philippines than in the United States. Students in the U.S. gamed the system about 5.3% of the time (SD = 9.9%), in line with gaming frequencies observed in previous studies of the scatterplot tutor lesson (Baker, Corbett, Koedinger, & Wagner, 2004). Students in the Philippines gamed the system about twice as much: 10.7% of the time (SD = 15.3%). The difference in gaming frequency between the two countries was considerable, though smaller than the difference in off-task behavior—a 0.54 SD difference. The difference in gaming frequency was statistically significant, t(111) = 2.17, p = 0.03.


As such, the frequency of off-task behavior was substantially lower in the Philippines than in the United States, corresponding to previous results along these lines in traditional curricula. However, students in the Philippines may have simply replaced off-task behavior with gaming the system, a different disengaged behavior. We return to this possibility—and its implications—in the discussion section of this paper.


STUDIES TWO AND THREE


The results of the first study suggest that students in the Philippines are off-task for a significantly lower proportion of the time than students in the United States when using the exact same curriculum and when studied using an identical protocol.


In the second and third studies presented in this paper, we investigate the possibility that this difference may be isolated specifically to the Cognitive Tutor software used in that study; that perhaps there is some factor of Cognitive Tutor software that specifically engages students in the Philippines. To this end, we studied students using two additional intelligent tutoring systems. These two studies were conducted with an identical observation protocol to that used to study off-task behavior in the Scatterplot Tutor. Specifically, we studied students using Aplusix and Ecolab/M-Ecolab. We also studied a broader sample of students, to establish the general nature of the findings of Study One—moving from public schools to private schools, looking at a broader age range, and including schools in a second province of the Philippines, Cavite. Details on the two systems and the populations are given in the following sections.


SYSTEM USED (STUDY 2): APLUSIX


Aplusix (Nicaud, Bouhineau, & Chaachoua, 2004; Nicaud, Bouhineau, Mezerette, & Andre, 2007) (http://aplusix.imag.fr/) is an intelligent tutoring system for mathematics.  Topics are grouped into six categories (numerical calculation, expansion and simplification, factorization, solving equations, solving inequations, and solving systems), with four to nine levels of difficulty each. Aplusix presents the student with an arithmetic or algebraic problem from a problem set chosen by the student, and allows the student to solve the problem one step at a time, as he or she would using a paper and pen.  At each step, Aplusix displays equivalence feedback: two black parallel bars mean that the current step is equivalent to the previous step, while two red parallel bars with an “X” mean that the current step is not equivalent to the previous step.  This informs the student about the state of the problem in order to guide him or her toward the final solution.  Students can end the exercise when they believe they are done.  Aplusix then tells the student whether errors still exist along the solution path or whether the solution is not in its simplest form yet.  The student has the option of looking at the solution, a “bottom out” hint with the final answer.


POPULATION OF RESEARCH (STUDY 2)


Data were collected in four private schools in Metro Manila and one in Cavite, a province to the south of Metro Manila. Participants included 140 students in the first year of high school, and 29 middle school students, aged 12 to 15. The high-school students had approximately equal proportions of males and females, while the middle-school students were all male (they were studying at an all-male school). All students were using the software for the first time.  The students used the software for 40 minutes.   Observations were conducted by pairs of two observers, out of a pool of four observers. A total of 3,640 observations were collected from the high school students, with average inter-rater reliability of k = 0.78 across pairs of observers.  696 observations were collected from the middle school students, with average inter-rater reliability of k = 0.62 across pairs of observers.


SYSTEM USED (STUDY 3): ECOLAB/M-ECOLAB


Ecolab is educational software designed to help students learn the ecological and feeding relationships between different animal species (Luckin & du Boulay, 1999). Ecolab is based on the metaphor of an ecology laboratory, and enables learners to add plants and animals to a virtual environment, as well as to view that environment from different perspectives, such as an energy view or a web diagram.


M-Ecolab extends Ecolab to incorporate motivationally adaptive scaffolding (Rebolledo-Mendez, du Boulay, & Luckin, 2006), adding an affective companion, Paul.  If M-Ecolab detects that the student is currently in a low state of motivation, Paul appears with a worried facial expression and says, “You’re doing well but now try to do even more actions within the activity and if you make an error try again to do the correct action!” A more detailed description of M-Ecolab’s motivational support is provided in (Rebolledo-Mendez et al., 2006).


POPULATION OF RESEARCH (STUDY 3)


The Ecolab/M-Ecolab observations were conducted in two private middle schools in the Philippines—one in Cavite and one in Quezon City. A total of 180 students participated, 90 per system, aged 9 to 13, with students randomly assigned to use one software package or the other.  The population included an approximately equal number of males and females, and none of the students had used either Ecolab or M-Ecolab before this study. Students used the software for 40 minutes. Observations were conducted by two observers at a time, out of a pool of three observers. A total of 4,566 observations were collected. Each pair of observers’ inter-rater reliability was tested, and the average k was ĸ = 0.75 for Ecolab observations and ĸ = 0.77 for M-Ecolab observations.  


RESULTS


From the quantitative field observations, we computed an approximate percentage of time that each student was off-task, and then calculated the average time off-task of each group under study. As in the first study, the incidence of off-task behavior was quite low for the students in the Philippines, as shown in Table 2. Students using Aplusix were observed to be off-task less than 1% of the time.  The students using M-Ecolab and Ecolab exhibited slightly more off-task behavior: respectively, 2.7% and 4.5%. These proportions of off-task behavior were comparable to the frequency of off-task behavior in the Philippines in the first study, and substantially lower than the frequency of off-task behavior in the United States in the first study or in prior research on off-task behavior in the United States. As such, these results continue to provide evidence for off-task behavior being generally lower in the Philippines than in the United States, among students using adaptive learning technologies.


Unlike in the first study, the frequency of gaming behavior within the second and third studies was low—3.3% of behavior in Aplusix was gaming the system, and under 1% of behavior in Ecolab and M-Ecolab was gaming the system. Since the samples differed between the first study, and the second and third studies, it is not appropriate to statistically compare between studies. It is not particularly surprising that gaming frequency varied between systems—gaming frequency is already known to vary considerably between different learning environments (Baker et al., 2009). However, this finding suggests that even if students in the Philippines game the system in some educational software, they do not do so in all educational software. Hence, it will be relevant to understand what factors lead Filipino students to game the system to different degrees in different learning systems.


Table 2. Study 2-3 average incidence of off-task behavior.

 

% Off-Task

% Gaming

Aplusix

0.8% (2.2%)

3.3% (12.9%)

Ecolab

4.5% (10.5%)

0.6% (3.5%)

M-Ecolab

2.7% (7.4%)

<0.1% (1.0%)

Notes. Parentheses indicate standard deviation.


DISCUSSION AND CONCLUSIONS


Within this paper, we have presented a study comparing off-task behavior in the United States and the Philippines, using the same curriculum and research protocol, and two further studies on these behaviors in the Philippines with different curricula but the same research protocol. The difference in the degree of off-task behavior between the United States and the Philippines, within the educational software studied in this article, is quite striking. In particular, the results of the first study suggest that the previously observed differences in off-task behavior between the United States and Asia cannot be attributed simply to differences in curricula. This allows us to focus on differences in the people involved, their culture, and their classroom culture.  


One factor that may limit the generalizability of these findings is that the research population in the studies presented here was drawn from a relatively small part of each country. Though the sample size in general was comparable to or larger than the other cross-cultural studies reported in this paper, it is important to note that differences within a country can be quite substantial. The United States samples were drawn from a relatively demographically average part of the United States (the suburbs of Pittsburgh, PA); however, previous research in the United States suggests that populations with much higher or lower socio-economic status would have had different degrees of off-task behavior and gaming (see Baker & Gowda, 2010), though all samples in Baker and Gowda (2010) had substantially more off-task behavior than seen in the Philippines in this paper. Most of the Philippines samples were drawn from Metro Manila, the capital of the Philippines.  Aside from being densely populated, the area also has the highest functional literacy rate in the country (Philippines National Statistics Office, 2006), and the highest proportion of households with electronic media devices such as televisions and personal computers (Philippines National Statistics Office, 1994).  Clearly, replicating this study in a wider variety of schools within both the United States and the Philippines will be an important step towards understanding how general these findings are.


One of the key interpretations for these findings comes from the research discussed earlier on cultural differences between the United States and the Philippines. Past research had characterized Philippine society as one in which respect for authority and obedience are considered more important than independence and self-direction (Grimm et al., 1999; Helmke & Vo, 1999; Inglehart, 2009; Welzel, 2006), a pattern seen throughout East and Southeast Asia. The same survey characterized the United States, by contrast, as a society that deemphasizes respect for authority, and values independence over obedience.  Correspondingly, socially acceptable behavior is considered more important in the Philippines than in the United States (Lynch, 1984). These observations are consistent with findings from other cross-cultural studies. North American children are raised to act in self-directed and autonomous manners, following individual personal styles (Smetana, 2002). Off-task behavior is also related to how willing a student is to be noisy and social, versus being shy and quiet. Being shy and quiet is construed differently in Western and East and Southeast Asian societies: Westerners tend to interpret shyness as proof of immaturity or social ineptness and therefore meet it with disapproval (Rubin, Burgess, & Coplan, 2002), whereas shy East Asian children receive affirmation and acceptance (Chen, Hastings, Rubin, Chen, Cen, & Stewart, 1998). In general, Filipino students tend to blend authority figures’ wishes, expectations, goals, and standards into their own personal motivations (Bernardo, 2008). Additionally, ethnographic research suggests that the values of cooperation and compliant behavior become internalized by children in East Asian cultures, leading them to emphasize social obligation, group orientation, harmony, and close family ties (see Eisenberg, Fabes, & Spinrad, 2006; Farver, Kim, & Lee, 1995).  These differences may lead Filipino students to be more reluctant to go off-task than students from the United States.


These same values may explain why Filipino students disengaged in another fashion, gaming the system, in Study 1. Whereas off-task behavior typically draws attention, gaming the system presents a surface appearance of being on-task. To a teacher or fellow classmate, a gaming student appears to be actively engaging with the software. In a culture that emphasizes the value of group dynamics and self-control, gaming behavior may be a way that a student can disengage without incurring the costs of overt off-task behavior.


Another difference between the United States and East and Southeast Asia is in terms of the value given to academic effort (Stevenson & Lee, 1990; Stevenson, Chen, & Lee, 1993); this difference could potentially explain the lower off-task behavior seen in the Philippines, but would not explain the relatively higher frequency of gaming the system in Study 1.


Alternatively, these same cultural differences may have caused observer effects to function differently in the two countries. Despite our attempts to make the observations non-intrusive, the students in the Philippines might have regarded the observers in the room as authority figures.  As mentioned earlier, Bernardo (2008) found evidence that Filipino students tend to blend their authority figures’ wishes, expectations, goals, and standards into their own personal motivations. Students might therefore have made an additional effort to remain on-task.  Students in the United States, on the other hand, may not have been influenced to the same degree by the observers, given their greater individualism and propensity for self-expression. However, as with valuation of academic effort, this explanation does not account for the prevalence of gaming the system in the Philippines within the first study.


It is worth noting, however, that similar problems exist for alternative methods for assessing off-task behavior. For instance, survey items might produce different demand effects between cultures, and video cameras might produce different observer effects between cultures. Weisz et al. (1995) argues that cultural differences are a more serious problem for survey measures of off-task behavior than for observational measures, as there can be very strong self-presentation or demand effects with surveys. Automated software-based detectors of off-task behavior (see Baker, 2007) may be a future possibility for a method that avoids these problems (as they are completely non-obtrusive), but these detectors need to be validated to be accurate across cultures, and such validation has typically relied upon field observation data (cf. Baker, 2007; Cetintas et al., 2009).


Another possible explanation for the rarity of off-task behavior in these studies is that intelligent tutor software—and educational software in general—is more novel in the Philippines than in the U.S. Computers themselves are rarer in Philippine schools than in the United States, and when students utilize computers it is predominantly in the context of courses on computer literacy or programming, as opposed to language arts, math, or science (Rodrigo, 2004). Within our studies in the Philippines, all students were using the Cognitive Tutor, Aplusix, Ecolab, and M-Ecolab for the first time.  The students in the United States, on the other hand, had been using intelligent tutor software as part of their regular classroom routine all year (though they had not previously seen the specific intelligent tutor studied). Given this discrepancy, the lower off-task behavior seen in the Philippines might be primarily due to the novelty of using an intelligent tutor for math or science. While this might be viewed as a confounding “novelty effect,” it can also be seen as a genuine difference between students in the two countries. Finding middle school students in the United States who are completely unfamiliar with educational software would require finding schools that serve populations well outside of the norm for American students. At the same time, it is worth noting that this study replicates past studies in other parts of East and Southeast Asia that found that off-task behavior was rare when using traditional curricula of a type commonly seen (cf. Abiko & George, 1986; Bennett, 1987; Winter, 1991; Weisz et al., 1995). By contrast, field observations of engagement in the earlier years of intelligent tutor usage in the United States did not replicate the behavior patterns seen here (see Schofield, 1995). Hence, it seems relatively unlikely that a novelty effect could explain all of the differences in off-task behavior between the United States and East and Southeast Asia. Furthermore, it is unclear why intelligent tutors are so much more common in U.S. classrooms than in other countries—cultural differences may in fact be one of the factors explaining this difference.


Overall, understanding the relative roles of cultural factors and past experiences with adaptive educational technology in student off-task behavior is an important area of future work. One potential way to follow up the research presented here would be to replicate this study in additional countries. Comparing two nations inherently produces a confounded comparison. Hence, it would be of value to replicate the first study in this paper in a set of countries that vary systematically in terms of cultural variables and exposure to educational technology. For example, we could address the dimension of individualism/collectivism from (Welzel, 2006; Inglehart, 2009; Hofstede et al., 2010), one of the factors hypothesized above to explain the differences in off-task behavior between the United States and the Philippines.  The U.S. is a country characterized by high individualism and high exposure to educational technology; the Philippines is characterized by low individualism and low exposure to educational technology. Japan, to give one of many examples, is characterized by low individualism and high exposure to educational technology. Interestingly, there are relatively few countries that are high in individualism and low in exposure to educational technology – Zimbabwe is one of the few countries meeting that profile. The study could also include a pre-test that measures the extent to which the study participants exhibit individualism/collectivism.  This pre-test would help establish whether the participants selected from each country indeed exhibit the cultural characteristics attributed to their respective countries.


One other factor that is worthy of future study is the increase in gaming the system seen within the Cognitive Tutor in the Philippines in the first study. Though there was relatively little gaming seen in the second and third studies, this finding is nonetheless worth investigating further.


In particular, if students in the Philippines are disengaged a similar total amount of the time as American students for a given learning system, but manifest their disengagement differently, it may be necessary to reinterpret the meaning of the differences in off-task behavior repeatedly seen between countries. Specifically, past theoretical accounts have suggested that American students are at a disadvantage compared with Asian students because American students spend more time off-task (e.g., Abiko & George, 1986; Weisz et al., 1995). However, if students spend comparable amounts of time disengaged but spend their disengaged time differently, the differences in learning between the two populations of students might be minimal. In fact, there may even be a moderate learning advantage for American students, as gaming the system appears to be more strongly associated with poor learning than off-task behavior (Baker, Corbett, Koedinger, & Wagner, 2004; Cocea, Hershkovitz, & Baker, 2009). In order to confirm (or disconfirm) this interpretation, it will be necessary to analyze whether gaming the system correlates to learning in the same fashion (and under the same conditions) in the Philippines as it does in the United States. Analyzing gaming behavior at this grain-size is facilitated through the development of automated detectors of gaming the system, which can infer the exact moments when gaming behavior occurs in student log files (Baker, Corbett, & Koedinger, 2004; Baker & de Carvalho, 2008; Cocea et al., 2009). Hence, it would be necessary to develop and validate detectors of this behavior to study this interpretation in full depth. It is also worth looking at how gaming changes among students in the Philippines over time. Gaming the system has been shown to increase in the United States as students become more familiar with a specific type of educational software (see Beck, 2005). If this is the case in the Philippines as well, then gaming the system may turn out to be a significant problem for the use of educational software in the Philippines.


Over recent decades, there have been several calls to look at classrooms in East and Southeast Asia as inspirations for behavior and classroom management in the United States and Europe (Abiko & George, 1986; Bennett, 1987; Chase & Mueller, 1989; Winter, 1991; Weisz et al., 1995). Though the research presented here carefully controls for curricular confounds, multiple possible explanations still remain for the differences in off-task behavior between the United States and the Philippines found in this paper. As proposed by Abiko & George (1986), Winter, (1991), and Weisz et al. (1995), these findings may reflect genuine differences in engagement stemming from cultural differences. However, further research is needed to determine whether these findings are instead due to differences in observer effects, or to disengagement manifesting differently (for instance, manifesting as gaming the system rather than off-task behavior)—though these differences are themselves likely to be cultural in nature. One way to reduce observer effects would be to develop and validate automated detectors of off-task behavior and gaming the system in log files (see Baker, 2007; Baker & de Carvalho, 2008), as discussed earlier in this section. These detectors could then be used to confirm (or disconfirm) the findings seen here. Furthermore, these detectors could be applied to data from larger numbers of students in the Philippines to study the exact situations in which gaming occurs, towards understanding if the two behaviors occur in the same situations in the Philippines (in the United States, the two behaviors occur in different situations—Baker, Corbett, & Koedinger, 2004; Baker, 2007). As such, further work is needed to understand the causes and implications of international differences in the prevalence of off-task behavior.


In the long-term, a better understanding of how disengagement manifests itself in the United States, the Philippines, and other countries as well – and how specific cultural factors drive the manifestation of disengagement – may support the design of better education worldwide. Different countries appear to produce students who disengage in different amounts and in different ways, and these differences are apparently not due to differences in curricular design (at least in the case of off-task behavior in the United States and Philippines). Understanding these differences will be key for developing culturally appropriate practices that reduce disengagement and improve learning in every country.

 

Acknowledgements


This research was supported by the Philippines’ Department of Science and Technology, the Ateneo de Manila University, and by the Pittsburgh Science of Learning Center (National Science Foundation) via grant “Toward a Decade of PSLC Research,” award number SBE-0836012.  We thank our observers and research assistants Jenilyn Agapito, Maria Beatriz Espejo-Lahoz, Sheryl Ann Lim, Julieta Nabos, Sheila Pascua, Ivan Jacob Pesigan, Salvador Reyes, Jr., Ramon Rodriguez, and Jessica Sugay. We thank Mrs. Carmela Oracion and the Ateneo Center for Educational Development, the Department of Information Systems and Computer Science of the Ateneo de Manila University and the administrators, teachers, staff, and students of our participating schools for their support in this project. We thank Jaclyn Ocumpaugh, Lyn Corno, and the anonymous reviewers for helpful comments and suggestions, which helped us more clearly articulate this article’s research findings and theoretical implications.


References


Abiko, T., & George, P. S. (1986). Education for early adolescents in Japan, U.S.: Cross cultural observation. NASSP Bulletin: Official Journal of the National Association of Secondary School Principals, 70, 74-81.


Abramowitz, A. J., O’Leary, S. G., & Rosen, L. A. (1987). Reducing off-task behavior in the classroom: A comparison of encouragement and reprimands. Journal of Abnormal Child Psychology, 15 (2), 153-163.


Aleven, V., McLaren, B., Roll, I., & Koedinger, K. (2006). Toward meta-cognitive tutoring: A model of help seeking with a Cognitive Tutor. International Journal of Artificial Intelligence and Education, 16, 101-128.


Allday, R. A., & Pakruar, K. (2007). Effects of Teacher Greetings on Student On-Task Behavior. Journal of Applied Behavior Analysis, 40(2), 317-320.


Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. The Journal of the Learning Sciences, 4(2) 167-207.


Baker, R. S. (2007). Modeling and Understanding Students’ Off-Task Behavior in Intelligent Tutoring Systems. Proceedings of ACM CHI 2007: Computer-Human Interaction, 1059-1068.


Baker, R. S., Corbett, A. T., & Koedinger, K. R. (2004). Detecting Student Misuse of Intelligent Tutoring Systems. Proceedings of the 7th International Conference on Intelligent Tutoring Systems, 531-540.


Baker, R. S., Corbett, A. T., Koedinger, K. R., & Wagner, A.Z. (2004). Off-Task Behavior in the Cognitive Tutor Classroom: When Students “Game the System”. Proceedings of ACM CHI 2004: Computer-Human Interaction, 383-390.


Baker, R. S., D'Mello, S. K., Rodrigo, M. M. T., & Graesser, A. C. (2010). Better to Be Frustrated than Bored: The Incidence, Persistence, and Impact of Learners' Cognitive-Affective States during Interactions with Three Different Computer-Based Learning Environments. International Journal of Human-Computer Studies, 68 (4), 223-241.


Baker, R. S., & de Carvalho, A.M.J.B. (2008). Labeling Student Behavior Faster and More Precisely with Text Replays. Proceedings of the 1st International Conference on Educational Data Mining, 38-47.


Baker, R.S., de Carvalho, A. M. J. A., Raspat, J., Aleven, V., Corbett, A. T., & Koedinger, K. R. (2009). Educational Software Features that Encourage and Discourage "Gaming the System." Proceedings of the 14th International Conference on Artificial Intelligence in Education, 475-482.


Baker, R. S. J. d., & Gowda, S.M. (2010). An Analysis of the Differences in the Frequency of Students' Disengagement in Urban, Rural, and Suburban High Schools. Proceedings of the 3rd International Conference on Educational Data Mining, 11-20.


Baker, R. S. J. d., Moore, G., Wagner, A., Kalka, J., Karabinos, M., Ashe, C., & Yaron, D. (2011). The Dynamics Between Student Affect and Behavior Occuring Outside of Educational Software. Proceedings of the 4th bi-annual International Conference on Affective Computing and Intelligent Interaction.


Baker, R., Walonoski, J., Heffernan, N., Roll, I., Corbett, A., & Koedinger, K. (2008) Why Students Engage in "Gaming the System" Behavior in Interactive Learning Environments. Journal of Interactive Learning Research, 19(2), 185-224.


Beck, J. (2005). Engagement tracing: using response times to model student disengagement. Proceedings of the 12th International Conference on Artificial Intelligence in Education (AIED 2005), 88-95.


Bennett, W. J. (1987). Looking at Japanese Education: Implications for American Education. NASSP Bulletin: Official Journal of the National Association of Secondary School Principals, 71, 102-108.


Berliner, D. C. (1990). What’s All the Fuss About Instructional Time? The Nature of Time in Schools: Theoretical Concepts, Practitioner Perceptions. New York: College Press.


Bernardo, A. B. I. (2008). Individual and social dimensions of Filipino students’ achievement goals. International Journal of Psychology, 43(5), 886-891


Blanchard, E., Ogan, A. (2010). Infusing Cultural Awareness in Intelligent Tutoring Systems for a Globalized World. In R. Nkambou, R. Mizoguchi, & J. Bordeau (Eds.), Advances in Intelligent Tutoring Systems, pp. 485-506. Berlin: Springer-Verlag.


Blatchford, P., Bassett, P., & Brown, P. (2005). Teachers’ and Pupils’ Behavior in Large and Small Classes: A Systematic Observation Study of Pupils Aged 10 and 11 Years. Journal of Educational Psychology, 97(3), 454-467.


Bloom, B. S. (1976) Human characteristics & school learning. New York: McGraw-Hill.


Bozack, A. R., Vega, R., McCaslin, M., & Good, T. L., (2008). Teacher support of student autonomy in comprehensive school reform classrooms. Teachers College Record, 110(11), 2389-2407.


Brooks, A., Horner, R. H., Todd, A. W., & Tofflemoyer, S. (2003). Use of Functional Assessment and a Self-Management System to Increase Academic Engagement and Work Completion. Journal of Positive Behavior Interventions, 5, 144-152.


Caldwell, J. H., Huitt, W. G., & Graeber, A. O. (1982). Time spent in learning: Implications from research. The Elementary School Journal, 82, 470-480.


Carroll, J. A. (1963) A Model For School Learning. Teachers College Record, 64, 723-733.


Cetintas, S., Si, L., Xin, Y.P., & Hord, C. (2009). Automatic Detection of Off-Task Behaviors in Intelligent Tutoring Systems with Machine Learning Techniques. IEEE Transactions on Learning Technologies, 3(3), 228-236.


Chen, X., Hastings, P., Rubin, K.H., Chen, H., Cen, G., & Stewart, S. L. (1998). Childrearing attitudes and behavioral inhibition in Chinese and Canadian toddlers: a cross-cultural study. Developmental Psychology, 34, 677–686.


Chase, C. I., Mueller, D. J. (1989). Off task behavior: An American loss to the Japanese. Journal of Human Behavior and Learning, 6(1), 19-21.


Clare, S. K., Jenson, W. R., Kehle, T. J., & Bray, M. A. (2000). Self-modeling as a treatment for increasing on-task behavior. Psychology in the Schools, 37(6), 517-522.


Cobb, J. A. (1972). Relationship of discrete classroom behavior to fourth-grade academic achievement. Journal of Educational Psychology, 63, 74-80.


Cocea, M., Hershkovitz, A., & Baker, R. S. J. d. (2009). The Impact of Off-task and Gaming Behaviors on Learning: Immediate or Aggregate? Proceedings of the 14th International Conference on Artificial Intelligence in Education, 507-514.


Corbett, A. T., & Anderson, J. R. (1995). Knowledge Tracing: Modeling the Acquisition of Procedural Knowledge. User Modeling and User-Adapted Interaction, 4, 253-278.


Currie, J. (1884). The principles and practice of common school education. Cincinnati, OH: R. Clarke.


Dalton, T., Martella, & R. C., Marchand-Martella, N.E. (1999). The effects of a self-management program in reducing off-task behavior. Journal of behavioral education, 9(3-4), 157-176.


Eisenberg, N., Fabes, R. A., & Spinrad, T. L. (2006). Prosocial development. In W. Damon (Ed.), Handbook of child psychology, volume 3: Social, emotional, and personality development. 5th edition. New York, NY: Wiley.


Farver, J. M., Kim, Y. K., & Lee, Y (1995). Cultural differences in Korean- and Anglo-American preschoolers’ social interaction and play behaviors. Child Development, 66, 1088–1099


Fisher, C. W., Berliner, D. C., Fully, N. N., Marliave, R. S., Cahen, L. S., & Dishaw, M. M. (1980). Teaching behaviors, academic learning time, and student achievement: An overview. In C. Denham & A. Lieberman (Eds.), Time to learn, 7-32. Washington, DC: National Institute of Education.


Fredrick, W. C., & Walberg, H. J. (1980). Learning as a function of time. Journal of Educational Research, 73, 183-194.


Fredrick, W. C., Walberg, H. J., & Rasher, S.P. (1979). Time, Teacher Comments, and Achievement in Urban High Schools. Journal of Educational Research, 73(2), 63-65.


Ginsburg, A., Leinwand, S., Anstrom, T., & Pollock, E. (2005). What the United States Can Learn From Singapore's World-Class Mathematics System (and What Singapore Can Learn from the United States): An Exploratory Study. Washington, DC: The American Institutes for Research.


Gobel, P. (2008). Student Off-task Behavior and Motivation in the CALL Classroom. International Journal of Pedagogies and Learning, 4(4), 4-18.


Goodman, L. (1990). Time and learning in the special education classroom. Albany, NY: SUNY Press.


Grimm, S. D., Church, A. T., Katigbak, M. S., & Reyes, J. A. S. (1999). Self-described traits, values and moods associated with individualism and collectivism: Testing I-C theory in an individualistic (US) and collectivistic (Philippine) culture. Jounal of Cross-Cultural Psychology, 30(4), 466-500.


Helmke, A., & Vo, T. A. T. (1999). Do Asian and Western students learn in a different way? An empirical study on motivation, study time, and learning strategies of German and Vietnamese university students. Asia Pacific Journal of Education, 19, 30-44.


Hofstede, G., Hofstede, G. J., & Minkov, M. (2010). Cultures and Organizations: Software for the Mind (3rd ed.). Columbus, OH: McGraw-Hill.


Inglehart, R. (2009). Inglehart-Welzel Cultural Map of the World. Accessed 17 February 2009 from the World Values Survey website:  http://margaux.grandvinum.se/SebTest/wvs/articles/folder_published/article_base_54


Karweit, N. L., & Slavin, R. E. (1981). Measurement and modeling choices in studies of time and learning. American Educational Research Journal, 18, 157-171.


Karweit, N., & Slavin, R. E. (1982). Time-On-Task: Issues of Timing, Sampling, and Definition. Journal of Experimental Psychology, 74(6), 844-851.


Kelly, S., & Turner, J. (2009). Rethinking the Effects of Classroom Activity Structure on the Engagement of Low-Achieving Students. Teachers College Record, 111(7), 1665-1692.


Kerr, M. M., & Nelson, C. M. (1998). Strategies for managing problem behaviors in the classroom (2nd ed.). Columbus, OH: Merrill.


Koedinger, K., & Corbett, A. (2006). Cognitive tutors: Technology bringing learning sciences to the classroom. In K. Sawyer (Ed.), Cambridge handbook of the learning sciences, 61-78. New York, NY: Cambridge University Press.


Kreijns, K. (2004). Sociable CSCL environments. Social affordances, sociability, and social presence. Unpublished Master’s Thesis. Open University of the Netherlands. Heerlen, The Netherlands.


Lahaderne, H. M. (1968). Attitudinal and Intellectual Correlates of Attention: A Study of Four Sixth-Grade Classrooms. Journal of Educational Psychology, 59(5), 320-324.  


Lee, S. W., Kelly, K. E., & Nyre, J. E. (1999) Preliminary Report on the Relation of Students’ Off-Task Behavior with Completion of School Work. Psychological Reports, 84, 267-272.


Lloyd, J. W., & Loper, A. B. (1986). Measurement and Evaluation of Task-Related Learning Behaviors: Attention to Task and Metacognition. School Psychology Review, 15(3), 336-345.


Luckin, R., & du Boulay, B. (1999). Ecolab: The development and evaluation of a Vygotskian design framework. International Journal of Artificial Intelligence in Education, 8, 30-43.


Lynch, F. (1984). Philippine Society and the Individual: Selected Essays of Frank Lynch, 1949-1976. Ann Arbor, MI: University of Michigan.


McKinney, J. D., Mason, J., Perkerson, K., & Clifford, M. (1975). Relationship between classroom behavior and academic achievement. Journal of Educational Psychology, 67, 198-203.


Moore, L. W., Sweeney, J. J., & Butterfield, P. H. (1993). Differential effects of primary, activity, and computer reinforcement on decreasing the off-task behavior of three special education populations. Journal of Instructional Psychology, 20(2), 132-144.


Nicaud J. F., Bittar, M., Chaachoua H., Inamdar P., & Maffei L. (2006). Experiments With Aplusix In Four Countries. International Journal for Technology in Mathematics Education, 13(1).


Nicaud, J. F., Bouhineau, D., & Chaachoua, H. (2004). Mixing microworld and CAS features in building computer systems that help student learn algebra. International Journal of Computers for Mathematical Learning, 9(2), 169-211.


Nicaud, J. F., Bouhineau, D., Mezerette, S., & Andre, N. (2007). Aplusix II [Computer software].


Ogan, A., Walker, E., Baker, R. S. J. d., de Carvalho, A., Laurentino, T., Rebolledo-Mendez, G., & Castro, M. J. (2012). Collaboration in Cognitive Tutor Use in Latin America: Field Study and Design Recommendations. Proceedings of ACM SIGCHI: Computer-Human Interaction, 1381-1390.


Philippines National Statistics Office. (1994). Proportion of households by ownership of radio, tv, vcr, and personal computer by region, urban-rural. Retrieved on 16 September 2011 from the NSO website: http://www.census.gov.ph/data/sectordata/1994/fl9401mm.txt


Philippines National Statistics Office. (2006). Functional literacy rate of population 10-64 years old, by sex, age group and region Philippines: 2003. Retrieved on 16 September 2011 from the NSO website: http://www.census.gov.ph/data/sectordata/2003/fl03tabE.htm


Public Agenda (2004). Teaching Interrupted: Do Discipline Policies in Today’s Public Schools Foster the Common Good? Report. Retrieved Online on August 6, 2007 from http://www.publicagenda.org/research/pdfs/teaching_interrupted_exec_summary.pdf   


Rebolledo-Mendez, G., du Boulay, B., & Luckin, R. (2006). Motivating the learner: an empirical evaluation. 8th International Conference on Intelligent Tutoring Systems, 545-554.


Reid, R., & Harris, K. R. (1993). Self-monitoring of attention versus self-monitoring of performance: effects on attention and academic performance. Exceptional Children, 60(1), 29-40.


Rodrigo, M. M. T. (2004). Quantifying the divide: A comparison of ICT usage in schools in Metro Manila and IEA-surveyed countries. International Journal of Educational Development, 25(1), 53-68.


Rodrigo, M. M. T., Baker, R. S. J. d., Lagud, M. C. V., Lim, S. A. L., Macapanpan, A. F., Pascua, S. A. M. S., Santillano, J. Q., Sevilla, L. R. S., Sugay, J. O., Tep, S., & Viehland, N. J. B. (2007). Affect and usage choices in simulation problem-solving environments. In R. Luckin, K. R. Koedinger, & J. Greer (Eds.), Artificial Intelligence in Education: Building Technology Rich Learning Contexts that Work, 145-152. Amsterdam, The Netherlands: IOS Press.


Rodrigo, M. M. T., Rebolledo-Mendez, G., Baker, R. S. J. d., du Boulay, B., Sugay, J. O., Lim, S. A. L., Lahoz, M. B. E., Luckin, R. (2008). The effects of motivational modeling on affect in an intelligent tutoring system. In Chan, T.-W., Biswas, G., Chen, F.-C., Chen, S., Chou, C., Jacobson, M., Kinshuk, Klett, F., Looi, C.-K., Mitrovic, T., Mizoguchi, R., Nakabayashi, K., Reimann, P., Suthers, D., Yang, S., & Yang, J.-C. (Eds.). International Conference on Computers in Education, 49-56.


Ross, S.M., McDonald, A.J., Alberg, M., & McSparrin-Gallagher, B. (2007). Achievement and Climate Outcomes for the Knowledge is Power Program in an Inner-City Middle School. Journal of Education for Students Placed at Risk, 12(2), 137-165.


Rossmiller, R.A. (1986). Resource utilization in schools and classrooms: Final Report. Program Report 86-7. Madison, WI: University of Wisconsin, Wisconsin Center for Education Research.


Rowe, J., McQuiggan, S., Robison, J., & Lester, J. (2009). Off-Task Behavior in Narrative-Centered Learning Environments. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Education, 99-106.


Rubin, K. H., Burgess, K. B., & Coplan, R. J. (2002). Social withdrawal and shyness. In P. K. Smith & C. H. Hart (Eds.), Blackwell Hand-book of Childhood Social Development, pp. 330–52. Malden, MA: Blackwell Sci.


Schofield, J. W. (1995). Computers and Classroom Culture. Cambridge, UK: Cambridge University Press.


Smetana J. (2002). Culture, autonomy, and personal jurisdiction. In R. Kail & H. Reese (Eds.), Advances in Child Development and Behavior, New York, NY: Academic Press.


Stevenson, H. W., Chen, C., & Lee, S. Y. (1993). Mathematics achievement of Chinese, Japanese, and American children: Ten years later. Science, 259, 53-58.  


Stevenson, H. W., & Lee, S. Y. (1990). Contexts of achievement. Monographs of the Society for Research in Child Development, 55, 1-119.


Stigler, J. W., Gonzales, P., Kawanaka, T., Knoll, S., & Serrano, A. (1997). The TIMSS videotape classroom study: Methods and findings from an exploratory research project on eighth-grade mathematics instruction in Germany, Japan, and the United States. Washington, DC: U.S. Department of Education, National Center for Education Statistics.


Suh, E., Diener, E., Oishi, S., & Triandis, H. C. (1998). The shifting basis of life satisfaction judgments across cultures: emotions versus norms. Journal of Personality and Social Psychology, 74, 482-493.


Tweed, R. G., & Lehman, D. R. (2002). Learning considered within a cultural context: Confucian and Socratic approaches. American Psychologist57, 89–99


Walker , H. M., Horner, R. H., Sugai, G., Bullis, M., Sprague, J. R., Bricker, D., & Kaufman, M. J. (1996). Integrated approaches to preventing antisocial behavior patterns among school-age children and youth. Journal of Emotional and Behavioral Disorders, 4, 194-209.


Weisz, J. R., Chaiyasit, W., Weiss, B., Eastman, K. L., & Jackson, E. W. (1995) A multimethod study of problem behavior among Thai and American children in school: teacher reports versus direct observation. Child Development, 66(2), 402-415.


Welzel, C. (2006). A human development view on value change trends [PowerPoint Presentation]. Accessed 17 February 2009 from the World Values Survey website: http://margaux.grandvinum.se/SebTest/wvs/articles/folder_published/article_base_83


Winter, S. (1991) Are Behavioural Classroom Management Approaches Appropriate to a Non-Western Educational System. School Psychology International, 12, 211-223.




Cite This Article as: Teachers College Record Volume 115 Number 10, 2013, p. 1-27
https://www.tcrecord.org ID Number: 17140, Date Accessed: 5/21/2022 9:16:16 AM

Purchase Reprint Rights for this article or review
 
Article Tools
Related Articles

Related Discussion
 
Post a Comment | Read All

About the Author
  • Ma. Mercedes Rodrigo
    Ateneo de Manila University
    E-mail Author
    MA. MERCEDES T. RODRIGO is a Professor at the Department of Information Systems and Computer Science and the head of the Ateneo Laboratory for the Learning Sciences. She received her PhD in Computer Technology in Education from Nova Southeastern University in 2002, her MS in Applied Computer Science from the University of Maryland Eastern Shore in 1992, and her BS in Computer Science (honorable mention) from the Ateneo de Manila University in 1988. From 2003-2008 was the Chair of the Department of Information Systems and Computer Science. In 2008, Dr. Rodrigo was a Fulbright Senior Research Fellow at the Pittsburgh Science of Learning Center. Dr. Rodrigo’s areas of interest are affective computing and artificial intelligence in Education. She has published in IEEE Transactions on Affective Computing, the International Journal of Human-Computer Studies, and the Journal of Computer Science Education.
  • Ryan Baker
    Teachers College
    E-mail Author
    RYAN S. J. D. BAKER is Visiting Associate Professor in the Department of Human Development and the Department of Mathematics, Science, and Technology at Teachers College. He received his PhD in Human-Computer Interaction from Carnegie Mellon University in 2005, and his ScB in Computer Science (senior prize) from Brown University in 2000. Prior to joining Columbia University, he was Assistant Professor at Worcester Polytechnic Institute, Technical Director of the Pittsburgh Science of Learning Center DataShop, and Research Fellow at the University of Nottingham. He is currently the President of the International Educational Data Mining Society and Associate Editor of the Journal of Educational Data Mining. His work on educational data mining has received awards at five different conferences, and over 1,500 citations.
  • Lisa Rossi
    Worcester Polytechnic Institute
    E-mail Author
    LISA ROSSI is Research Analyst in the Department of Social Science and Policy Studies at Worcester Polytechnic Institute. She received her B.S. in Psychological Science from Worcester Polytechnic Institute in 2011.
 
Member Center
In Print
This Month's Issue

Submit
EMAIL

Twitter

RSS