Mapping the Inequity Implications of Help-Seeking in High School Online Credit-Recovery Classrooms
by Jennifer Darling-Aduana, Annalee G. Good & Carolyn Heinrich - 2019
Background: Expectations that students should request assistance from teachers when needed, a set of classroom behaviors termed “help-seeking,” have the potential to contribute to inequitable access to quality learning experiences in traditional classroom settings.
Purpose: This study extends current literature by mapping the nature of help-seeking interactions between students and teachers in online high school credit-recovery classrooms, where the implications of help-seeking have yet to be examined systemically.
Research Design: Drawing on qualitative and quantitative analysis of data collected from the 2014–2015 through 2016–2017 school years in a large, urban school district serving predominantly low-income student of color, we identify patterns in these interactions and their implications for disparities in academic opportunities.
Findings: We find that few of the high school students enrolled in online credit-recovery courses had access to consistent, constructive interactions in instructional spaces, even though most students required instructor support to obtain full access to the learning environment. Our observations point to disparate access to quality educational experiences in online credit-recovery labs that mirror those documented by others in traditional classroom settings.
Conclusions: Based on these findings, we identify strategies to support more equitable learning in online courses including explicit expectations and proactive assistance to students as well as the use of real-time data by teachers, lower student-teacher ratios, and assigning teachers certified in course subjects to improve educational quality.
Student-teacher interactions help define student learning opportunities in the classroom, and these interactions are shaped in part by the social identities and cultural contexts of both students and teachers (Downey & Pribesh, 2004; Lareau & Weininger, 2003; Rist, 1970). Over the last two decades, online learning has fundamentally changed the instructional space students experience in school, and therefore, the nature of these critical interactions. With over 75 percent of U.S. school districts serving one or more students enrolled in an online course (Gemin, Pape, Vashaw, & Watson, 2015), it is critical to consider how the growth of online courses changes student interactions in classrooms and how these interactions might reflect, exacerbate, or mitigate persistent racial and socioeconomic inequities in schools. The purpose of this study is to examine interactions between high school students and teachers in online credit recovery labs within a large, urban, Midwestern school district, with an emphasis on "help-seeking", a process by which student attempt to gain assistance from teachers.
Existing research on digital learning prompts questions about the nature of online classroom interactions. For example, despite trends of technology-based instruction replacing several of the central tasks traditionally assigned to teachers, some research suggests that online learning programs that incorporate live instructors contribute to better student outcomes (Hannum, Irvin, Lei, & Farmer, 2008; Means, Toyama, Murphy, & Baki, 2013; Taylor et al., 2016; Zhao, Lei, Yan, Lai, & Tan, 2005). This research indicates an important, but not yet fully understood role, for interactions in mediating student access to learning in online spaces. The stakes for students enrolled in high school online credit recovery courses are particularly high. With rapidly expanding online recovery programs and high school completion a prerequisite for most employment (Torpey & Watson, 2014), broadening understanding of the role of interactions in mediating student access to quality educational experiences is essential to inform practice and policies in these settings.
Providing students with opportunities to earn previously incomplete course credits required for high school graduation, credit recovery is one of the most common uses of online platforms (Clements, Stafford, Pazzaglia, & Jacobs 2015; Queen & Lewis 2011). In most instances, vendors develop the curriculum for their software (Patrick, Kennedy, & Powell, 2013), raising potential concerns about adaptability and relevance, particularly for students who have not yet mastered grade-level content, fully developed self-regulated learning strategies, or struggled previously with engagement. With students from low socioeconomic status backgrounds more likely to be at risk of dropping out than students in the general population (Rumberger, 2004), the extent to which online credit recovery programs provide access to quality educational experiences for students is pertinent to understanding the creation and maintenance of gaps in academic opportunity, achievement and attainment. Other concerns have been raised that with little oversight, education standards in online credit recovery programs may be lowered, and students at risk for failure may be directed into online learning as a means of cost-savings, potentially further exacerbating unequal access to quality learning opportunities (Gardiner, 2014; Thevenot & Butrymowicz, 2010).
We examined the following research questions:
What are the potential spaces for student-teacher interactions in online high school credit-recovery labs?
What specific patterns do we observe in help-seeking interactions between students and teachers in the same online credit recovery program?
In our examination, we draw on interpretive social reproduction scholarship (Mehan, 1992; Lareau & Weininger, 2003; Rist, 1970), which argues that social institutions (e.g., schools, government, religious communities) reproduce societal structures and power dynamics through persistent cultural norms that ultimately advantage dominant groups (Bourdieu, 1986/2010; Lareau, 2003; Van den Bergh, Denessen, Hornstra, Voeten, & Holland, 2010). We then explore the implications of our findings, with the aim to motivate and guide future empirical research on inequity in online credit-recovery courses. We extend the current literature by mapping interactions between students and teachers in online learning to spaces associated with disparities in academic opportunities and attainment in traditional classroom settings (Calarco, 2011; Downey & Pribesh, 2004; Heath, 1982; Ladson-Billings, 2004; McLaren, 1994; Rist, 1970).1 As an increasing number of students receive instruction online, we identify strategies for school districts and teachers to address inequities in these new learning environments, particularly given that these shifts disproportionately affect students already underserved by current educational systems.
HELP-SEEKING AS A CONCEPTUAL FRAMEWORK
Prior research indicates that student-teacher interactions matter, and they matter differently based on social identities (Calarco, 2011; Downey & Pribesh, 2004; Heath, 1982; Ladson-Billings, 2004; McLaren, 1994; Rist, 1970). The following literature identifies the extent to which a form of teacher-student interactionshelp-seekingmay be influenced by cultural signals. Help-seeking is a self-regulatory skill typified by assertiveness and comfort with requesting accommodations and assistance from individuals in a positive of authority (Calarco, 2011; Zimmerman, 2008). Researchers have identified class-based differences in help-seeking, with middle class parents more likely to employ parenting strategies that encourage the development of associated skill, behaviors, and ways of interacting within middle class institutions (Calarco, 2011; Lareau, 2003; Streib, 2011). Addressing concerns raised by others (i.e., Peck et al., 2015) that the structure and limited personalization and human interaction in online credit-recovery courses may only serve to reproduce current class categorizations, our study draws on interpretive social reproduction scholarship (Mehan, 1992; Lareau & Weininger, 2003; Rist, 1970) to guide an examination of the nature of student-teacher help-seeking interactions in online spaces. This study furthers understanding into how these online course-based interactions may advantage (or disadvantage) students from various backgrounds.
Social reproduction theory suggests that as social institutions, schools reproduce societal structures through embedded cultural norms (e.g., teacher expectations of behavior and definitions of achievement), which in turn advantage students from dominant groups (Bourdieu, 1986/2010; Lareau, 2003; Van den Bergh, Denessen, Hornstra, Voeten, & Holland, 2010). The result is social reproduction disguised and legitimized through the appearance of school success as a factor of individual, versus class-specific, characteristics (Bourdieu, 1986/2010; Lareau & Calarco, 2012). As social actors in social institutions that tend to identify with middle-class norms, teachers spend more time with and react more positively to the interactional styles and language patterns exhibited by middle-class students (Bernstein, 1975; Calarco, 2011; Streib, 2011). For instance, many teachers equate student engagement and intelligence with active participation in classroom activities, verbal assertiveness, and help-seeking behaviors, all which can be differentially expressed based on socioeconomic status, racial identity, and cultural background (Calarco, 2011; Heath, 1982; Lareau, 2003). Although associated with student background, students' "cultural toolkit" of attitudes, behaviors, and preferences are perceived by others, including teachers, as individual skills, talents, or capacities (Bourdieu, 1986/2010). For instance, teachers prioritize students who ask more questions because teachers view them as more interested in learning, even though research demonstrates that help-seeking reflects class-background more than interest in learning (Calarco, 2011, 2014).
Help-seeking is a rehearsal learning strategy, whereby students must correctly evaluate their need for help and actively communicate with an individual capable of assisting (Newman, 2000; Zimmerman, 2008). Effective help-seeking thus requires many interpersonal attributes: communication skills, assertiveness, and the ability to identify when and from whom to request assistance (Calarco, 2011, 2014). As a result, help-seeking often supports students in staying on-task, increases the speed of learning, and builds confidence (Calarco, 2011; Newman, 2000). Beyond cultural norms, the decision to ask for help is also informed by perceptions of trust, relational style, and expertise, with students more likely to ask for assistance if they believe an instructor will treat them with respect and communicate pertinent information effectively (Brion-Meisels, 2015, 2016).
The increased prevalence of online courses in the United States has the potential to disrupt current, systematically biased interactional norms and expectations by modifying how students and teachers interact in an instructional setting. This disruption may take several forms with different implications for inequity. Online courses fundamentally reframe the role of the teacher, as teachers are no longer primarily responsible for content delivery. Larger class sizes and fewer, more constrained student-teacher interactions may lead teachers to rely more often on unconscious cultural cues when interpreting student behaviors and actions (Altonji & Pierret, 2001; Robinson & Lubienski, 2011). Accordingly, student help-seeking may play an increased role in shaping how teachers interact with, assist, and evaluate students in an online classroom setting (Ahn, 2011).
An important implication of interactions within computer-based instruction are changes in the quantity and quality of information about students available to educators. Often, teachers have less access to information through in-person interactions in online classroom settings but increased access to information on student progress and assessment results. When lacking information, evaluators often unconsciously employ statistical discrimination, relying on average characteristics of others belonging to a similar socio-demographic group (Altonji & Pierret, 2001; Ewens, Tomlin, & Wang, 2014). Statistical discrimination reinforces the existing status-quo and disadvantages historically lower-achieving subgroups. However, the effect fades as evaluators gain access to information about an individual, at which time evaluators substitute knowledge about the individual for average group characteristics (Altonji & Pierret, 2001; Devine, Forscher, Austin, & Cox, 2012). Thus, there is reason to believe elements of online courses may equalize the quantity and quality of student-teacher interactions. The availability of real-time data and frequent assessments typical of online courses may reduce reliance on incomplete or inaccurate information based on cultural signals (Altonji & Pierret, 2001; Devine et al., 2012). Specific to this study, we are interested in whether access to presumably more objective information on students might result in teachers relying on social and cultural cues less often when initiating or responding to help-seeking requests.
Further, while teachers expect students to be proactive and seek out help when struggling, this expectation is predominantly expressed implicitly (Ahn, 2011; Calarco, 2011; Patrick, Anderman, Ryan, Edelin, & Midgley, 2001). Delpit (2006) established that when these implicit rules are made explicit, all students are better able to succeed (i.e., Luykx et al., 2007; Parks, 2010). In this regard, the standardized structure and rules of labs supporting online instruction may offer many advantages by making the implicit expectations in traditional classrooms more explicit. Instead of focusing predominantly on teaching, teachers can, and do, prioritize maintaining student motivation in online classrooms (Ahn, 2011). More explicit expectations about weekly course progress and what type of learning is required may also be communicated more efficiently due to standardized course structures and requirements. Even if an instructor does not communicate expectations more explicitly in an online classroom, students may be more insulated from changing teacher expectations or subtle interpersonal cues (Arnot & Reay, 2007; Calarco, 2014) when a third-party provides instructional delivery and grading services.
Lastly, the use of standardized, asynchronously delivered course content may allow instructors to focus on encouraging the development of learning and study skills, such as note-taking. Decreased time devoted to direct instruction and standardized course structures by teachers may also facilitate the more explicit communication of expectations, eliminating cultural insider knowledge on how best to learn and earn course credit (Bernstein, 1975; Delpit, 2006). What help-seeking looks like may also vary in online versus traditional classroom settings, as students enrolled in online courses have ready access to additional non-instructor-based resources for assistance, such as the educational program delivering content and Internet resources. By presenting a descriptive analysis of patterns in student help-seeking and subsequent student-teacher interactions, we explore which and in what contexts these hypothesized disruptions to classroom interactional norms and expectations appeared across credit recovery labs in a large, urban district.
In this paper, we apply frameworks from social reproduction theory and prior research on the role of help-seeking in traditional school settings to the new digital, online context, which has yet to be fully explored, and indeed, may be more challenging to observe. We examine how changing student-teacher interactional norms, expectations for both students and teachers, and access to information may redefine previous spaces of inequity and highlight possible levers for improving student access to quality education with digital tools.
Situated within a multi-year, mixed methods study on the implementation and outcomes associated with digital tools in K12 classrooms in the United States, this paper draws on 156 qualitative observations of instructional sessions and 24 interviews. Data were collected across three years and 18 schools implementing an online credit recovery program in a large, urban school district in the Midwest. We collected 17 observations across two high schools during the 20142015 school year, 31 observations across seven high schools during the 20152016 school year, and 108 observations across 18 high schools during the 20162017 school year. Across the study years, high schools in the district, and particularly those offering credit recovery options, serve a predominately black, low income student population, as seen in Table 1. Around one-fourth of all high school students in the district accessed one or more courses online, with the students enrolled in online courses slightly more likely to be identified as African American and from low-income backgrounds.
Table 1. Student Characteristics among District High School Students and Credit Recovery Students in High Schools (20142017)
MAP scores reported in 20142015 and STAR scores in 20152016 and 20162017.
The well-tested, research-based observation instrument (Burch, Good, & Heinrich, 2016) enabled observers to evaluate the extent to which an instructional session (and integration of educational technology) facilitated quality learning opportunities for students.2 The observation instrument contains a set of indicators or dimensions of quality elements that capture the type of interactions occurring between teachers, students, and educational technology. We recorded ratings of 10 core elements of digital and blended instruction (described in Appendix A) on a 5-point Likert-type scale (04).3 Observers also recorded narrative comments and vignettes, as well as information on the total instructional time, time on task, time a student interacted with an instructor, and whether the format facilitated live interaction between instructors and students around instructional tasks. Although we documented descriptors of students and teachers within the observation instrument, including estimations of gender, race, and ethnicity, we do not report these categories in our analysis of qualitative data, as these were based on researcher judgments versus self-identification. We also facilitated regular training to establish interrater consistency for all raters conducting classroom observations.
We collected interview data using a semi-structured interview protocol containing interview topics, probes, and sample questions. Interview topics included instructor background, instructional practices, support for the use of digital tools, digital tool access, use by student subgroups, assessment of the effectiveness of digital tools in the classroom, and plans for ongoing use of digital tools. (Refer to Appendix B for the full interview protocol). Instructor responses provided insights on program goals, implementation, and trends that complement the detailed, snap-shot information gathered through observations. During each classroom observation, we asked instructors if they would be willing to answer a few questions about their experiences. We sometimes conducted the interview on the same day as we observed the instructors classroom or lab. Other times we exchanged contact information and scheduled an interview time during subsequent data collection visits or over the phone. We recorded and transcribed each formal interview. We also summarized informal conversations with lab instructors during observations with the instructors permission.
We analyzed qualitative data from these 156 observations and 24 interviews in NVivo coding software using pre-established thematic codes to organize passages around common themes (Gibbs, 2007). Sample thematic codes included the physical environment, curriculum, instructional model, interactions, assessment, engagement, digital citizenship, and digital tools. Refer to Appendix C for a full list of these codes, along with samples of coded data excerpts. We used spot-checking to check coding consistency. Focusing on the program model and staff parent nodes, we followed with an inductive coding process, reading and then assigning interpretive codes and thoughts to each excerpt. As we proceeded through the excerpts, we solidified labels and descriptions. Once themes emerged as prevalent (i.e., once we achieved saturation), we continued to check for these themes, but only marked illuminating examples or those that demonstrated variation or diversity within the identified theme. We also searched for alternative explanations to challenge preconceptions and personal biases.
We then drafted analytic memos focused on an emerging theme (e.g., access, instructor capacity, etc.), organizing the document around the interpretive codes. Throughout the drafting of analytic memos, we added sub-headers where the distinction appeared necessary and expounded on key points from detailed descriptions to pull out themes of interest. This phase also provided a means through which to identify more and less frequent occurrences. The original coding and analytic memo writing occurred based on observations and interviews from the 20142015 and 20152016 school years. We subsequently confirmed, revised, and expounded upon these themes based on interviews and observations collected during the 20162017 school year.
We used quantitative data to test and supplement emergent findings. For instance, after observing that many teachers did not possess sufficient background knowledge to assist students struggling with course content, we examined interaction ratings from the observation instrument in conjunction with information on teacher qualifications. We only conducted statistical analyses and reported quantitative findings when a we recorded a numerical rating across all observations or a common item on the interview protocol assured that we had information across all observations or interviews. Due to the ordinal nature of the scales measuring each dimension in the observation instrument, we used ANOVAs with chi-squared tests to identify significant differences between classrooms where we did and did not observe various forms of student help-seeking and teacher assistance. Where applicable, we reported the p values from statistical tests. We do not report the prevalence of analytic themes that emerged through qualitative coding of narrative vignettes to prevent the overinterpretation of these findings.
Triangulation across qualitative and quantitative data was used to confirm the validity and reliability of analytic themes. Prior to inclusion, we established that the opinions, experiences, and observations expressed by lab instructor in interviews were consistent with observation data collected by researchers. We also examined findings by school and year to confirm our findings reflected in a range of lab settings.
SUMMARY OF FINDINGS
In this section, we illustrate potential spaces for student-teacher interactions focused on help-seeking within online credit-recovery classrooms. Drawing primarily on rich, observational data, we highlight patterns, focusing on interactional strategies that may either reproduce or mitigate gaps in student access to quality educational opportunities.
INSTRUCTIONAL SPACES IN CREDIT RECOVERY LABS
In all credit recovery labs observed, students were provided a laptop or desktop computer and expected to progress through the online program independently. Course progression required watching video lectures, responding online via clicks and written responses, and taking notes. The labs were supervised by one or more teachers whose primary role was to ensure students were making progress in the course. These instructors also provided technical support, and on occasion, instructional support. Learning occurred primarily through student interactions with the online course platform, which housed and delivered the curricular content. Once students logged in, content relevance, cognitive demands, and feedback informed subsequent student engagement, self-regulation, and persistence. Students largely determined their pacing and could repeat sections with instructor permission. The software also offered lecture notes in multiple languages. Teachers described in-program accommodations as minimal; most involved teacher-initiated actions such as removing multiple-choice options in quizzes or resetting lessons so that students could attempt them again. Any further adaptation to students' needs, interests, or context had to be facilitated by a live instructor, with any differential access to instructors resulting in disparate access to equitable educational opportunities.
Below is a composite vignette that contains observation notes from several classrooms and is representative of our qualitative data. The vignette was created to illustrate the instructional setting and interactions of a typical computer lab reserved for students enrolled in online courses. The vignette also serves as a foil in discussions of variants of the instructional models and settings observed.
At the beginning of the first period, students straggle in and go directly to the desktops. There are 30 computers in the large basement classroom. All students sit at their desktop computers, working on various course modules that depend on where they need to recover credit. Twelve of the 15 students have headphones on and plugged into the computer. Students are talking quietly, occasionally laughing. Ten minutes into the class period, the teacher stands up and walks around to check on the students, at which point nine of 15 students are actively working in the online course system. The teacher emphasizes to the students that they need to strive for the goal of completing three percent of their coursework per week. He tells them to focus more and to take advantage of the resources they have both during and after the school day. The students are a distraction to each other, with some students walking around and disturbing others or talking out loud. There is no redirection of students on the part of the teacher. Toward the end of the period, five of the students are still actively clicking, looking up at the screen, typing, etc. Two of these students also have a paper notebook out. These students are engaged in an iterative process of reading content off the screen and then writing it down in their notebooks. One student is toggling between the online course program and Google to look up terms. At any one time, four to five students are checking their phones, and one or more are sleeping.
Typical of our observations, the instructor in the above vignette interacted with students in a predominately motivational versus instructional role. The extent to which instructors monitored student engagement varied across classrooms, with a little over half of the interviewed teachers describing monitoring engagement and progress as one of their daily strategies to encourage student achievement. Above, the instructor did not attempt to redirect students, while in other observations instructors verbally redirected students, albeit often with limited success.
As shown in the vignette above and across observations, student time off-task increased, and interactions with the online course system decreased substantially over the observation period. Few students maintained the focus to take consistent advantage of the educational resources available. Examples of fully engaged students were rare. Students exhibited full engagement in instruction throughout the entire class period in only 21 percent of observations. One such student from an observation in an alternative school setting,4 "worked through the assessment questions, checking her notes and selecting responses carefully," without interruption and without interacting with any instructors or peers in the classroom environment. The student possessed sufficient self-regulation skills, including focus and persistence, that she maintained productive interactions with the software interface. Furthermore, she appeared to possess requisite academic skills, such as minimum reading proficiency and study skills, further facilitating access to course content.
The student described above was atypical in successfully accessing and interacting with course content without requiring instructor assistance. Within this select group, many students completed coursework outside of the school day, indicating home access to digital devices and the Internet, as well as a minimal need for instructor assistance to master content. Our quantitative analysis of student behaviors within the online system suggested that those accessing the program at home, outside of school hours were less likely to qualify for free or reduced lunch, more likely to have achieved junior or senior standing, and more likely to have scored highly on previous standardized assessments (Heinrich, Darling-Aduana, Good, & Cheng, 2018). When examined in conjunction with barriers to learning, the typical profile of students who ultimately earned credit highlights the many ways in which transitioning from teacher-driven to technology-driven courses may further disadvantage those students in need of additional assistance, amplifying current disparities in achievement.
In the discussion and illustration of more typical interactional patterns below, we focus predominantly on the experiences of those students requiring assistance to learn course content. We represent visually the observed interactional patterns and possible help-seeking pathways of students struggling to learn content in Figure 1. This figure and subsequent findings highlight how interactions with lab instructors may have resulted in different learning experiences for these students. Whether a student decided to ask for help or completed coursework without assistance, there were barriers to learning and opportunities for demoralization and subsequent disengagement.
Figure 1. Observed Patterns and Possible Help-Seeking Pathways among Students Struggling with Content in Online-Credit Recovery Computer Labs
When Students Did Not Seek Help from Teachers
When students didn't ask for help, they often required more time to finish assignments or were unable to learn content. During one observation in an alternative school, a student and teacher "worked together and found that the program's supposed-to-be correct answer is not correct," after a student voluntarily asked for help. A student who didn't ask for assistance on the same problem would likely have either learned the content incorrectly or been unable to complete the assignment. In the following example, a student attending a different specialty school did not request assistance despite appearing unable to complete the required task.
At the time the observation began, the student was working in a Thermochemical Equations course. During the 20 minutes observed (before the class change), he progressed slowly in the lesson. In particular, he seemed to stall in the activity where more self-initiative was required (to practice the enthalpy of reaction equations). He did not leave the computer but did not practice what he had been shown in the video (solving problems). He did not request any assistance.
Whether the students slow progress was due to low engagement or difficulty comprehending content, a proactive instructor could have diagnosed and mitigated the underlying issue. Instead, the student received no credit for his time in the platform and left at the end of the class period without appearing to master content. In addition to often taking longer, the learning trajectories of students without instructor assistance were filled with opportunities for demoralization. The modal student from individual observations had some constructive interactions (72 percent) with the software but progressed through course content slowly, with occasional distractions (41 percent). For instance, in the representative, composite vignette presented at the beginning of this section, nine of 15 students interacted with the online course system at the beginning of the class period, with approximately half disengaging as the class period progressed.
One of the primary means by which students struggling with course content made progress without instructor assistance was through Internet searches or guessing. One such student from an observation in an alternative school read a source document and took notes on a lesson on the Mongol Empire before beginning an assessment about halfway through the observation. When completing the assessment, "the student copied and pasted the exact assessment question into Google to find the answers." We observed similar behavior across settings, which suggests a different type of help-seeking than the traditional version between students and teachers. Easier access to online resources used in this manner might result in assessment scores that don't reflect learning. In the instance above, the student's course notes might have been insufficient or required more effort to review than an Internet search; in other similar cases, students chose not to take notes at all, despite district policy guidance that urged classroom instructors to enforce note-taking practices. Notwithstanding the ethical concerns raised by this strategy, students who completed online assessments in this manner made course progress and might avoid demoralization. If the goal of credit-recovery is solely to provide students a second chance to earn course credits required for graduation, then this process achieves that goal. If the goal of credit recovery is to give students a second opportunity to learn course content because mastery of that material is deemed necessary for post-secondary success, then help-seeking only from online sources rarely contributed toward that end.
When Students Asked Teachers for Help
Similar pathways emerged among students who asked a teacher for help. Many observations that identified students asking for assistance were accompanied by comments indicating reactive instructor behavior, such as sitting behind a computer at the front of the classroom. In interviews, over half of the instructors described monitoring student progress as one of their primary responsibilities, although only a quarter of instructors reported following-up with students based on information gathered through their monitoring. This corresponds with the approximately three-quarters of instructors who expressed the belief that students should be intrinsically motivated and have high levels of self-motivation and self-control. One instructor shared his instructional strategy as follows, I tend to stay in the back, watch what they are doing, help as needed. In other observations, instructors focused on classroom management and administrative tasks unless a student voluntarily asked the instructors to check their answers or help with the questions." As such, the format of the online course system often required more initiative on the part of participating students than traditional instruction methods. A passive participant in a class incorporating a lecture component might still absorb knowledge, while student learning in a classroom that allowed students to determine their pacing might be more sensitive to low student engagement or motivation (Ahn, 2011).
As demonstrated in the passages above, instructors focused their attention on students who actively voiced the need for assistance, resulting in inequitable access to one of the students' most valuable instructional resourcesinstructor attention. We found a strong association between interaction and instruction ratings in our quantitative analysis of the observation data. On both dimensions, about 77 percent of the observations were rated a "2" on a zero to four-point scale, and a chi-square test confirmed the statistically significant association (p = 0.000). An interaction rating of 2 indicated that instructors or resources had some constructive interaction with students (i.e., facilitating some quality learning opportunities but not adapting to observed (or known) student needs), compared to mostly (3) or constant constructive interactions (4), no constructive interactions (1), or destructive interactions (0). The distribution of ratings of interactions in individual student observations differed from that of whole-class observations (p = 0.006). There were noticeably more (21 vs. 5 percent) low ratings in observations of individual student's learning experiences. This disparity was supported by observation notes, many of which explained that instructors interacted with students in a manner that enhanced learning throughout the class period. However, few instructors supported the instruction of all students in their classroom during any given observation.
In our observations, instructors responded to all but a handful of student requests for assistance. In one classroom, "The student asks the instructor for assistance about 44 minutes into the observation, but the teacher doesn't hear her. At the end of the observation, she is waiting by the teacher's desk for assistance." In another example, we observed an instructor repeatedly respond to requests for assistance from the one student identified as gifted in her classroom, limiting the teacher's ability to assist the students in her lab working on credit recovery. These findings are consistent with Calarco's (2011) finding that teachers provided more assistance to students with cultural capital associated with dominant groups, who were more likely to request help repeatedly and make eye contact and speak loudly while doing so.
When Teachers Offered Assistance
Even in instances where teachers proactively sought out students, teachers were more likely to follow up with students who had previously asked for help. We observed this method of student identification often when students asked for assistance early in the class period, as seen in the following excerpt.
The student was stationary for a few minutes and then went up to the front to ask the teacher a question. Another support teacher came around and noted that the student was at a 40 percent quiz score. He took some time to discuss the content with the student and to help him in considering the answers to a particular question. He encouraged the student to apply his test-taking skills, e.g., to determine the solution through a process of elimination. At the conclusion of the observation, the student was still working on the quiz, and he went up to the front to ask the teacher a question.
Asking for assistance earlier in the class period, as the student did above, might prime teachers to see the student as needing assistance or signal student engagement, indicating that time invested in assisting the student would likely translate into achievement. Similarly, although we do not have sufficient information to indicate directionality, there was a strong, statistically significant positive relationship between interaction and student engagement ratings (p = 0.000). These patterns appear to advantage further those students who asked for and gained instructor assistance.
Expectations of Students and Teachers
Another reason teachers relied on students to ask for help might be that teachers believed program expectations were clear to students. One teacher shared in an interview, "Teachers don't typically have a plan. Teachers refer to their online course system screen as students come in. Students know what they need to work on and are supposed to get started on it." Teachers expressed similar sentiments in around three-fourths of interviews. Further, when asked about daily instructional plans and strategies, over one-fourth of teachers only mentioned technical and logistical responsibilities. In an observation where a new student was assigned to the online course system, the teacher set the student up with a login. The entire orientation process involved only a few minutes of student-teacher interaction and was focused solely on the technical components of the platform. Without explicit guidelines in all online labs, the program model required student intuition to determine how to use the available resources effectively. Keeping expectations surrounding course completion and help-seeking implicit appeared to disadvantage students who did not know those expectations and with the least prior experience or success in dominant cultural settings (Bernstein, 1975; Delpit, 2006; Mehan, 1992).
Access to instructional assistance also varied based on the number of students and instructors assigned to each online credit-recovery lab. We observed a variety of student-teacher ratios across the 18 schools, with ratios ranging from 2:1 to 28:1. On average, we observed a student-teacher ratio of 10:1 in specialized and alternative schools compared to 14:1 in neighborhood schools. In whole classroom observations, we saw a definitive pattern and statistically significant association between larger student-teacher ratios and lower ratings of digital citizenship, as shown in Figure 2. Within the observation instrument, we defined digital citizenship as the responsible use of the technology by students. Ratings of the classroom environment, which included considerations of who else in the physical environment was available to assist students with technological problems and support learning, were also significantly (positively) associated with digital citizenship (p = 0.013). These descriptive results suggest that schools should prioritize a lower student-teacher ratio in online instructional environments to increase the number of students who might receive assistance and accommodations from instructors at a given time.
Figure 2. Digital Citizenship Ratings in Whole Classroom Observations with the Smallest and Largest Third of Student-Teacher Ratios
TYPES OF HELP RECEIVED
Among students who obtained assistance, interactions with teachers might be either assessment or learning focused, demonstrated by the pathway fork in Figure 1. Assessment assistance included providing students information on which questions they answered incorrectly but also at times included providing students the answers to assessment questions. Learning assistance included scaffolding knowledge or problem-solving with students to access and digest content. Whether students received learning assistance depended on their instructors' content and instructional capacity.
The most common reason students sought help was to ask a teacher to check their quizzes. As district policy only allowed students two quiz retakes, many students asked teachers to check quiz responses before submitting assessments for online grading, a process that was systematized by the district in the 20162017 school year. The new policy required teachers to review quiz answers with students before submitting responses to encourage instructional assistance and improve student pass rates. The most frequently observed response to this policy was to encourage students to engage in a process of elimination when completing quizzes, with many students asking an instructor to review responses two or more times during a single period, despite that fact that almost all quizzes consisted of multiple-choice questions with four answer options. The following excerpt describes one student who used this process strategically to progress.
The student spent some of the class period with videos running and answering problems, but she was quickly distracted. She talked with classmates, used her phone, and did not have headphones in to hear the audio. She made minimal progress in the videos. After filling in answers to the assessment (mostly incorrect), she went up to the teacher's desk multiple times for a list of the questions that she had incorrectly answered before changing them and going back to check again. She did not spend a lot of time thinking about the problems she previously answered incorrectly.
Many observations highlighted a systematized process like the one described above where teachers wrote or verbally shared the numbers of the questions students answered incorrectly without providing accompanying instructional support. In some instances, we observed classroom instructors staying at their desks and calling out question numbers, and on rarer occasions, directly stating the correct answers. Often, lines formed near the end of the class period as students worked to complete an assessment, with the same students standing in line multiple times until they determined the correct answers to the predominantly multiple-choice assessment questions through a process of elimination.
This type of interaction did not seem to support student learning, given that there was no assistance provided on how to find or learn content, only how to correctly respond to assessment questions. In fact, one teacher explicitly stated in an interview that he did not believe students were learning in the courses: They are not really getting anything out of it. Some do, but majority do not. Alternatively, although less frequent, some instructors offered instructional assistance in response to requests to review quizzes. As previously mentioned about online assessment assistance, access to in-person assessment assistance often prevented demoralization and facilitated course progression. For this reason, assessment assistance without instructional assistance might be preferable to no student-teacher interactions. At the same time, there is an opportunity when students initiate contact in this manner to use assessment results to inform teaching moments that encourage not just course progression, but also content mastery.
While support for understanding content is incontrovertibly fundamental to learning, observations indicated that instructors were often unable to assist students with content-related questions, and access to qualified and experienced instructors varied across classrooms. One instructor shared in an interview that to help a student with a genetics module, the student and teachers used YouTube and Internet searches to find the answer. In the excerpt below, we observed two teachers attempt to assist a student. Without sufficient content knowledge, the teachers spent the class period searching for the answer.
The student raised her hand and requested assistance from a teacher at the beginning of the observation time. The first teacher is unable to assist. The teacher copies and pastes the question in Google and attempts to find resources. The teacher then asks the other teacher for assistance. The second teacher takes some time to review the project and is able to find the answer to one of the problems. When the teachers left, the student reverted to playing with her phone or watching a TV show. The teachers left and returned numerous times, only finding the answer to one question throughout the class period.
Instead of involving the student in the learning process, the teachers tracked down the answer alone while the student waited, playing with her phone. We found that even when teachers were familiar with content, they rarely helped students learn the material, providing answers instead. In the above excerpt, the teachers lacked not only the content knowledge but also the expertise or belief that they should instruct students on the process of learning. After repeated experiences like the one described above, it is possible that many students might decide there is little value in requesting instructor assistance, decreasing subsequent help-seeking (Brion-Meisels, 2015, 2016).
Further supporting this assertion, we observed more favorable rates of instructor engagement, interactions, physical environment, and student engagement in classrooms where we noted in our observations that students had access to one or more certified teachers, as shown in Figure 3. The sizeable proportion of substitute teachers serving as instructors in credit recovery labs might contribute to the variability in prior experience and qualifications observed. We identified substitute teachers in 18 percent of observations where we had information on instructor background (n = 77). The presence of a substitute teacher, whether long-term or single-day, in a credit recovery classroom, was strongly, significantly associated with less favorable ratings of instructor engagement; 86 percent of observations with a substitute teacher received the lowest ratings (0 or 1) on instructor engagement, compared to 38 percent of observations without a substitute (p = 0.004). Ratings of instructor-student-digital tool interactions were also significantly lower in classrooms with a substitute teacher (p = 0.002). In one class, an observer noted, the "students weren't accessing the software during the session, primarily because it was a sub that day who didn't have access to the program and couldn't help." In another observation, a substitute teacher refrained from monitoring student engagement to prevent "starting something," communicating low expectations in the process. In a more extreme case, "The substitute teacher did not play an active role and at some point, just left the classroom." These instances highlight the limited capabilities of some substitute teachers, with the sizeable proportion of classrooms served by substitute teachers (higher than general education classrooms) suggesting possible discrepancies in access to quality learning experiences for credit recovery students.
Figure 3. Proportion of Observation Ratings at Each Scale Level in Classrooms with and without a Certified Instructor Present
ATYPICAL, BUT PROMISING, STRATEGIES
Above, we discussed how variations in student-teacher interactions might have created unequal access to quality learning opportunities. Below, we highlight interactions that minimized or eliminated many of the previously discussed barriers to learning, which if applied more universally, may reduce educational inequities in these and similar spaces. Although exceptional rather than the norm, the following classroom observation highlights all three characteristics that instructors used to facilitate more equitable access to quality learning opportunities in online credit recovery labs: (1) systematically building trust, (2) consistently offering assistance, and (3) providing content-specific expertise.
The instructor rotates around the room a number of times, asking each student if everything is going okay. There is quiet talking. A pair of students working together calls the instructor over, who works through problems with them using a process of elimination and explaining underlying concepts (in fluent Spanish). When the instructor finishes working with the students, he rotates the room, checking in with the other students in the room again before returning to the English Learner students. About ten minutes into the lesson, the instructor sits at his desk for the first time. A few minutes later a different student calls the teacher over to help his friend who failed a test. The instructor reviews responses with the student, focusing on the underlying content, which the teacher says the student gets before encouraging him to apply that knowledge to the quiz questions. The instructor then rotates around the room checking in with students who haven't yet asked for help. During the observation, the instructor had two extended conversations with students, one where a student explained a connection he made between his geometry assignment and a personal interest and the other where the instructor helped a student process her brothers arrest the previous night.
Above, the instructor demonstrated a genuine and holistic interest in his students well-being. He proactively reached out to each student to offer assistance instead of depending on his students to seek help. At the same time, and likely not unrelatedly, his students asked for help more often than typically observed. Lastly, when students struggled with content or on an assessment, the instructor broke down concepts to determine what students understood, providing alternative examples, scaffolding content, or affirming knowledge as needed. We discuss the merits of these strategies in greater detail below.
Building Trust to Facilitate Help-Seeking
First and foremost, the students in credit recovery labs are individuals with agency and out-of-school lives. Acknowledging this reality, approximately half of all instructors discussed in interviews the importance of taking on roles unrelated to the effective use of digital tools. One teacher explained, "I'm their administrator, counselor, and teacher." As a credit recovery program, the teacher explained that he counseled the students least engaged in school, which stemmed from a host of reasons. For instance, one student came into class crying on a Monday because her grandmother was shot over the previous weekend. The student needed to process this experience with her instructor before being ready to engage in instruction. Students sharing experiences such as these with their instructors were not uncommon. Unsurprisingly, we observed a significant, positive association between interaction ratings and instructors who took the time to build rapport with their students by demonstrating an interest in their lives (p = 0.008). Although at first glance, non-academically focused conversations might appear to distract from course progression, this might instead be an essential first step to earning students trust. The research of Brion-Meisels (2015, 2016) indicated developing trust between students and teachers encourages both help-seeking and engagement, two critical components of success in online courses (Ahn, 2011).
[Not] Proceeding Without Instructor Assistance
While the modal teachers relied on students to ask for help, around a quarter of instructors reported in interviews identifying and offering students assistance based on course progress or behavior. In over half of the observations where teachers offered assistance, they used technology-based resources to identify the students targeted. The availability of real-time progress and assessment information on each student helped instructors determine which students were actively interacting with the software, and of those students, which ones were struggling to master content. To monitor student engagement, one teacher used LanSchool (a classroom management software), which allowed her to log into any of the students' desktops and see a screenshot of their desktop at that moment. Although only observed in 10 percent of whole-class observations, we found statistically significant associations between teachers use of these computer-based tools to monitor students and both classroom interaction (p = 0.015) and instructor engagement ratings (p = 0.023).
The classroom teachers facilitating the online credit-recovery in this school district also had regular access to a broad array of data on student progress through the course system. For example, instructors could identify how far a student progressed through a course, their scores on quizzes and tests, and which questions students answered incorrectly. In the following example, instructors used the progress monitoring reports provided by the online course program to facilitate individualized conversations with each student about his or her progress.
When students log in each class, they can see their progress. The instructor has a different screen to monitor where they are. Kids check in with her and set a goal for where they want to be, looking for six percent progress per week. The instructor tends to show students the resources they have to track their progress.
While the above instructor used progress monitoring tools to develop goals and personal connection with students, other instructors used these tools to identify students requiring just-in-time assistance. In one such example, The instructor was monitoring the student's progress, as he noticed the student's low quiz score and came over to discuss content with him and help him determine the correct answers. Instead of providing more instructional assistance to more assertive students or students that the instructors perceived as engaged, both teachers used the information available through the online course system to identify students requiring assistance and drive one-on-one conversations with students in the classroom. While relying on student help-seeking might exacerbate existing advantage based on student access to and embodiment of middle-class behavioral norms, taking advantage of detailed data available to educators when students complete courses online, could help teachers facilitate individualize learning for all students, potentially providing more equitable access to instructors time and expertise. A similar result could be accomplished by regularly checking-in with all students and proactively offering assistance instead of relying on students to seek it.
Improving the Quality of Assistance Received
The value of an instructors' time in an online credit recovery lab is based in part on their capacity to connect to students, but instructors must also be able to provide content-specific assistance in a format that students can comprehend (Brion-Meisels, 2015, 2016). Many of the credit recovery labs that provided the highest quality educational experiences for students supported students completing courses in a single subject with a teacher certified in that subject area assigned to the lab (Taylor et al., 2016). For instance, the instructor described in the excerpt at the beginning of this section was a certified math and bilingual teacher with over a decade of experience teaching every math course from Algebra 1 through Calculus. It is unreasonable to expect an instructor without comparable subject and instructional expertise to provide the same caliber of assistance. Schools do their best to staff traditional courses with certified, experienced teachers; always, but particularly when students from underserved populations are disproportionately assigned to online credit-recovery courses, school should do the same to support equitable educational experiences in online courses (Hannum et al., 2008; Means et al., 2013; Taylor et al., 2016; Zhao et al., 2005).
In this section, we described student-teacher interactions in a digital credit-recovery program across 18 schools in a large, urban district in the Midwest. Interactional spaces included the decision to and process of instructors offering assistance or students asking for help and the type of support provided. The quantity and quality of these interactions likely influenced the extent to which students learned content, made progress in completing courses, or experienced demoralization, as represented in Figure 1. Many of these interactional spaces aligned with mechanisms for class and race-based achievement gap reproduction identified in interpretive social reproduction scholarship conducted in traditional classroom settings. Where data permitted, we observed similar patterns of disparate access to quality educational opportunities within this predominately online, technology-driven educational environment. Below, we discuss in greater detail the research and practical implications of our findings, including opportunities for expanding the use of promising instructional strategies.
This study extends current literature on spaces of educational inequality by mapping help-seeking interactions between students and teachers in online credit-recovery labs to spaces associated with disparities in academic opportunities and attainment in traditional classroom settings. Twenty percent of all secondary course credits completed in the observed school district during the 2016-2017 school year were earned online, with historically underserved populations disproportionately assigned to online courses. Across our 156 classroom observations, only one percent of students enrolled in online credit-recovery courses had access to consistent, constructive interactions with the online interface delivering instruction. All but the most engaged and well-prepared students required instructor support to obtain full access to the learning environment. Inequitable access to quality learning experiences in this context has profound inequity implications and importance for the overall quality of education in the district.
Observations exposed disparate access to quality educational experiences in online credit-recovery labs that mirrored those documented by others in traditional classroom settings (i.e., Calarco, 2011; Lareau, 2003; Streib, 2011). To mitigate these inequities, we identified guidelines for policy and practice that should improve equitable educational access in online courses:
Instructors should provide explicit expectations and proactive assistance to students, with students most likely ask for and accept help if instructors demonstrate trustworthiness and respect (Brion-Meisels, 2015, 2016).
The use of technological tools and real-time data can facilitate student-teacher interactions, such as goal setting and targeted support.
Low student-teacher ratios and assigning teachers certified in course subjects can enhance educational quality.
More specifically, instructors can encourage help-seeking by communicating expectations explicitly (Delpit, 2006) and encouraging proactivity (Calarco, 2011). Expectations for instructors must also be clear. Instructors must do more than prevent behavioral disturbances and provide technical support. Instead of sitting behind a desk waiting for students to approach with requests for assistance, teachers should monitor the classroom, either physically or with software, to identify and seek out students requiring instructional or motivational support. Instructors should also be prepared to fill non-instructional roles as counselor or confidant to build trust and demonstrate respect, which improves the likelihood students ask for and accept instructional assistance (Brion-Meisels, 2015, 2016). Transformation of the role of the instructors in this manner will likely require professional development and the minimization of administrative demands.
While many student-teacher interactions highlighted in our findings resulted in inequitable instructor assistance, other digital resources showed potential to reduce interactions identified by prior research as spaces that reproduced inequality. In online learning environments, instructors have access to real-time data on student progress, engagement, and learning, but the effective use of this information requires training and practice. While not the sole means to identify students needing assistance, computer-assisted monitoring of students allowed teachers to identify students who were off-task and students who required instructional aid. These findings indicated increased use of computer-based tools might assist teachers with classroom management and enhance the quantity and quality of instructional assistance. Instructors could use this information to initiate conversations with students about their progress and deepen instructor understanding of student knowledge and engagement based on objective versus subjective measures. For instance, with clear expectations and support, the checking of quizzes before submission could be transformed into an opportunity to provide targeted instructional assistance through blended learning.
From a structural standpoint, prioritizing low student-teacher ratios in credit recovery labs has the potential to increase the quantity and subsequent quality of student-teacher interactions. This is consistent with the work of Lazear (2001) who demonstrated that class size reductions would have the largest impact in classrooms serving students classified as disruptive, although teacher quality mediates that benefit. Similarly, assigning instructors with the content and teaching background to serve the instructional needs of students is necessary to ensure high-quality learning experiences. At minimum, there appears to be a positive association between factors associated with student learning and teacher certification and a negative relationship between learning environment and instruction by a substitute teacher.
FUTURE RESEARCH DIRECTIONS
Future research should expand understanding of the factors students consider when asking for assistance in online classrooms, including pre-existing student-teacher relationships and the type of support an instructor may provide. Researchers with access to observations of the same student over time may be able to identify how students' academic behaviors, including help-seeking, change across repeated student-teacher interactions. In turn, there are equity implications based on the extent to which teachers perceptions of students inform the quantity and type of assistance volunteered, which we were not able to examine with our cross-sectional data. Perhaps most substantially, our data collection process prevented us from making claims about disparities by socioeconomic status or racial identities. Consequently, the assumption that differences observed in traditional classroom settings transfer entirely to online classrooms merits further examination. Similarly, additional documentation on the extent to which differential student-teacher interactions by race or class characteristics mediate academic achievement and engagement would strengthen the motivation for future study.
We thank the William T. Grant Foundation and Mr. Jaime Davila for generous funding of this research; Vanderbilt University and the Wisconsin Evaluation Collaborative at the Wisconsin Center for Education Research, University of Wisconsin-Madison, for ongoing support of this initiative; as well as administrators and educators at our school district partner for their assistance in providing student record data and support of the project data analysis and fieldwork. We also want to thank district collaborators for their willingness to participate in research for continuous improvement.
1. Although not a focus of this study, interactions between students and curriculum are equally important to learning opportunities. Frameworks such as critical multiculturalism (Ladson-Billings, 2004; McLaren, 1994) can help examine the ways in which these interactions reflect and reproduce normative narratives along race, class, and gender lines, among others.
2. A copy of the instrument and information on instrument validity is available at the following website: https://my.vanderbilt.edu/digitaled/files/2016/08/Observation-Instrument.pdf.
4. The observed school district defined neighborhood schools as giving priority to students who live close to the school. Specialty schools focused on a program or area of study such as the arts or gifted and talented programming, while alternative schools targeted students requiring flexibility or attention, including students at-risk of dropping out.
Burch, P., Good, A., & Heinrich, C. (2016). Improving access to, quality, and the effectiveness of digital tutoring in K12 education. Educational Evaluation and Policy Analysis, 38(1), 65-87.
Heinrich, C. J., Darling-Aduana, J., Good, A., & Cheng, H. (2019). A look inside online educational settings in high school: Promise and pitfalls for improving educational opportunities and outcomes. American Educational Research Journal.
APPENDIX A: DIMENSIONS OF DIGITAL AND BLENDED INSTRUCTION RATED IN OBSERVATIONS
We used an instrument that directed observers to rate the following dimensions of digital and blended instruction.
Physical environment: How and where students access the instructional setting, including the technological setting and any associated limitations, and who else in the same physical environment as the student could assist with technological problems and support learning;
Technology and digital tools: How students access instruction, including internet connectivity, hardware and software in use, and the safety, operability and accessibility of the technology;
Curricular content and structure: Content and skill focus, who developed it and where it is located (e.g., software loaded onto a tablet, paper workbook), stated learning objectives, sequence and structure, level of rigor or intellectual challenge, and ability to meet and adapt curricular content to student needs;
Instructional model and tasks: Role of instructor and software in instruction (what drives instruction); purpose or target of instruction; student/instructor ratio and grouping patterns, multimodal instruction; order of thinking required and application of technology in instructional tasks, and ability to meet/adapt instructional model and tasks to student needs;
Interaction: How much interaction with a live person, and does the technology affect the ability of the instructor or student to positively interact with one another and the instructional resources?
Digital citizenship: Are students using the technology as intended by the instructor and/or instructional program?
Student engagement: Overall student engagement levels, level of student self-regulation and persistence, and level of community within the instructional setting;
Instructor engagement: Overall instructor engagement levels (passive or active) and instructor efforts to encourage engagement;
Assessment/feedback: Who develops and manages the assessment (instructor, provider via software), structure, and whether it is individualized to student learning and relevant to stated learning goals.
APPENDIX B: INSTRUCTIONAL STAFF INTERVIEW PROTOCOL
This interview protocol contained topics, probes and sample questions. We indicated required questions (versus suggested questions and probes) with an asterisk.
1. Instructor background
*Teaching experience and current instructional role/position:
How many years have you been teaching or working in education? Do you have any formal training in education? Are you pursuing additional education?
*What is your specific role or title with ____________? How long have you been in this position with __________? What other roles do you have around supporting digital instruction or using technology with students?
[Other related experience]
Do you have training particularly relevant to digital education, such as computer technology, media studies, software development, coding, etc.?
In addition to teaching, are you involved in (or responsible for) other educational and/or extracurricular programs or activities in the school district? How much time do you spend in an average week (outside of your classes) with students?
2. Instructional core
In a typical day with these students, what is the goal of the instructional session?
*How do you come up with your daily lesson plans? What is the length of a typical instructional period, and how many times does this group of students meet for classroom instruction?
*What digital tools do your students use in your classroom?
What are your goals for using these tools?
What are your strategies for using these tools?
Describe a typical instructional session in which the students use digital tools (time spent, days/week).
[Probe about tools observed and if it was a typical session, including activities, teacher/student roles, engagement, progress]
Source and use of digital tools:
Where do the digital educational tools that your students are using come from? Are you required to use them? If so, by whom?
*Do you or your students ever have problems accessing or using these tools?
Support for use of digital tools:
*Describe any training or other professional development specific to these digital educational tools. What aspects of the training were most useful to you in preparing for the use of the digital educational tools?
Thinking back to the start of this school year, which of these terms best describes your past experience with using digital educational tools in instruction: no experience, minimal experience, some experience, extensive experience or expert at using digital educational tools?
*Are there technology support staff available on site to help with these digital educational tools?
If yes, what types of support do they offer?
If no, what do you do when you need support?
What additional support for digital tools would you want or need?
Digital tool access and use by student subgroups:
*What additional resources are needed when using digital tools with English language learners? What about students with disabilities?
*Are the curriculum, instructional plans or digital tools adapted in any way for students with special needs, i.e., English language learners and/or students with disabilities?
Besides resources and curriculum adaptation, are there other differences in the ways certain groups of students use digital tools in your classroom?
3. Assessment and future use
Assessing the effectiveness of digital tools in the classroom:
*Do digital educational tools offer learning opportunities that face to face instruction does not? What are some examples of such opportunities?
In your opinion, how do digital tools impact student learning? How does it impact their school engagement?
*Are there any particular groups of students for whom you think digital tools have more potential for increasing student learning than others?
What are the greatest challenges to reaching the potential of digital educational tools for increasing student learning?
Plans for ongoing use of digital tools:
Do you plan to continue using digital tools in your classroom? Why or why not? How much input do you have into the extent to which digital tools are used in your school?
*What changes would you like to see in digital educational programming?
APPENDIX C: FULL LIST OF ANALYSIS CODES
We list the thematic and interpretive codes used in qualitative coding process below.
The following qualitative coding tree describes the parent and child codes used in our initial coding of observations and interviews.
The following table lists the main interpretive codes that emerged from an inductive coding process within the above codes. The second column provides an example or two that typifies this code as well as contrasting examples when available.