Subscribe today to the most trusted name in education.  Learn more.
Current Issue Current Issue Subscriptions About TCRecord Advanced Search   

 

From Checklists to Heuristics: Designing MOOCs to Support Teacher Learning


by Karen Brennan, Sarah Blum-Smith & Maxwell M. Yurkofsky

Background: Although much is known from educational research about factors that support K–12 teacher professional learning, it has been an ongoing challenge to incorporate these factors into practice in new contexts and environments. We argue that these factors are too often treated like a checklist of discrete elements, either present or not, insufficiently attending to the complexities of design and experience. To understand how Massive Open Online Courses (MOOCs) might support K–12 teacher learning, it is critical to move beyond application of discrete factors to nuanced navigation of the interplay among researcher examination and theorization, designer intention and implementation, and learner use and experience—balancing considerations of learning theory, instructional objectives and specific learning context, and the desires, needs, and experiences of participants.

Focus of Study: This study examines MOOCs as a medium for supporting teacher professional learning. What did K–12 teachers identify as meaningful about their participation in the Creative Computing Online Workshop (CCOW), a large-scale, constructionist, online learning experience for teachers? How do the teachers’ experiences relate to each other, to learning research, and to the affordances of MOOCs?

Research Design: This qualitative, interview-based study draws on 15 semistructured interviews with participants 1 year after they completed CCOW, as well as course artifacts. We used an iterative approach to develop common themes reflecting what teachers found meaningful and key tensions present in these themes.

Findings: Teachers described four qualities as most meaningful to their learning: activity, peers, culture, and relevance. Although these qualities were often mutually supporting, three key tensions among the qualities and the implications for the design of online teacher learning experiences are discussed: autonomy, with structure; diversity, with commonality; and experimentation, with validation.

Conclusions: This paper challenges the notion that implementing successful professional development for K–12 teachers is simply a matter of following a checklist of design elements. This study presents qualities that teachers found meaningful in an online learning experience, offering heuristics that designers might consider when designing for their specific contexts. Future research might assess to what extent the qualities and tensions identified in this study apply to other contexts, and explore the reasons why contextual changes may or may not influence results.

Radio. Television. Personal computers. The Internet. Each new wave of technologies has been accompanied by aspirations for new ways to support learning—including promises of new opportunities for supporting teachers’ professional learning (Cuban, 1986; Dede, Ketelhut, Whitehouse, Breit, & McCloskey, 2009; Selwyn, 2011). The recent excitement surrounding Massive Open Online Courses (MOOCs) is no exception, with high-profile MOOC initiatives like Udacity, Coursera, and edX receiving enormous attention over the past several years.


We approach this space as educational researchers and as designers, motivated by a curiosity about how MOOCs can be used as a medium for supporting professional learning experiences for teachers, but cautious to avoid design and methodological missteps that have undermined previous teacher professional development efforts. Although much is known about how to support teachers’ learning, the translation of learning research into design, particularly in a new medium such as MOOCs, is neither simple nor straightforward. Chief among our concerns is that despite a consensus around the complex, varied, and nonroutine nature of teachers’ work (Borko & Livingston, 1989; Bransford, Brown, Cocking, Donovan, & Pellegrino, 2000; Lampert, 2001) and the concomitant need for professional development experiences that are learner-centered and sensitive to teachers’ contexts (Borko, 2004; Bransford et al., 2000; Putnam & Borko, 2000), research on teacher professional development has tended to cohere around an inventory of characteristics (e.g., opportunities for active learning) that are theorized to lead to changes in teachers’ knowledge and practice, as well as improved student outcomes (Desimone, 2009). This focus on defining effective professional development as the presence or absence of discrete elements obscures the important role that designers and facilitators of professional development play in adapting learning principles to serve individual teachers, instructional objectives, and learning environments—and the important role that participating learners play in shaping the experience.


We argue that this disconnect between current framings of effective professional development and the complexities inherent in learning has been a significant contributing factor to disappointing results from professional development design and research (Garet et al., 2008). Accordingly, in this work, we move away from an approach that relies on research being distilled to a design checklist and toward an approach that foregrounds the necessary interplay between research, design, and experience. We are interested not only in whether particular learning principles are incorporated into a professional development experience for teachers, but how designers and facilitators of professional development experiences negotiate those principles. Our aim is to identify heuristics—principles and ideas that are iteratively negotiated in relation to research, design, and experience—and to explore the tensions and tradeoffs inherent in instantiating these heuristics in specific learning experience designs.


A heuristics-based approach is well attuned to the inherent unpredictability in design endeavors that results from the complexity of both learners and environment (Scott, 1998). By focusing on the ways that designers incorporate and participants experience design elements within specific learning contexts, this approach allows us to better explore the affordances and challenges of new technologies, such as MOOCs, in supporting teacher learning (Fishman et al., 2014). And by focusing on the ways that designers balance competing priorities related to learning principles, instructional content, and the medium, this approach allows researchers to explore the tensions or dualities endemic to learning experiences (Barab, MaKinster, & Scheckler, 2003).


In this study, we examine MOOCs as a medium for professional learning through the lens of teacher experience, relating those experiences to learning research and design implementation. Specifically, we explore teachers’ descriptions of their participation in the Creative Computing Online Workshop (CCOW), a 6-week online learning experience designed to help teachers incorporate introductory computing with the Scratch programming language into their classroom activities. Our work is guided by a central research question: What did teachers identify as meaningful about their participation in CCOW? We use “meaningful” as a central term not to create a new analytical construct or to privilege an existing one, but rather to foreground the subjective experience of teachers (in contrast, for example, with “effective,” a word that we will discuss in greater detail in the literature review). By focusing on teachers’ own perspectives about what was meaningful to their learning, we hope to illuminate the qualitative diversity of the interactions between learner and medium around a specific content area, in the interests of discovering how elements of meaningful learning experiences can be integrated by designers and be taken up by participants.


We have organized this text into six parts. In the first part, we review the seeming consensus that has developed around the elements of effective teacher learning and the ongoing challenges in incorporating those elements into professional development experiences. In the second part, we describe CCOW as design context. In the third part, we outline our approach to understanding teachers’ experiences. In the fourth part, we present four qualities of CCOW that teachers identified as meaningful to their learning. In the fifth part, we consider how these qualities can also result in tensions, particularly within a MOOC context. We end with thoughts on lingering questions and potential future work.


LITERATURE REVIEW


The past few decades have been characterized both by important consensus around the conditions that enable teacher learning, as well as challenges and frustrations around operationalizing that knowledge in the research and practice of teacher professional development. A foundational synthesis of the state of knowledge about learning from the National Research Council noted that teacher learning, like student learning, should be learner centered, knowledge centered, assessment centered, and community centered (Bransford et al., 2000). Similarly, Ball and Cohen (1999) summarized that substantive learning experiences for teachers often include active engagement; framing tailored to the specific context of teachers; opportunities to practice, revise, and reflect; exemplar models; and support from others in the immediate environment. Other research has emphasized the importance of attention to teacher knowledge (Kennedy, 1998) and the ways in which teacher learning takes place within a specific context that includes interactions with others (Putnam & Borko, 2000). These themes are reflected in Shulman’s (2004) framework of five principles that account for “effective and enduring” learning experiences for teachers: activity, reflection, collaboration, passion, and culture.


Along with assertions about what teachers need in order to learn, nearly all of the research cited above included lamentations about the challenges of ensuring that these qualities are present in professional development experiences (Ball & Cohen, 1999; Bransford et al., 2000; Kennedy, 1998; Putnam & Borko, 2000; Shulman, 2004). As noted by Borko (2004), one of the central problems is that work in the field of professional development frequently does “not take into account what we know about how teachers learn” (p. 3). And as Ball and Cohen (1999) argued, many of the characteristics of successful learning experiences for teachers, such as support from others in the immediate learning community, are at least partially out of the control of those designing interventions. Additionally, most successful professional development programs, the results of which inform the consensus around recommended characteristics, were designed and implemented by specialists in the field whose expertise has not always been generalizable to programs implemented by state or local agencies, or corporate service providers (Kennedy, 1998). Underlying many of these challenges is the enduring tension between the complexity of variables that impact learning environments and the desire to show the impact of specific variables with an eye toward informing policy and practice (Labaree, 1998).


These challenges are evident in efforts over the past 15 years to operationalize qualities of effective learning experiences in the design and evaluation of large-scale teacher professional development. The most prominent example of these efforts is a series of studies that used this research base to develop survey items that helped identify specific characteristics of professional development programs that are associated with teachers’ self-reported learning (Garet, Porter, Desimone, Birman, & Yoon, 2001) and discrete changes in teachers’ instructional practices (Desimone, Porter, Garet, Yoon, & Birman, 2002; Penuel, Fishman, Yamaguchi, & Gallagher, 2007). Based on this large-scale and primarily observational evidence base, Desimone (2009) called for designing professional development with the “core set of characteristics that we know are related to effective professional development, and measuring them every time we study professional development” (p. 186). Despite such epistemic certainty around these features, large-scale professional development programs informed by these specific characteristics have struggled to yield substantive changes in teaching practice or student learning (Garet et al., 2008; Garet et al., 2011).


We attribute these disappointing results in part to the oversimplification inherent in efforts to identify discrete practices that yield improvements in teacher or student learning, independent of context. That is, by focusing on discrete practices, we fail to recognize the importance of how program designers understand and enact these practices in response to their intentions, the content of the program, and teachers’ contexts and particular needs. Teachers, for example, bring varied knowledge, skills, needs, and interests that may shape the specific characteristics most helpful in supporting their learning (Bransford et al., 2000). Ball and Cohen (1999) have similarly argued that the specific ways designers implement the important qualities of professional development experiences should also depend on the program’s instructional objectives.


Although there is certainly a role for larger scale evaluations of specific approaches to professional development (Hill, Beisiegel, & Jacob, 2013), we are concerned about the dominance of these approaches to research design at the expense of those that illuminate the nuance and complexity of teacher learning and the diversity and variety of the individuals engaged in that process. As noted by Scott (1998), “the lack of context and particularity is not an oversight; it is the first premise of any large-scale planning exercise” (p. 346). We see an opportunity to complement these efforts by encouraging those designing and researching professional development to more centrally consider prior research on human learning and context-specific factors, such as the variability of what learners bring to the experience, the specific substance of the work in which they engage, and the community of learners in which that engagement takes place (Bransford et al., 2000; Putnam & Borko, 2000; Shulman, 2004). These context-specific factors, as well as the variety of ways in which learners actually make use of the elements of their environment, can be more difficult to capture in large-scale, quantitative studies, but they are an affordance of qualitative investigations that foreground specificity and diversity (de Certeau, 1984). We argue that a qualitative, heuristic-based approach better supports the need for designers to continually negotiate the design of professional development experiences in response to research, the content being taught, and the specific learning context.


The tension between making legible the “best practices” of teacher learning so that they can be applied more broadly and exploring the variability of learners and dynamic interaction of learner and context (Scott, 1998) has played out in specific ways in the field of online teacher learning. In particular, research has been structured around the extent to which the online medium does or does not make a difference in teacher learning, rather than exploring the particular challenges and affordances of online learning. We are again concerned with the ways in which this approach attempts to identify the broad effectiveness of characteristics of a learning environment (i.e., the online medium) rather than starting with general principles of what is important in teacher learning and exploring the different ways those principles can be supported within a given medium. For example, one of the most influential and comprehensive reviews of online learning experiences asked, as one of four primary research questions, “What practices are associated with more effective online learning?”—but intentionally limited the scope of the investigation to experimental studies that compared the impact of online and face-to-face professional development on student learning (Means, Toyama, Murphy, Bakia, & Jones, 2009). Similarly, in a comparison RCT of online and face-to-face mathematics professional development, M. Russell, Carey, Kleiman, and Venable (2009) evaluated whether a face-to-face course was as effective as an online course with the same instructors and an identical syllabus. In making the treatment and control conditions as similar as possible, so that any differences in student learning can be attributed to the online nature of the learning experience, such studies miss opportunities to reevaluate and restructure learning experiences to take advantage of the online medium. Although such designs are helpful in answering certain kinds of research questions (e.g., all else equal, what are the effects of online professional development compared to face-to-face experiences?), they are less useful in exploring the range of practices that support teacher learning in the online medium, illuminating how those practices might operate in combination with one another, and understanding the consequences of such interaction, particularly considering the unpredictable ways that individual learners might find value and enact their learning in practice.


Recognizing that tools for online learning have been gaining attention absent a deep understanding of how these tools might support learning in new ways (Roskos, Jarosewich, Lenhart, & Collins, 2007), a growing number of researchers have begun to explore the affordances of online learning platforms, illustrating how online environments can support teacher learning in new and rich ways that may be less feasible in face-to-face experiences (Bonk & Cunningham, 1998; Rudestam & Schoenholtz-Read, 2010; Whitehouse, 2011). Research has highlighted key challenges and opportunities for online professional development, such as the need to garner increased trust to support knowledge sharing in this more anonymous platform (Booth, 2012; Ukpokodu, 2008), the opportunity to better engage with others’ work to support learning and reflection (Ukpokodu, 2008), and the importance of designing for participants who may range dramatically in terms of their skill and comfort with engaging in course activities (Farooq, Schank, Harris, Fusco, & Schlager, 2007).


Our study aims to extend this nascent body of literature, exploring how online environments can support teacher learning. Placing ourselves in the tradition of foregrounding attention to the processes of learning within both traditional (Putnam & Borko, 2000) and online (Bonk & Cunningham, 1998) environments, we investigate experiences that teachers did and did not find meaningful, or supportive of their learning, and why. This approach allows us to move beyond checklists of discrete elements, focusing instead on how teachers experience elements of the learning environment. We distill these experiences to heuristics for learning design—broad learning principles in need of continuous negotiation and reflection in response to the interaction of learner and environment. By investigating heuristics or tensions that must be negotiated, rather than discrete elements that must be incorporated, we endeavor to offer other researchers, teachers, and designers an example that they can use to “more readily identify patterns occurring in their own interventions and navigate the challenges they face more intelligently” (Barab et al., 2003, p. 239).


RESEARCH CONTEXT


Scratch (http://scratch.mit.edu) is a free graphical computer programming environment that enables young people to create their own interactive games, stories, animations, and art—and then share their creations with others in an online community. The Scratch online community, launched in May 2007, now includes millions of members (mostly between the ages of 8 and 18, from all around the world) and more than 20 million projects. With increasing use of Scratch in K–12 classrooms, teachers need support for understanding Scratch as an authoring tool and an online community, as well as for understanding approaches to teaching that support creative design activities (Brennan, 2013). Whereas many introductory computing experiences remain highly didactic and procedural, Scratch was intentionally designed for a constructionist approach to learning (Brennan, 2015).


From 2009 until 2012, with support from Google’s Computer Science for High School (CS4HS) program, we hosted in-person Scratch professional development workshops, enabling teachers to participate in the kinds of constructionist learning experiences that would benefit students working with Scratch in the classroom. In these multiday Creative Computing workshops, teachers engaged with Scratch through designing (e.g., engaging in hands-on, project-based learning), personalizing (e.g., developing resources and projects based on their contexts and interests), sharing (e.g., discussing their experiences with colleagues), and reflecting (e.g., maintaining a design notebook that documented and unpacked their questions and progress).


Though we received hundreds of applications each year, the physical space available limited annual enrollment to 50 participants. Motivated by a desire to make the Creative Computing workshop accessible to a broader audience, and with additional support from Google’s CS4HS program, we developed CCOW, a large-scale online learning experience developed on Google’s Course Builder platform. Like the in-person workshop, CCOW was organized as an experience for teachers to learn about Scratch, both as a tool and as an approach to learning. CCOW was hosted for 6 weeks, from June 3 until July 12, 2013. Approximately 2,100 people from around the world enrolled in the workshop, with 51% indicating that they intended to participate beyond “just browsing.”


During the workshop, participants created projects with the Scratch programming language, from focused debugging challenges to more open-ended design explorations. They maintained online design notebooks that served as a record of and reflection on their participation. We provided both general prompts to reflect upon the big ideas and themes of each week and specific prompts for reflection accompanying each suggested activity. For example, in a session on programming games and other activities, participants were invited to solve nine different Scratch puzzles that required skills that may apply to game design. Participants were then asked to reflect on the strategies they used to solve these puzzles, and how, if at all, those strategies helped them in designing their own games. Participants would also interact with workshop colleagues through comments on design notebooks and discussions in the course’s online forums. For each set of tasks, participants could share solutions or pose questions to one another in a specific discussion thread.


The course invited participants to interact with one another through the sharing of activities, commenting on one another’s design notebooks, and engaging in discussions of course ideas. For example, in the second week of the course, participants were prompted as follows: “If you haven’t been able to share your notebook or see others’ notebooks, I encourage you to share your notebook in this dedicated design notebook forum,” and “I thought it might be fun to have a discussion about ‘thinking about thinking’ and about powerful ideas. If you’re interested, I invite you to read Seymour Papert’s (2000) article entitled ‘What’s the big idea: Towards a pedagogy of idea power’ and discuss it in the Week 2 general questions/discussions forum.” This language reflected our intention as course designers to provide some structure for the tone, form, and substance of participant reflection and interaction, but also leave space for learners to make their own choices. During the second half of the workshop, participants defined and pursued independent learning projects, such as designing curriculum, hosting workshops for kids, and exploring connections between programming and art.


Over the 6 weeks of the workshop, there were a total of 24,000 separate views of workshop videos (excluding multiple views by the same individual, but including partial views). Workshop participants created 4,700 Scratch projects and wrote 5,000 discussion posts on the course forums, Twitter, and Google+. Although such metrics are informative as to aggregate levels of participation, they should not be used to form assumptions about typical participant experiences. There are many forms of learner engagement in MOOCs, from enrollment through to achievement (DeBoer, Ho, Stump, & Breslow, 2014). The teachers whose experiences form the basis of this study were part of a core group of more intensively engaged and active participants—127 people who persisted through the 6 weeks of the learning experience and completed the majority of workshop activities.


METHOD


The goal of the present study was to explore the diversity of how individual teachers experienced their learning in CCOW, a large-scale, online professional development workshop. Therefore, we adopted an in-depth, qualitative interview approach which foregrounded teachers’ own descriptions of and viewpoints on their experience (Kvale, 1996). The purpose of this work was not to produce an overall evaluation or description of CCOW, nor to ascertain the presence or absence of specific qualities identified in prior research. Rather, we sought to understand what experiences this group of individuals found meaningful, valuable, and important to their learning, and how the qualities reflected in those experiences interacted with one another. Our aim in this work was to inform the design of future learning environments; we thus used this approach to “[instantiate] the possible, not only documenting that it can be done but also laying out at least one detailed example of how it was organized, developed, and pursued” (Shulman, 1983, p. 495).


DATA COLLECTION


We conducted the study 1 year after CCOW had taken place. Though such a time delay has disadvantages (e.g., participants may not remember in as rich detail 1 year out), it also allows participants to reflect on their experience with more perspective. We came to the interviews with artifacts from the course (e.g., the participants’ design notebooks), to help them recall more specific details about the experience.


Obtaining a representative sample of MOOC participants can be challenging due to high rates of attrition in MOOCs (DeBoer et al., 2014). We began with the 127 CCOW participants who completed a workshop exit survey, indicating a willingness to be interviewed and providing contact information. We then narrowed the list to 57 individuals who appeared to be full-time teachers. We intentionally sampled this group for diversity across curricular areas, ages taught, country of origin, programming and Scratch-specific experience, and extent of participation in CCOW. This sampling produced a list of 37 teachers whom we contacted via email in May 2014 to ask if they would like to participate in an interview study of their experiences during the workshop.


Sixteen of the 37 teachers whom we contacted agreed to be interviewed. One individual was excluded from our sample because she was not a full-time teacher. As documented in Table 1, which contains interviewee demographic information, our sample (in spite of efforts to sample for diversity) tended toward technology teachers with prior programming and Scratch experience. None of the teachers with whom we spoke were in their initial years of teaching. One third had less than 10 years of teaching experience, one third had between 10 and 20 years of experience, and one third had more than 20 years of experience. Interviews were conducted in May and June 2014. The names used throughout the rest of the document to describe particular participants are pseudonyms. Interviews were conducted through Skype or Google Hangout, audio recorded, transcribed by a third-party service, and checked for accuracy by one or more of the authors. We interviewed each participant once, with a typical interview lasting 90 minutes.


Table 1. Participant Demographics, Within the Interviewee Sample and Across All CCOW Participants


Characteristic

# of interviewees

% in sample

% in CCOW

Elementary school teacher

6

40

56.6

Secondary school teacher

9

60

Primarily teaches technology-related subjects

11

73.3

45.4

Lives in the United States

11

73.3

54.3

Completed course

15

100

5.3

Scratch experience

   

Very experienced

6

40

7

Experienced

5

33.3

18

Familiar

2

13.3

25

New

2

13.3

50

Programming experience

   

Extensive experience, feels comfortable programming

3

20

17

Some experience, feels comfortable programming

8

53.3

30

Some experience, does not feel comfortable programming

3

20

27

No experience

1

6.7

24


We used semistructured interviews as our primary source of data to understand teachers’ experiences in CCOW. The protocol was structured into four major sections: (a) teachers’ demographic and professional information, (b) descriptions of past meaningful learning experiences, (c) discussion of the qualities of CCOW that they did and did not find meaningful, and (d) the ways in which teachers did (or did not) use or incorporate elements of their learning from the course into their present practices and ways of thinking (see Appendix A for full interview protocol). The findings from this study relate primarily to the first three sections of the interview. In other work (Yurkofsky, Blum-Smith, & Brennan, 2016), we have explored the ways in which teachers conceptualized the various kinds of outcomes, or things they took away, from their CCOW experience—including impact on content knowledge (e.g., greater fluency and familiarity with the Scratch programming language), pedagogical practice (e.g., greater comfort and confidence with their role as facilitator of more open-ended learning experiences), and self-concept (e.g., greater sense of agency and advocacy for self and others within and beyond the classroom). Interview data were accompanied by an examination of learning artifacts that teachers created through CCOW activities, such as written reflections, Scratch programs, and forum discussions.


Consistent with qualitative research interview tradition (Kvale, 1996), the structure and language of the interview protocol reflected our commitment to understanding teachers’ perceptions of their CCOW experiences, rather than imposing predetermined meanings. This was our attempt to “bracket” our own beliefs and assumptions about the experience of meaningful teacher learning so as to leave space for those of the participants (Willig, 2008). We began each section of the interview with a broad prompt (e.g., “Describe a powerful learning experience. What made it meaningful for you?”; “What qualities of CCOW were/were not meaningful to you as a learner?”) in order to encourage intrinsic descriptions of experiences before probing for specific areas of interest.


DATA ANALYSIS


Following the steps for analysis of qualitative data described by Boyatzis (1998), we began by open coding a small sample of the interviews. After coding each sample interview individually, the three members of the research team met to discuss emerging overarching themes. Through discussion, we decided that the aspects of meaningful learning described by interview participants fell broadly under Shulman’s (2004) framework of five principles that account for “effective and enduring” learning experiences for teachers: activity, reflection, collaboration, passion, and culture. This framework had surfaced during our literature review of research on meaningful teacher learning. We proceeded to code the remainder of the interviews using these five categories. This combination of emic and etic analysis can be especially useful in interview data analysis, where the limited data set can constrain the development of more robust conceptual themes (Boyatzis, 1998).


To support reliability between coders, we developed precise definitions of activity, reflection, collaboration, culture, and passion. We developed subcodes that reflected the particulars of how these themes manifested in individual teachers’ learning experiences and what seemed to enable or constrain the presence of those qualities, both in and out of CCOW (e.g., “learning by doing” and “making something useful” as subcodes for “activity”). After a first round of coding in which we made distinctions between the context of the learning experience (in or out of CCOW), we noted the continuity between qualities teachers identified across various contexts, and therefore proceeded with a second round of coding that collapsed these distinctions. The two authors coding the interviews met periodically to check for reliability, adjusting the names and definitions of subcodes accordingly. During this process, we also noticed several tensions between the major qualities teachers identified (e.g., participants’ desire for autonomy in their learning experiences, while still appreciating some course structures and constraints).


To further ensure the validity of our data throughout this process, we used various course artifacts to triangulate the emerging themes and tensions, and searched purposefully for examples of experiences that did not match the overall findings (Maxwell, 2010). The course artifacts included (a) participants’ individual online design notebooks (which consisted primarily of written reflections on their course learning experiences, accompanied by images of and links to their course creations, e.g., their Scratch projects), (b) participants’ Scratch projects (as mentioned earlier, more than 4,700 Scratch projects were created through course activities), and (c) participants’ discussion posts on the course forums, Twitter, and Google+ (as mentioned earlier, more than 5,000 posts were shared among all enrolled students). The identified themes were evidenced both by interview data and by these course artifacts.


During the coding process, we wrote approximately 70 pages of memos to support our interpretation of the data (Luttrell, 2010). Through memoing, we realized that although Shulman’s framework was useful for understanding the broad qualities of meaningful learning (and we continued to see those qualities reflected throughout interview and artifact data), it was limited in its ability to explain important specifics of participant experiences. That is, we wanted to move beyond detection of the important qualities articulated by Shulman to understanding how these qualities manifested for different learners, and the interplay between learning principles, design and content of the course, and learners’ needs and interests. To that end, we iteratively reconceptualized the thematic and organizational structure of the findings with the aim of making distinctions between categories and better reflecting the most important dimensions of participant experience. We returned to the raw data to test the validity of our revised structure (see Table 2 for examples of changing codes over time). The final coding structure presented in the analysis here (activity, peers, culture, and relevance) reflects the conclusion of our iterative process.


Table 2. Examples of Coding Schema Over the Course of Data Analysis


First Iteration

Second Iteration

Third Iteration

Activity

Learning by doing

Engaging in authentic and legitimate work

Opportunities to tinker and experiment

Reflection

Prompted by interactions

Prompted by structures

Engaged in over time as part of iterative process

Collaboration

Cognitive benefits

o

Sharing ideas

o

Getting feedback

Affective benefits

o

Feeling supported

o

Validating perspective

Passion

Useful and relevant to work

Choice as motivation

Culture

Located in relationships with facilitators

Located in relationships with peers

Embrace of failure

What you were doing and how you were doing it

Creative and open-ended tasks that allowed people of different levels to enter and engage in meaningful work

Time and space to engage in a reflective process of iteration

Symmetry between participant work and student work generates reflection on students’ experiences

The role of others in doing and thinking

Cognitive and affective benefits of receiving feedback on your work from peers

Opportunities to get new ideas from a diverse group of people

Engaging with others’ work and thinking

Setting the foundation

Making connections

Embodying norms in words and actions

Connecting learning to personal and professional goals

What people brought with them

What people took away

Activity

Open-ended

Accessible

Aligned

Peers

Inspiration

Feedback

Encouragement

Culture

Connected

Experimentation

Participatory

Relevance

Intentions

Broader outcomes


Nonetheless, the initial framing from Shulman did invite a way of looking at the data that remained with us throughout coding, analysis, and writing. In the original presentation of the qualities of “enduring and effective” learning in The Wisdom of Practice, Shulman (2004) emphasized their nested and interrelated nature, calling attention not only to the concrete work that learners do but to how that work is embedded in tangible and intangible norms and structures, inviting particular ways of thinking, doing, and interacting. This approach of attending to multiple, interconnected layers influenced how we developed understandings of what was meaningful to teachers about their CCOW learning experiences.


LIMITATIONS


Our approach to data collection had several limitations. Due to the accessible population (i.e., teachers who completed the exit survey) and the relatively low response rate (39.5%), our sample was biased toward teachers who taught technology-related subjects, who had prior knowledge of Scratch and computer programming, and, most likely, who had positive CCOW experiences. Further, we only interviewed teachers who were able to speak with us in English, although this was not the first language for three of the interview participants. Although we did not purposefully exclude non-English speakers from our sample, such limitations undoubtedly impacted those who had filled out the exit survey in the first place and responded to our requests to speak in more detail. Nevertheless, there was individual and experiential variety among the people we interviewed. There were no aspects of CCOW that garnered entirely positive or negative reactions from the interviewees, and many participants appreciated similar aspects of CCOW for different reasons.


In addition, the teachers we spoke with may have been reluctant to share negative feelings about their CCOW experiences. Although the two authors who conducted the interviews neither designed nor facilitated CCOW, and included scripted statements at multiple points in the interview protocol asserting a lack of stake in the responses as well as inviting negative comments, many teachers viewed the interviewers as part of the “CCOW team.” This was reflected in references to “your course” and actions taken by “you guys,” along with assumptions that the interviewers had personal experiences with individuals and activities from the workshop. Although the feedback we received from participants was overwhelmingly positive, all teachers highlighted multiple aspects of CCOW that were not meaningful for them. Nevertheless, given the promise of online learning to broaden access to a greater number and diversity of learners, it is essential for further work to explore how other MOOC participants (e.g., “lurkers,” “dropouts”) conceptualize meaningful learning and how those different conceptualizations may demand different design qualities.


Finally, our exploration of large-scale online learning was limited to a workshop focused on a particular kind of content (i.e., computer programming) and pedagogical approach (i.e., constructionist). We are cautious about arguing that the qualities teachers highlighted as meaningful, described in the following section, would be as important, for example, in learning experiences not about computer programming or not grounded in constructionist traditions. That said, our approach did engage teachers in talking not only about CCOW, but about their other in-person and non-computer-programming learning experiences, which suggests broader applicability of these findings to other mediums, content areas, and pedagogical approaches. We recommend a similar methodological approach for examining other types of online courses, especially those focused on different content areas and pedagogical approaches.


FINDINGS


This research is guided by a central question: What did teachers identify as meaningful about their participation in the Creative Computing Online Workshop (CCOW)? In this section, we present the understandings that we developed about this question through interviews with teachers and analysis of course artifacts. The section is organized thematically into the four main qualities that teachers identified as meaningful to their CCOW experiences. In Activity, we examine the substance of teachers’ CCOW work through the daily tasks and activities in which they engaged. In Peers, we move from individual activity to examine how teachers’ interactions with others in the course supported their engagement and learning. In Culture, we explore how teachers experienced and contributed to the development of a distinct culture of learning. In Relevance, we finish by considering the role teachers’ initial intentions and professional needs played in their learning.


ACTIVITY


A key theme in our conversations with teachers was their desire to be actively engaged in their learning through activity. In CCOW, course activities included a variety of Scratch programming tasks, a culminating self-directed project, and ongoing reflections recorded in an online design notebook. Specifically, teachers identified activities as meaningful to their learning when they were open-ended with multiple pathways for exploration, accessible to people with various levels of skill and experience, and aligned with student learning experiences in a way that provided insight into student perspectives. The theme of finding meaning through activity is consistent with existing learning research. In particular, research on conceptual change emphasizes that it is only through trying to enact our understandings in practice, through doing, that we recognize the inadequacy of our current thinking and are motivated to seek out new ways of thinking (Carey, 1999; Chinn & Brewer, 1993; diSessa, 2006; Strike & Posner, 1985).


The open-ended nature of the tasks provided teachers an opportunity to try things, reflect on the success or failure of their approaches, and try again. Teachers had different names for this process—including “experimenting,” “tinkering,” and “iterating”—but the shared underlying theme was the importance of being able to work with an idea or concept over time, and to think about one’s evolving understandings. For example, Katie (a middle school technology teacher) and Justine (a middle school digital art and design teacher) described how Debug-It tasks, in which participants determined why a Scratch program was not running correctly, enabled this kind of process:


Katie: I was eating up things like the Debug-It sections and stuff like that where things weren’t working and I needed to make them work. Because in the middle of a classroom when your students are like—“Ms. P, it doesn’t work!”—you try as quickly as you can to be able to go and pinpoint why that child’s program doesn’t work or things kind of fall apart. That really was really helpful because I got to spend time trying different techniques on my own, and things wouldn’t work and I would have to figure out why.

Justine: The Debug-Its helped me read scripts, think iteratively, decompose or pull things apart, understand how to read something.


While Justine described the cognitive processes that Debug-Its promoted, Katie emphasized the role that time and privacy played in allowing her to engage in the iterative process of figuring out why her initial solutions were not successful. She contrasted her experience in CCOW with the lack of progress she had made in the classroom, where she did not have adequate time to tinker when confronted with a problem. But although Katie and Justine’s experiences were shared by others, some teachers, often with more expertise, found Debug-Its less meaningful for their learning.


One course activity that participants valued as both open-ended and accessible, enabling multiple points of entry, was the 10 Blocks activity. In Scratch, creators combine a series of instruction blocks to program a sprite (the object that performs the instructions), where each block has a different functionality (e.g., moving forward, rotating, or making a noise). In this activity, participants were invited to create a Scratch project using a set of 10 specific blocks. Individuals could produce very different projects that still satisfied the constraints of the activity. Depending on individual expertise and adventurousness, participants could create simple programs that fulfilled the requirements in straightforward ways, or more elaborate programs that fulfilled the requirements in novel, elegant, or unintuitive ways.


Examining the completed 10 Blocks projects and reflections of two teachers—a novice to Scratch and a more experienced participant—highlighted these qualities (see Figure 1). The first participant built a fairly simple project and relied on others in the community for help (discussed further in the next section). The second participant built a more elaborate program about a cat trying to get out of a box, featuring a cat complaining in rhyme, “Stuck in this box! Trying to think outside of it with only ten blocks!” Their design notebook reflections illustrated how the activity was flexible enough to enable each teacher to find unique, engaging challenges that were meaningful to their own learning. The constraint of only being able to use 10 specific blocks served simultaneously as a scaffold and as an added challenge. The more experienced participant described in her design notebook how the limited set of blocks kept her from relying on her “favorite tricks” to create the program and prompted her to think of new approaches. In contrast, other participants described how the very same constraints made the activity less daunting. For example, Martin, an elementary learning resources teacher, reflected on how being forced to use just 10 kinds of blocks made the process less “paralyzing”:


I have found often with classes I teach, and with myself, that giving some constraints can often be better for learning. An environment like Scratch has so many options that it can actually be paralyzing to try and make a choice. Fewer options means I can really think of inventive ways to use the blocks I had available and increased my understanding of them and of Scratch.


Figure 1. A novice and expert’s respective reflections in their design notebooks, accompanied by their 10 Blocks programs


[39_22103.htm_g/00001.jpg]


[39_22103.htm_g/00002.jpg]


[39_22103.htm_g/00003.jpg]


[39_22103.htm_g/00004.jpg]


Teachers also emphasized the importance of engaging as learners with the kinds of activities that their students would undertake. The opportunity to explore and reflect on such activities allowed teachers to empathize with students’ learning experiences and connect those insights to their classroom practices. For example, in the prior quote, Martin’s reflection on his own experience with the 10 Blocks activity begins with his connection to the kinds of learning experiences he designs for students. Another teacher similarly discussed how working through CCOW activities helped her realize that she underestimated the time required for the projects she assigned to students. In reflecting on the impact of doing the kinds of activities she asks of students, Layla, an elementary STEAM (Science Technology Engineering Arts Math) integrator, explained:


[This] is what we are asking our kids to do all the time. If we are not experiencing it as adults, there is a concern on my part that teachers will forget what it is like to be a student. I believe you need to be able to empathize with the learning process. So if you don’t try and learn something that is way out of your comfort zone or brand new to you, how are you going to empathize with a kid who is trying to learn something new?


These experiences resonate with sociocultural learning research, which emphasizes knowledge as situated within an activity in a specific environment rather than a decontextualized set of practices or understandings (Barab & Duffy, 2000). Teachers in CCOW found meaning through activities they saw as grounded in the daily tasks of their classroom and students.


It was important that these programming activities were accompanied by space for metacognition, enabling teachers to reflect on their thought processes and contemplate their students’ thought processes. Though there were exceptions, most teachers appreciated the course design notebooks as a place to think through their approach to solving problems and make connections to their teaching practice. The open-ended, flexible, and aligned nature of the activities and spaces for reflection were supportive of meaningful learning experiences and consistent with a conception of the learning process that involves doing, considering results, adjusting thinking, and then trying again (Koschmann, 2011).


PEERS


CCOW attracted participants from around the world, bringing together varied expertise, knowledge, and interests. A second theme that recurred in our conversations with teachers was how their actions and thinking were impacted by interactions with other participants in the course. Teachers discussed the ways in which they used feedback from others to advance their learning, benefited from peer encouragement in persisting with challenging work, and gained inspiration from the work and thinking of others. The importance of peers in professional learning experiences was not exclusive to CCOW; teachers’ descriptions of past experiences with professional learning frequently included enthusiastic stories about the role of colleagues and peers.


Teachers described the importance of feedback from peers as a way to both promote new thinking and provide support in enacting change. They frequently emphasized the diversity of peer knowledge and skill as an important foundation for helpful feedback. Drawing on positive past experiences with professional development, Stephanie, a middle school technology coordinator, described the disequilibrating effect of peer feedback on her learning:


When you were getting feedback from these 12 other people, they were just genuinely questioning. “What is your thought process here? Why did you go this way? Why didn’t you go that way? Have you thought of this?” That disequilibrium was really helpful … that state of constantly being off balance.… Because every time you thought you had it, it turned out there was always a way to improve.


For Stephanie, having peers who would interrogate her work helped her continually improve her own thinking and practice. This theme echoed across our conversations with teachers, who emphasized that peer feedback was particularly effective in helping to guide iterative cycles of creating, sharing, reflecting, and revising. Many teachers spoke of peer feedback in the context of exploring new kinds of instructional approaches, trying them out with students, and sharing some aspect of the process with colleagues.


With respect to CCOW, teachers described the value of exchanging feedback with others, but also identified challenges to giving and receiving substantive feedback—feedback that fostered new ways of thinking. This was often because there was less opportunity to engage with other participants’ work in a deep and sustained way over time. Kelly, a high school computer science teacher who particularly valued peer feedback in face-to-face courses, explained how the design notebooks (the public online artifacts where teachers shared their progress and reflections) did not allow for the kind of substantive and reciprocal feedback necessary to advance learning:


I wanted to make it meaningful feedback. [But] it was almost like a one-way conversation.… There wasn’t a back and forth, so it felt a little strange to make a comment, and you didn’t know exactly what to say. Sometimes it was hard to find something, not just, “Oh, that was good.”


Other teachers who reported more meaningful exchanges of feedback in CCOW sometimes acted beyond the course structures to support those interactions. For example, Sharon, an elementary computer lab manager, and a group of other participants decided to regularly gather for synchronous, online, video-based meetings to discuss one another’s work. Though no members of the research team witnessed these interactions, Sharon described how she was able to rely on this small group even after the course ended to provide her with feedback on her ongoing practice.


Still, others did highlight the value of receiving substantive feedback from peers in the CCOW online forums, but often described such exchanges in more affective terms. For example, Donna, a middle school technology teacher, credited encouragement from other participants as key to her success in the course:


I will say that what made me continue was the support from the other class members. I remember I posted a question … and someone wrote back and gave me my answer, like two seconds later, and it was one of those, “Wow, this is so great. There are so many other people doing this, and they might be just as lost as I am.” And that’s what made me want to continue. There was such a support system from the other participants in the class that I thought, “Well, if I don’t know how to do something, I’ll just ask, and I won’t look so stupid. Someone else might be in the same shoes as me.”


Donna’s comment highlights the encouragement that can result from even brief exchanges. By getting a positive response to her question, Donna recognized that asking introductory questions could benefit not only herself, but other participants who might be afraid to speak up.


Many interactions in CCOW seemed to be valuable for reasons that went beyond the specific information that participants shared with one another. Below, an extended exchange between Layla and Diane illustrates how they used the design notebook as a medium to give and receive feedback that was substantive but also layered with encouragement. Figure 2 presents a page from Layla’s design notebook; a transcription of their comments follows.


Figure 2. Layla planning her final project in her design notebook


[39_22103.htm_g/00005.jpg]


Diane: I love your idea of doing a workshop for seniors! Look forward to hearing more!

Layla: Thanks, Diane! If you have time, I’d be delighted, please, if you’d take a look at my planning document and provide feedback on any portion of it.

Diane: Layla, Wow! Loved reading through your plan. Thanks so much for sharing! I added a few comments.

Layla: Hi Diane, Thanks for stopping by and providing feedback. I saw your Dance activity suggestion and love it! Am thinking that could be the Self Spa activity for that session. ��

Diane: P.S. I’m including your project and beautiful notebook in my Week 5 Reflections.

Layla: Thank you! I’ve thoroughly enjoyed reading through your notebook, have added a link to it, and hope that we can collaborate in the fall!


In this example, in addition to the substantive feedback they provided on one another’s work, Diane and Layla also provided the kind of recognition and encouragement that play an important role in fostering enthusiasm and building relationships that may yield productive collaborations after the course ends.


Teachers also highlighted other types of interactions that were important to their learning. CCOW was designed to be open, with participants encouraged to share their work early and often; therefore, teachers had numerous opportunities to explore other participants’ work and thinking. For example, in their design notebooks, participants shared and described their activities, reflected on their progress, asked questions of others, and linked to their Scratch projects. Teachers described “looking under the hood” of their peers’ projects to explore the code, and “remixing” others’ programs by adding new code or adjusting existing code to create new programs.


Teachers described two primary benefits of having direct access to one another’s code. First, when stuck on a problem or task, looking at others’ programs and reflections served as scaffolding to help create projects that were otherwise beyond reach. For example, Donna initially found the Debug-It activities (in which participants fixed broken projects) extremely intimidating, reflecting that she had “no clue how to fix” the program, but eventually learned to use other people’s responses to support her problem solving. She commented that peers’ reflections were helpful because “not everybody fixed [the Debug-Its] exactly the same way” and allowed her to compare “other people’s thought processes” as they approached the same problem. Second, looking inside others’ projects enabled teachers to access techniques or approaches that might be useful in the future. For example, Stephanie emphasized the evolution in her thinking after exposure to different problem solving approaches:


[It] is interesting when you see people doing things that you’re really not going to do. And this happens to me quite a bit where I try to write it off in my head, but it ends up kind of rolling around in the background. And then it can generate something where you go, “Oh, I wouldn’t ever have thought of it if I hadn’t seen this person’s train of thought, which is going in a different direction than my train of thought. But maybe I can bring that on board.”


Stephanie’s experience reflects essential elements of conceptual change processes (Strike & Posner, 1985); exposure to a new approach may not influence an individual’s own practice until they find that this new approach offers an advantage to their existing strategies. Here we see how the work of peers, and not just their direct feedback, can support learners in reflecting upon their own work in comparison with others (Collins, Brown, & Newman, 1989) and serve as a starting point for new creation and exploration (Harel & Papert, 1991).


The peer interactions that teachers described as meaningful to their learning in CCOW involved feedback to support changes in practice, encouragement to persist with challenge and difficulty, and inspiration to think in new ways. We turn now to the ways in which those interactions were made possible by, and given meaning through, the culture of learning that developed in the course.


CULTURE


For some teachers, feelings of isolation initially prompted their participation in the online course, as they aspired to connect across institutional and geographic boundaries with others who shared their professional goals, values, and interests. Teachers emphasized that “cultural” aspects of CCOW served as a foundation for supporting these connections. Throughout our conversations, teachers shared a desire and appreciation for aspects of what has been termed an “integrated professional culture”—a culture in which it is safe for novices to admit to what they do not know and cannot yet do, and to receive support from more experienced and knowledgeable colleagues to persist in challenging and complex work (Kardos & Johnson, 2007). Elements of such a culture were described in the previous section, where interactions between participants around their learning could be interpreted as reflecting a sense of shared responsibility. The following section explores how culture-building practices, activities, and interactions supported teachers in their learning. Specifically, teachers described the importance of course structures and practices that helped them feel connected to the course community, valued experimentation in learning, and invited participatory engagement in the learning experience.


Teachers established connections through course structures that invited community building and interaction. For example, the first Scratch project that most teachers created was an interactive About Me program, which shared words and images that reflected something about themselves. Aamir, a middle school computer science teacher, described the role of this project in building connections among participants:


When I saw the About Me project, I said, “Oh, this is a good exercise for the concept of sequence, and for the concept of events in computational thinking.” But what is good about [it], I saw different About Me projects, and how they worked. You really see that people may express themselves in that project, and every project is different from the other.


Aamir characterized the act of sharing an About Me project as particularly meaningful for course participants—in one step, entering a community, engaging with others, and expressing a willingness to approach tasks in creative ways. He noted that the project enabled him to understand not only demographic information about participants, but also subtler aspects of their personalities, forms of expression, and approaches to programming that could support future collaborations and discussions.


Synchronous live events, such as office hours or “unfocus” groups (where participants formed groups to share project ideas) over Google Hangouts, also played an important role in helping participants feel more connected to one another. Many participants described these events as helpful in building a stronger community because they facilitated understanding of other participants’ experiences and goals, and served as opportunities to begin collaborative efforts. However, whereas some saw these live events as a useful motivating tool, other participants—particularly those with less experience with Scratch or online courses more generally—were very intimidated by these live hangouts and intentionally avoided them. One participant even chose to only audit the course specifically because she did not feel comfortable joining the live online events.


Teachers also emphasized the importance of a course culture that supported experimentation and embraced failure and mistakes as part of the learning process. Research has explored how difficult it can be to create such a culture of learning, especially in a large-scale environment, and the value of distinct course norms and structures in doing so (Nelson, Slavit, Perkins, & Hathorn, 2008). Some teachers highlighted that this culture allowed them to feel safe in taking risks and to worry less about their performance of knowledge to others in the course, either peers or facilitators. Others further expressed a desire to foster experimentation in their own classrooms and appreciated efforts by the CCOW facilitation team to model this type of learning culture. For example, Layla described how the CCOW facilitation team explicitly positioned themselves as learners alongside the teachers in the course, which in turn supported teachers in experimenting with the course activities and one another’s work:


The fact that there is not a right and a wrong comes with play. It is like you are learning by trying, you are learning by doing, by asking questions, by remixing, by looking at other people’s work.… The fact that even the experts … very much present themselves as peers and collaborators, not as like, “OK, we know it all and we are dumping it—doing a core dump on you, but we are learning along with you.” And that is huge, I think.


Teachers highlighted the importance of a facilitation team that embraced experimentation and participation by inviting CCOW participants to shape and improve the course experience in substantive ways. Broadly, the teachers we spoke with valued professional learning experiences in which they, and their ideas, were taken seriously by leaders or facilitators, and often complained of prior experiences in which they felt disempowered or undervalued by facilitators. In discussing the importance of the participatory culture of CCOW, many teachers referred to an incident early in the course, in which participants experienced difficulties with design notebook feedback. To discuss the problem and collaboratively develop solutions, the facilitation team engaged CCOW participants via online “office hours” (twice-weekly live video gatherings where participants could interact directly with course facilitators and one another). Sharon reflected on the experience:


In the beginning, there were a few bugs with the design notebooks—not being able to look at people’s design notebooks—and there was confusion about who you should be reviewing, and how, and they were making the changes. Not only were the facilitators of the class making changes, the participants were coming up with ideas.… I remember there was one participant—he had made a spreadsheet where everybody was putting their links in, so you could view other people’s notebooks. There was quite a bit of collaborative projects that people were kind of creating, and asking for you to participate, or review it, or remix it, or things of that—and I think that helped build the connections.


By establishing a tone of participation and openness to feedback, the facilitators turned a glitch that could have disrupted learning and sapped enthusiasm into an opportunity for participatory problem solving. Efforts by the CCOW facilitation team to position themselves as learners alongside course participants, to encourage experimentation, and to provide opportunities for participants to play active roles as legitimate problem-solvers took place in concert with the efforts of participants themselves, like the individual described by Sharon, who contributed by supporting fellow learners and taking ownership of the course.


RELEVANCE


The majority of teachers that we interviewed characterized professional learning experiences as often lacking relevance to their classroom interests and needs. These negative experiences were often associated with mandated PD; when afforded choice, teachers used their autonomy to select more personally and professionally meaningful learning experiences. Unsurprisingly, teachers reported deeper engagement with self-selected professional learning experiences, a finding that is consistent with research on the motivation that learners possess when setting their own goals and finding intrinsic value in a learning experience (Bransford et al., 2000; Lepper & Henderlong, 2000; Ryan & Deci, 2000).


All the teachers whom we interviewed had voluntarily enrolled in CCOW and had specific, practical needs that motivated their continued engagement with the course. Teachers emphasized that they were taking the course to learn about Scratch, the tool itself and/or its constructionist approach to learning. Some teachers were motivated by a recent major release of Scratch (Scratch 2.0), while others were motivated by specific learning needs for themselves or their schools. Many teachers, particularly those with less experience, intentionally sought out the course because of the structures and support it provided. For example, Donna had not been able to finish reading books about Scratch, and enrolled in CCOW because she “do[es] much better in an environment where [she has] a timeline and a deadline.”


In addition to the expected connections between CCOW activities and classroom practice, many teachers reported unexpectedly meaningful or (in the words of one teacher) “resonant” learning experiences in CCOW. Frequently, this resonance went beyond surface-level connections between professional learning and classroom work to deeper connections between values and approaches to teaching and learning. Laura, a sixth-grade math and science teacher, spoke of a desire to better support students’ independent thinking and action in non-Scratch and non-programming-related work based on her experiences as a learner in the course. Sharon was one of many teachers who described gaining unanticipated insights into teaching and learning from her experiences:


There was a lot more to CCOW than just—because, like I told you, initially I was doing it to learn about Scratch, but I ended up, which surprises me, I was getting teaching information; information on how to present, how to give the ideas, like all the ideas in design. Design thinking, and things of that nature, the whole learning cycle, spiral—these were things that were new to me, and that was great. It just got me thinking in a different way.


Often, the qualities of the course described as most meaningful by participants were unanticipated; thus, there were several moments in our interviews when participants discovered changes in their own thinking as they retraced their learning trajectory from their initial intentions. For example, Martin observed that he had been initially skeptical of the value of exploring others’ projects. But in reviewing a series of his own design notebook entries, he self-deprecatingly commented on the evolution of his thinking:


Obviously, I spent a lot of time exploring things after because my next several notes state, “I spent a lot of time exploring. I found a bunch of cool things that are good inspirations for making my final project.” So that [initial skepticism] was pretty ridiculous.


Martin’s thoughtful and honest reflection provides an important counterpoint to the strongly professed desire for choice in professional learning activities. As Martin’s experience shows, the value of choice in a learning experience is complicated by limitations in learner foresight.


Alignment between the CCOW content and the personal and professional goals of the teachers served as motivation to enroll, to persist in challenges, and to engage deeply in the course work. But unanticipated qualities of the learning experience also resonated with teachers, such as the opportunity to explore and engage with new approaches to teaching and learning, or what other researchers have termed “orientations,” an overarching way of thinking about teaching practice embedded in a professional development experience (Whitney & Friedrich, 2013).


DISCUSSION


In this study of a constructionist MOOC, we used in-depth interviews to examine what teachers found meaningful to their learning. As explored in the previous section, the qualities highlighted by this group of teachers—activity, peers, culture, and relevance—aligned with prior research on the character of meaningful learning and the conditions that invite it (Bransford et al., 2000; Lepper & Henderlong, 2000; Strike & Posner, 1985). We note this alignment not to dismiss the importance of the learning medium, but as evidence that it is helpful to begin from learning principles when exploring the affordances and challenges of a given medium. Moreover, the intersections and interrelationships between the qualities of meaningful learning (e.g., how the culture of learning in the course enabled peer interactions) reflect not only Shulman’s (2004) articulation of the nested nature of “enduring and effective” learning principles, but also the nature of online learning as an “activity system,” in which the “complex assemblage” of activities, participants, context, tools, and underlying principles is what matters (Haythornthwaite et al., 2007, para. 14). This conception of the learning environment as a space in which the qualities of meaningful learning interact with one another, with the learning environment, and with the needs of individual learners in complex and unpredictable ways lends itself to thinking about the design of professional development not in terms of discrete characteristics to be included, but in terms of heuristics to be continuously negotiated.


Our findings build on existing research about the affordances and challenges of supporting teacher learning in online spaces. We share the concerns of other researchers, noted in the literature review, that the research and design of professional development can become disconnected from an understanding of teacher learning. And in the excitement around MOOCs as a new medium for teacher learning, we recognize the potential for this disconnect to be further amplified. Thus, we ground discussion of our findings in the connections between CCOW participant experiences and research knowledge of learner experiences in online spaces.


First, we found that teachers valued opportunities to engage with and reflect on activities that were open-ended, accessible, and aligned with the kinds of learning experiences they were hoping to provide to their students. Research has shown that iterative processes of activity and reflection in online learning enable learners to think about things in new ways (Ukpokudo, 2008). In addition, the teachers with whom we spoke described how deeply this process was influenced by the other learners in the course who provided cognitive and affective support for new learning both through direct feedback and through the provision of models and inspiration. Others have noted the importance of peer interaction and feelings of connectedness in online environments, with their potential for loneliness and isolation, and the ways in which such social bonds enable the cognitive dimensions of learning (Booth, 2012; Farooq et al., 2007; Ukpokudo, 2008). As described by CCOW participants, a course culture that invited the building of such connections, as well as democratic forms of participation, helped make their learning possible. The role of community norms and values in learning and participation is both important in the online professional development space (Karagiorgi & Lymbouridou, 2009) and has been understudied relative to more straightforward instructional elements (Haythornthwaite et al., 2007). Finally, the importance of teachers’ learning in CCOW being relevant to their personal and professional goals reflects long-standing convictions in the promise of online learning spaces to more directly address individual learner passions and needs (Siemens, 2005). Nevertheless, teachers appreciated leaving the course with new ways of thinking about teaching and learning that they may not have intended to find, reflecting the way in which it can often be through the process of learning that a learner finds out what they want or need (Biesta, 2006).


Though these findings could be interpreted as confirming the consensus around characteristics of environments that invite meaningful learning experiences, we argue that reducing these findings to a list of discrete characteristics would obfuscate the ongoing negotiation and reflection that take place as designers make choices about how to realize learning principles in specific environments. Our in-depth, qualitative methodological approach illuminated that although the qualities teachers identified as meaningful were often mutually supporting, there were also underlying tensions—tensions that played out within and across the experiences of individual teachers, reflecting the diversity of individual experience; tensions that emerged in relation to the design intentions of CCOW, reflecting the interactions of design elements with one another; and tensions related to the technological affordances of a MOOC platform, reflecting the interactions of learner with environment. Here, we consider three specific tensions and their implications for the design of online teacher professional learning experiences: (a) autonomy, with structure, (b) diversity, with commonality, and (c) experimentation, with validation. While we offer specific examples of how CCOW designers sought to mitigate these tensions, we recognize them as ongoing issues to be negotiated, rather than resolved, in the design of learning experiences. We believe that an understanding of endemic tensions and heuristics for negotiating these tensions will be helpful, not only to those designing online professional development for teachers, but also to those designing online learning experiences more broadly (Barab et al., 2003).


TENSION 1: AUTONOMY, WITH STRUCTURE


Many teachers commented on their own tendency to be dismissive of mandated professional development. Teachers identified autonomy as essential to meaningful learning experiences—opportunities that encourage the pursuit of self-defined goals and respect individual learning needs and preferences. Online environments have enormous potential for supporting learner autonomy (Anagnostopoulos, Basmadijan, & McCrory, 2005; Jonassen, Davidson, Collins, Campbell, & Haag, 1995 Kay, Reimann, Diebold, & Kummerfeld, 2013) and CCOW participants emphasized the benefits of voluntary enrollment, optional and customizable course activities, and a final project where they could explore, in depth, a topic that was important to them.


At the same time, teachers acknowledged the value of structure for providing a sense of progress in their learning experiences. Many participants sought out the course because they wanted a more structured environment (e.g., Donna’s admission that she needed deadlines), whereas others initially doubted, but later came to value, structure (e.g., Martin’s initial skepticism and ultimate enthusiasm for exploring other people’s projects). Unsurprisingly, the ideal balance between autonomy (choice and customization) and structure (scaffolding that helped CCOW participants develop their abilities over time) varied from participant to participant. Further, we saw ways in which structure could enable autonomy, for example, when clarity about expectations, along with support, provided conditions for meaningful self-direction and expression.


In CCOW, the balance between autonomy and structure was implemented primarily through multiple learning pathways for participants. For example, a “goal” activity was identified for each week of the workshop. For those with greater expertise, jumping directly to the goal activity was encouraged. But for participants with less expertise, the goal activity was accompanied by a collection of optional starter activities to build greater familiarity and fluency with the ideas and practices of the week. This type of flexibility is certainly possible in face-to-face learning experiences, but the predominantly asynchronous interactions of MOOCs can enable greater freedom for learners, particularly along the dimension of time (i.e., those who need more time have the freedom to take it).


However, the presence of multiple pathways also increases navigational complexity. In CCOW, we struggled with the tension between serving a diverse audience and potentially creating an overwhelming number of choices. Our decision to build choice into the experience was appreciated by some learners but alienated others. In particular, two teachers noted that the number of choices could feel overwhelming, with one describing a feeling that they had to “do everything” and another noting how much they appreciated “a little bit of direction” rather than having to make constant decisions among open-ended choices. We also puzzled over whether participants’ choices were always relevant and meaningful, and how to provide them with support and direction when needed, especially with large numbers of learners. We found that one way to mitigate the tension between autonomy and structure was through regular communication with community members, both to invite feedback (e.g., asking participants to share their challenges through the CCOW discussion forums) and to offer further explanation of the choices (e.g., providing multiple examples of different types of final projects).


TENSION 2: DIVERSITY, WITH COMMONALITY


Teachers identified peers as a crucial part of their CCOW experiences; peers offered new ideas and problem solving approaches, as well as a sense of community with shared values, interests, and approaches to learning. Many teachers emphasized the benefit of having access to peers who were more advanced or had very different approaches to teaching or using Scratch. The work of these diverse peers served as scaffolding or inspiration that pushed teachers to think, create, and teach in new ways (e.g., Stephanie’s discussion of the valuable disequilibrium that occurred when receiving feedback from others). At the same time, many teachers highlighted the benefits of belonging to a community of teachers who had similar aspirations and interests, in contrast with their feelings of isolation at school. Shared values supported the exchange of feedback and encouragement among teachers.


Online learning environments offer rich opportunities to build connections among diverse participants, varying by geography, expertise, etc. But the low “signal” (i.e., the limited means for representing one’s self) in online environments (Donath & boyd, 2004) can make it difficult to support the kind of substantive, meaningful two-way interactions that help participants evolve their thinking and practice, as evidenced by Kelly’s frustrations with course feedback mechanisms. That is, even though the greater anonymity and invisibility of personal characteristics in the online environment can mitigate some individuals’ fears of contributing, they can also make it harder to build the sense of shared community that enables learning (Haythornthwaite et al., 2007). Accordingly, designers must take intentional steps to design mechanisms of connection that can sustain activity, reflection, and collaboration.


For example, we counteracted the limitations of large enrollments by designing opportunities for interactions among smaller groups of participants. But we often puzzled over the tension between diversity and commonality when grouping participants. There is a pragmatic appeal to grouping people by geography, curricular area, age of students they support, or expertise—but homophilic interactions, particularly related to expertise, are not always best for learning (Burt, 2004). At the same time, grouping people with very little in common presented different challenges. For example, when randomly pairing people to provide feedback on design notebooks, some participants felt anxiety about offering feedback to people they perceived as possessing greater expertise. Even with structured feedback templates, participants’ self-consciousness sometimes prevented exchanges among learners. We found that one way to mitigate the tension between diversity and commonality was to avoid remaining in any particular peer configuration for too long. By encouraging constantly shifting interactions among participants, teachers were able to ensure that participants engaged with a variety of similar and dissimilar people throughout the course.


TENSION 3: EXPERIMENTATION, WITH VALIDATION


Many of the CCOW learning activities were open-ended—activities that could not be assessed as right or wrong—and teachers repeatedly expressed appreciation for being freed from worrying about getting the “correct” answer. Teachers found it meaningful to engage in activities where there were multiple pathways for learning, particularly within a culture of learning that valued experimenting with ideas and making mistakes. Learners could spend the time they needed or wanted looking inside others’ projects, reading others’ reflections, and tinkering with work before sharing their creations or reflections with others.


Despite appreciating a release from the intellectual constraint and psychological pressure of needing to produce a correct answer, teachers still expressed a desire for some kind of external validation. Teachers described benefiting, both cognitively and emotionally, from receiving feedback on their work from peers, especially when this attention moved beyond superficial comments. Additionally, teachers described the importance of feeling recognized for their questions or ideas, and highlighted the value of timely and helpful responses from the facilitation team or other participants. The ways in which these feelings of validation seemed to enable openness and risk-taking in the learning environment echo research findings that have emphasized the importance of trust for learning in online communities, the relationship between feelings of trust and the development of substantive “knowledge sharing,” and the role that course facilitators play in creating that sense of trust (Booth, 2012). However, some teachers brought with them expectations of traditional forms of validation in learning experiences, such as grading or certification, which could conflict with the learning culture of CCOW.


Large-scale online learning environments present designers with a validation puzzle (D. M. Russell et al., 2013). The online environment offers the flexibility and privacy required for low-stakes tinkering with ideas and practices. But it becomes challenging to validate learners’ experimentation, as a way of supporting their development, as the number of participants grows. In early MOOCs, validation was often achieved by designing activities in such a way that they could be assessed as right or wrong (e.g., the green checkmark of edX). New forms of validation are actively being developed and explored, from models of large-scale peer assessment (Kulkarni, Bernstein, & Klemmer, 2015) to distributed/decentralized models of large community learning (Fisher, 2014).


Whatever the approach, the tension between experimentation and validation is one that needs to be carefully negotiated. Given the ease and frequency with which people drop out of online courses, validation can be an important factor for minimizing attrition. In CCOW, we employed two primary strategies for mitigating the tension between experimentation and validation. First, instead of relying on “expert” or facilitator feedback, we focused on peer feedback as validation. This approach was appealing because it was both scalable and modeled the type of peer learning that we hoped participants might bring to their own classrooms. Second, we highlighted, through whole-group emails or synchronous online “office hours” video sessions, interactions that had been identified by participants as particularly meaningful, exciting, or valuable. This type of highlighting seemed to satisfy (un)spoken desires for “expert” validation, while still encouraging a culture of experimentation and multiplicity of outcomes.


CONCLUSION


MOOCs have the potential to transform teacher learning, reaching large numbers of teachers while simultaneously responding to individual needs and desires. However, as we have demonstrated in this work, the novel aspects of this new medium have not obviated decades of research about how teachers learn and how to design learning environments for teachers; the qualities highlighted by CCOW participants as meaningful to their learning—activity, peers, culture, and relevance—are consistent with foundational findings in the field of teacher professional learning.


Yet we argue that these qualities can only be realized by negotiating the challenges and affordances of the medium, recognizing the individual variability of teachers as learners, and balancing what teachers want and need from the learning experience. Our findings illustrate that although the design of online professional learning experiences should initially be informed by learning theories and technological affordances, designs should be iteratively refined in response to experience and interaction—how designs are taken up, used, and appropriated by learners. Too often, professional learning experiences (independent of the medium) position teachers as consumers who need to be compliant with the desires and intentions of the designers or researchers. But, as de Certeau (1984) argued, designs are more complexly engaged by their “audience”:


It is nonetheless implicit in the “producers’” claim to “inform” the population, that is, to “give form” to social practices.… To assume that is to misunderstand the act of “consumption.” This misunderstanding assumes that “assimilating” necessarily means “becoming similar to” what one absorbs, and not “making something similar” to what one is, making it one’s own, appropriating or reappropriating it. (p. 166)


In this study, we have offered a detailed example of what this complex engagement can look like—describing the interplay between technology, learning theory, and participant experience and offering a set of heuristics to inform and inspire future designs. It is through attention to this interplay that researchers, designers, and teachers will unlock the potential of new (and old) mediums for learning, build on existing understandings of learning, and support meaningful learner experiences.


Acknowledgments


We would like to thank the TCR reviewers for their invaluable feedback. We would also like to express our sincere appreciation to all the Creative Computing Online Workshop participants, particularly those who were so generous with their time to respond to surveys and participate in conversations with members of the research team. This material is based upon work supported by Google, by the Scratch Foundation, and by the National Science Foundation under Grant No. DRL-1019396. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of Google, the Scratch Foundation, or the National Science Foundation.

 

References


Anagnostopoulos, D., Basmadjian, K., & McCrory, R. (2005). The decentered teacher and the construction of social space in the virtual classroom. Teachers College Record, 107(8), 1699–1729.


Ball, D. L., & Cohen, D. K. (1999). Developing practice, developing practitioners: Towards a practice-based theory of professional education. In L. Darling-Hammond & G. Sykes (Eds.), Teaching as the learning profession: handbook of policy and practice (pp. 3–32). San Francisco, CA: Jossey-Bass.


Barab, S. A., & Duffy, T. (2000). From practice fields to communities of practice. Theoretical Foundations of Learning Environments, 1(1), 25–55.


Barab, S. A., MaKinster, J. G., & Scheckler, R. (2003). Designing system dualities: Characterizing a web-supported professional development community. The Information Society, 19(3), 237–256.


Biesta, G. J. J. (2006). Beyond learning: Democratic education for a human future. Boulder, CO: Paradigm Publishers.


Bonk, C. J., & Cunningham, D. J. (1998). Searching for learner-centered, constructivist, and sociocultural components of collaborative educational learning tools. In Electronic collaborators: Learner-centered technologies for literacy, apprenticeship, and discourse (pp. 25–50). New York, NY: Erlbaum.


Booth, S. E. (2012). Cultivating knowledge sharing and trust in online communities for educators. Journal of Educational Computing Research, 47(1), 1–31.


Borko, H. (2004). Professional development and teacher learning: Mapping the terrain. Educational Researcher, 33(8), 3–15.


Borko, H., & Livingston, C. (1989). Cognition and improvisation: Differences in mathematics instruction by expert and novice teachers. American Educational Research Journal, 26(4), 473–498.


Boyatzis, R. E. (1998). Transforming qualitative information: Thematic analysis and code development. Thousand Oaks, CA: SAGE.


Bransford, J., Brown, A. L., Cocking, R. R., Donovan, M. S., & Pellegrino, J. W. (2000). How people learn: Brain, mind, experience and school (Rev. ed.). Washington, DC: National Academies Press.


Brennan, K. (2013). Learning computing through creating and connecting. Computer, 46(9), 52–59. doi:10.1109/MC.2013.229


Brennan, K. (2015). Beyond technocentrism: Supporting constructionism in the classroom. Constructivist Foundations, 10(3), 289–296.


Carey, S. (1999). Knowledge acquisition: Enrichment or conceptual change? In E. Margolis & S. Laurence (Eds.), Concepts: core readings (pp. 459–487). Cambridge, MA: MIT Press.


Chinn, C. A., & Brewer, W. F. (1993). The role of anomalous data in knowledge acquisition: A theoretical framework and implications for science instruction. Review of Educational Research, 63(1), 1–49.


Collins, A., Brown, J. S., & Newman, S. E. (1989). Cognitive apprenticeship: Teaching the crafts of reading, writing, and mathematics. In L. B. Resnick (Ed.), Knowing, learning, and instruction: Essays in honor of Robert Glaser (pp. 453–494). Hillsdale, NJ: Erlbaum.


Cuban, L. (1986). Teachers and machines: The classroom use of technology since 1920. New York, NY: Teachers College Press.


de Certeau, M. (1984). The practice of everyday life. Berkeley: University of California Press.


DeBoer, J., Ho, A. D., Stump, G. S., & Breslow, L. (2014). Changing “course”: Reconceptualizing educational variables for massive open online courses. Educational Researcher, 43(2), 74–84.


Dede, C., Ketelhut, D. J., Whitehouse, P., Breit, L., & McCloskey, E. M. (2009). A research agenda for online teacher professional development. Journal of Teacher Education, 60(1), 819.


Desimone, L. M. (2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher, 38(3), 181–199.


Desimone, L. M., Porter, A. C., Garet, M. S., Yoon, K. S., & Birman, B. F. (2002). Effects of professional development on teachers’ instruction: Results from a three-year longitudinal study. Educational Evaluation and Policy Analysis, 24(2), 81–112.


diSessa, A. A. (2006). A history of conceptual change research: Threads and fault lines. Cambridge, UK: Cambridge University Press.


Donath, J., & boyd, d. (2004). Public displays of connection. BT Technology Journal, 22(4), 71–82.


Farooq, U., Schank, P., Harris, A., Fusco, J., Schlager, M. (2007). Sustaining a community computing infrastructure for teacher professional development: A case study of designing tapped in. Computer Supported Cooperative Work, 16(4), 397–429.


Fisher, W. W. (2014). HLS1X: CopyrightX course report. HarvardX Working Paper Series No. 5. Retrieved from http://ssrn.com/abstract=2382332


Fishman, B., Konstantopoulos, S., Kubitskey, B. W., Vath, R., Park, G., Johnson, H., & Edelson, D. (2014). The future of professional development will be designed, not discovered: Response to Moon, Passmore, Reiser, and Michaels, “Beyond comparisons of online versus face-to-face PD.” Journal of Teacher Education, 65(3), 261264.


Garet, M. S., Cronen, S., Eaton, M., Kurki, A., Ludwig, M. Jones, W., … Sztejnberg, L. (2008, September). The impact of two professional development interventions on early reading instruction and achievement (NCEE 2008-4030). Washington, DC: National Center for Education Evaluation and Regional Assistance.


Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915945.


Garet, M. S., Wayne, A. J., Stancavage, F., Taylor, J., Eaton, M., Walters, K., … Doolittle, F. (2011, May). Middle school mathematics professional development impact study: Findings after the second year of implementation (NCEE 2011-4024). Washington, DC: National Center for Education Evaluation and Regional Assistance.


Harel, I., & Papert, S. (1991). Constructionism. Cambridge, MA: Ablex.

Haythornthwaite, C., Bruce, B. C., Andrews, R., Kazmer, M. M., Montague, R., & Preston, C. (2007). Theories and models of and for online learning. First Monday, 12(8). http://dx.doi.org/10.5210/fm.v12i8.1976


Hill, H. C., Beisiegel, M., & Jacob, R. (2013). Professional development research consensus, crossroads, and challenges. Educational Researcher, 42(9), 476–487.


Jonassen, D., Davidson, M., Collins, M., Campbell, J., & Haag, B. B. (1995). Constructivism and computer-mediated communication in distance education. American Journal of Distance Education, 9(2), 7–26.


Karagiorgi, Y., & Lymbouridou, C. (2009). The story of an online teacher community in Cyprus. Professional Development in Education, 35(1), 119–138.


Kardos, S. M., & Johnson, S. M. (2007). On their own and presumed expert: New teachers’ experience with their colleagues. Teachers College Record, 109(9), 2083–2106.


Kay, J., Reimann, P., Diebold, E., & Kummerfeld, B. (2013). MOOCs: So many learners, so much potential... IEEE Intelligent Systems, 28(3), 70–77.


Kennedy, M. (1998, December). Form and substance in inservice teacher education (Research Monograph No. 13). Madison, WI: National Institute for Science Education.


Koschmann, T. (2011). Theorizing practice. In T. Koschmann (Ed.), Theories of learning and studies of instructional practice (pp. 3–17). New York, NY: Springer.


Kulkarni, C., Bernstein, M. S., & Klemmer, S. (2015). PeerStudio: Rapid peer feedback emphasizes revision and improves performance. In Proceedings of the Second (2015) ACM Conference on Learning @ Scale (pp. 75–84). https://doi.org/10.1145/2724660.2724670


Kvale, S. (1996). InterViews: An introduction to qualitative research interviewing. Thousand Oaks, CA: SAGE Publications.


Labaree, D. F. (1998). Educational researchers: Living with a lesser form of knowledge. Educational Researcher, 27(8), 4–12.


Lampert, M. (2001). Teaching problems and the problems of teaching. New Haven, CT: Yale University Press.


Lepper, M. R., & Henderlong, J. (2000). Turning “play” into “work” and “work” into “play”: 25 years of research on intrinsic versus extrinsic motivation. In C. Sansone & J. Harackiewicz (Eds.), Intrinsic and extrinsic motivation: The search for optimal motivation and performance (pp. 257–307). San Diego, CA: Academic Press.


Luttrell, W. (2010). Reflexive writing exercises. In W. Luttrell (Ed.), Qualitative educational research: Readings in reflexive methodology and transformative practice (pp. 469480). New York, NY: Routledge.


Maxwell, J. (2010). Validity: How might you be wrong? In W. Luttrell (Ed.), Qualitative educational research: Readings in reflexive methodology and transformative practice (pp. 279–287). New York, NY: Routledge.


Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2009). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. Washington, DC: U.S. Department of Education.


Nelson, T., Slavit, D., Perkins, M., & Hathorn, T. (2008). A culture of collaborative inquiry: Learning to develop and support professional learning communities. Teachers College Record, 110(6), 1269–1303.


Penuel, W. R., Fishman, B. J., Yamaguchi, R., & Gallagher, L. P. (2007). What makes professional development effective? Strategies that foster curriculum implementation. American Educational Research Journal, 44(4), 921–958.


Putnam, R. T., & Borko, H. (2000). What do new views of knowledge and thinking have to say about research on teacher learning? Educational Researcher, 29(1), 4–15.


Roskos, K., Jarosewich, T., Lenhart, L., & Collins, L. (2007). Design of online teacher professional development in a statewide Reading First professional development system. Internet and Higher Education, 10, 173–183.


Rudestam, K. E., & Schoenholtz-Read, J. (2010). The flourishing of adult online education: An overview. In K. E. Rudestam & J. Schoenholtz-Read (Eds.), Handbook of online learning (pp. 1–28). Thousand Oaks, CA: Sage Publications.


Russell, D. M., Klemmer, S., Fox, A., Latulipe, C., Duneier, M., & Losh, E. (2013). Will massive online open courses (MOOCs) change education? In CHI’13 Extended Abstracts on Human Factors in Computing Systems (pp. 2395–2398). https://doi.org/10.1145/2468356.2468783


Russell, M., Carey, R., Kleiman, G., & Venable, J. D. (2009). Face-to-face and online professional development for mathematics teachers: A comparative study. Journal of Asynchronous Learning Networks, 13(2), 7187.


Ryan, R. M., & Deci, E. L. (2000). Self–determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68–78.


Scott, J. C. (1998). Seeing like a state: How certain schemes to improve the human condition have failed. New Haven, CT: Yale University Press.


Selwyn, N. (2011). Education and technology: Key issues and debates. New York, NY: Continuum.


Siemens, G. (2005). Connectivism: A learning theory for the digital age. Retrieved from http://www.itdl.org/journal/jan_05/article01.htm.


Shulman, L. S. (1983). Autonomy and obligation: The remote control of teaching. In L. S. Shulman & G. Sykes (Eds.), Handbook of teaching and policy (pp. 484–504). New York, NY: Longman.


Shulman, L. S. (2004). The wisdom of practice: Essays on teaching, learning, and learning to teach. San Francisco, CA: Jossey-Bass.


Strike, K. A., & Posner, G. J. (1985). A conceptual change view of learning and understanding. In L. H. T. West & A. L. Pines (Eds.), Cognitive structure and conceptual change (pp. 211–231). New York, NY: Academic Press.


Ukpokudo, O. N. (2008). Teachers’ reflections on pedagogies that enhance learning in an online course on teaching for equity and social justice. Journal of Interactive Online Learning, 7(5), 227–255.


Whitehouse, P. (2011). Networked teacher professional development: The case of Globaloria. Journal of Interactive Learning Research, 22(1), 139–165.


Whitney, A. E., & Friedrich, L. (2013). Orientations for the teaching of writing: A legacy of the National Writing Project. Teachers College Record, 115(7), 1–37.


Willig, C. (2008). Introducing qualitative research in psychology. New York, NY: McGraw Hill.


Yurkofsky, M., Blum-Smith, S., & Brennan, K. (2016). Expanding outcomes: Exploring varied forms of teacher learning in an online professional development experience. In C. K. Looi, J. L. Polman, U. Cress, & P. Reimann (Eds.), Transforming learning, empowering learners: The international conference of the learning sciences (ICLS) 2016, Volume 1 (pp. 330–337). Singapore: International Society of the Learning Sciences.

 

APPENDIX

CCOW PARTICIPANT INTERVIEW PROTOCOL


PART 1. The Individual as a Teacher and Learner

1.

Tell us about your current role and position.


Part 1a. As a learner

2.

Tell us about a time when you remember an experience of learning something that really stayed with you over time. What made it stick with you?


3.

Tell us about a time when you remember an experience of learning about teaching that stayed with you over time, where you were able to use what you learned to inform your teaching practice or approach to teaching? What do you think it was about that experience that made it so meaningful to you? What about it made it easier to apply to your teaching practice?


4.

To what extent have the learning experiences you have had as a teacher reflected these qualities? Do you notice any patterns in when or where they have or have not been present?


5.

If you got to design learning experiences for teachers like yourself, what qualities or characteristics would you find important? What would it be about? What would it look like? What would it enable you to do?


Part 1b. As a teacher

6.

Could you walk me through a typical lesson and your reasoning behind the choices you make?


7.

Did you feel as though there are barriers, either within yourself or the larger school/district environment that get in the way of you having the classroom you might have wanted to have?


PART 2. The Individual Encounters the Experience


Contextual Introduction: Now we want to ask you about your experiences with the Creative Computing Online Workshop. Before we begin, we want to emphasize that we are genuinely trying to understand what did and didn’t work for participants. We have no stake in your responses, so please do not worry about being negative or critical. It’s also okay to say you don’t know, don’t remember, don’t have an opinion.


1.

To begin, do you remember what brought you to CCOW? How did you learn about CCOW? What made you decide to enroll and continue with the program? What did you hope to take away from the learning experience?


2.

How would you describe CCOW to someone else? What stands out to you about CCOW? What should people considering enrolling in CCOW know?


3.

What kinds of activities did you do the most? What did you spend time on during your participation (e.g., watching activity overviews, exploring Scratch on your own, spending time in the discussion forums)?


4.

What activities were most meaningful, or felt most connected to what you cared about or were trying to accomplish? Describe one or two experiences from the workshop that were most meaningful to you (e.g., the discussion boards, the video lectures, the examples, the journal, the final project, or revising other people’s work).


5.

What was not meaningful?


6.

Were there times during the workshop that you felt uncomfortable, with either what you were being asked to do, or how? By uncomfortable, we do not necessarily mean bad, we are interested in whether there were times you were asked to do things in a particular way that wasn’t familiar, intuitive, or in keeping with your typical ways of doing things. Was this a familiar experience? Did any of these feelings change over the course of the workshop? Were there kinds of thinking and/or practices that you got more or less comfortable with over time?


7.

Did you do a final project? If you did, describe your final project to us. How did you decide on a topic? Why was it important to you? Have you used this project since CCOW? Are you happy you chose that topic/approach?


8.

Have you continued interacting with CCOW in any way (e.g., revisiting website, maintaining contact with participants)? Have those experiences influenced your mindset or practice?


9.

Did the fact that this was an online course influence how you interacted with it? It might be helpful to think about CCOW in comparison with face-to-face PD experiences. In what ways did it differ? Were those ways for better or for worse in terms of your learning and/or experience? What did you notice about the online experience? How might it have impacted how you spent or organized your time (e.g., how much of the workshop you chose to do)? Did it influence the kinds of activities you did or the ways you interacted with people?


10.

How was CCOW similar to or different from other online PD experiences you’ve had? How was the content different? How did learning about Scratch influence your learning experience? How did the structure or organization influence the kinds of activities you did or the ways you interacted with people?


PART 3. The Experience Changes the Individual


Contextual Introduction: In the final section of the interview, we want to ask you about the influence CCOW may or may not have had on your current approach towards teaching. Again, we are genuinely trying to understand what did and did not influence your practice. We have no stake in your responses, so please do not worry about being negative or critical. It can also be hard sometimes to identify how things have changed over time for ourselves. Please feel free to say that you aren’t sure or don’t know in response to any question.


1.

What do you feel like you learned from CCOW? How did [a particular knowledge, skill, mindset] influence your classroom practice? If it did not, what might have gotten in the way of implementing that learning in the ways you wanted to? What about the CCOW experience led to each of these changes? Was this what you expected?


2.

Do you feel as though your approach to your own learning about teaching has changed following your participation in CCOW (e.g., your desire to try out new kinds of lessons, tinkering/experimenting with new teaching ideas, changing how you use colleagues to support this process)?


3.

[Discussion of artifact that represents current approach to teaching. If no artifact, just skip question.] Tell us about how this artifact is reflective of you as a teacher, how you think about your work, or your current approach to instruction. How did you end up with this approach to teaching? In what important ways have you changed?


Cite This Article as: Teachers College Record Volume 120 Number 9, 2018, p. 1-48
http://www.tcrecord.org/library ID Number: 22103, Date Accessed: 11/17/2018 6:54:15 PM

Article Tools

Related Articles

Catch the latest video from AfterEd, the new video channel from the EdLab at Teachers College.
Global education news of the week in brief.; NCLB; international education; software; This episode explores ten interesting and little known facts about Social Studies.; social studies; humor; media; research; schools; Three seniors at Heritage High School talk about education and what the next President should do about it.; Debates; Heritage High School; NCLB; NYC schools; education; election; girls; interview; politics; presidential election; schools; speak out; students; testing; EdWorthy Theater starring MIT Physics Professor Professor Walter Lewin.; MIT; physics; We feature new content about the future of education. Put us on your website ­ whether you're a student, teacher, or educational institution, we aim to create great content that will entertain and enlighten your audience. http://link.brightcove.com/services/link/bcpid1078591423http://www.brightcove.com/channel.jsp?channel=1079000717

Site License Agreement    
 Get statistics in Counter format