|
|
|
|
A Randomized Controlled Trial of Professional Development for Interdisciplinary Civic Education: Impacts on Humanities Teachers and Their Studentsby Dennis J. Barr, Beth Boulay, Robert L. Selman, Rachel McCormick, Ethan Lowenstein, Beth Gamse, Melinda Fine & M. Brielle Leonard - 2015 Background/Context: Billions of dollars are spent annually on professional development (PD) for educators, yet few randomized controlled trials (RCT) have demonstrated the ultimate impact PD has on student learning. Further, while policymakers and others speak to the role schools should play in developing students’ civic awareness, RCTs of PD designed to foster civic learning are rare. This randomized controlled trial contributes to the knowledge base on the effectiveness of PD designed to integrate civic learning, ethical reflection, and historical thinking skills into high school humanities courses. Focus of Study: The study examined the impact of a PD intervention in two areas: (a) teacher self-efficacy, burnout, and professional engagement and satisfaction; and (b) the academic, civic, social, and ethical competencies of 9th and 10th grade students in the teachers’ classes. Population/Participants/Subjects: The study involved 113 teachers and 1,371 9th and 10th grade students in 60 high schools from eight metropolitan regions in the United States. Intervention/Program/Practice: The intervention, Facing History and Ourselves, provides PD through a five-day seminar, curricular materials, and follow-up coaching and workshops to help teachers develop their capacities to implement an interdisciplinary historical case study unit using student-centered pedagogy. Research Design: The study used a school-level, randomized, experimental design to investigate impacts of the intervention for teachers and their 9th and 10th grade students. Findings/Results: Intervention teachers showed significantly greater self-efficacy in all eight assessed domains, more positive perceptions of professional support, satisfaction and growth, and greater personal accomplishment. Intervention students demonstrated stronger skills for analyzing evidence, agency, and cause and effect on an historical understanding performance measure; greater self-reported civic efficacy and tolerance for others with different views; and more positive perceptions of the classroom climate and the opportunities afforded for engaging with civic matters. Fidelity analysis found these causal effects despite the fact that roughly half of the intervention teachers did not fully implement the program. Conclusions/Recommendations: Educators need evidence-based approaches for teaching complex social, civic, and political issues enabling students of diverse mindsets and backgrounds to engage constructively with one another while obtaining necessary skills and knowledge. These findings provide empirical support for a professional development approach that engages teachers in fostering academic and civic competencies critical to both participation in a democracy and success in college and career. Civic education is an essential, though often marginalized, component of educating Americas youth. Our countrys commitments to the democratic ideals of equality, accountability, public deliberation, and a political culture based on shared values all require widespread civic competencies among its members (Gould et al., 2009; Youniss & Levine, 2009). Fundamental civic skills and dispositionssuch as the abilities to engage in public discourse, cooperate, respect the rights of others, and solve problems with people from diverse backgrounds or with different beliefsare also competencies essential for success in higher education and the workplace (The National Task Force on Civic Learning and Democratic Engagement, 2012; Partnership for 21st Century Skills, 2006; Torney-Purta, 2009.) Further, there is evidence that civic learning and engagement are positively related to academic progress and the likelihood of graduating from college (Davila & Mora, 2007; Finlay & Flanagan, 2009; Levine, 2009). Classroom-based civic education at the secondary level usually occurs in humanities courses, most frequently history or social studies and, less often, language arts (Hepburn, 2000; Levstik & Tyson, 2008). To be effective, educators in these subject areas need high-quality preservice and in-service professional development in civic education to develop the relevant pedagogical content knowledge (Shulman, 1986), and self-efficacy to employ effective instructional practices that foster civic learning (Bickmore, 2005; Gould et al., 2009; Hess & Zola, 2013; Levstik & Tyson, 2008; Torney-Purta, Richardson, & Barber, 2005; Vontz & Leming, 2005). In actuality, humanities teachers rarely receive the preservice or in-service professional development needed to foster their effectiveness at integrating civic learning with subject-specific goals and practices (Levstik & Tyson, 2008; Schwartz, 2008; Vontz & Leming, 2005). Successful integration of civic education within an academic discipline, such as history, requires that teachers understand the potential promises and pitfalls of interdisciplinary education (Boix-Mansilla, 2000). For example, a teacher might use the history of pivotal past events and processes that have shaped our nation (e.g., the Civil Rights Movement) to help students understand the civic controversies and conflicts of today (e.g., current debates over whether or not the Voting Rights Act of 1965 is still necessary). This approach holds great promise: student interest in history and current events is engaged, and their ability to understand both past and present is enhanced. However, if not well executed, students can make simplistic connections, not developing critical historical thinking skills, such as the ability to analyze evidence, cause and effect, and agency in historical contexts (Seixas, 2000; Wineberg, 2000). Can professional development approaches help teachers to maximize the potential of using history to foster students academic, social, ethical, and civic competencies while minimizing distortions of the past? This paper describes the results of a randomized controlled trial assessing the impact of professional development designed to help humanities teachers effectively integrate civic education in their humanities courses to enhance both discipline-based and civic learning outcomes.1 The research examines teacher outcomes hypothesized as critical to the effective classroom implementation of the approach and student outcomes hypothesized to improve as a result, including civic skills, dispositions and behaviors, historical thinking skills, and social and ethical reflection. Although many of the core features of effective professional development have been identified for civic education (Bickmore, 2005; Hess & Zola, 2013; Vontz & Leming, 2005), math, science, and language arts (Desimone, 2011; Ingvarson, Meiers, & Beavis, 2005; Yoon, Duncan, Lee, Scarloss, & Shapley, 2007), relatively few studies have provided rigorous evidence of the ultimate impact on students of professional development for teachers. A recent review of 1,345 studies of professional development in math, science, and language arts conducted between 1986 and 2006 identified nine studies meeting the What Works Clearinghouse standards of evidence, all of which focused on elementary schools (Gurskey & Yoon, 2009; Yoon et al., 2007). Since 2006, the Institute of Education Sciences of the U.S. Department of Education has funded many studies of professional development and student outcomes, and some encouraging findings have emerged on literacy at both the elementary (Jones, Hoglund, Brown, & Aber, 2010) and secondary levels (Kim, Olson, Scarcella, Kramer, & Pearson, 2011; Matsumura, Garnier, & Spybrook, 2012; Slavin, Cheung, Groff, & Lake, 2008). Yet, relatively little is known about how the intent to provide professional development for humanities teachers at the secondary level ultimately impacts students. This is the first randomized study to examine the impact on both teachers and students of professional development for interdisciplinary civic learning at the secondary level. THE PROFESSIONAL DEVELOPMENT MODEL THAT WAS EVALUATED Facing History and Ourselves (hereafter referred to as Facing History) is a nonprofit teacher professional development organization whose mission is to help teachers engage students of diverse backgrounds in an examination of racism, prejudice, and antisemitism in order to promote the development of a more humane and informed citizenry (Facing History and Ourselves, 2012). Facing History provides an educational model that supports secondary teachers and students in making connections between social history, norms of human behavior, and the civic and moral choices confronted in their own daily lives. Facing Historys approach to professional development is designed to facilitate teachers capacities to integrate three key elements of educating for informed civic reflection into their classroom pedagogy and goals. First, teachers learn to engage and foster students civic skills and dispositions, historical thinking skills, and social and ethical reflection (Barr & Facing History and Ourselves, 2010; Selman & Barr, 2009; Selman & Kwok, 2010). Second, teachers learn to use the student-centered pedagogy and content of the approach to improve classroom climate; promote academic discussion; and increase students cognitive, ethical, and emotional engagement to optimize student learning and development. Third, Facing History helps teachers to use an in-depth case study approach to examine periods in history when civil society has broken down and collective violence has erupted. Throughout, the approach relies upon examples of individual and collective efforts to preserve and strengthen civil society during times of unrest. The program employs a sequence of study and methods described below in more detail. SCOPE AND SEQUENCE Facing Historys approach involves a sequence of interdisciplinary humanities studies beginning with an examination of psychosocial identityfirst individual identity and then group and national identities with their definitions of membership and dynamics of identity-based labeling, stereotyping, prejudice, and discrimination, as well as intergroup understanding and respect (Strom, 2004). Next, students and their teachers examine an historical case study in depth. For this valuation, the historical content involved the failure of democracy in pre-World War II Germany and, specifically, the steps leading up to the Holocaust. The case study examines hatred, racism, antisemitism, and examples of courage, care, and compassion. Students explore difficult questions about moral judgment in times of collective violence, the roles of both individual and societal memory, historical legacies, and implications of the history for their own, current social and civic participation. One central goal is for students to discover that historical events are not inevitable and that preventing injustice and preserving democracy requires citizens to be informed, ethically reflective, and active participants. Program materials provide language and vocabulary as tools for entry into the exploration of human behaviorterms like perpetrator, victim, defender, bystander, opportunist, rescuer, and upstanderthat can help students understand complicated social history and connect the lessons of that history to their own lives and current events. The Facing History pedagogy for engaging with this content emphasizes establishing a classroom climate characterized by intellectual, emotional, and ethical engagement and interpersonal trust and respect. Instructional methods emphasize reflection, interaction, cooperation, deliberation, and discussion of complex social and civic issues. The content and methods jointly emphasize gaining in-depth understanding of historical processes and events, making personal connections to the subject matter, and linking the past to current social and civic issues. The approach contrasts with humanities courses that emphasize teaching factual information and de-emphasize drawing connections between the past and the present, or the past and ones own life (Leming, Ellington, & Schug, 2006). PROFESSIONAL DEVELOPMENT ACTIVITIES Though not prescriptive, Facing History provides educators with professional development through seminars and workshops, ongoing coaching and mentoring, and access to print and digital resources with content and strategies in support of the programs scope and sequence and their own ongoing professional growth. Seminars are typically five days long (3540 hours) and allow participants to engage with the content, themes, practices, and key resources of the program and to learn how to apply them in their own classrooms. Teachers are provided with a resource book for each historical case with materials and activities from which to choose. The resource book used in the study described here, Facing History and Ourselves: Holocaust and Human Behavior, includes varied primary and secondary source documents and other readings drawn from art, social psychology, biography, memoirs, etc. organized to support the sequence of study and the suggested teaching methods (see Sleeper & Stern Strom, 2006). Program staff members provide at least one hour-long initial follow-up meeting to help teachers plan their implementation. Additional ongoing coaching, print and digital resources, modeling of lessons, and guest speakers are provided as requested to help teachers refine implementation and address challenges that may arise. THE THEORY OF CHANGE GUIDING THE EVALUATION This evaluation of Facing History was guided by a theory of change informed by scholarship related to teacher professional development; adolescent historical understanding; and moral, social, and civic growth. The study was also informed by program theory as well as findings and measures from previous evaluations of the program. The theory of change guiding this research links professional development activities, proximal teacher outcomes, intermediate classroom outcomes, and longer term student academic learning and development. Figure 1. Theory of change for research on Facing History
THEORY AND EVIDENCE RELATED TO TEACHER PROFESSIONAL DEVELOPMENT Over the past decade, researchers have made significant progress in identifying the critical features of effective professional development, even if the causal relationships have not been fully established (Borko, 2004; Darling-Hammond, Wei, Andree, Richardson, & Orphanos, 2009; Desimone, 2009; Ingvarson et al., 2005; Sykes, 1996; Wei, Darling-Hammond, & Adamson, 2010; Yoon et al., 2007). Although researchers may use different terminology to describe these features or emphasize the importance of different components of successful models, five areas are generally recognized as critical to effective professional development: " content-focus: addressing the specific challenges related to teaching and learning specific academic subject matter (Desimone, 2011); " active learning: providing the opportunity to get involved as opposed to passively sitting through lectures (Hochberg & Desimone, 2010); " coherence: ensuring that the focus of the professional learning is connected to the demands teachers face within their schools and is adaptable to their context (Anderson & Herr, 2011; Gurskey & Yoon, 2009); " sufficient duration: intensive and ongoing professional development activities consisting of 30100 hours over a 12-month period (Yoon et al., 2007); " collective participation: engaging educators in strong professional learning communities that include feedback from a coach or supporting teacher (Desimone, 2011). For humanities teachers aiming to foster civic competencies, it is especially important that professional training incorporate ample opportunity to engage with professional colleagues in discussions about complex, controversial, and sensitive civic, moral, and political issues as they will with their students (Bickmore, 2005; Hess & Zola, 2013). Facing History professional development includes these five characteristics (Barr & Bardige, 2013) and two additional emphases. The first draws from adolescent psychosocial and moral development theory and research to enhance teachers knowledge (Erikson, 1950; Gilligan & Lyons, 1989; Kohlberg, 1984; Selman, 2003). The second stresses teachers multifacted growth in self-efficacy, reflecting research on the relationships between teacher beliefs about effectiveness and ability to engage in complex teaching tasks, persevere in the face of teaching challenges, reflect on and modify instructional practices, and, ultimately, promote student motivation and academic achievement (Henson, 2001; Ross & Bruce, 2007; Ross & Gray, 2007; Woolfolk-Hoy & Davis, 2006). Self-efficacy beliefs are also positively correlated with teacher professional engagement and satisfaction, and negatively related to teacher burnout (Skaalvik & Skaalvik, 2007). Prior to this study, evaluations of Facing History professional development had demonstrated positive impacts on teacher self-efficacy and perceptions of professional support, engagement, growth, and satisfaction (for example Barr, 2003; Frey & Barr, 2004; Romer, 2006; Sechser & Barr, 2005). These factors are, therefore, hypothesized to be important links between Facing History professional development and changes in instruction that ultimately promote student learning achievement and development. THEORY AND EVIDENCE RELATED TO ADOLESCENT DEVELOPMENT Adolescence is a period of rapid and significant development in capacities for self-reflection, social awareness, and reflective judgment (Kuhn, Wang, & Li, 2011; Selman & Bellino, 2012). At the same time, adolescents have heightened interest and concern about personal and social identity, belonging, and their role in and responsibility to others in society (Erikson, 1950; Gilligan & Lyons, 1989; Kohlberg, 1984; Youniss, 1980). To deeply engage in learning, adolescents need to see the relevance of their studies to their lives (National Research Council, 2003; Toshalis & Nakkula, 2012) and to experience social trust and belonging in the classroom (Farrington et al., 2012). Such engagement is essential for teens to attach to school, which is, in turn, an essential factor for school success and later civic engagement (Cohen, Pickeral, & Levine, 2010; Flanagan, Stoppa, Syversten, & Stout, 2010). Successful pedagogies for teens, then, whether learning history or developing social, moral, and civic capacities, should be student-centered and involve self-reflection, cooperative interaction with peers, and active participation in meaningful discussions (Bickmore, 2005; Gould et al., 2009; Hahn, 1994; Pianta & Allen, 2008; Solomon, Watson, & Battistich, 2002). Previous research on the Facing History approach has shown that it helps teachers create classroom climates that engage students in grappling personally and publicly in discussions about complex civic and moral issues from both the past and present (Bardige, 1983; Barr et al., 1998; Fine, 1995).2 The theory of change guiding this research, then, posits a positive classroom climate as an important program outcome believed to emerge when teachers successfully use content and methods that engage students in the ways described above. Finally, the theory of change hypothesizes that full implementation of Facing History content and methodology increases students historical understanding, ethical awareness, and civic competencies, the components of the development of informed civic reflection. Facing History targets historical thinking skills believed essential to thoughtful civic engagement, including the abilities to think critically about evidence, cause and effect, and the agency of people in the past as well as a healthy skepticism of simplistic explanations for complex events, whether historical or current (Bellino & Selman, 2012; Carretero, 2011; Seixas, 1996). In the social and ethical domain, the approach targets self and social awareness, perspective taking, interpersonal negotiation (Schultz, Barr, & Selman, 2001), and ethical awareness (Feigenberg, Steel King, Barr, & Selman, 2009; Selman & Kwok, 2010). In the civic domain, the approach seeks to foster increased interest in and tolerance for others with different beliefs and backgrounds (Schultz et al., 2001); concern for the welfare of others (Bardige, 1988; Domitrovich et. al., in process); and a sense that ones actions can make a difference in society (Fine, 1995). Although not the focus of this study, the Facing History approach also targets literacy skillsfor example, the ability to access and comprehend text in order to deliberate with others about complex, difficult, or controversial social, civic, and political issues (Lowenstein, 2005; Snow, Lawrence, & White, 2009). Together, the constellation of historical, civic, social, and ethical competencies outlined in the theory of change for this study are believed to be essential to young peoples capacity and commitment to participate, both individually and collectively, in activities that help to preserve and strengthen democracy (Barr & Bardige, 2013; Selman & Kwok, 2010). Previous program evaluation evidence has supplied only partial support for the assumptions underlying the programs theory of change. Early evaluations had demonstrated that Facing History students gain relevant knowledge of history (Glynn, 1982; Lieberman, 1981). A more recent quasi-experimental evaluation of Facing History involving eighth graders in New England indicated positive effects on students social understanding, their reflection on social skills, and a heightened awareness of the personal meaning of social relationships and a decrease in racist attitudes and self-reported fighting behavior (Schultz et al., 2001). But, other important aspects of the program had not been examined within the context of a rigorous design, including the programs effect on developing specific skills for understanding history. Nor had other critical elements been studied, such as civic skills (e.g., deliberation on controversial issues), dispositions (e.g., civic efficacy), and actions (e.g., civic discourse in and out of school) recognized by civic researchers and theorists as critical to mature civic engagement (Bickmore, 2005 ; Hess, 2009; Youniss & Levine; 2009). RESEARCH QUESTION AND HYPOTHESES Building on the theory and research described above, this study examines the following research questions and hypotheses: 1. What is the impact of assignment to this approach to professional development and follow-up coaching on high school teachers sense of self-efficacy, burnout, and perceptions of professional engagement and satisfaction compared to a group of control teachers who were not assigned to participate in the professional development activities? We hypothesized that teachers without prior experience with the intervention and randomly assigned by school to the intervention would, as compared to control group teachers, develop greater self-efficacy and professional engagement and satisfaction, and lower levels of burnout than control teachers. 2. What is the impact of assigning teachers to professional development aimed at supporting the classroom implementation of the Facing History and Ourselves program on 9th and 10th grade students: " perceptions of the classroom climate and civic learning opportunities; " civic skills, dispositions and behaviors; " historical thinking skills; and, " social and ethical awareness, compared to a group of control students taught by teachers who were not assigned to such professional development? Our hypotheses related to students in this domain were that the intervention students would " perceive their Facing History class as having a more open climate and providing more opportunities to learn about civic matters; " develop greater civic skills, dispositions, and behaviors; " demonstrate the capacity to reflect more deeply on social relationships and ethical decisions and be more likely to value active, prosocial solutions to social conflict (e.g., collaboration) and ethical dilemmas (e.g., standing up for others); and, " given the lack of previous research on how this educational approach impacts adolescents historical thinking skills, we expected program students to perform at least as well as control students in their academic subject area. METHOD AND PROCEDURES The study employed a group randomized design to address both research questions and sets of hypotheses. Schools that had no or extremely limited previous exposure to Facing History were recruited to participate in the study and were randomly assigned to one of two conditions: (a) participation in traditional Facing History summer professional development and traditional school year follow-up services in year 1, or (b) participation in an as is control condition with such participation in Facing History professional development and follow-up services in year 2. Ninth and tenth grade humanities teachers in the first group, the intervention schools, attended a Facing History and Ourselves: Holocaust and Human Behavior professional development seminar during the summer or fall of 2007 and implemented the program during the 20072008 school year, during which they received follow-up support from Facing History staff as they developed their lesson plans and implemented their units. Teachers were also provided with a classroom set of the resource book, Facing History and Ourselves: Holocaust and Human Behavior, and access to the programs print and digital resources, including unit outlines and historical content. In year 1, the second group, or control teachers, each in different schools to protect against informal interactions with other teachers implementing Facing History, received no services and continued to use the standard practices for history and language arts courses in their districts. SCHOOL RECRUITMENT AND RANDOMIZATION The study team defined the population of interest as a set of schools in close enough proximity to a Facing History regional office to facilitate training, ongoing teacher support, and data collection. Regional offices in the United States included Chicago, Cleveland, Denver, Los Angeles, Nashville/Memphis, New England, New York/New Jersey, and the San Francisco Bay Area. Schools were recruited from the population of schools within one hour of an office. To improve the external validity of the study, only the most necessary restrictions were put on eligibility. The study sample was limited to schools that, when approached by Facing History and the research staff and provided an overview of the program, became interested in having their teachers trained by Facing History but had not previously taken proactive steps to adopt the program. Facing History staff from the regional offices participated in recruitment and provided professional development to intervention teachers. Within the regions, a school was eligible if it had no teacher or administrator who was teaching a Facing History unit or who had previously attended a Facing History seminar, although informational knowledge of Facing History or attendance at presentations about Facing History did not make a school ineligible for the study. Teachers were eligible if they taught a humanities course in which they could implement a Facing History unit, made a commitment to implementing a minimum of six weeks of Facing History (the equivalent of thirty 45-minute classes) during the 20072008 school year, and were teaching primarily 9th or 10th graders.3 Students were eligible for the study if they were in the 9th or 10th grade in a randomly selected classroom of a teacher participant. One classroom per 9th and/or 10th grade teacher participant was randomly selected to participate in the study. As mentioned earlier, teachers in control schools were offered professional development and follow-up training in the second year of the study (beginning in summer 2008). Eligible teacher participants were identified prior to random assignment and all participating teachers agreed to participate in both the study and the professional development regarding the implementation of Facing History, for which they received a stipend. Random assignment was at the school level. Schools were first stratified by region and, using a computer-generated, randomly ordered list, schools were randomly assigned to either an intervention group or a control group within each region. Attrition from the student sample was minimized using a variety of strategies, including maintaining close contact with the teachers involved in the study by phone and email. This allowed research staff to keep abreast of issues as they arose and to resolve them before any school dropped out of the study. Efforts were also coordinated with regional offices, particularly for Facing History teachers and schools with which program staff cultivated close relationships. Nonetheless, as reported below, some schools and teachers did leave the study. CONSENT AND PARTICIPANTS Participating schools signed memoranda of understanding detailing the procedures of the study and asking for written consent for the school, teachers, and students to participate. A teacher in each school was paid a stipend to collect parental permission forms from students of participating 9th and 10th grade teachers and send them to the research team. Parental permission forms were collected from classrooms of 82% of participating 9th and 10th grade teachers. The study recruited 84 schools and randomly assigned them to either an intervention group or a control group (see Figure 2). Four schools (two intervention schools and two control schools) dropped out of the study after random assignment. At baseline, then, a total of 80 schools participated in the study. A year later, 62 schools were involved in the study (74% of the original randomized sample); 8 intervention schools and 10 control schools dropped out due to scheduling conflicts and teacher attrition over the summer in 2007. The schools participating at follow up were distributed across the eight regions, with no region accounting for greater than one-fifth of the respondents. Approximately two-thirds (69%) of schools in the study sample were district public schools, about one-fifth (23%) were other types of public schools (e.g., charter schools), and the rest (8%) were private schools. The average school in the study sample had an enrollment of 1,014 students, with variation ranging from 87 to 4,000 students. Figure 2. Tracking the study sample
Sixty-six percent of the schools in the sample were underperforming schools4 based on federal criteria; and most were high-poverty schools.5 Table 1 presents the demographics of the schools in the study at the first follow-up data collection point at the end of the first year of the intervention (spring, 2008). Table 1. Demographic Characteristics of Schools at Follow-up (N=62)
Notes. a Missing school size data for 2 schools, therefore average/median based on 60 schools. At baseline, 180 teachers from 80 schools were enrolled in the study; 78 teachers and 102 teachers were enrolled from intervention and control schools, respectively (see Figure 2). One year later, 113 teachers were participating in the study (38 from 22 schools that exited the study, and 29 teachers from 62 schools that stayed in the study; 80% of the 142 eligible teachers from the 62 schools that stayed in the study were included in the follow-up analysis. So, while some schools and teachers did not participate in follow-up data collection, the amount of overall and differential attrition at both the school and the teacher level was not large enough to jeopardize the integrity of the randomized design. 6 Teacher demographics are summarized in Table 2. The teachers had taught from 1 to 34 years (mean = 8.49 years), and more than 75% of the sample had 12 or more years of experience. The average years teaching at ones current institution was 4.38. Sixty-one percent of the sample had completed at least a masters degree, and the majority of teachers identified themselves as White (81%). Sixty-seven percent reported teaching history, 22% reported teaching English language arts, and 11% reported teaching other types of humanities classes. Table 2. Teacher Demographic Characteristics (N=113)
Randomly selecting one classroom of students for each of the eligible 9th or 10th grade teachers in the schools resulted in a sample of 1,401 students with parental permission to participate in the study. Student demographic characteristics are summarized in Table 3. Two-thirds of students in the sample were 10th graders. The sample contains a roughly 60/40 split of females to males. Students self-identified as Hispanic (36%), White (32%), Black (13%), Asian (13%), and Other (6%).7 English was the first language for 71% of the students. Of the remaining 29%, 75% of these students indicated Spanish as their first language and the remaining 25% reported Chinese, Vietnamese, or Tagalog, among other languages. Slightly less than half of both mothers and fathers of students had completed high school or less. Table 3. Student Demographic Characteristics at Follow-up 1(N*=1,371)
Notes. *The student sample includes 1,257 9th and 10th grade students who submitted both Book 1 and Book 2, 114 students who submitted only Book 1, and 30 students who submitted only Book 2. Given that the student demographic questions were found in Book 1, we have calculated the demographics in this table from the sample of students (N=1,371) who submitted Book 1. **Ns do not always add up to 1,371 due to missing data. ***Percentages were calculated using the N observations for which data were available as the denominator; therefore, percentages always add up to 100. MEASURES Outcomes were measured in two teacher domains and seven student domains. When available and appropriate to the study, existing measures were used; otherwise, measures were developed specifically for this study. There were 12 teacher outcomes and 15 student outcomes. The measure of student perception of classroom climate was treated as a student outcome. Tables 4 and 5 list the outcomes by domains, measures, and their sources. Whenever possible, we report the Cronbachs alpha for each of the measures described below. Table 4. Description of Teacher Domains and Outcome Measures
Notes. *Emotional Exhaustion (MBI) and Depersonalization (MBI) are the two teacher outcome measures for which Facing History hypothesizes Facing History teachers will receive a lower score than control teachers; lower scores on these measures indicate lower levels of emotional exhaustion and depersonalization, respectively. Table 5. Description of Student Domains and Outcome Measures
Notes. *The Modern Racism Scale is the only student outcome measure for which Facing History hypothesizes Facing History students will receive a lower score than control students; lower scores on this measure indicate fewer racist/intolerant attitudes. TEACHER MEASURES Teacher self-efficacy The domain of teacher self-efficacy was evaluated using eight outcomes, including the Character Education Efficacy Belief Instrument (CEEBI) and seven subscales of the Teaching for Informed Civic Engagement Efficacy Belief Inventory developed specifically for this study (Lowenstein & Facing History, 2007). Self-efficacy scale items ask teachers to report the extent to which they agree or disagree, on a scale from 1 (strongly disagree) to 5 (strongly agree), with a series of statements about their sense of confidence in having the relevant knowledge or skills for implementing instruction that fosters student learning and development in each of the targeted domains (civic, history, and ethics) in the context of a secondary level humanities course. The Character Education Efficacy Belief Instrument (CEEBI) The study used the 12-item Personal Teacher Efficacy subscale of the CEEBI (Milson, 2003). Teachers respond to 12 statements about their perceived ability to model and develop character in their students, such as, I am usually comfortable discussing issues of right and wrong with my students. In a pilot study conducted prior to the studys baseline data collection, the CEEBI demonstrated high reliability (i.e., internal consistency, Cronbachs alpha = .79). The Teaching for Informed Civic Engagement Efficacy Belief Inventory (TICE-EBI) The TICE-EBI (Lowenstein & Facing History and Ourselves, 2007) assesses teacher self-efficacy in teaching in ways that foster students growth in the specific domains explored in this study: historical understanding, tolerance and psychosocial development, civic literacy, and deliberation skills. While deliberation skills are conceptualized as an aspect of civic learning, the research team created a separate deliberation self-efficacy scale, reflecting its importance to the program and the field. The TICE-EBI also assesses three self-efficacy scales related to instructional practices and classroom processes thought necessary for such student learning and development to occur, and the scales were guided by the How People Learn Framework (Darling-Hammond & Bransford, 2005): self-efficacy in creating learner-centered, community-centered, and knowledge-centered learning environments. Key components in this area include developing efficacy in using knowledge-centered, learner-centered and community-centered instructional practices. The Cronbachs alpha for these seven measures ranged from .79 to .92. Teacher perceptions of professional engagement and satisfaction The Maslach Burnout Inventory (Maslach et al., 1996) includes three subscales treated as three separate outcomes for teachers: emotional exhaustion, depersonalization, and personal accomplishment. The inventory consists of a series of statements describing feelings or attitudes teachers may have about their profession, such as, I dont really care what happens to some students. Teachers indicate the frequency with which they experience these feelings from 0 (never) to 6 (every day). Summary scores are calculated as the mean of the items, which returns the score to the original scale of the items (a score between 0 and 6). A higher score for the first two constructs (emotional exhaustion and depersonalization) indicates more negative feelings toward the profession, while a higher score on the third construct (personal accomplishment) indicates more positive feelings toward teaching. Studies have demonstrated high reliability across all three subscales in similar teacher populations (Cronbachs alpha for emotional exhaustion is .88 to .90, depersonalization is .74 to .76, and personal accomplishment is .72 to .76). The Teacher Perception of Professional Support, Engagement, and Growth Scale, developed for this study, assesses the extent to which teachers agree with 11 statements about the quality of professional development support received and about the engagement and growth they feel in their discipline on a five-point scale. Items ask teachers to agree or disagree that they received needed support to engage students in sensitive and difficult moral and civic conversations, received professional development experiences that engaged their mind and heart, and feel greater expertise in their subject area. STUDENT MEASURES Student measures assessed student outcomes in the domains of civic learning (10 outcomes), historical understanding (1 outcome), and social and ethical reflection (4 outcomes). Civic Learning Five civic learning domains were selected to encompass core components of civic literacy salient to the Facing History program: civic responsibility, tolerance, civic efficacy, civic participation, and civic learning opportunities. In several cases, new scales were created by the research team where no instruments existed, and some existing scales were adapted to fit the particular focus of the study (see Table 5 for details). Civic Responsibility The Civic Responsibility Instrument contains three subscales that identify different dimensions of students sense of their primary roles and responsibilities as a citizen: personal, participatory, and social-justice orientations (Kahne et al., 2006). All three subscales have high internal consistency (Cronbachs alpha = .80, .76, and .78). Although students completed all three scales, only the Justice-Oriented Citizen subscale was included in the main study, given the interventions distinctive focus on issues of social justice.8 Items ask students to rate, on a five-point scale, the extent to which they agree with a series of statements about the importance of understanding and addressing the root causes of social problems. Tolerance The tolerance domain consisted of three measures. The Modern Racism Scale assesses tacit racism toward ethnic/racial minorities (McConahay, 1986). Students are asked the extent to which they agree with 12 statements, such as, This country would be better off if there were more acceptance of the good things in minority cultures (Alpha coefficients for the Modern Racism Scale range from .79 to .86). Avery et al.s (1997) Political Tolerance Scale measures students tolerance for others who express political views with which the students themselves disagree (also see Sullivan, 1953). Students pick a group whose views differ the most from their own from a list of groups, such as groups that oppose the legal right to an abortion or groups that supports the legal right to an abortion. Students are then asked the extent to which they agree with statements about whether or not the group is entitled to civil liberties such as the right to run for public office.9 For example, one item is Members of the group should not be able to run for president or other elected offices (Cronbachs alpha = .74). The Deliberation Convictions Scale examines students tolerance for engaging with others in deliberation of controversial public issues in class, such as Students should hear each other out, even when they disagree (Fine et al., 2007). Students rate the extent to which they agree with the statements on a five-point scale ranging from strongly disagree to strongly agree. Civic Self-Efficacy The Civic Self-Efficacy Scale examines how efficacious students feel in understanding and/or engaging in civic matters (Kahne et al., 2005; Education Commission of the States, n.d.). The scale asks students to rate their agreement with such statements as whether or not they feel well-informed and capable of speaking up, and whether or not they believe their public actions will make a difference (Cronbachs alpha = .86). Civic Participation Civic participation was measured with two scales. The Deliberation Practice Scale asks students to rate the frequency with which they engage in varied civic-focused deliberative practices, such as discussing social and political problems with others, using a four-point scale (Fine, Bermudez and Facing History & Ourselves, 2007). The Civic Discourse Scale (Fine et al., 2007; Haste, 2005) asks students to rate the frequency with which they engage in conversations with others about civic matters (Cronbachs alpha for these two measures were .74 and .81, respectively). Classroom Climate and Civic Learning Opportunities The Classroom Open Climate Scale measures a variety of dimensions of classroom climate on a five-point scale (Flanagan et al., 1998; Flanagan et al., 2007). The first subscale, Teacher Practices, focuses on teacher practice in creating an open classroom environment, addressing such things as whether teachers encourage discussion among students who hold different opinions, whether they expect students to listen to one anothers opinions, and whether they treat students respectfully. The second subscale, Student Practices, focuses on student practices that define an open classroom environment, including such things as whether students feel they have a voice in what happens, whether they are encouraged to express their opinions, and whether they can disagree with the teacher as long as they are respectful. The Engaging with Civic Matters Scale identifies the extent to which students perceive that they have opportunities to engage with civic issues, have conversations about civic matters, and learn about civic participation in class (Kahne & Sporte, 2008). For example, survey items ask students to rate whether they have been given opportunities to learn about people who have worked to make society better, to strengthen their community, and to explore the dangers of prejudice and discrimination (Cronbachs alpha for these three measures ranged from .76 to .85). Academic Learning: Historical Thinking Skills In recent times, psychologists have applied theory and methods to historical thinking, linking childrens and adolescents emerging cognitive development to their capacity for historical understanding, both as it develops naturally and as it is taught in middle and high school. Accordingly, this growing body of research illustrates how historical understanding develops through a sequence of progressive differentiations and integrations, rather than simply as an aggregation of information (Ashby, Gordon, & Lee, 2005; Duhlberg, 2002; Hartmann & Hasselhorn, 2008; Hartmann, Sauer, & Hasselhorn, 2009; Jensen, 2008; Lee & Ashby, 2000; Lee, Dickinson, & Ashby, 2001; Lee & Shemilt, 2003; Reisman & Wineburg, 2012; Seixas, 1996). Based on the evidence from these theory-oriented research studies, the evaluation team developed and validated an assessment measure for students' historical understanding (Stoskopf et al., 2007) using a progression model rubric to assign levels of complexity to students responses to questions that assess their historical understanding of evidence, causality, and agency (Seixas, 1996). The measure uses a historical case study approach to assessment that asks students to interpret and integrate information from seven documents, some primary source and other actual economic data, that provide historical information relevant to the inter-ethnic conflict in the former Yugoslavia in the 1990s (Bellino & Selman, 2012).10 Students are asked three true/false questions to assess the level of their comprehension of the historical facts contained in the documents (Bellino & Selman, 2011). Five items ask students to analyze historical information and rate the adequacy of a number of different explanations provided for the historical interpretation of that information. For example, after reading the verbatim statement from a Serbian Orthodox priest, students are asked to rate the adequacy of three historical claims within the statement about the role of religion during the war. Student responses were scored using a rubric based on a progression model for the development of historical thinking skills, such that explanations reflecting a more sophisticated understanding of evidence, causality, or agency received higher scores (Stoskopf et al., 2007, Bellino & Selman, 2011). An overall historical understanding score, a composite across the three historical understanding skills, was calculated based on the average of students scores across all of the items. SOCIAL AND ETHICAL COMPETENCIES Social Awareness The Relationship Questionnaire (Rel-Q) is a validated measure that assesses the development of social perspective coordination and its application to three interdependent malleable social competencies: interpersonal understanding, interpersonal negotiation, and the awareness of the personal meaning of relationships . Each item on the Rel-Q presents a brief story depicting an interpersonal dilemma or issue and asks students to rate the adequacy of four potential responses of the protagonist in the situation. Students are also asked to select what they see as the best response. The Rel-Q was adapted for this study by included items thematically relevant but not exclusive to Facing History: social identity, social perspective-taking, social conflict resolution, social awareness, friendship and inclusion, and autonomy and respect. In previous studies, validity and reliability for the measure has been assessed with a diverse sample of youth and the Cronbachs alpha reliability coefficients for the composite score (.87), the best choice score (.71), and the response ratings (.82) suggest the measure is internally consistent .11 Ethical Awareness The Choices in Social Context Measure is an exploratory measure developed for this research using a similar item-rating methodology to the Rel-Q to tap students reflections on ethical decisions and choices (Selman, Barr, Feigenberg, & Facing History and Ourselves, 2007). The measure consists of four school-based situations involving some kind of social injustice, such as peer victimization or racism. Students rate the adequacy of strategies for responding to the situations and justifications for responses. The measure produces two scores, one focusing on strategy choices and the second on justification. Because Facing History explores the dire consequences for democracy of individual and collective indifference and passivity in the face of injustice, and democracys role in protecting vulnerable groups, the research team posited that intervention students would be more likely to favor interrupting social injustice, directly or indirectly, over standing by or joining the perpetrators of injustice. Further, we expected Facing History students to favor justifications for strategies that reflect an awareness of relational and contextual factors more than control students, given the programs emphasis on studying how individuals and groups can positively transform social dynamics through their actions. DATA COLLECTION All participating teachers were asked to complete a baseline teacher survey in spring 2007, and all participating teachers and students were asked to complete follow-up surveys in spring 2008. The second wave of data collection was timed to occur after the intervention teachers had attended a Facing History seminar and implemented an entire unit. The teachers in control schools conducted their classes as usual during the 20072008 school year. Student survey measures were divided into two survey booklets, which took approximately 45 minutes each to complete. (Survey booklet one contained the Civic Learning and Choices in Context measures, while survey booklet two contained the Relationship Questionnaire and Historical Understanding measures.) Survey mailing was timed such that intervention teachers and students completed surveys after teachers had finished their entire Facing History unit.12 Supplemental information (e.g., school type, size, location, etc.) was also collected from study participants. FIDELITY OF IMPLEMENTATION The research team, in consultation with program staff, defined fidelity of implementation at both the teacher and the classroom levels. At the teacher level, implementation was defined as participation in a full four- or five-day Facing History seminar and the use of follow-up services, including at least one planning session with a staff member. All intervention teachers participated in Facing History summer seminars and had at least one planning meeting with a staff member. Nearly all teachers reported that they received at least some support for finding online, print, video, and/or other resources (98%), developing a Facing History unit (96%), and developing particular lessons (85%). About two-thirds of teachers reported taking advantage of Facing History conferences or community events (62%) at least once. Fewer teachers indicated receiving support managing their classrooms (52%), or through workshops (42%) or modeling lessons (37%). Overall, these findings suggest that Facing History teachers received some form of follow-up support during the development phase of their unit, but that program staff were less often directly involved in classroom implementation (see Appendix A, Figure A for more detail). Full implementation at the classroom level was defined as teaching a minimum of a six-week Facing History and Ourselves: Holocaust and Behavior unit with at least two hours of study of each part of the sequence of study, using the Facing History and Ourselves: Holocaust and Human Behavior resource book and some Facing History DVDs and/or videos in their teaching. Forty-nine of the 53 teachers in the Facing History group provided information at follow-up about how they had implemented the program in their classrooms (see Appendix A, Figures B and C), and were assigned a fidelity score. These scores were translated into three discrete categories based on predetermined benchmarks: low, medium, and high fidelity. Approximately half (47%) of Facing History teachers followed the programs time and resource expectations with high fidelity. The remaining 53% delivered the program to students with medium or low fidelity. ANALYTIC STRATEGY All impacts were estimated using a two-level hierarchical linear model (Raudenbush & Bryk, 2002) to account for the fact that teachers and students are clustered within schools. The impact of the intervention was modeled at the school level, the level of random assignment. Indicator variables for the geographic region, or region fixed-effects, were also included at the school level to control for differences in outcomes across regions. Demographic characteristics were included at both the teacher/student level and the school level. Teacher characteristics included the highest level of education attained and the subject the teacher reported teaching. Student characteristics included gender and race. School characteristics included the school type, such as a district public school, an alternative public school (e.g., a charter school), or a private school. In models estimating impacts on teachers, the baseline measure of the outcome was included as a covariate. Student demographics were missing for 30 students. Gender was imputed by looking at the students name and making a determination as to whether the student was a boy or a girl. Race was imputed by computing a mean value for race at each school and assigning this value to all cases where race was missing. To help assess the relative magnitudes of the effects and to aid in interpretation, all impact estimates are reported as effect sizes (calculated as Glasss delta). Recent advances in the evaluation literature provides guidance about limiting the number of outcomes on which hypothesis tests are conducted as a way of balancing risk (see Schochet, 2008). In this study, prior to data collection, the research team divided the primary outcomes into substantive domains; each domain of outcomes was considered as a group when answering research questions about impacts. Tests of statistical significance reported here reflect an adjusted alpha level, using the Benjamini-Hochberg correction to bring the effective alpha level across outcomes within a domain down to .05. ESTIMATES OF EQUIVALENCE BETWEEN TEACHERS IN INTERVENTION AND CONTROL SCHOOLS AT BASELINE Table 6 shows that, at baseline, the teacher groups were similar on measured demographic characteristics. There were no statistically significant differences in years of teaching experienceyears spent teaching at ones current institution, highest level of education, race, or the types of courses they typically teach. Table 6. Estimated Baseline Differences on Teacher Characteristics
Table 7 shows that there were also no statistically significantly differences between the two groups on all of the outcomes of interest at baseline, including self-efficacy, and burnout or professional engagement and satisfaction. Table 7. Estimated Baseline Differences between Facing History and Control Teachers in Analysis
Notes. The CEEBI and NPDEP measures are scored on a scale from 1 (strongly disagree) to 5 (strongly agree). The MBI measures are scored on a scale from 0 (never) to 6 (every day). *Emotional Exhaustion (MBI) and Depersonalization (MBI) are the two teacher outcome measures for which Facing History hypothesizes Facing History teachers will receive a lower sore than control teachers; lower scores on these measures indicate lower levels of emotional exhaustion and depersonalization, respectively. ESTIMATES OF EQUIVALENCE BETWEEN STUDENTS IN INTERVENTION AND CONTROL SCHOOLS AT FOLLOW-UP The study team also tested the equivalence of the students in intervention and control schools at follow-up (spring, 2008, after the intervention) to further test the veracity of our assumptions about random assignment. The process of random assignment created two equivalent groups with regard to student demographic characteristics (grade level, sex, first language race, mothers education level, and fathers education levelsee Table 8).13 Table 8. Estimated Differences on Student Demographic Characteristics between Facing History and Control Students a
Notes. a Data on students demographic characteristics were collected at Time 1, when students completed other study measures. The student sample includes 1,257 9th and 10th grade students who submitted both Book 1 and Book 2, 114 students who submitted only Book 1, and 30 students who submitted only Book 2. Given that the student demographic questions were found in Book 1, we have calculated the demographics in this table from the sample of students (N=1,371) who submitted Book 1. IMPACTS OF THE INTERVENTION Tables 9 and 10 summarize the estimated impacts of the Facing History professional development program on the 12 teacher outcomes and the 14 student outcomes. The first two columns provide the estimated mean outcome for the control and intervention groups respectively in the original metric for the measure. The third column provides the estimated impact of the intervention in the original metric, and the fourth column provides the estimated p-value, as well as an assessment of the statistical significance of the impact after adjusting for the number of tests conducted in each domain. The final column provides the impact estimate in effect size units. As seen in Table 9, positive, statistically significant impacts were found for 10 of 12 teacher outcomes, including all of the self-efficacy outcomes (effect sizes range from .53 to .85); teacher satisfaction with professional support, engagement, and growth (ES = .94); and one of the three teacher burnout subscales (personal accomplishment ES= .53). There were no main effects for the other two subscales of the Maslach Burnout Inventory: emotional exhaustion and depersonalization. Table 9. Estimated Impacts on Teachers
Notes. * indicates that the estimated impact is significant after applying the Benjamini-Hochberg correction to bring the effective critical value of p across outcomes to .05. ~Emotional Exhaustion (MBI) and Depersonalization (MBI) are the two teacher outcome measures for which Facing History hypothesizes Facing History teachers will receive a lower score than control teachers; lower scores on these measures indicate lower levels of emotional exhaustion and depersonalization, respectively. For students (Table 10), the estimated impact of the intervention was positive and statistically significant for overall historical understanding (ES=.14) and for four civic literacy variables: political tolerance (tolerance for others with different social or political views; ES=.18), civic efficacy (ES=.20), opportunities to engage in civic matters in class (ES= .23), and open classroom climate, teacher practices (ES=.17). There were no main effects for the other civic learning variables or for the four variables in the social and ethical awareness domain. However, one additional civic literacy outcome, participation in civic discourse, approached significance (p<.058) and had a comparable effect size to other positive student outcomes (ES=.17). Table 10. Estimated Impacts on Students
Notes. The student sample for all outcomes in the Civic Responsibility, Tolerance, Efficacy, Engagement, and Opportunities domains, as well as the Social Choices in Context outcomes include the 759 control students and 612 Facing History students who completed booklet one. The remaining outcomes are based on the 700 control students and 587 Facing History students who completed booklet two. The vast majority of students completed both survey booklets. ~ For Racism/Tolerance (MRS) and none, Facing History hypothesizes Facing History students will receive a lower score than control students; lower scores on the MRS indicate fewer racist/intolerant attitudes, and lower scores on none indicate that a smaller proportion of students responded none to the question about whether their class had raised their awareness of any groups experiences with discrimination and prejudice. DISCUSSION This paper presents impacts of professional development for implementing an interdisciplinary, student-centered approach to civic learning on teachers, and the corresponding impact of these teachers first classroom implementation of the intervention on their 9th and 10th grade students civic learning, academic learning (historical understanding skills), and social and ethical reflection. We posited both teacher- and student-level effects from the intervention. Teacher effects were anticipated on teacher self-efficacy in a range of domains seen as critical to fostering students informed civic reflection. Increased teacher professional engagement and satisfaction and reduced teacher burnout was also hypothesized as important aspects of teacher effectiveness in this area. Student effects were posited for five civic learning domains, including civic learning opportunities (including classroom climate), civic responsibility, tolerance, civic efficacy and civic engagement, and for social and ethical awareness. Historical understanding was hypothesized to be similar in both groups. With respect to teacher results, we found strong, statistically significant, positive effects on all eight areas of teacher self-efficacy and on two of four teacher professional engagement and burnout outcomes, including teacher-reported personal accomplishment and perceptions of professional support, engagement, and growth in the teachers academic discipline (including satisfaction with professional development experiences). The fact that intervention students outperformed control students in some of the domains in which teachers also reported greater self-efficacy (e.g., tolerance) is suggestive of possible links between increased teacher self-efficacy and student learning, which should be explored in future research. No differences were found between intervention and control teachers for two aspects of teacher burnout: depersonalization and emotional exhaustion. Perhaps these aspects of teacher burnout are adversely affected by factors over which teachers often have little control, such as course load, class size, mandates to prepare students for standardized tests, and compensation and teacher evaluation policies, limiting the potential impact of the professional development. The evidence of the impact of teachers participation in Facing History professional development in this study is based on self-reported measures. Because the teachers knew whether they were in either the intervention or the control sample, there is a risk of potential bias in reporting. The battery of student civic measures was predominantly drawn from measures previously validated by research in the field, although very few had been used as measures of a programs civic impact. Despite the fact that roughly half of the intervention teachers did not fully implement the program, the study detected significant effects on students perceptions of the classroom climate as more open (based on teacher practices) and providing more opportunities to learn about civic matters, students tolerance for the rights of others who hold beliefs that are strongly contrary to their own, and students civic self-efficacy. The significant impact on civic efficacy is noteworthy because research has demonstrated a positive relationship between civic efficacy and actual civic and political participation (Kahne & Westheimer, 2003). Moreover, the programs impact on actual engagement in civic discourse was marginally significant (p=.058). No main effects were found for the remaining civic learning variables, including justice-oriented civic responsibility, tacit racism, deliberation convictions and practices, and the students role in creating an open classroom climate.14 As a result, the overall findings related to civic attitudes and behaviors paint a mixed picture. While there were no main effects for deliberation, intervention students did perceive greater civic learning opportunities, a more open classroom climate, and greater involvement in civic discourse than control students. Similarly, there were no main effects for racist attitudes, and yet intervention students held more tolerant attitudes about others with views dramatically different from their own. Further, while intervention students did not report a greater sense of civic responsibility, they reported having more civic efficacy. Future studies should include qualitative research to better understand the interrelationships among these various civic dispositions and behaviors and to assess which classroom-based practices are most salient to their development. With respect to the measure of students historical understanding, we found statistically significant program impacts on students historical thinking skills, in particular, the combined score for students analysis of historical agency, evidence, and cause and effect. This finding provides evidence that an educational approach for adolescents that integrates intellectual rigor, engaged academic discussion, and ethical reflection processes for learning historyand then connecting that history to ones current social and civic concerns, commitments, and participationis effective in fostering fundamental historical thinking skills that students can apply to understanding new content. This finding informs debates about whether rigorous high school history education should involve personal, ethical, and emotional engagement and have civic purposes or whether it should focus solely on fostering intellectual skills necessary for analyzing the past (Barton & Levstik, 2004; Bermudez & Jaramillo, 2001; Carretero, 2011). Some scholars argue that personal, emotional, and ethical engagement with history often leads to distorted (e.g., presentistic and anachronistic) and incomplete historical understanding and superficial pastpresent connections. In this study, intervention students exposed to an integrated, interdisciplinary approach to analyzing history and civic engagement using student-centered methods outperformed control students on a measure of historical understanding; this finding raises questions about the complex relationship that may exist between adolescents development of historical understanding skills and social, ethical, and civic awareness (Bellino &Selman, 2012; Selman and Kwok, 2010). The historical understanding measure used in this study, however, has only begun to be validated in comparison to other traditional measures of historical learning focused on content knowledge (Ashby et al., 2005; Bellino & Selman, 2011) and recently validated psychometric measures of historical understanding based on similar theoretical assumptions (Hartmann & Hasselhorn, 2008; Hartmann et al., 2009). Conclusions must be drawn with caution, therefore, and the measure needs continued validation. With respect to social and ethical reflection, the 9th and 10th graders in the intervention group were expected to outperform control students. The lack of main effects in this area could suggest that teachers, on average, need more experience implementing the pedagogy and content of the program to successfully foster students' social and ethical reflection relative to the experience needed to effectively promote civic learning and historical understanding. Or, it may be the case that the targeted areas of social and ethical development (i.e., social and ethical reflection on topics of social inclusion and exclusion) are more difficult to operationalize as particular skills and, therefore, more difficult to assess, especially using exploratory measures. Finally, it may be that a higher degree of program fidelity would have resulted in the hypothesized outcomes in the social and ethical reflection domain. In sum, we found statistically significant main effects of the program for 5 of 15 student outcomes (four civic and one historical). This provides promising evidence that, after only one year of participation in the professional development model and with their first experience implementing the intervention, teachers were able to create more open classroom climates, provide increased civic learning opportunities, and impact students learning and growth in areas critical to participation in a democracy. This includes the capacity to analyze historical evidence, causation, and human agency, and developing ones sense of civic efficacy and tolerance. It should be noted that 66% of the schools in the sample were underperforming schools based on federal criteria. Most were high-poverty schools and, for 30% of the students, English was their second language. The study, therefore, also provides initial evidence of the efficacy of this professional development approach for teachers and students in settings where quality civic education has historically been less available, suggesting that it is a promising strategy for ameliorating what has been referred to as a civic empowerment gap between students in high-poverty schools and their more privileged counterparts (Kahne et al., 2005; Levinson, 2010).15 It is also important to note that, rather than self-selecting for this kind of professional development experience, teachers who were recruited into the study by their schools had minimal knowledge of the Facing History program. The professional developmentboth its pedagogy and contentis complex, demanding, and highly interactive. It does not consist of a uniform set of materials/lessons each participant is expected to implement consistently, much less identically. Seminars are designed to lead teachers through a process of changing how they understand, think about, and deliver the content they teach. It requires a reflective practice and a commitment to a way of teaching that could well represent a departure from past practices. Indeed, it is not unreasonable to expect that it would take more than a single school year for teachers to implement the program with high fidelity. Therefore, future research should examine whether subsequent cohorts of students taught by teachers with more experience implementing the program and those who implement it with greater fidelity demonstrate stronger outcomes for students than found in the current study. Such research should employ a fidelity measure that takes into account the quality of program implementation rather than just the level of implementation. Finally, it will also be important to look for latent student effects in a longitudinal study. Are initial program effects enhanced or attenuated with time, and do social and ethical reflection effects emerge later as these competencies are applied in real-life situations? Ideally, we would contextualize this study by comparing these findings to the results of other research on the impacts of similar programs for similar populations of teachers and students and similar outcome variables. However, the study is part of a new and emerging field; no large body of research yet exists to which we can compare the current results. This point is supported by the review of studies of programs in the broad realm of character education conducted by the U.S. Department of Educations What Works Clearinghouse. Only 13 of 41 (31.7%) programs had research studies that met rigorous evidence standards, highlighting the fact that relatively few programs have undergone rigorous evaluation at all and even fewer have demonstrated positive effects. Further, these studies vary widely in the target population (e.g., elementary-age students, middle school students, and high school students), nature of the intervention (e.g., stand-alone curriculum, time-limited program participation, direct interactions with students, and professional development for teachers), and sample size. Finally, few studies have measured the impact of professional development seeking to help humanities teachers integration of civic education into their practice and, more importantly, none have examined both teacher and student outcomes using a randomized controlled experimental design. CONCLUSIONS This study documents substantial causal impacts for teachers, and more moderate impacts for students of an interdisciplinary educational intervention. Notably, this intervention helps teachers provide compelling historical content and student-centered methods to foster their students academic and civic growth. Equally notable, randomized controlled trials of such interventions are rare. Few studies have found meaningful learning outcomes for students as a function of their teachers participation in professional development. In this study, to the contrary, moderate impacts for students were found in multiple domains despite the fact that roughly half of the intervention teachers did not fully implement the program. Educators need evidence-based approaches for teaching complex social, civic, and political issues in ways that allow students of diverse mindsets and backgrounds to engage constructively with one another. The need for such approaches to professional development in civic education is especially urgent, given the current politically polarized climate in the United States and elsewhere. Preserving our democracy and more fully enacting the principles and values on which it is based requires deep investments in high-quality, evidence-based, civic learning opportunities for youth, opportunities that help youth feel they can make a difference and equip them to do so. Acknowledgments This research was made possible with the generous support of the Richard and Susan Smith Family Foundation. The authors acknowledge the following Facing History staff for their meaningful contributions and support: Terry Tollefson, Marc Skvirsky, Anna Romer, Marty Sleeper, Margot Strom, and Abbey Mann. We also thank Angela Bermudez, Ulrike Hartmann and Alan Stoskopf for their contributions to measures used in the study, and Marc Moss and Kristina Kliorys of Abt Associates, Inc. for their contributions to the design of the study and data analysis. A number of other individuals gave invaluable input, including: Larry Aber, Betty Bardige, Veronica Boix-Mansilla, Joshua Brown, Laura Desimone, Celene Domitrovitch, Connie Flanagan, Helen Haste, Stephanie Jones, Joe McDonald, Firkins Reed, Roger Weissberg, and John Willett. In addition, we thank the following current or former Facing History staff members, including Dr.Barr, who helped Abt Associates, Inc. recruit subjects for the study: Dan Alba, Fran Colletti, Dunreith Kelly Lowenstein, Peter Nelson, Bonnie Oberman, Molly Schen, Ted Scott, Rachel Shankman, Fran Sterling, Mark Swaim-Fox, Corey Todaro, and Jack Weinstein. Notes 1. Dennis J. Barr served as the principal investigator, and Robert L. Selman, Ethan Lowenstein, and Melinda Fine served as co-investigators. Dennis J. Barr, Robert L. Selman, and Terry Tollefson served as the Steering Committee. The full research team included researchers from three different kinds of organizations who played different roles: (a) researchers from Abt Associates, Inc. led on study design, data collection, and data analyses: Beth Boulay, Beth Gamse, Rachel McCormick, Kristina Kliorys, and Marc Moss; (b) university-based scholars led on measurement and contextualizing the study in previous scholarship on teacher professional development, and civic learning, historical understanding, and social and ethical awareness in adolescence: Robert L. Selman, Ethan Lowenstein, Melinda Fine, Alan Stoskopf, and Angela Bermudez; and (c) Dennis Barr, director of evaluation at Facing History and Ourselves, managed the overall project, including the participation of other Facing History staff members who supported the project. To avoid potential bias, Facing History staff members were not involved in data collection or analyses. 2. For a review of prior research on the program that informed the current study, please see: Barr, D. J. & Facing History and Ourselves. (2010). The Facing History and Ourselves National Professional Development and Evaluation Project: Continuing a tradition of research on the foundations of democratic education. Brookline, MA: Facing History and Ourselves National Foundation, Inc. 3. In order to collect student data in every participating school, we excluded schools that did not have at least one 9th or 10th grade teacher participating. Once one 9th or 10th grade teacher agreed to participate, we allowed teachers of other grades to participate in the teacher study, even though their students were ineligible for the student study 4. Schools that have not met annual yearly progress, based on standardized test scores, for two consecutive years are considered underperforming. 5. In one quarter of the schools, 90% or more of the students were eligible for Free or Reduced Price Lunch (FRPL), making them eligible for schoolwide federal Title I funding. In 60% of the schools, 4089% of students were eligible for FRPL. 6. The reported levels of attrition are within the acceptable range established by the U.S. Department of Educations What Works Clearinghouse. For more information, please see Appendix A-Assessing Attrition Bias, of the WWC Procedures and Standards Handbook, Version 2.1 available at http://ies.ed.gov/ncee/wwc/help/idocviewer/Doc.aspx?docId=19&tocId=7 7. The category of other is largely comprised of individuals who identified themselves as members of multiple racial groups. The research team selected primary outcomes from a larger set of outcomes of interest. Secondary outcomes were included in exploratory analyses and were not subjected to multiple hypotheses-test corrections. For example, though students completed the entire civic responsibility measure, the research team selected only one of the subscales, social justice orientation, for hypotheses testing in the primary study based on the programs emphasis on raising awareness about the root causes of social injustice. 8. For this study, we updated the groups included in the list, in order to make the options more relevant to current events. 9. Facing History does not address that period in history so the measure is not biased in favor of intervention students. 10. This study used only the best choice and response ratings scores. 11. For the majority of schools, this meant that data were collected in the spring. However, a few teachers completed their Facing History unit in the fall, and their students were surveyed shortly after the completion of the unit 12. We can only examine group equivalence by examining student demographic characteristics as we do not have baseline outcome data for students. 13. Trends in favor of Facing History students were found, however, for the two subscales of civic responsibilitypersonally responsible (p=.07) and participatory (p.=.06)that were included in the exploratory rather than the experimental part of the study. 14. A post hoc analysis revealed that the intervention was equally effective in underperforming schools as it was in the full sample of schools. 15. To investigate this question, we constrained the sample to those schools designated as underperforming, and re-ran the student outcome analysis. The pattern of results from this re-analysis is virtually indistinguishable from the pattern for the sample as a whole. References Anderson, G. L., & Herr, K. (2011). Scaling up Evidence-Based practices for teachers is a profitable but discredited paradigm. Educational Researcher, 40(6), 287289. Ashby, R., Gordon, P., & Lee, P. (Eds.). (2005). Understanding history: Recent research in history education. In International review of history education (Vol. 4). London, UK: Woburn Press. Avery, P. G., Sullivan, J. L., & Wood, S. L. (1997). Teaching for tolerance of diverse beliefs. Theory into Practice, 36(1). Bardige, B. (1983). Reflective thinking and prosocial awareness (Unpublished doctoral dissertation). Harvard Graduate School of Education, Cambridge, MA. Bardige, B. (1988). Things so finely human: Moral sensibilities at risk in adolescence. In C. Gillian, J. V. Ward, & J. M. Taylor (Eds.), Mapping the moral domain. Cambridge, UK: Harvard University Press. Barr, D. J. (2003). Evaluation of the fostering understanding project: Race and membership in the United States (Unpublished report). Brookline, MA: Facing History and Ourselves National Foundation, Inc. Barr, D. J. (2005). Early adolescents reflection on social justice: Facing History and Ourselves in practice and assessment. Intercultural Education, 16(2), 145160. Barr, D. J., & Bardige, B. (2013). Case study 18A: Facing History and Ourselves. In P. M. Brown, M. W. Corriga, & A. Higgins-D'Alessandro (Eds.), Handbook of prosocial education (Vol. 2). Lanham, MD: Rowman & Littlefield Publishers, Inc. Barr D. J., Bender, J., Fine, M., Schultz, L. H., Tollefson, T., & Selman, R. L. (1998). A case study of Facing History and Ourselves in an eighth grade classroom: A thematic and developmental approach to the study of inter-group relations in a programmatic context (Unpublished report). New York, NY: Carnegie Corporation of New York. Barr, D. J. & Facing History and Ourselves. (2010). The Facing History and Ourselves National Professional Development and Evaluation Project: Continuing a tradition of research on the foundations of democratic education. Brookline, MA: Facing History and Ourselves National Foundation, Inc. Barton, K. C., & Levstik, L. S. (2004). Teaching history for the common good. Mahwah, NJ: Lawrence Erlbaum Associated, Publishers. Bellino, M., & Selman, R. L. (2012). The intersection of historical understanding and ethical reflection during early adolescence: A place where time is squared. In M. Carretero (Ed.), History education and the construction of national identities (pp. 189202). Charlotte, NC: Information Age Publishing, Bellino, M. J. & Selman, R. L. (2011). High school students understanding of betrayal in an historical context: Implications for teaching history. International Journal of History Teaching, Learning and Research 10(1), 2943. Bermudez, A., & Jaramillo, R. (2001). Development of historical explanation in children, adolescents and adults. In A. Dickinson, P. Gordon, & P. Lee (Eds.), International review of history education: Vol. 3. Raising standards in history education (pp. 146167). Portland, OR: Woburn Press. Bickmore, K. (2005). Teacher development for conflict participation: Facilitating learning for difficult citizenship education. International Journal of Citizenship and Teacher Education, 1(2). Boix-Mansilla, V. (2000). Historical understanding: Beyond the past and into the present. In P. N. Stearns, P. Seixas, & S. Wineburg (Eds.), Knowing, teaching, and learning history: National and international perspectives (pp. 390418). New York, NY: New York University Press. Borko, H. (2004). Professional development and teacher learning: Mapping the terrain. Educational Research, 33(8), 315. Carretero, M. (2011). Constructing patriotism: Teaching history and memories in global worlds. Charlotte, NC: Information Age Publishing. Cohen, J., Pickeral, T., & Levine, P. (2010). The foundation for democracy: Social, emotional, ethical, cognitive skills, and dispositions in K12 schools. Inter-American Journal of Education for Democracy, 3(1), 7497. Colby, A., Beaumont, E, Ehrlich, T., & Corngold, J. (2007) Educating for democracy: Preparing endergraduates for responsible political engagement. San Francisco: Jossey-Bass. Darling-Hammond, L., & Bransford, J. (2005). Preparing teachers for a changing world: What teachers should learn and be able to do. San Francisco, CA: Jossey-Bass. Darling-Hammond, L., Wei, R. C., Andree, A., Richardson, N., & Orphanos, S. (2009). Professional learning in the learning profession: A status report on teacher development in the United States and abroad. Dallas, TX: National Staff Development Council. Davila, A., & Mora, M. T. (2007). Working paper 52: Civic engagement and high school academic progress: An analysis using NELS data. College Park, MD: University of Maryland, The Center for Information and Research on Civic Learning and Engagement. Retrieved from http://www.civicyouth.org/PopUps/FactSheets/FS_Mora.Davila.pdf Desimone, L. M. (2009). Improving impact studies of teachers professional development: Toward better conceptualizations and measures. Educational Research, 38(3), 181199. Desimone, L. M. (2011). A primer on effective professional development. Phi Delta Kappan, 92(6), 68. Domitrovich, C. E., Syversten, S., Cleveland, M., Moore, J. E., Jacobson, L., Glenn, J., & Greenberg, M. T. (in process). The effects of the Facing History and Ourselves on classroom climate and middle school students social cognition and behavior. Dulberg, N. (2002, April). Engaging in history: Empathy and perspective-taking in children's historical thinking. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. Education Commission of the State. (n.d.). Civics assessment database [Web page]. Denver, CO. Retrieved from http://www.ecs.org/qna/BrowseQuestions.asp?ak=1&all=1 Erikson, E. (1950). Childhood and society. New York, NY: WW Norton and Co. Facing History and Ourselves. (2012, October). Mission statement [Web page]. Brookline, MA. Retrieved from http://www.facinghistory.org/aboutus/missionstatement Farrington, C. A., Roderick, M., Allensworth, E., Nagaoka, J., Keyes, T. S., Johnson, D. W., & Beechum, N. O. (2012). Teaching adolescents to become learners. The role of noncognitive factors in shaping school performance: A critical literature review. Chicago: University of Chicago Consortium on Chicago School Research. Feigenberg, L. F., Steel King, M., Barr, D. J., & Selman, R. L. (2008). Belonging to and exclusion from the peer group in schools: Influences on adolescents moral choices. Journal of Moral Education, 37(2), 165184. Fine, M. (1995). Habits of mind. New York, NY: Teachers College Press. Fine, M., Bermudez, A., & Facing History and Ourselves (2007). Civic learning instrument. Unpublished manuscript. Finlay, A., & Flanagan, C. (2009). Making educational progress: Links to civic engagement during the transition to adulthood (Working Paper 67). Medford, MA: Tufts University Center for Information and Research on Civil Learning and Engagement. Retrieved from www.civicyouth.org Flanagan, C., Bowes, J., Jonsson, B., Csapo, B., & Sheblanova, E. (1998). Ties that bind: Correlates of adolescents civic commitments in seven countries. Journal of Social Issues, 54(3), 457475. Flanagan, C., Cumsille, P., Gill, S., & Gallay, L. (2007). School and community climates and civic commitments: Processes for ethnic minority and majority students. Journal of Educational Psychology, 99(2), 421431. Flanagan, C., Stoppa, T., Syvertsen, A., & Stout, M. (2010). Schools and social trust. In L. Sherrod, J. Torney-Purta, & C. Flanagan (Eds.), Handbook of research on civic engagement in youth (307329). Hoboken, NJ: Wiley. Frey, F., & Barr, D. J. (2004). Evaluation of Facing History and Ourselves in Cleveland schools: A follow-up study of teachers (Unpublished report). Brookline, MA: Facing History and Ourselves Foundation, Inc. Gilligan, C., & Lyons, N. (1989). Making connections: The relational worlds of adolescent girls at the Emma Willard School. Cambridge, MA: Harvard University Press. Glynn, M. (1982). American youth and the Holocaust: A study of four major curricula. New York, NY: National Jewish Resource Center. Gould, J., Jamieson, K. H., Levine, P., McConnell, T., Smith, D. B., McKinney-Browning, M., & Cambell, K. (2009). Guardian of democracy: The civic mission of schools. The Campaign for the Civic Mission of Schools. Available at www.civicmissionofschools.org Gurskey, T. R., & Yoon, K. S. (2009). What works in professional development? Phi Delta Kappan, 90(7), 495500. Hahn C. L. (1994). Controversial issues in history instruction. In M. Carretero & J. F. Voss (Eds.), Cognitive and instructional processes in history and the social sciences. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers. Hartmann, U., & Hasselhorn, M. (2008). Historical perspective taking: A standardized measure for an aspect of students historical thinking. Learning and Individual Differences, 18(2), 264270. Hartmann, U., Sauer, M., & Hasselhorn, M. (2009). Perpektive nübernahme als Kompetenz für den Geschichtsunterricht [Perspective taking as a competence for history education: Theoretical and empirical relations between subject-specific and socio-cognitive characteristics in students]. Zeitschrift für Erziehungswissenschaft 12, 321342. Haste, H. (2005). My voice, my vote, my community: A study of young peoples civic action and inaction. London, UK: Nestle Social Research Programme & The Economic and Social Research Council. Henson, R. K. (2001, January). Teacher self-efficacy: Substantive implications and measurement dilemmas. Keynote address given at annual meeting of the Educational Research Exchange, Texas A&M University, College Station, Texas. Hepburn, M. (2000). Service learning and civic education in the schools: What does recent research tell us? In S. Mann & J. Patrick (Eds.), Education for civic engagement in democracy: Service learning and other promising practices (pp. 4559), Bloomington, IN: ERIC Clearinghouse for Social Studies/Social Science Education. Hess, D., & Zola, J. (2013). Professional development as a tool for improving civic education. In D. E. Campbell, M. Levinson, & F. M. Hess (Eds.), Making civics count: Citizenship education for a new generation (183206). Cambridge, MA: Harvard Education Press. Hess, D. E. (2009). Controversy in the classroom: The power of discussion. New York, NY: Routledge. Hochberg, E. & Desimone, L. (2010). Professional development in the accountability context: Building capacity to achieve standards. Educational Psychologist, 45(2), 89106. Ingvarson, L., Meiers, M., & Beavis, A. (2005). Factors affecting the impact of professional development programs on teachers knowledge, practice, student outcomes & efficacy. Professional Development for Teachers and School Leaders, 13(10). Jensen, J. (2008). Developing historical empathy through debate: An action research study. Social Studies Research and Practice, 3(1), 55-67. Jones, S. M., Hoglund, W. L. G., Brown, J. L., & Aber, J. L. (2010). A school-randomized clinical trial of an integrated social-emotional learning and literacy intervention: Impacts after 1 school year. Journal of Consulting and Clinical Psychology, 78(6), 829842. Kahne, J., Chi, B., & Middaugh, E. (2006). Building social capital for civic and political engagement: The potential of high school government courts. Canadian Journal of Education, 29(2), 387409. Kahne, J., Middaugh, E., & Schutjer-Mance, K. (2005). California civic index [Monograph]. New York, NY: Carnegie Corporation and Annenberg Foundation. Kahne, J., & Sporte, S. (2008). Developing citizens: The impact of civic learning opportunities on students commitment to civic participation. America Educational Research Journal, 45(3), 738766. Kahne, J., & Westheimer, J. (2003). Teaching democracy: What schools need to do. Phi Delta Kappan, 84(1): 3440, 5767. Kim, J. S., Olson, C. B., Scarcella, R., Kramer, J., & Pearson, M. (2011). A randomized experiment of a cognitive strategies approach to text-based analytical writing for mainstreamed Latino English language learners in grades 6 to 12. Journal of Research on Educational Effectiveness, 4(3), 231263. Kohlberg, L. (1984). The psychology of moral development: The nature and validity of moral stages. In Essays on moral development (Vol. 2). San Francisco, CA: Harper & Row. Kuhn, D., Wang, Y., & Li, H. (2011). Why argue? Developing understanding of the purposes and values of argumentative discourse. Discourse Processes, 48(1), 2629. Lee, P., & Ashby, R. (2000). Progression in historical understanding among students ages 714. In P. Stearns, P. Seixas, & S. Wineburg (Eds.), Knowing, teaching, and learning history: National and international perspectives (199222). New York, NY: New York University Press. Lee, P., Dickinson, A., & Ashby, R. (2001). Childrens ideas about historical explanation. In A. Dickinson, P. Gordon, & P. Lee (Eds.), Raising standards in history education (97115). London, UK: Woburn Press. Lee, P., & Shemilt, D. (2003). A scaffold, not a cage: Progression and progression models in history. Teaching History, 113, 1322. Leming, J., Ellington, L., & Schug, M. (2006). The state of social studies: A national random survey of elementary and middle school social studies teachers. Social Education, 70(5), 322327. Levine, P. (2009). The civic opportunity gap. Educational Leadership, 66(8), 2025. Levinson, M. (2010). The civic empowerment gap: Defining the problem and locating solutions. In L. R. Sherrod, J. Torney-Purta, & C. A. Flanagan (Eds.), Handbook of research on civic engagement. Hoboken, NJ: John Wiley & Sons, Inc. Levstik, L. S., & Tyson, C. A. (Eds.). (2008). Handbook of research in social studies education. New York, NY: Routledge. Lieberman, M. (1981). Facing History and Ourselves: A project evaluation. In L. Kuhmerker & D. Cochrane (Eds.), Moral education forum. New York, NY: Hunter College, City University of New York. Lowenstein, E. (2005). Teacher development in education for social and political responsibility: The case of Facing History and Ourselves. Paper presented at annual meeting of the College and University Faculty Assembly of the National Council of the Social Studies, Kansas City, MO. Lowenstein, E., & Facing History and Ourselves. (2007). Teaching for informed civic engagement efficacy belief inventory. Brookline, MA: Ethan Lowenstein & Facing History and Ourselves National Foundation, Inc. Maslach, C., Jackson, S. E., & Schwab, R. L. (1996). Maslach Burnout InventoryEducators Survey (MBI-ES). In C. Maslach, S. E. Jackson, & M. P. Leiter (Eds.), MBI Manual (3rd ed.). Palo Alto, CA: Consulting Psychologists Press. Matsumura, L. C., Garnier, H. E., & Spybrook, J. (2012). The effect of content-focused coaching on the quality of classroom text discussions. Journal of Teacher Education, 63(3), 214228. McConahay, J. B. (1986). Modern racism, ambivalence, and the Modern Racism Scale. In J. F. Dovidio & S. L. Gaertner (Eds.), Prejudice, discrimination, and racism (91125). Orlando, FL: Academic Press. Milson, A. J. (2003). Teacher efficacy beliefs for the formation of students' character. Paper presented at the annual meeting of the Association for Moral Education, Krakow, Poland. National Research Council Institute of Medicine. (2003). Engaging schools: Fostering high school students motivation to learn. Washington, DC: The National Academy Press. The National Task Force on Civic Learning and Democratic Engagement. (2012). A crucible moment: College learning and democracys future. Washington, DC: Association of American Colleges and Universities. Partnership for 21st Century Skills. (2006). Results that matter: 21st century skills and high school reform. Retrieved from www.21stcenturyskills.org Pianta, R. C., & Allen, J. P. (2008). Building capacity for positive youth development in secondary school classrooms: Changing teachers interactions with students. In M. Shinn & H. Yoshikawa (Eds.), Toward positive youth development: Transforming schools and community programs. New York, NY: Oxford University Press. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods. In Advanced quantitative techniques in the social sciences (Vol. 1). Thousand Oaks, CA: SAGE Publications, Inc. Reisman, A., & Wineberg, S. (2012). Ways of knowing and the history classroom: Supporting disciplinary discussion and reasoning about texts. In M. Carretero, M. Arsenio, & M. Rodriguez-Moneo (Eds.), History education and the construction of national identities (171189). Charlotte, NC: Information Age Publishing. Romer, A. L. (2006). Choices in Little Rock: Follow-up study with 15 urban teachers (Unpublished report). Brookline, MA: Facing History and Ourselves Foundation, Inc. Ross, J., & Bruce, C. (2007). Professional development effects on teacher efficacy: Results of randomized field trial. The Journal of Educational Research, 101(1), 5060. Ross, J. A. & Gray, P. (2007). Transformational leadership and teacher commitment to organizational values: The mediating effects of collective teacher efficacy. School Effectiveness and School Improvement, 17, 179199. Schochet, P. Z. (2008). Technical methods report: Guidelines for multiple testing in impact evaluations (NCEE 2008-4018). Washington, DC: National Center for Education Evaluation and Regional Assistance. Schultz, L. H., Barr, D. J., & Selman, R. L. (2001). The value of a developmental approach to evaluating character development programmes: An outcome study of Facing History and Ourselves. Journal of Moral Education, 20(1), 327. Schultz, L. H. & Selman, R. L. (1998). Ego development and interpersonal development in young adulthood: A between-model comparison. In P. M. Westenberg, A. Blasi, & L. D. Cohn, (Eds.), Personality development: Theoretical, empirical, and clinical investigations of Loevingers conception of ego development (183203). Hillsdale, NJ: Erlbaum. Schultz, L. H., Selman, R. L., & LaRusso, M. (2003). The assessment of psychosocial maturity in children and adolescence: Implications for the evaluation of school-based character education programs. Journal of Research in Character Education, 1(2), 6787. Schwartz, M. (2008). Teacher education for moral and character education. In L. P. Nucci & D. Narvaez (Eds.), Handbook of moral and character education. New York, NY: Routledge, Taylor & Francis. Sechser, C. C. & Barr, D. J. (2005). Tennessee safe and drug-free schools: 2004 educator follow-up study (Unpublished report). Brookline, MA: Facing History and Ourselves Foundation, Inc. Seixas, P. (1996). Conceptualizing the growth of historical understanding. In D. Olson & N. Torrance (Eds.), Handbook on education and human development: New models of learning, teaching, and schooling. Oxford, UK: Blackwell. Seixas, P. (2000). Schweigen! Die Kinder! Or, does postmodern history have a place in the schools? In P. Stearns, P. Seixas, & S. Wineburg (Eds.), Knowing, teaching, and learning history: National and international perspectives (1937). New York, NY: New York University Press. Selman, R. L. (2003). The promotion of social awareness: Powerful lessons from the partnership of developmental theory and classroom practice. New York, NY: Russell Sage Foundation. Selman, R. L., & Barr, D. J. (2009). Can adolescents learn to create ethical relationships for themselves in the future by reflecting on ethical violations faced by others in the past? In M. Martens, U. Hartmann, M. Sauer, & M. Hasselhorn (Eds.), Interpersonal understanding in historical context (1941). Rotterdam, The Netherlands: Sense Publishers. Selman, R. L., Barr, D. J., Feigenberg, L., & Facing History and Ourselves. (2007). Choices in context measure. Brookline, MA: Robert L. Selman and Facing History and Ourselves National Foundation, Inc. Selman, R. L. & Bellino, M. (2012). The intersection of historical understanding and ethical reflection during early adolescence: A place where time is squared. In M. Carretero, M. Arsenio, & M. Rodriguez-Moneo (Eds.), History education and the construction of national identities (171189). Charlotte, NC: Information Age Publishing. Selman, R. L., & Kwok, J. (2010). Informed social reflection: Its development and importance for adolescents' civic engagement. In Handbook of research on civic engagement in youth (651–683). Hoboken, NJ: Wiley. Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14. Skaalvik, E. M., & Skaalvik, S. (2007). Dimensions of teacher self-efficacy and relations with strain factors, perceived collective teacher efficacy, and teacher burnout. Journal of Educational Psychology, 99, 611–625. Slavin, R. E., Cheung, A., Groff, C., & Lake, C. (2008). Effective reading programs for middle and high schools: A best-evidence synthesis. Reading Research Quarterly, 43(3), 290322. Sleeper, M., & Stern Strom, M. (2006). Facing history and ourselves. In M. J. Elias & H. Arnold (Eds.), The educators guide to emotional intelligence and academic achievement: Social-emotional learning in the classroom. Thousand Oaks, CA: Corwin Press. Snow, E., Lawrence, L. F., & White, C. (2009). Generating knowledge of academic language among urban middle school students. Journal of Research on Educational Effectiveness, 2(4), 325344. Solomon, D., Watson, M. S., & Battistich, V. A. (2002). Teaching and schooling effects on moral/prosocial development. In V. Richardson (Ed.), Handbook for research on teaching. Washington, DC: American Educational Research Association. Stoskopf, A., Bermudez, A., Hartmann, E., Selman, R. L., Barr, D. J., & Facing History and Ourselves. (2007). An assessment measure for students historical understanding. Brookline, MA: Facing History and Ourselves National Foundation Inc. Strom, M. S. (Ed.). (2004). Facing history and ourselves: Holocaust and human behavior. Brookline, MA: Facing History and Ourselves National Foundation, Inc. Sullivan, H. S. (1953). The interpersonal theory of psychiatry. New York, NY: Norton. Sullivan, J. L., Piereson, J., & Marcus, G. E. (1982). Political tolerance and AmericandDemocracy. Chicago, IL: University of Chicago Press. Sykes, G. (1996). Reform of and as professional development. Phi Delta Kappan, 77, 465467. Torney-Purta, J. (2009). International research that matters for policy and practice. American Psychologist, 64, 822837. Torney-Purta, J., Richardson, W. K., & Barber, C. H. (2005). Teachers educational experience and confidence in relation to students civic knowledge across countries. International Journal of Citizenship and Teacher Education, 1(1), 3257. Toshalis, E. & Nakkula, M. J. (2012). The integration of motivation, engagement, and voice in student-centered learning. Invited paper for the Students at the Center: Teaching and Learning in the Era of the Common Core project, sponsored by Jobs for the Future and funded by the Nellie Mae Foundation. Vontz, T. S., & Leming, R. S. (2005). Designing and implementing effective professional development in civic education. International Journal of Social Education, 20(2), 6787. Wei, R. C., Darling-Hammond, L., & Adamson, F. (2010). Professional learning in the United States: Trends and challenges. Dallas, Texas: National Staff Development Council. Wineberg, S. (2000). Making historical sense. In P. N. Stearns, P. Seixas, & S. Wineburg (Eds.), Knowing, teaching, & learning history: National and international perspectives. New York, NY: New York University Press. Woolfolk-Hoy, A., & Davis, H.A. (2006). Teacher self-efficacy and its influence on the achievement of adolescents. In F. Pajares & T. Uran (Eds.), Self-efficacy beliefs of adolescents (117137). Greenwich, CT: Information Age Publishing. Yoon, K. S., Duncan, T., Lee, S. W., Scarloss, B., & Shapley, K. L. (2007). Reviewing the evidence on how teacher professional development affects students achievement (Issues & Answers Reports, REL 2007-No. 033). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest. Retrieved from http://ies.ed.gov/ncee/edlabs Youniss, J. (1980). Parents and peers in social development: A Sullivan-Piaget perspective. Phoenix, AZ: The University of Chicago Press. Youniss, J., & Levine, P. (Eds.). (2009). Engaging young people in civic life. Nashville, TN: Vanderbilt University Press. APPENDIX A.
|
|
|
|
   |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||
![[39_17470.htm_g/00002.jpg]](/Html/39_17470.htm_g/00002.jpg)
![[39_17470.htm_g/00004.jpg]](/Html/39_17470.htm_g/00004.jpg)
![[39_17470.htm_g/00006.jpg]](/Html/39_17470.htm_g/00006.jpg)
![[39_17470.htm_g/00008.jpg]](/Html/39_17470.htm_g/00008.jpg)
![[39_17470.htm_g/00010.jpg]](/Html/39_17470.htm_g/00010.jpg)
Executive Summary
Full Text in PDF Format
Email this article
Print this article
Post a Comment
