Home Articles Reader Opinion Editorial Book Reviews Discussion Writers Guide About TCRecord
transparent 13
Topics
Discussion
Announcements
 

Identifying Levers Related to Student Performance on High-Stakes Science Exams: Examining School, Teaching, Teacher, and Professional Development Characteristics


by Christian Fischer, Brandon Foster, Ayana McCoy, Frances Lawrenz, Chris Dede, Arthur Eisenkraft, Barry J. Fishman, Kim Frumin & Abigail Jurist Levy - 2020

Background: Many students enter into postsecondary education without the preparation to face the demands of postsecondary coursework in science. Increasingly, policymakers and educational researchers are responding to calls for reforming secondary education to provide more opportunity for all students to receive high-quality education and to become career and college ready.

Purpose: This study attempts to identify levers to increase student learning in secondary education. In particular, it examines relationships between school, teaching, teacher, and teacher professional development characteristics and student scores on high-stakes Advanced Placement (AP) examinations in the sciences.

Setting: This study is situated in the context of the large-scale, top-down, nationwide AP curriculum and examination reform in the sciences (biology, chemistry, physics) in the United States. This is an unprecedented opportunity to analyze changing educational landscapes in the United States with large-scale national student-, teacher-, school-, and district-level datasets across multiple science disciplines and different stages of the curriculum reform implementation connected to a standardized and high-stakes student outcome measure.

Population: This study analyzes nationwide data samples of the AP Biology, AP Chemistry, and AP Physics population during the first, second, and third year of the curriculum reform implementation. Across disciplines and years, the analytical samples include a total of 113,603 students and 6,046 teachers.

Research design: This empirical quantitative study uses data from web-based surveys sent to all AP science teachers. Additionally, the College Board provided student- and school-level data for all students taking AP examinations. Data preparation methods included exploratory and confirmatory factor analysis. Associations with student achievement were analyzed through a multilevel ordered logistic regression analysis, separately by science discipline and year of the curriculum reform implementation. Afterwards, results were aggregated through a meta-analysis.

Findings: Even after controlling for student background variables, roughly 60% of the AP score variance could be explained at the teacher and school levels. In particular, teachers’ perceived administrative support, self-efficacy, teaching experience, and elements of classroom instruction were related to student performance. Notably, teachers’ professional development participation—which has been a major focus of interventions—has a small, mixed impact on student achievement.

Conclusion: The identified levers for improving student achievement provide a strong rationale for the continued efforts of policymakers to improve school environments and to support science teachers, with the ultimate goal of improving student learning to help all students to be prepared for college and ready for their future careers.



There is continued concern about whether students in the United States are ready to meet the demands of postsecondary education. Recent statistics indicate that roughly 19% of college students who matriculate into four-year institutions and roughly 39% of college students in two-year institutions do not return for further studies in the following fall term (National Student Clearninghouse, 2017; Snyder, de Brey, & Dillow, 2016). For students who began college in 2008, only 60% who set out to complete a bachelor’s degree finish within six years, and only 31% of students who set out to complete a two-year degree finish within 150 percent of normal time (Snyder et al., 2016). For students in STEM fields between 2003 and 2009, 48% of bachelor’s degree students and 69% of associate’s degree students left the STEM field by 2009 (Chen, 2013). These statistics indicate that a large number of students enter into postsecondary education without the college-readiness skills that allow them to face the demands of postsecondary education. Consequently, policy efforts have targeted the improvement of student science achievement and postsecondary persistence rates of students in the United States (i.e., Framework for K-12 Science Education [National Research Council, 2012a], Next Generation Science Standards [NGSS; NGSS Lead States, 2013]). Further, policymakers, as well as the business community, believe that the foundation of a successful 21st-century American labor market will depend on the effectiveness of these actions (National Academy of Sciences, National Academy of Engineering, & Institute of Medicine, 2007).


Policy-focused efforts to improve the college readiness of American students rely on having valid indicators for college readiness. Many policymakers believe that student performance on Advanced Placement (AP) tests, and in particular the AP science examinations, are viewed as indicators of college readiness (Cromwell, McClarty, & Larson, 2013). The AP program provides rigorous, college-level material to high school students across a broad range of subjects. In recent years, the College Board, the provider of the AP program, revised the AP examinations in the sciences (biology, chemistry, physics) to respond to recommendations from the National Research Council (2002) to reduce its former emphases on rote learning and algorithmic procedures, and to increase emphases on scientific inquiry, reasoning, and depth of understanding. These changes are in line with other recommended large-scale national curriculum shifts in the sciences such as the Framework for K-12 Science Education (National Research Council, 2012a) and the NGSS (NGSS Lead States, 2013). The AP Biology examinations were redesigned in 2013, followed by AP Chemistry in 2014 and AP Physics in 2015. Both before and after the AP redesign, a unique aspect of the AP program is that there are no “official” College Board curriculum materials for courses. Before receiving the designation of an “Advanced Placement” course, teachers must develop their own curriculum plan, which must be certified by the College Board, or choose one presented by the College Board. The College Board defines the curriculum standards for AP courses, has them reviewed at over 100 different universities by professors teaching parallel courses, and offers corresponding examinations (i.e., the AP exams) to assess students’ content mastery that are scored on a 1–5 scale. Depending on university policies, students who earn a 3 or higher may use their scores toward college-level credit and/or for placement in higher-level college courses. It should be noted that there are individuals in high schools and universities that do not value the AP science examinations. This may be similar to individuals at universities who do not agree that transfer students should receive credit for courses they have taken at another institution. Be that as it may, a great number of universities (including Tier 1 schools) do accept scores on AP science examinations and provide students with credit and/or placement; in this way, they treat the AP science scores as equivalent to courses at their own institutions.

With respect to equity-related perspectives, the AP program has been criticized as not sufficiently narrowing achievement and opportunity gaps and even excluding students who are traditionally underserved as they have not been afforded equal access; thus, students’ class and race have been shown to affect success in AP programs (e.g., Klopfenstein, 2004a, 2004b; Klugman, 2013; Ladson-Billings & Tate, 1995; Lichten, 2010; Schneider, 2009). There have been multiple efforts by the College Board, multiple U.S. states, and local initiatives to substantially increase access to AP courses (e.g., Conger, Long, & Iatarola, 2009; Roegman & Hatch, 2016; The College Board, 2014; Wyatt & Mattern, 2011). Still, the use of academic “tracking” and local school cultures might limit AP course enrollments for students who might benefit from such AP expansion initiatives (Klopfenstein, 2004b; Klugman, 2013; Solorzano & Ornelas, 2004; Zarate & Pachon, 2006). Once enrolled in AP courses, students might not be provided with equitable learning opportunities due to variations in teacher knowledge and qualifications, instructional quality, and school environments in communities with underserved students (Hallett & Venegas, 2011; Kyburg, Hertberg-Davis, & Callahan, 2007; Taliaferro & DeCuir-Gunby, 2008). For instance, schools with large proportions of students who are historically marginalized often face challenges to recruit and retain highly qualified teachers (e.g., Borman & Dowling, 2008; Clotfelter, Ladd, & Vigdor, 2005; Ronfeldt, Loeb, & Wyckoff, 2013). Also, AP courses may not always present science learning in ways that honor or acknowledge cultural practices of all students nor provide students opportunities for diverse sense-making and intellect generation (Brown, 2004, 2006; Warren, Ballenger, Ogonowski, Rosebery, & Hudicourt-Barnes, 2001). Furthermore, increased access to AP courses does not simply equate to greater success on the AP examinations. In the class of 2013, out of all students receiving at least one passing score (3 or higher) on an AP exam during high school, only 21.7% were eligible for free or reduced-priced lunch programs compared to 78.3% of students not eligible for free or reduced-priced lunch programs (The College Board, 2014). According to College Board data (our calculation based on data retrieved from The College Board, 2014), out of all students eligible for free or reduced-priced lunch programs, 47.8% received at least one passing score. This is considerably lower compared to the group of students not eligible for free or reduced-priced lunch programs, in which 65.4% received at least one passing score.


Nonetheless, the College Board’s AP exams are often viewed as an indicator of college readiness. Despite the popularity of the AP program, relatively little is known with regard to associations between school context factors and the revised AP science examinations. If the AP science exams are to improve the college readiness of students, it is important to identify factors that may predict students’ success on these (and other high-stakes) examinations that are susceptible to intervention or policy reform initiatives. Knowledge of these associations might enable an understanding of how they could be leveraged to affect policy initiatives aimed at improving student learning and achievement. In particular, education policy research that focuses on positively affecting student outcomes typically makes the distinctions between different types of variables; namely, those factors of a student or his or her environment that are more easily affected by interventions, and those features that are less easily affected (i.e., socioeconomic status, race, gender) (Fives & Buehl, 2016; Jackson, Rockoff, & Staiger, 2014; Roegman & Hatch, 2016). Most typically, researchers are interested in understanding the association between more easily changeable factors and student outcomes, after controlling for student characteristics or contextual features from students’ environments that are understood to be more difficult to change, but are associated with student outcomes (e.g., socioeconomic status). In terms of important features of a student’s environment, much focus has been given to the role of teachers in promoting students’ educational success (Cohen & Ball, 1999; Hattie, 2012; Seidel & Shavelson, 2007). Even more recently, attention has been given to understanding how teachers respond to large-scale education reform agendas, such as the College Board’s redesign of the AP examinations, with particular interests in understanding teacher and school characteristics that lead to successful implementation of reform initiatives (Fives & Buehl, 2016). This is a natural progression for inquiry, as teachers are primarily responsible for enacting policy decisions, and their beliefs about teaching, beliefs about the abilities and strengths of their students, and subject-specific knowledge all can combine to impact the fidelity and efficacy of the reform initiative and subsequent student outcomes (Cohen & Ball, 1999; Hattie, 2012; Seidel & Shavelson, 2007). A specific goal of this paper is to investigate a range of potentially modifiable teacher, teaching, and school characteristics that are correlated with students’ performance on the redesigned AP science examinations, which could be used as levers for policy interventions.


LITERATURE REVIEW


IMPORTANCE OF STUDENT CHARACTERISTICS FOR STUDENT PERFORMANCE

Student characteristics account for large shares of variance in students’ performance on standardized high-stakes examinations. Therefore, when modeling relationships with student achievement, it is imperative to account for such student characteristics to reduce omitted variable biases. In particular, students’ prior performance on standardized tests such as the Preliminary Scholastic Aptitude Test (PSAT) are among the largest predictors of students’ AP scores (Ewing, Camara, & Millsap, 2006; Ewing, Huff, & Kaliski, 2010; Zhang, Patel, & Ewing, 2014). Furthermore, researchers have used advanced econometric techniques to show that much of the predictive power of student aptitude tests is driven by student-level socioeconomic factors (Atkinson & Geiser, 2009; Rothstein, 2004). In addition, students’ family background is often considered an important factor in predicting student success. For instance, several research studies indicate associations of parental educational attainment with student achievement and success (Davis-Kean, 2005; Desforges & Abouchaar, 2003; Fischer et al., 2018; Woessmann, 2004). However, despite the importance of student-level variables in predicting students’ performance on tests like the AP science exams, these factors have typically been considered much less amenable to policy interventions. Therefore, researchers have increasingly focused on identifying potential levers on the school and teacher levels that might be associated with increases in student science achievement.


IMPORTANCE OF SCHOOL CHARACTERISTICS FOR STUDENT PERFORMANCE


At the school level, researchers have identified several factors associated with student science achievement. It is theorized that school composition can be associated with several aspects of school and classroom context, which in turn can impact the quality of instruction students are exposed to, and subsequently their performance on high-stakes tests like the AP exam (Willms, 2010). Current data indicates that 15% of the student score variance on the PISA benchmark tests can be explained by students’ socioeconomic status (SES) (OECD, 2013). Because selection into schools is non-random, schools in high-poverty areas tend to have the highest concentration of low-SES students. The achievement gap in science and mathematics tends to be larger for these students when compared to their more affluent peers, and they also experience higher incidents of psychosocial challenges (Crosnoe, 2009). Further, the socioeconomic stratification of students into schools is associated with high school graduation and college enrollment rates (Palardy, 2013). Also, school socioeconomic stratification has been shown to affect associations with teacher expectations for their students’ performance, how teachers enact instructional practices, as well as other indicators of teacher quality and student achievement (Brault, Janosz, & Archambault, 2014; Nye, Konstantopoulos, & Hedges, 2004; Sass, Hannaway, Xu, Figlio, & Feng, 2012; Supovitz & Turner, 2000; Willms, 2010).

In order to buffer against these deleterious effects, many researchers have highlighted the importance of contextual factors such as administrative and principal support, which might be leveraged to provide necessary support to science teachers in order to improve their working conditions, and subsequently improve student achievement. For example, studies indicate that supportive work environments; coherent, mission-driven school cultures; and effective administrative leadership structures (i.e., structures that provide teachers with resources, opportunities to collaborate, and instill trust) are associated with increased teacher effectiveness and improved student performance (Johnson, Kraft, & Papay, 2012; Kraft, Marinell, & Shen-Wei Yee, 2016; Supovitz, Sirinides, & May, 2010; van Geel, Keuning, Visscher, & Fox, 2016; Waters, Marzano, & McNulty, 2003). Taken in sum, the aforementioned research suggests that, while school socioeconomic status might appear to have a strong and hard-to-influence effect on student achievement, some features of the school context can be leveraged to improve teacher working conditions, efficacy, and ultimately student achievement.


IMPORTANCE OF TEACHER AND TEACHING CHARACTERISTICS FOR STUDENT PERFORMANCE


The role of teachers as the primary driver of student learning and performance has been widely accepted. As such, researchers have long focused on (1) identifying systematic differences between teachers in their ability to raise student test scores, (2) examining how important any potential systematic differences are for raising student test scores while accounting for fixed factors (i.e., family education, family income, etc.), and (3) whether any potential differences in teacher quality are associated with observable characteristics of teachers and schools (Rivkin, Hanushek, & Kain, 2005). The latter is popular among researchers within education reform circles, largely due to the belief that observable characteristics that are associated with teacher quality can be leveraged by policy initiatives to improve overall teacher quality and subsequent student learning (Cohen & Ball, 1999; Hattie, 2012; Seidel & Shavelson, 2007).


Among teacher characteristic variables, teachers’ knowledge (i.e., subject matter knowledge, pedagogical content knowledge, curricular knowledge; Shulman, 1986) and teaching experience have been hypothesized as important predictors of both teacher quality and student outcomes (Hattie, 2012; National Research Council, 2005, 2012b). Teachers’ instructional capacity grows with frequent and continuous interactions with students and materials (Cohen & Ball, 1999). Thus, research consistently demonstrated that teachers’ knowledge and experience is associated with increases in teachers’ educational effectiveness and subsequent student achievement gains in the sciences and beyond (Boyd et al., 2008; Carlsen, 1993; Daly, Moolenaar, Der-Martirosian, & Liou, 2014; Keller, Neumann, & Fischer, 2017; Nye et al., 2004). Similarly, teachers who can concentrate their years teaching on one grade level are able to further increase their students’ performance compared to teachers with similar overall experience teaching a wider range of grades (Ost, 2014). In addition, internal psychological constructs, such as personality, motivation, and normative beliefs, influence teachers’ adoption of curriculum reforms and instructional enactments (Klassen & Tze, 2014; van Aalderen-Smeets & Walma van der Molen, 2015; Veal, Riley Lloyd, Howell, & Peters, 2016). For instance, self-efficacy beliefs not only relate to teachers’ perceptions of their working conditions, job satisfaction, and burnout, but also with student learning and achievement (Caprara, Barbaranelli, Steca, & Malone, 2006; Klassen & Tze, 2014; Skaalvik & Skaalvik, 2010).

Teachers’ classroom instruction is often seen as a lever to directly influence student learning, as evidenced in countless educational policy reforms aiming for changes in curriculum, pedagogy, and teaching practice. A wide range of research studies support these educational policy reforms by identifying associations of instructional practices and curriculum designs with differential student outcomes (Desimone, Smith, & Phillips, 2013; Hattie, 2012; Seidel & Shavelson, 2007). In particular, science education research strands emphasized positive associations of reform-, standards-, or inquiry-based instructional practices with students’ science achievement (Bismack, Arias, Davis, & Palincsar, 2015; Furtak, Seidel, Iverson, & Briggs, 2012; Hamilton et al., 2003; Houseal, Abd-El-Khalick, & Destefano, 2014; Mikeska et al., 2017; Secker, 2002). In the context of high school science courses, laboratory investigations represent a common example of such inquiry-based instructional enactments (Abd-El-Khalick et al., 2004; National Research Council, 2006). Although a tremendous wealth of research documents the importance of each individual teacher and teaching characteristic in influencing student learning and achievement, there seems to be an underdeveloped research base evaluating the relationships between this multitude of teacher-level constructs and student performance on high-stakes examinations in a multilevel framework in the sciences.


IMPORTANCE OF TEACHER PROFESSIONAL DEVELOPMENT FOR STUDENT PERFORMANCE


Many different forms of capacity building can be grouped under the term “professional development” (PD), including conventional PD activities (i.e., face-to-face workshops, online courses, and peer-based professional learning communities), as well as informal PD learning opportunities (i.e., one-on-one mentoring/coaching, conference participations, and use of instructional materials such as textbook teacher guides or journal/magazine articles). Theoretical conceptualizations of PD posit that teacher participation in PD can increase teachers’ knowledge and skills, which leads to increased confidence in enacting curriculum elements and improved self-efficacy. That improvement in turn can increase the effectiveness of instruction, and subsequently promote positive outcomes in students (Desimone, 2009).

Decades of research on the effectiveness of PD for improving teacher learning, which is strongly rooted in research on science teacher education (e.g., Banilower, Heck, & Weiss, 2007; Fishman, Marx, Best, & Tal, 2003; Garet, Porter, Desimone, Birman, & Yoon, 2001; Penuel, Fishman, Yamaguchi, & Gallagher, 2007), have resulted in a consensus on the design characteristics that might constitute “high quality” PD. Desimone (2009) summarizes these design characteristics as active learning, coherence, content focus, collective participation, and duration. This summary is similar to other reviews of the PD literature (e.g., Borko, Jacobs, & Koellner, 2010; Darling-Hammond & Richardson, 2009). The empirical research base on the effectiveness of PD in directly enhancing student learning and performance is mixed (Desimone & Garet, 2015; Kennedy, 2016). Several studies indicate positive effects of PD participation on student performance in science contexts (e.g., Doppelt et al., 2009; Fishman et al., 2013; Frumin et al., 2018; Lee, Deaktor, Enders, & Lambert, 2008; Llosa et al., 2016; Price & Chiu, 2018; Roth et al., 2011; Schuchardt, Tekkumru-Kisa, Schunn, Stein, & Reynolds, 2017), especially if PD activities have a content focus, model teaching, and/or include active learning components (Gersten, Dimino, Jayanthi, Kim, & Santoro, 2010; Penuel, Gallagher, & Moorthy, 2011). However, other studies were not able to detect positive effects of PD participation on student achievement, even if PD activities are aligned to the high-quality PD design characteristics (Garet et al., 2008, 2011; Jacob & McGovern, 2015). This mixed evidence base for PD effectiveness might evoke questions about the likely impact PD might have on student outcomes in real-world scenarios. Research that systematically identifies potential percentages of student score variation that could be explained with teachers’ PD participation might provide a promising novel perspective on this research base.


RESEARCH QUESTIONS

This study is connected to a large-scale, multi-methodological, longitudinal National Science Foundation-funded research project that investigates the choice(s) of PD that AP science teachers made and the correlation of teachers’ PD with their students’ AP science scores. The project takes advantage of an unprecedented opportunity to match student AP scores with teacher and school data for three different science disciplines over time. This paper uses data from this broader study to identify potential levers to increase student performance on high-stakes standardized examinations. Our study addresses the following two research questions:


How is the variance in students’ AP science scores distributed across different conceptual components (i.e., PSAT scores, mother’s educational attainment, school and teacher characteristics, PD participation) at the student- and teacher/school-levels, respectively?


What school, teaching, teacher, and teacher PD variables are positively associated with students’ AP science scores, after controlling for student characteristics?

METHODS


DATA SOURCES AND SAMPLE


The data used in this study was provided by the College Board and collected through web-based surveys. The College Board’s data includes information about students’ AP and PSAT scores, students’ family background, and contextual school and district information. The  surveys were sent to all AP Biology, AP Chemistry, and AP Physics teachers in the nation (unless placed on a do-not-contact list by the College Board). Key categories of the survey instrument included teachers’ teaching background, classroom instruction, classroom and school context, attitudes, and participation in PD. An exemplary full survey in Fischer (2017) includes all items of the survey instrument used in this study. Response rates ranged from 23% to 34%, which represents a good response rate for web-based surveys with this population size (Shih & Fan, 2009).


Validity and reliability of the survey data was improved through a multitude of measures. Survey pilots were reviewed by a national panel of researchers with expertise on the AP program, PD, science content, science education, and measurement. Also, AP teachers were invited to respond to survey items utilizing a talk-aloud cognitive interview approach to validate that intended item meanings were congruent with teacher interpretations (Desimone & Le Floch, 2004). Furthermore, survey responses were validated by cross-checking randomly selected teachers who indicated on the surveys to have participated in the College Board’s five-day AP summer institute with the College Board’s official PD attendance records, which resulted in a greater than 90% accuracy. In general, survey research methodologies serve an important role in better understanding teachers and teaching practices, in particular when considering a balance between the feasibility of large-scale data collection and overall reliability and validity of teacher self-reports (Fenstermacher, 1994; Rowan, Correnti, & Miller, 2002).


This study uses data from the second and third year of the AP Biology redesign (2014, 2015) (Year 2: Nteacher = 1,265, Nstudent = 25,108; Year 3: Nteacher = 1,243, Nstudent = 21,429), the first two years of the AP Chemistry redesign (2014, 2015) (Year 1: Nteacher = 1,443, Nstudent = 26,449; Year 2: Nteacher = 1,219, Nstudent = 21,204), and the first year of the AP Physics redesign (2015) (Nteacher = 876, Nstudent = 19,413). Although the larger study collected data from the first year of the AP Biology redesign, the survey was revised significantly between the first and second/third years, making first-year AP Biology data substantially less consistent with the analyses in this article. Consequently, the first year of the AP Biology redesign was not included in this study. Also, while there is only one form of the AP Biology and AP Chemistry exams, there are several different forms for AP Physics. This study only examined AP Physics 1.


Observations with missing data on variables included in the analysis were removed from the analytical samples. Due to the absence of student–teacher identifiers, only teachers and students in schools with only one AP Biology or one AP Chemistry or one AP Physics 1 teacher were included in the analytical samples. The effect sizes of differences between analytical and comparison samples on student performance on PSAT and AP science examinations, as well as the school-level percentage of student enrollment in free or reduced-priced lunch programs, are very small (Table 1). Therefore, analytical samples can be assumed to be a good representation of the entire student and school AP science population in the United States.

Table 1. Non-response Analysis

 

z

p

r

Students’ PSAT scores

Biology Year 2

6.036

< 0.001

0.015

Biology Year 3

5.436

< 0.001

0.013

Chemistry Year 1

2.898

< 0.001

0.009

Chemistry Year 2

7.407

< 0.001

0.022

Physics Year 1

8.003

< 0.001

0.022

Students’ AP scores

Biology Year 2

–22.164

< 0.001

0.050

Biology Year 3

–19.190

< 0.001

0.042

Chemistry Year 1

–25.728

< 0.001

0.072

Chemistry Year 2

–16.093

< 0.001

0.043

Physics Year 1

–8.475

< 0.001

0.022

Percentage free or reduced-priced lunch programs

Biology Year 2

–8.088

< 0.001

0.089

Biology Year 3

–9.327

< 0.001

0.101

Chemistry Year 1

–9.670

< 0.001

0.122

Chemistry Year 2

–8.695

< 0.001

0.106

Physics Year 1

–8.792

< 0.001

0.132



Overall, 61% of AP teachers in this study are female, on average 45 years old, and predominantly identified as White (91%), followed by Asian or Asian American (5%), Hispanic or Latino (4%), and Black or African American (3%). AP teachers in this study most commonly hold master’s degrees (67%), followed by bachelor’s degrees (21%) and doctoral degrees (7%). Approximately 86% of AP teachers in this study majored in the corresponding disciplinary area (i.e., physical/life science) and 45% majored in secondary education. On average, AP teachers in this study had 14 years of high school science teaching experience and six years of AP teaching experience. Table 2 describes the above teacher characteristics of AP teachers included in this study separately for each discipline and year of the AP reform implementation.

Table 2. Descriptive Information of the Teachers Included in This Study

 

Biology Y2

Biology Y3

Chemistry Y1

Chemistry Y2

Physics Y1

 

N / Mean

% / (SD)

N / Mean

% / (SD)

N / Mean

% / (SD)

N / Mean

% / (SD)

N / Mean

% / (SD)

Gender

 


 


   


 


    Male

355

28.77

363

29.54

538

38.05

456

37.78

549

63.32

    Female

878

71.15

865

70.38

874

61.81

749

62.05

318

36.68

    Other

1

0.08

1

0.08

2

0.14

2

0.17

0

0.00

Age

45.21

(9.86)

45.01

(10.18)

45.90

(10.44)

45.90

(10.43)

45.04

(10.65)

Racial/ethnic background


 


   


 


    White

1,137

92.21

1,121

91.21

1,284

90.74

1,083

89.65

790

91.54

    Black or African

       American

37

3.00

36

2.93

47

3.32

43

3.56

20

2.32

    Asian or Asian

       American

48

3.89

48

3.91

76

5.37

72

5.96

43

4.98

    Hispanic or Latino

49

3.97

51

4.15

48

3.39

45

3.73

27

3.13

    American Indian,

       Alaska Native, or

       Pacific Islander

22

1.78

21

1.71

34

2.40

26

2.15

20

2.32

Educational Attainment


 


   


 


    Associate’s degree

1

0.08

1

0.08

1

0.07

1

0.08

0

0.00

    Bachelor’s degree

254

20.09

256

20.61

295

20.44

245

20.10

213

24.34

    Master’s degree

875

69.22

849

68.36

974

67.50

821

67.35

572

65.37

    Certificate of

       Advanced Study

62

4.91

58

4.67

60

4.16

50

4.10

29

3.31

    Doctoral degree

72

5.70

78

6.28

113

7.83

102

8.37

61

6.97

Discipline-specific     

    major

1,206

95.49

1,186

95.72

1,184

82.05

969

79.56

642

73.62

Secondary education

    major

759

60.10

803

64.81

125

8.66

412

33.83

483

55.39

Years of high school

   science teaching

   experience

13.69

(7.26)

14.20

(8.08)

15.04

(8.81)

15.22

(8.81)

13.63

(8.98)

Years of AP teaching

    experience

6.34

(5.56)

6.33

(6.05)

6.70

(6.23)

6.58

(6.27)

5.85

(6.04)



MEASURES


The dependent variable used in these analyses is student achievement as measured through students’ scores on the AP Biology, AP Chemistry, and AP Physics examinations. AP scores are measured as ordinal variables with a 1–5 range, with 3 considered “passing.”


Independent variables included student-level, teacher-level, and school-level variables. On the student level, variables included a continuous variable representing students’ PSAT scores as a prior performance indicator, and a single-indicator ordinal variable describing students’ family background as represented by students’ self-report on their mother’s educational attainment. Prior research identified both students’ prior performance and parental educational attainment as influential for student learning and achievement (e.g., Ewing et al., 2006; Woessmann, 2004).


On the teacher level, teaching and teacher characteristics were described by single-item continuous variables including the number of enacted laboratory investigations, teachers’ years of AP teaching experience, teachers’ years of experience with the AP redesign, and teachers’ perceived importance of PD for students’ AP science performance. Also, the total hours of AP science course instruction throughout the academic year was included as an independent variable. This variable is based on teachers’ self-reported start and end date of AP instruction, as well as the weekly minutes of class time. In addition, composite variables described teachers’ enactment of AP redesign practice elements, enactment of AP redesign curriculum elements, teachers’ perceived administrative support, and self-efficacy. These variables were included in the models as numerous strands of research emphasize the importance of instruction and teacher quality for student learning (e.g., Cohen & Ball, 1999; Hattie, 2012; Nye et al., 2004; Seidel & Shavelson, 2007). The research particularly emphasizes the influence of reform- and inquiry-based instruction, internal psychological teacher characteristics, and supportiveness of the work environment for science learning (e.g., Abd-El-Khalick et al., 2004; Furtak et al., 2012; Klassen & Tze, 2014).

The above described composite variables were computed based on exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) on randomly sampled and equal-sized datasets, separated by science discipline and year of the AP redesign implementation. The number of retained factors was determined using scree plots and the Guttman-Kaiser criterion. Subsequently, variables were gradually excluded from the composite variables if factor loadings were below an absolute value of 0.25. This threshold is more conservative than traditionally applied thresholds, which are commonly between 0.3 to 0.4 (Grice, 2001). Normalized oblimin oblique rotation methods were applied as factors were assumed to be correlated to each other. CFA utilized maximum-likelihood estimations. Extraction of items to be included in the composite variables were based on model fit characteristics such as several goodness-of-fit measures (RMSEA, AIC, BIC, CFI, TLI, SRMR) and likelihood-ratios tests. Furthermore, conceptual consideration based on the results of the EFA were taken into account such that identical items across science disciplines and years of the AP redesign implementation were used in the computation of composite scores. Finally, composite variables were computed utilizing Bartlett factor scores (DiStefano, Zhu, & Mindrila, 2009). The internal consistency was determined with Cronbach’s α. Table 3 lists all individual items included in the computation of these composite variables, as well as the internal consistency ratings. The factor structure of the resulting composite variables with factor loadings, uniqueness, scoring coefficients, and Cronbach’s α values is described in Table A1 in the appendix.

Table 3. Variables Included in Computations of Composite Variables

 

Included variables

α

Administrative support

(a) Principal understands challenges for AP science students×, (b) principal understands challenges for AP science teachers×, (c) principal supports PD×, (d) lighter teaching load for AP science teachers×, (e) fewer out-of-class responsibilities for AP science teachers×, (f) AP science is given additional funding×, (g) availability of equipment to perform labs, (h) availability of expendable (consumable) supplies to perform labs

0.72– 0.74

Enactment:   AP practices

(a) Students work on laboratory investigations×, (b) provide guidance on integrated content questions×, (c) provide guidance on open/free response questions×, (d) students report laboratory findings to another×, (e) students perform inquiry laboratory investigations×, (f) use science practices outside of the classroom×

0.67–0.72

Enactment:  AP curriculum

(a) Refer to the “Big Ideas”×, (b) refer to how enduring understandings relate to the “Big Ideas”×, (c) refer to learning objectives from AP curriculum×, (e) refer to the curriculum framework×

0.83–0.88

Self-efficacy

(a) Students perform better because of my extra effort×, (b) student scores improve because of my teaching×, (c) my teaching can overcome student backgrounds×, (d) extra effort in teaching produces little change×

0.57–0.63

Number / duration of conventional PD activities

Face-to-face: AP Summer InstituteCB, AP Fall WorkshopCB, transition to inquiry-based labs workshopCB, day with AP readerCB, laying the foundation by NMSI, BSCS Leadership Academy by BSCS and NABT, reasoning skills workshopCB

Online courses: Transition to inquiry labsCB, Introduction to AP Biology/Chemistry/PhysicsCB, AP Central webcast – Exploring atomic structure using photoelectron spectroscopyCB, AP insightCB

Online communities: AP online teacher communityCB, NSTA online teacher community

Number of unconventional PD activities

Face-to-face: District/regional/local college/teacher-initiated meetings, mentoring/coaching one-on-one or with other teachers, conference or conference sessions, serving as AP exam reader, serving as AP consultant

Materials: AP course and exam descriptionCB, AP lab manualCB, teacher textbook guide and related materials, student guide: data analysisCB, teacher guide: quantitative skills and analysisCB, AP practice exams, materials developed from colleagues, articles from magazines/journals, video resources, computer-based simulations

Teachers were not asked to indicate their participation on all listed PD activities for each discipline and year. ×5-point Likert scale item. 4-point Likert scale item. CBProvided by the College Board.



PD participation was described through both the quantity and quality of teachers’ PD participation. Measuring PD quantity, variables included summative continuous variables on teachers’ self-reported number of participations in conventional PD activities (e.g., summer institutes, one-day workshops, self-paced online courses, College Board’s online teacher community, etc.) and unconventional PD activities (e.g., conferences, one-on-one mentoring, teacher textbook guides, articles from magazines, etc.). Table 3 lists all PD activities classified as “conventional” and “unconventional” PD. PD quality was measured utilizing teachers’ average rating of six PD design features across all conventional PDs a teacher participated in. The PD design features included five-point Likert scale items measuring whether each conventional PD activity (a) offered teachers an experience of active learning, (b) was responsive to teachers’ needs and interests, (c) focused on student work or materials from meetings with other teachers, (d) provided observations of modeled teaching, (e) afforded opportunities to build relationships with colleagues, and (f) effectively supported teachers in teaching redesigned AP science courses. These PD design feature variables are inspired by Desimone’s (2009) characteristics of “high quality” PD design, whereas “responsive agenda” refers to Desimone’s “coherence,” “focus on student work” refers to “content focus,” and “relationship building” refers to “collective participation.” Generally, PD participation is often linked with increases in student performance (e.g., Banilower et al., 2007; Fishman et al., 2003; Garet et al., 2001; Penuel et al., 2007).


On the school level, independent variables included a continuous variable describing school affluence as represented by the percentage of students enrolled in free or reduced-priced lunch programs, and an ordinal variable describing school districts’ school support as measured by districts’ per-student funding allocations for instructional materials. School affluence and socioeconomic stratification are frequently viewed as an important factor influencing teacher and teaching quality and student achievement (e.g., Crosnoe, 2009; OECD, 2013; Willms, 2010). Table 4 provides descriptive information on all dependent and independent variables included in the analyses.

Table 4. Descriptive Information of Variables Used in Analyses

 

Biology Y2

Biology Y3

Chemistry Y1

Chemistry Y2

Physics Y1

 

N / Mean

% / (SD)

N / Mean

% / (SD)

N / Mean

% / (SD)

N / Mean

% / (SD)

N / Mean

% / (SD)

 Dependent variable

AP Score

 

 

 

 

   

 

 

 

   1

1,564

 6.23

1,105

 5.16

4,243

16.04

3,804

 17.94

5,739

 29.56

   2

6,053

 24.11

5,418

 25.28

6,800

25.71

5,325

 25.11

5,946

 30.63

   3

9,102

 36.25

8,069

 37.65

7,394

27.96

6,335

 29.88

4,115

 21.20

   4

6,379

 25.41

5,231

 24.41

4,961

18.76

3,721

 17.55

2,716

 13.99

   5

2,010

 8.01

1,606

 7.49

3,051

11.54

2,019

 9.52

897

 4.62

 Student Characteristics

PSAT scores

164.2

 (26.8)

163.3

 (26.5)

173.4

(26.8)

172.2

 (26.8)

167.1

 (26.4)

Mother’s educational attainment

 

 

 

   

 

 

 

   No post-secondary

4,765

 18.98

4,180

 19.51

4,345

16.43

3,562

 16.80

3,771

 19.43

   Some post-secondary

5,742

 22.87

4,770

 22.26

5,420

20.49

4,305

 20.30

4,169

 21.48

   Bachelor’s degree

8,210

 32.70

6,895

 32.18

9,163

34.64

7,202

 33.97

6,555

 33.77

   Graduate degree

6,391

 25.45

5,584

 26.06

7,521

28.44

6,135

 28.93

4,918

 25.33

School Characteristics

Percentage free or   

   reduced-price lunch

33.6

 (22.8)

37.6

 (23.2)

31.3

(22.0)

34.3

 (22.1)

36.7

 (21.9)

District funding for instructional   materials

 

 

   

 

 

 

   Funding < $200

600

 47.43

589

 47.39

669

46.36

519

 42.58

422

 48.17

   Funding    $200–$300

434

 34.31

435

 35.00

474

32.85

463

 37.98

288

 32.88

   Funding > $300

231

 18.26

219

 17.62

300

20.79

237

 19.44

166

 18.95

Administrative support

–0.07

 (1.09)

–0.10

 (1.09)

–0.11

(1.09)

–0.10

 (1.11)

–0.08

 (1.10)

 Teacher and Teaching Characteristics

Hours of AP instruction

   in 10 hours

17.90

 (5.51)

17.74

 (5.49)

17.51

(5.39)

17.14

 (5.33)

15.92

 (4.39)

Enactment of AP

   curriculum elements

0.03

 (1.10)

0.05

 (1.11)

0.03

(1.08)

0.04

 (1.05)

0.04

 (1.05)

Enactment of AP

   practice elements

0.03

 (1.17)

0.03

 (1.15)

–0.01

(1.16)

0.01

 (1.14)

0.00

 (1.21)

Number of labs

13.69

 (5.64)

14.25

 (5.58)

15.39

(5.52)

15.35

 (5.73)

17.33

 (6.20)

Years AP teaching

6.34

 (5.56)

6.33

 (6.05)

6.70

(6.23)

6.58

 (6.27)

5.85

 (6.04)

Years AP redesign

   experience

1.90

 (0.30)

2.62

 (0.72)

1.87

 (0.34)

 —

Self-efficacy

0.03

 (1.33)

0.06

 (1.30)

0.03

(1.32)

0.03

 (1.32)

–0.05

 (1.27)

Importance of PD for

   AP scores

4.08

 (0.76)

4.04

 (0.78)

3.93

(0.80)

3.93

 (0.85)

3.87

 (0.85)

 PD Characteristics

Number conventional PD

1.80

 (1.22)

1.73

 (1.24)

1.85

(1.12)

1.71

 (1.19)

1.57

 (1.12)

Number unconventional PD

5.00

 (1.37)

6.70

 (1.70)

4.13

(1.41)

5.83

 (1.69)

6.22

 (2.24)

PD includes active

   learning

2.05

 (1.29)

1.89

 (1.29)

2.10

(1.16)

1.81

 (1.19)

2.14

 (1.37)

PD has responsive agenda

2.90

 (1.41)

2.81

 (1.48)

2.83

(1.29)

2.70

 (1.44)

2.66

 (1.51)

PD focuses on student

   work

2.21

 (1.26)

2.25

 (1.34)

2.12

(1.15)

2.17

 (1.28)

1.99

 (1.28)

PD models teaching

2.24

 (1.30)

2.31

 (1.39)

2.23

(1.20)

2.20

 (1.32)

2.21

 (1.40)

PD helps relationship

   building

3.03

 (1.48)

2.95

 (1.56)

2.92

(1.34)

2.70

 (1.47)

2.81

 (1.59)

PD supports instruction

3.06

 (1.41)

3.03

 (1.52)

3.10

(1.31)

2.96

 (1.45)

2.74

 (1.51)

Duration of conventional PD

36.0

 (48.3)

32.6

 (39.8)

38.5

(38.0)

27.7

 (31.3)

34.5

 (33.2)



ANALYTIC METHODS


The analyses for this study were conducted in two phases. In the first phase, the associations between student, teacher, and school covariates and students’ AP science scores were examined by utilizing a multilevel ordered logistic regression approach (Hedeker & Gibbons, 1994; Snijders & Bosker, 2012). Specifically, for each of the regressions that follow, a proportional odds model was utilized (Agresti, 2002; Hox, 2010). The proportional odds model is the most widely used approach for analyzing multilevel ordinal data. The model assumes that the effect of any given explanatory variable remains constant regardless of the probability of any given level of the outcome. Models were fit for each discipline and year of the AP redesign separately to provide additional support for validity and reliability of findings. Modeling was carried out through staging the predictors into the model in blocks using conventional modeling approaches (Hox, 2010). All continuous variables were grand-mean centered to aid in interpretation. The sequence of variables used in the models was as follows: (a) a null model was fitted to the data to establish the percent of variance split between level-1 and level-2 in the model, (b) students’ PSAT, (c) mother’s highest level of education, (d) teacher- and school-level contextual variables, (e) PD characteristic and dosage variables. This modeling sequence emphasizes the variance contribution of teachers’ PD participation on students’ AP science scores. Model fit for each stage of modeling was compared using the log likelihood ratio test. To answer the first research questions, the percent change in level-2 variance was examined across all models to aid in understanding the variance left unaccounted for by variables in the model. The intraclass correlation coefficient (ICC) statistic was used to calculate the percent of variance split between levels. For ordinal outcomes, ICC statistics are calculated as follows: ICC = (Variance at level-2) / (Variance at level-2 +(π⁄3)).

In the next phase of modeling, the results were pooled across disciplines and years using meta-regression (also referred to as a meta-analytic model). This is possible because variables used in all models are identical. The goal of this analysis is to understand the aggregated associations of covariates with students’ AP science scores. This meta-regression analysis pools the log odds ratios of the individual multilevel ordered logistic regression analyses across all disciplines and years. The pooled log odds ratios parameters were estimated with a weighted fixed-effects model utilizing restricted maximum-likelihood estimation (Viechtbauer, 2005, 2010, 2016). The applied weights correspond with the sample sizes of each individual study. This analysis does not assume that the true effects are homogenous, but rather restricts inferences to the sets of models for each discipline and year in the sample. Additionally, Wald tests and confidence intervals were obtained to ascertain the statistical significance of parameters in the model. The statistical significance and confidence intervals are reported for the pooled effect estimate (i.e., log(OR)). Finally, Cochran’s Q test for heterogeneity of the effect is reported in order to ascertain if variation between the individual study effects is likely due to factors outside of sampling error (Higgins & Green, 2011).


RESULTS


VARIANCE CONTRIBUTIONS OF STUDENTS’ AP SCIENCE SCORES


Information about the variance components for the models is reported in the form of the ICC for each model. ICC statistics approximate the average correlation between any two students’ test scores within the same classroom. Large ICC statistics indicate that the within-cluster variability is low, and consequently there is a higher variability between the clusters. High between-cluster variability can be accounted for with student, teacher, and school effects, but typically is susceptible to contextual effects (i.e., PD).


The ICC statistic for students’ AP science scores ranges from .37 to .46 (Biology Year 2: .39, Biology Year 3: .37, Chemistry Year 1: .46, Chemistry Year 2: .46, Physics Year 1: .39). This indicates that across all disciplines and years, the within-cluster variability is low. This means that 37% to 46% of the total variance in students’ AP science scores is on the teacher and school levels, whereas the remaining 54% to 63% is at the student level. This shows that a considerable amount of variance could potentially be influenced through educational policy interventions targeted at changes regarding teachers’ instructional enactment, as well as teacher- and school-level factors.


In order to identify contributions to the level-2 variance, blocks of covariates were staged into the models (Table 5). The goal with these results is to provide information about the amount of variance remaining in students’ AP science scores that could be plausibly contributed by other contextual variables not in the models. Results for the percent of variance accounted for by blocks of covariates indicate that across all models the PSAT scores (i.e., block 1) account for the largest percentage of level-2 variability, accounting for level-2 variance in the range of 28% to 44%. Students’ PSAT scores greatly vary across teachers and schools. Therefore, the inclusion of students’ PSAT scores in the model also influences the variance explained at the teacher and school levels despite being a student-level variable (Snijders & Bosker, 2012). Block 2 examined the addition of maternal educational attainment (another important proximal predictor for student achievement), which only accounted for less than 1% to 2.3% of level-2 variance across all disciplines and years. Block 3 examined associations between teacher characteristics and perceptions, as well as school factors affecting student outcomes. This block of covariates accounts for the second-largest percent of level-2 variance in student outcomes, accounting for 11% to 23% of level-2 variance. The final block of covariates examined were the PD characteristic variables. Overall, this block of covariates has only a small impact on students’ AP science scores, accounting for less than 1% to 2.3% of the remaining level-2 variance in students’ AP science scores.

Table 5. Percent of Level-2 Variance Accounted for by Blocks of Covariates

Model

Block 1      [%]

Block 2 [%]

Block 3 [%]

Block 4 [%]

Remaining [%]

Full model ICC

Biology Year 3

44.42

2.29

11.09

1.95

40.26

0.19

Biology Year 2

41.27

1.68

13.66

1.59

41.80

0.21

Chemistry Year 2

31.86

1.33

22.68

1.20

42.93

0.27

Chemistry Year 1

28.30

<1.00

19.47

2.33

49.12

0.29

Physics Year 1

36.73

1.09

15.30

<1.00

46.23

0.23

Note. Block 1 – students’ PSAT scores; Block 2 – mother’s educational attainment; Block 3 – school, teacher, and teaching variables; Block 4 – teacher PD participation characteristics.


In sum, the results demonstrate that there is substantial variance in students’ AP science scores between teachers and schools for all disciplines and years. Whereas PSAT scores and teacher and school context variables included in the models account for a sizeable portion of the variance in students’ AP science scores, the contribution of the included PD characteristics variables only accounted for a comparatively small percentage of level-2 variance. In addition, the remaining unexplained variance on the teacher and school levels ranges from 40% to 49%, which indicates that there are likely still other contextual variables that this model does not capture that could account for this remaining level-2 variance.

ASSOCIATIONS WITH STUDENTS’ AP SCIENCE SCORES

In order to examine the second research question focused on the relationships between school, teaching, teacher, and teacher PD characteristics and students’ AP science scores controlling for student characteristics, multilevel ordinal logistic regression analyses were conducted separately for each discipline and year; the parameter estimates, model fit indices, and changes in explained variances for each model are described in the appendix (Appendix Tables A2–A6). The present study examined the aggregate effects of each predictor of interest on students’ AP science scores combined across all disciplines and years in a meta-analysis. Results for the pooled parameter estimates are reported in Table 6.


Table 6. Meta-regression Results Predicting Relationships on Students’ AP Scores

Variable

log(OR)

SE

z

p

Lower 95% CI

Upper 95% CI

Test for Heterogeneity (Q)

p

Student Characteristics

PSAT scores

 1.795

 0.104

 17.181

 <0.001

 1.590

 2.000

488.294

 <0.001

Mother’s educational attainment (vs. no post-secondary)

 

 

 


 

  Post-secondary

 0.012

 0.021

 0.579

 0.563

 –0.029

 0.054

4.137

 0.388

  Bachelor’s degree

 0.134

 0.026

 5.204

 <0.001

 0.083

 0.184

6.987

 0.137

  Graduate degree

 0.175

 0.038

 4.580

 <0.001

 0.100

 0.250

13.716

 0.008

 School Characteristics

Percentage free or reduced-price lunch

 –1.917

 0.152

 –12.594

 <0.001

 –2.215

 –1.619

16.882

 0.002

District funding for materials (vs. $200–$300)

 

 

 


 

  Funding < $200

 0.121

 0.035

 3.470

 <0.001

 0.052

 0.189

3.524

 0.474

  Funding > $300

 –0.019

 0.045

 –0.421

 0.674

 –0.106

 0.069

4.111

 0.391

Administrative support

 0.032

 0.016

 2.046

 0.041

 0.001

 0.063

2.318

 0.678

 Teacher and Teaching Characteristics

Self-efficacy

 0.076

 0.019

 4.041

 <0.001

 0.039

 0.113

5.638

 0.228

Importance of PD for AP scores

 –0.061

 0.021

 –2.844

 0.004

 –0.103

 –0.019

6.777

 0.148

Years teaching AP

 0.021

 0.005

 4.657

 <0.001

 0.012

 0.030

10.638

 0.031

Years AP redesign experience

 0.155

 0.084

 1.835

 0.067

 –0.011

 0.320

4.106

 0.128

Number of labs

 0.029

 0.009

 3.340

 <0.001

 0.012

 0.045

32.698

 <0.001

Hours of AP instruction in 10 hours

 0.029

 0.003

 9.375

 <0.001

 0.023

 0.035

1.968

 0.742

Enactment of AP curriculum elements

 –0.145

 0.017

 –8.465

 <0.001

 –0.178

 –0.111

3.168

 0.530

Enactment of AP practice elements

 0.055

 0.018

 3.102

 0.002

 0.020

 0.089

1.178

 0.882

Curriculum X Practices

 0.020

 0.015

 1.343

 0.179

 –0.009

 0.048

2.794

 0.593

 PD Characteristics

Number conventional PD

 0.017

 0.021

 0.796

 0.426

 –0.025

 0.059

5.601

 0.231

Number unconventional PD

 0.011

 0.011

 1.007

 0.314

 –0.011

 0.033

1.125

 0.890

PD includes active learning

 –0.136

 0.038

 –3.609

 <0.001

 –0.210

 –0.062

 10.986

 0.027

PD focuses on student work

 –0.092

 0.023

 –3.919

 <0.001

 –0.138

 –0.046

1.701

 0.791

PD models teaching

 –0.062

 0.025

 –2.534

 0.011

 –0.111

 –0.014

1.833

 0.767

PD helps relationship building

 0.000

 0.036

 0.009

 0.993

 –0.070

 0.070

6.226

 0.183

PD has responsive agenda

 0.071

 0.028

 2.520

 0.012

 0.016

 0.126

1.158

 0.885

PD supports instruction

 0.149

 0.060

 2.463

 0.014

 0.030

 0.267

13.613

 0.009

Duration of conventional PD

 0.001

 0.001

 1.765

 0.078

 0.000

 0.002

1.387

 0.845

 

 

 

 

 

 

 


 


Associations with Student Characteristics


Students’ PSAT scores were positively associated with their subsequent AP science scores (log(OR) = 1.795, SE = 0.104, p < 0.001). In comparison to no post-secondary education, mother’s educational attainment of either a bachelor’s (log(OR) = 0.134, SE = 0.026, p < 0.001) or graduate degree (log(OR) = 0.175, SE = 0.038, < 0.001) also showed positive associations with students’ AP science scores. Of these parameter estimates, the estimated students’ PSAT scores, = 488.294, < 0.001, and mothers with a graduate degree exhibited significant heterogeneity across disciplines and years, = 13.716, < 0.001. This result indicated that the estimated effect sizes likely differed because of factors other than sampling error within the studies.


Associations with School, Teacher, and Teaching Characteristics


Several contextual school-level variables showed significant positive associations with students’ AP science scores. Specifically, the percentage of students in schools enrolled in free or reduced-priced lunch programs was negatively associated with AP science scores, log(OR) = –1.917, SE = 0.152, < 0.001. Further, the Cochran’s Q statistic indicated that there was significant difference in the estimate generated for each discipline and year, = 16.882, p < 0.01. In addition, the administrative support composite (i.e., principal understands challenges for AP science students/teachers, principal supports PD, lighter teaching load/fewer out-of-class responsibilities for AP science teachers, additional funding for AP science, availability of equipment/expendable supplies to perform labs) was significant, and positively associated with students’ AP science scores, log(OR) = 0.032, SE = 0.016, < 0.05.


Similarly, several variables describing teacher characteristics were significantly associated with students’ AP science scores. Teachers’ perceived importance of PD for increasing student scores was negatively associated with students’ AP science scores, log(OR) = –0.061, SE = 0.021, < 0.01, which indicated that teachers who rated this item high have students who performed worse on the AP science exams. Additionally, the self-efficacy composite (i.e., student performance is based on my effort as a teacher, students get better scores due to effective teaching, teaching overcomes students’ inadequate science backgrounds, and extra teaching effort does not change AP scores) was positively associated with students’ AP science scores, log(OR) = 0.076, SE = 0.019, < 0.001. This result indicated that the students of teachers with a high rating on this composite performed on average better on the AP science exams. Increases in teachers’ years of AP science teaching experience are significantly associated with higher student scores, log(OR) = 0.021, SE = 0.005, < 0.001. With respect to teachers’ instruction, the models indicated several significant associations with student scores. For example, the hours of AP science course instruction students were exposed to were positively associated with students’ AP science scores, log(OR) = 0.029, SE = 0.003, < 0.001. The number of AP labs teachers utilized in their instruction was associated with higher AP science scores, log(OR) = 0.029, SE = 0.009, < 0.001. Similarly, the enactment of AP practices composite (i.e., students work on laboratory investigations, provide guidance on integrated content questions, provide guidance on open/free response questions, students report laboratory findings to another, students perform inquiry laboratory investigations, and use science practices outside of the classroom) was positively associated with students’ AP science scores, log(OR) = 0.055, SE = 0.018, < 0.01. However, the enactment of AP curriculum composite (i.e., refer to the “Big Ideas” of science, refer to how enduring understandings relate to the “Big Ideas” of science, refer to the learning objective from the AP curriculum, refer to the curriculum framework) was negatively associated with students’ AP science scores, log(OR) = –0.145, SE = 0.017, < 0.001.

Associations with PD Characteristics

Several of the Desimone-inspired PD characteristic variables based on teachers’ self-reported PD quality were significantly associated with students’ AP science scores, while other PD variables showed a negative association. Specifically, the Desimone-inspired PD variable characteristics that were negatively associated with students’ AP science scores included: teachers’ rating regarding the presence of active learning, log(OR) = –0.136, SE = 0.038, < 0.001; a focus on student work, log(OR) = –0.092, SE = 0.023, < 0.001; and modeling teaching, log(OR) = –0.062, SE = 0.025, < 0.05. Notably, the parameter estimate for active learning varied across disciplines and years, = 10.986, p < 0.05. Further, the Desimone-inspired PD variable characteristics that were positively associated with students’ AP science scores included the teachers’ rating regarding the presence of a responsive agenda item, log(OR) = 0.071, SE = 0.028, < 0.05, and the effectively supporting instruction for the redesigned AP science course item, log(OR) = 0.149, SE = 0.060, < 0.05. The parameter estimate for the PD effectively supporting instruction showed variation across disciplines and years, Q = 13.613, p < 0.01.

DISCUSSION


SCHOLARLY SIGNIFICANCE

This work is unique in the educational research context in the United States as it constitutes the first science education research project that has had access to national student-, teacher-, school-, and district-level datasets in the midst of a large-scale, top-down mandated science curriculum reform connected to standardized high-stakes examinations across multiple science disciplines and years. This study has rare access to a broad spectrum of variables ranging from student scores on nationwide standardized tests to in-depth teacher self-reports of participation in more than 20 different PD programs in the landscape of the revised AP Biology, AP Chemistry, and AP Physics examinations. This is different from most other research that focuses only on very specific standalone, high-profile PD programs (Hill, 2015). The structure of the datasets with almost identical variables across different science disciplines and years of the AP redesign afforded the unusual opportunity to utilize a meta-analytic perspective in addition to analyzing each dataset individually. The consistency in the outcomes of the statistical models across years and disciplines provides evidence to support the validity and reliability of the findings, providing an uncommon opportunity for research on these important topics in science education. Given the national scope of the data in this study, the findings are likely to be relevant to other large-scale curriculum reforms in the sciences (i.e., Next Generation Science Standards, NGSS Lead States, 2013) and beyond.


IMPLICATIONS


This study intends to identify levers that might lead to improvements in student learning and achievement in the sciences beyond student characteristics such as prior PSAT score and socioeconomic status. The nationwide samples allowed the identification of relationships to student outcomes that were stable over time and across science disciplines. As such, the present study established the variance in students’ AP science scores accounted for at the teacher and school levels, which might be amenable to policy changes. Percentages of variance were explained by blocks of variables at the student, teacher, and school levels, and relationships of student, school, teaching, teacher, and teacher PD characteristics with students’ AP science performance were identified. The hope is to guide school leadership and educational policy in science in the mission of improving student performance. The three main findings are the following:


First, the inclusion of well known and influential student-level factors, i.e., PSAT score and mother’s highest level of education (Ewing et al., 2006; Woessmann, 2004), accounted for about 40% of the variance in students’ AP science scores, leaving roughly 60% of the student-level variance unexplained. This supports the examination of associations between school context and teacher characteristics and students’ AP science scores. Policy interventions could utilize these associations as levers to improve performance, similar to what has been reported in several previous research studies (Diamond, Maerten-Rivera, Rohrer, & Lee, 2014; Kane & Staiger, 2008; Nye et al., 2004; Papay & Kraft, 2015; Rivkin et al., 2005; Rockoff, 2004).


Second, this study identified several school, teacher, and teaching characteristics that were positively associated with student performance on the AP science examinations. Educational policy interventions could utilize these levers in the attempt to increase student learning and achievement. The findings in this study align with prior science education research, but those prior studies’ findings are more localized and do not have the comprehensive range across science disciplines and time that is available with the data in this study.


School-level factors associated with students’ AP science scores included teachers’ perceived administrative support and the percentage of students enrolled in free or reduced-priced lunch programs. The more teachers feel supported by their principals and other school administrators, the better their students’ performance. This result is consistent with previous research emphasizing the importance of principals and other school leaders for influencing teachers’ work environment, classroom instruction, and ultimately student scores (Johnson et al., 2012; Kraft et al., 2016; May & Supovitz, 2011; Supovitz et al., 2010), and suggests school leaders should facilitate the creation of supportive work environments for their science teachers. Also, this study indicates that increased access to AP programs does not necessarily yield equitable learning opportunities that allow all students to succeed. The importance of socioeconomic factors for student learning and achievement is well known and documented in the context of the AP program and beyond (e.g., Hallett & Venegas, 2011; Klugman, 2013; OECD, 2013; Reardon, 2013). Consequently, educational administrators and policymakers should feel encouraged to further equity-related initiatives to broaden benefits to all students, and especially to students who are traditionally underrepresented as participants in the AP program. As these students tend to also be culturally and linguistically diverse, policymakers and administrators as well as teachers can work to integrate cultural relevant (Ladson-Billings, 1995, 2006) or cultural sustaining (Paris, 2012) pedagogical approaches into AP science courses, as these approaches can empower, engage, and offer equitable learning opportunities to students (Buxton, 2006; Dimick, 2012; Lee & Buxton, 2011).


Teacher-level factors associated with higher student performance on the AP science examinations included teachers’ years of AP science teaching experience and perceived self-efficacy. This finding corresponds with previous research that documents the positive relationships between teacher knowledge and experience, teaching effectiveness, and student performance (Keller et al., 2017; Nye et al., 2004; Papay & Kraft, 2015), as well as research that highlights the importance of teachers’ self-efficacy beliefs for instruction and student learning in the sciences (Tschannen-Moran, Hoy, & Hoy, 1998; van Aalderen-Smeets & Walma van der Molen, 2015).


Teaching-level factors associated with students’ AP science scores included the total hours of classroom instruction, the number of laboratory investigations, and the enactment of particular practice and curriculum elements of the AP redesign. These findings also mirror previous research that identifies relationships between the amount and type of classroom instruction and student performance (e.g., Desimone & Long, 2010; Marcotte & Hansen, 2010). A policy recommendation emerging from this result is that the total number of instructional hours prior to the AP exam should be noted next to AP scores of students. This would, at least, make consumers of AP score data (e.g., college admissions) cognizant of possible instructional disparities, which students had no control over, that could be used to address issues of educational equity. For example, a student in an under-resourced school may only get five class periods per week for AP science while a student in a well-resourced school may get seven class periods per week. Despite similar on-paper access to the AP program, receiving 30% less instruction could be a reason why students in under-resourced schools received a 4 instead of a 5 on the AP exam. The positive associations of lab enactments and AP practices align with research that identified relationships emphasizing the importance of teaching and teaching characteristics for students’ science learning and performance (e.g., Diamond et al., 2014; Furtak et al., 2012; Hamilton et al., 2003; Secker, 2002). The negative association of teachers’ enactment of AP curriculum elements (i.e., Big Ideas) seems counterintuitive, since it is assumed that an emphasis on Big Ideas as organizing principles for the subject matter content promotes learning. Experts utilize Big Ideas when approaching new problems (National Research Council, 2000). Consensus documents speak to the need for more emphasis on Big Ideas in coursework (National Research Council, 2012a). That leaves two quite different explanations for this unexpected result. It could be that the teachers answering the questions regarding emphasis on Big Ideas teach these as separate nuggets of information rather than weaving them constantly and consistently through their curriculum. Teachers saying that they don’t focus on the Big Ideas may be using them continuously but not setting aside specific times to discuss each Big Idea and isolating it from their instruction of the content. Another possible explanation regards the assessment of knowledge of the Big Ideas. It is possible that the emphasis and importance of Big Ideas is not mirrored adequately in the AP examinations. Observational data would be useful in studying the quality of teacher implementation, but unfortunately was not part of this study design.


The third main finding is that the contribution of teachers’ PD participation to student achievement is mixed. The addition of Desimone-inspired variables, which describe teachers’ perceptions of their PD experiences, only accounted at most for 2.3% of the level-2 variance in students’ AP science scores across all science disciplines and years. However, it is important to note that this study is situated in the context of the AP program. Often, school administrators ask the most experienced and skilled teachers to teach AP courses. Similarly, students taking AP courses and examinations are often among the highest-achieving students in their schools. Therefore, this unique population, in contrast to all other students and teachers in the nation, might make detecting additional benefits of teacher PD participation for student achievement more difficult. Nonetheless, this finding corresponds with research that describes the mixed evidence base of detecting direct effects of teacher PD participation on student achievement measures, especially for studies that frame PD in terms of Desimone’s (2009) list of high-quality PD characteristics (e.g., Desimone & Garet, 2015; Garet et al., 2008, 2011; Jacob & McGovern, 2015). While some associations were statistically significant—which might also be in part related to the large sample sizes utilized in the statistical models—the strength of these effects on student science scores is comparatively small. This result supports calls for further research on PD effectiveness.

Finally, while the variables included in the models captured a wide range of concepts related to student performance, a substantial percentage of the variance remained in students’ scores at the teacher and school levels. In sum, these results suggest that work still remains in identifying teacher characteristics and behaviors that might serve as additional levers to increase student learning and achievement.

LIMITATIONS AND FUTURE WORK


The main limitations of this study relate to the data sources. Teacher-level data was limited to teachers’ self-reports on the web-based surveys, which might have led to some positive response bias in the responses to survey questions. As a result, parameter estimates might represent upper bounds of plausible associations between many of the teachers’ responses to survey items and students’ AP science scores. The largest threat to external validity of this study was the absence of student–teacher identifiers. As a result, students could only be matched to the schools, but not to their teachers. In order to safeguard against this threat to validity, a sampling frame was selected by eliminating from the analyses all schools with more than one AP science teacher teaching in the same AP science discipline to uniquely match students with teachers. Non-response analyses indicated no notable differences to the wider AP science populations.


Another limitation is that substantial variance at the teacher and school levels was not accounted for by the variables included in the models. Future studies should attempt to identify sources of this remaining unexplained variance. For instance, Crosnoe and Benner (2015) outline several additional variables that might affect student science achievement, such as school climate, class size, ethnicity, challenging coursework, and parental involvement. Additionally, relationships between classroom observations and student achievement have been shown to be strong (Kane & Staiger, 2012). Work on databases that would allow the separation of teacher and school effects would also be useful. Furthermore, the unique population of students and teachers in the AP program needs to be taken into consideration when generalizing the findings of this study, as access and success of the AP program in schools can be stratified by students’ class and race (Klopfenstein, 2004a; Klugman, 2013; Schneider, 2009). This stratification occurs despite multi-stakeholder-led efforts to broaden access to the AP program for students who are traditionally underrepresented in AP courses (Conger et al., 2009; Roegman & Hatch, 2016; The College Board, 2014). As providing equitable opportunities for students to be best prepared to succeed in college is of great importance to the prosperity and well-being of the country, future studies should examine pathways to narrow achievement and opportunity gaps. While a subset of the data analyzed in this study was used to identify factors related to “better-than-expected” AP science performance for students who are in schools with a higher percentage of students with lower-socioeconomic status (Fischer et al., 2016), continued threads of research are needed to comprehensively provide evidence-based recommendations to teachers, administrators, and policymakers to ensure that equitable learning opportunities are afforded to all students. For instance, more studies could assess AP teachers’ equity-related beliefs and examine associations with teaching practices, professional development, and student achievement (Grimberg & Gummer, 2013). Additional studies could explore the experiences of students who are from underrepresented communities in STEM through anti-deficit frameworks (Harper, 2010). As an example, researchers can speak with the 21.7% of students who were eligible for free or reduced-priced lunch programs and received a passing score (3 or higher) on an AP exam, as reported by the College Board in 2014, in order to learn of strategies and practices used by the students. Consequently, inferences from this study to all students and teachers in the United States might need further studies with student and teacher samples different from the AP population to ensure the robustness of the results of this study.

From a methodology perspective, studies that examine relationships of multi-faceted characteristics with student performance could investigate more complex associations. The present study utilizes what Opfer and Pedder (2011) describe as process-product logic to identify teacher effects on students’ performance. However, many theoretical perspectives are used to understand potential levers of educational policy interventions within longer chains of mediational pathways that ultimately lead to increases in student achievement. For example, Desimone’s (2009) logic model for the effects of PD relationships indicates that effects of teachers’ PD participation on student learning and achievement are mediated by teachers’ knowledge gains, changes in teachers’ belief systems, and teachers’ classroom instruction. Therefore, follow-up studies could utilize longitudinal data with a multilevel structural equation modeling approach to examine these mediational pathways. Alternatively, future studies could even further avoid potential oversimplification of the existing interdependent, dynamic, and multidimensional processes in the real world (Cochran-Smith, Ell, Ludlow, Grudnoff, & Aitken, 2014) by identifying potential levers to improve student performance from a complex systems theory lens (Byrne & Callaghan, 2014; Lemke & Sabelli, 2008).


CONCLUSION

This study makes contributions to the field’s understanding of policy-based interventions around student learning in several ways. First, it confirms the existence of potential levers that influence student performance at both the teacher and school level. Furthermore, it confirms these levers using a national dataset based on a high-stakes annual standardized science examination, allowing for examination of effects (which were shown to be mostly consistent) across science disciplines and across time. A second important contribution of this work is the light it sheds on the relatively small impact of teacher PD on student outcomes. While these findings should not be used to make claims about the general efficacy of well-designed PD in highly particularized cases, this study does indicate the difficulty of essentializing teacher PD across a range of offerings, even when tied to a relatively well-defined set of curriculum objectives as is the case with science AP courses. If, for instance, this study had been able to focus on a single well-designed PD offering as it relates to student AP outcomes, the findings may have been different. The data allowed only for the examination of features represented across a broad range of different AP-related PD offerings, and the results were an attenuation of any potential effects on student AP test outcomes or self-reported classroom practices. Nonetheless, the findings that laboratory investigation and the enactment of AP redesign practice in classroom instruction were positively associated with student performance can be viewed as an encouragement for PD developers to emphasize these instructional elements in their designs of PD for science teachers, as well as for teachers to primarily attend PD activities focusing on such instructional elements. Similarly, the positive association of teachers’ perceived administrative support with student performance alludes to a potential of PD being more effective when coupled with administrative support. Thus, this study argues for more focused research on teacher PD related to top-down curriculum reforms like the AP science redesign. Overall, the findings about teacher- and school-level factors that are related to student performance represent areas where policymakers and school leaders could make a meaningful difference in student outcomes with concerted effort. These changes could help to alleviate existing inequities in student science achievement to help all students make progress toward being better prepared for college-level science and their future careers.


Acknowledgements


The authors thank the following people for their contributions to this work: Amy Wheelock and Ted Gardella of the College Board, former members of the research team Yueming Jia and Janna Fuccillo Kook, and the thousands of AP teachers who helped shape and participated in this project. This work is supported by the National Science Foundation through the Discovery Research PreK-12 program (DRK-12), Award 1221861. The views contained in this article are those of the authors, and not their institutions, the College Board, or the National Science Foundation.


References

Abd-El-Khalick, F., BouJaoude, S., Duschl, R., Lederman, N. G., Mamlok-Naaman, R., Hofstein, A., . . . Tuan, H. (2004). Inquiry in science education: International perspectives. Science Education88(3), 397–419.

Agresti, A. (2002). Categorical data analysis (2nd ed.). New York, NY: Wiley-Interscience.

Atkinson, R. C., & Geiser, S. (2009). Reflections on a century of college admissions tests. Educational Researcher38(9), 665–676.

Banilower, E. R., Heck, D. J., & Weiss, I. R. (2007). Can professional development make the vision of the standards a reality? The impact of the National Science Foundation’s local systemic change through teacher enhancement initiative. Journal of Research in Science Teaching44(3), 375–395.

Bismack, A. S., Arias, A. M., Davis, E. A., & Palincsar, A. S. (2015). Examining student work for evidence of teacher uptake of educative curriculum materials. Journal of Research in Science Teaching52(6), 816–846.

Borko, H., Jacobs, J., & Koellner, K. (2010). Contemporary approaches to teacher professional development. In P. Peterson, E. Baker, & B. McGaw (Eds.), International encyclopedia of education (pp. 548–556). Oxford, UK: Elsevier.

Borman, G. D., & Dowling, N. M. (2008). Teacher attrition and retention: A meta-analytic and narrative review of the research. Review of Educational Research78(3), 367–409.

Boyd, D., Grossman, P. L., Hammerness, K., Lankford, R. H., Loeb, S., McDonald, M., . . . Wyckoff, J. (2008). Surveying the landscape of teacher education in New York City: Constrained variation and the challenge of innovation. Educational Evaluation and Policy Analysis30(4), 319–343.

Brault, M.-C., Janosz, M., & Archambault, I. (2014). Effects of school composition and school climate on teacher expectations of students: A multilevel analysis. Teaching and Teacher Education44, 148–159.

Brown, B. A. (2004). Discursive identity: Assimilation into the culture of science and its implications for minority students. Journal of Research in Science Teaching41(8), 810–834.

Brown, B. A. (2006). “It isn’t no slang that can be said about this stuff”: Language, identity, and appropriating science discourse. Journal of Research in Science Teaching43(1), 96–126.

Buxton, C. A. (2006). Creating contextually authentic science in a “low-performing” urban elementary school. Journal of Research in Science Teaching43(7), 695–721.

Byrne, D., & Callaghan, G. (2014). Complexity theory and the social sciences: The state of the art. New York, NY: Routledge.

Caprara, G. V., Barbaranelli, C., Steca, P., & Malone, P. S. (2006). Teachers’ self-efficacy beliefs as determinants of job satisfaction and students’ academic achievement: A study at the school level. Journal of School Psychology44(6), 473–490.

Carlsen, W. S. (1993). Teacher knowledge and discourse control: Quantitative evidence from novice biology teachers’ classrooms. Journal of Research in Science Teaching30(5), 471–481.

Chen, X. (2013). STEM attrition: College students’ paths into and out of STEM fields(Statistical Analysis Report No. NCES 2014-001). Washington, DC: National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education.

Clotfelter, C. T., Ladd, H. F., & Vigdor, J. (2005). Who teaches whom? Race and the distribution of novice teachers. Economics of Education Review24(4), 377–392.

Cochran-Smith, M., Ell, F., Ludlow, L., Grudnoff, L., & Aitken, G. (2014). The challenge and promise of complexity theory for teacher education research. Teachers College Record116(5), 1–38.

Cohen, D. K., & Ball, D. L. (1999). Instruction, capacity, and improvement (CPRE Research Report Series No. RR-43). Philadelphia, PA: University of Pennsylvania, Consortium for Policy Research in Education.

Conger, D., Long, M. C., & Iatarola, P. (2009). Explaining race, poverty, and gender disparities in advanced course-taking. Journal of Policy Analysis and Management28(4), 555–576.

Cromwell, A., McClarty, K. L., & Larson, S. J. (2013). College readiness indicators. Bulletin, 25, 1–8.

Crosnoe, R. (2009). Low-income students and the socioeconomic composition of public high schools. American Sociological Review74(5), 709–730.

Crosnoe, R., & Benner, A. D. (2015). Children at school. In R. M. Lerner (Ed.), Handbook on child psychology and developmental science (Vol. 4, pp. 268–304). Oxford, UK: Wiley.

Daly, A. J., Moolenaar, N. M., Der-Martirosian, C., & Liou, Y.-H. (2014). Accessing capital resources: Investigating the effects of teacher human and social capital on student achievement. Teachers College Record116(7), 1–42.

Darling-Hammond, L., & Richardson, N. (2009). Research review / Teacher learning: What matters? Educational Leadership66(5), 46–53.

Davis-Kean, P. E. (2005). The influence of parent education and family income on child achievement: The indirect role of parental expectations and the home environment. Journal of Family Psychology19(2), 294–304.

Desforges, C., & Abouchaar, A. (2003). The impact of parental involvement, parental support and family education on pupil achievement and adjustment: A literature review (Research Report No. RR433). Nottingham, UK: Department for Education and Skills.

Desimone, L. (2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher38(3), 181–199.

Desimone, L., & Garet, M. S. (2015). Best practices in teachers’ professional development in the United States. Psychology, Society and Education7(3), 252–263.

Desimone, L., & Le Floch, K. C. (2004). Are we asking the right questions? Using cognitive interviews to improve surveys in education research. Educational Evaluation and Policy Analysis26(1), 1–22.

Desimone, L., & Long, D. (2010). Teacher effects and the achievement gap: Do teacher and teaching quality influence the achievement gap between Black and White and high-and low-SES students in the early grades? Teachers College Record112(12), 3024–3073.

Desimone, L., Smith, T. M., & Phillips, K. (2013). Linking student achievement growth to professional development participation and changes in instruction: A longitudinal study of elementary students and teachers in Title I schools. Teachers College Record115(5), 1–46.

Diamond, B. S., Maerten-Rivera, J., Rohrer, R. E., & Lee, O. (2014). Effectiveness of a curricular and professional development intervention at improving elementary teachers’ science content knowledge and student achievement outcomes: Year 1 results. Journal of Research in Science Teaching51(5), 635–658.

Dimick, A. S. (2012). Student empowerment in an environmental science classroom: Toward a framework for social justice science education. Science Education96(6), 990–1012.

DiStefano, C., Zhu, M., & Mindrila, D. (2009). Understanding and using factor scores: Considerations for the applied researcher. Practical Assessment, Research & Evaluation14(20), 1–11.

Doppelt, Y., Schunn, C. D., Silk, E. M., Mehalik, M. M., Reynolds, B., & Ward, E. (2009). Evaluating the impact of a facilitated learning community approach to professional development on teacher practice and student achievement. Research in Science & Technological Education27(3), 339–354.

Ewing, M., Camara, W. J., & Millsap, R. E. (2006). The relationship between PSAT/NMSQT scores and AP examination grades: A follow-up study. New York, NY: The College Board.

Ewing, M., Huff, K., & Kaliski, P. (2010). Validating AP exam scores. Current research and new directions. In P. M. Sadler, G. Sonnert, R. H. Tai, & K. Klopfenstein (Eds.), AP: A critical examination of the Advanced Placement program (pp. 85–105). Cambridge, MA: Harvard Education Press.

Fenstermacher, G. D. (1994). The knower and the known: The nature of knowledge in research on teaching. Review of Research in Education20, 3–56.

Fischer, C. (2017). Examining forms and frames for science teacher learning related to large-scale reforms: A multi-manuscript dissertation (Doctoral thesis). University of Michigan, Ann Arbor, MI.

Fischer, C., Fishman, B., Dede, C., Eisenkraft, A., Frumin, K., Foster, B., . . . McCoy, A. (2018). Investigating relationships between school context, teacher professional development, teaching practices, and student achievement in response to a nationwide science reform. Teaching and Teacher Education72, 107–121.

Fischer, C., Fishman, B., Levy, A., Eisenkraft, A., Dede, C., Lawrenz, F., . . . McCoy, A. (2016). When do students in low-SES schools perform better-than-expected on a high-stakes test? Analyzing school, teacher, teaching, and professional development characteristics. Urban EducationOnlineFirst.

Fishman, B., Konstantopoulos, S., Kubitskey, B. W., Vath, R., Park, G., Johnson, H., & Edelson, D. C. (2013). Comparing the impact of online and face-to-face professional development in the context of curriculum implementation. Journal of Teacher Education64(5), 426–438.

Fishman, B., Marx, R. W., Best, S., & Tal, R. T. (2003). Linking teacher and student learning to improve professional development in systemic reform. Teaching and Teacher Education19(6), 643–658.

Fives, H., & Buehl, M. M. (2016). Teachers’ beliefs, in the context of policy reform. Policy Insights from the Behavioral and Brain Sciences3(1), 114–121.

Frumin, K., Dede, C., Fischer, C., Foster, B., Lawrenz, F., Eisenkraft, A., . . . McCoy, A. (2018). Adapting to large-scale changes in Advanced Placement Biology, Chemistry, and Physics: The impact of online teacher communities. International Journal of Science Education40(4), 397–420.

Furtak, E. M., Seidel, T., Iverson, H., & Briggs, D. C. (2012). Experimental and quasi-experimental studies of inquiry-based science teaching: A meta-analysis. Review of Educational Research82(3), 300–329.

Garet, M. S., Cronen, S., Eaton, M., Kurki, A., Ludwig, M., Jones, W., . . . Sztejnberg, L. (2008). The impact of two professional development interventions on early reading instruction and achievement (No. NCEE 2008-4030). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal38(4), 915–945.

Garet, M. S., Wayne, A. J., Stancavage, F., Taylor, J., Eaton, M., Walters, K., . . . Doolittle, F. (2011). Middle school mathematics professional development impact study: Findings after the second year of implementation (No. NCEE 2011-4024). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

Gersten, R., Dimino, J., Jayanthi, M., Kim, J. S., & Santoro, L. E. (2010). Teacher study group: Impact of the professional development model on reading instruction and student outcomes in first grade classrooms. American Educational Research Journal47(3), 694–739.

Grice, J. W. (2001). Computing and evaluating factor scores. Psychological Methods6(4), 430–450.

Grimberg, B. I., & Gummer, E. (2013). Teaching science from cultural points of intersection. Journal of Research in Science Teaching50(1), 12–32.

Hallett, R. E., & Venegas, K. M. (2011). Is increased access enough? Advanced Placement courses, quality, and success in low-income urban schools. Journal for the Education of the Gifted34(3), 468–487.

Hamilton, L. S., McCaffrey, D. F., Stecher, B. M., Klein, S. P., Robyn, A., & Bugliari, D. (2003). Studying large-scale reforms of instructional practice: An example from mathematics and science. Educational Evaluation and Policy Analysis25(1), 1–29.

Harper, S. R. (2010). An anti-deficit achievement framework for research on students of color in STEM. New Directions for Institutional Research2010(148), 63–74.

Hattie, J. (2012). Visible learning for teachers: Maximizing impact on learning. New York, NY: Routledge.

Hedeker, D., & Gibbons, R. D. (1994). A random-effects ordinal regression model for multilevel analysis. Biometrics50(4), 933–944.

Higgins, J., & Green, S. (2011). Cochrane handbook for systematic reviews of interventions(4th ed.). Chichester, UK: John Wiley & Sons.

Hill, H. C. (2015). Review of the mirage: Confronting the hard truth about our quest for teacher development. Boulder, CO: National Education Policy Center. Retrieved from http://www.greatlakescenter.org/docs/Think_Twice/TT-Hill-TNTP.pdf

Houseal, A. K., Abd-El-Khalick, F., & Destefano, L. (2014). Impact of a student-teacher-scientist partnership on students’ and teachers’ content knowledge, attitudes toward science, and pedagogical practices. Journal of Research in Science Teaching51(1), 84–115.

Hox, J. J. (2010). Multilevel analysis: Techniques and applications (2nd ed.). New York, NY: Routledge.

Jackson, C. K., Rockoff, J. E., & Staiger, D. O. (2014). Teacher effects and teacher-related policies. Annual Review of Economics6(1), 801–825.

Jacob, A., & McGovern, K. (2015). The mirage: Confronting the hard truth about our quest for teacher development. New York, NY: The New Teacher Project.

Johnson, S. M., Kraft, M. A., & Papay, J. P. (2012). How context matters in high-need schools: The effects of teachers’ working conditions on their professional satisfaction and their students’ achievement. Teachers College Record114(10), 1–39.

Kane, T. J., & Staiger, D. O. (2008). Estimating teacher impacts on student achievement: An experimental evaluation. Cambridge, MA: National Bureau of Economic Research.

Kane, T. J., & Staiger, D. O. (2012). Gathering feedback for teaching: Combining high-quality observations with student surveys and achievement gains (Research paper). Seattle, WA: Bill & Melinda Gates Foundation. Retrieved from http://eric.ed.gov/?id=ED540960

Keller, M. M., Neumann, K., & Fischer, H. E. (2017). The impact of physics teachers’ pedagogical content knowledge and motivation on students’ achievement and interest. Journal of Research in Science Teaching54(5), 586–614.

Kennedy, M. M. (2016). How does professional development improve teaching? Review of Educational Research86(4), 945–980.

Klassen, R. M., & Tze, V. M. C. (2014). Teachers’ self-efficacy, personality, and teaching effectiveness: A meta-analysis. Educational Research Review12, 59–76.

Klopfenstein, K. (2004a). Advanced Placement: Do minorities have equal opportunity? Economics of Education Review23(2), 115–131.

Klopfenstein, K. (2004b). The Advanced Placement expansion of the 1990s: How did traditionally underserved students fare? Education Policy Analysis Archives12(68), 1–13.

Klugman, J. (2013). The Advanced Placement arms race and the reproduction of educational inequality. Teachers College Record115(5), 1–34.

Kraft, M. A., Marinell, W. H., & Shen-Wei Yee, D. (2016). School organizational contexts, teacher turnover, and student achievement: Evidence from panel data. American Educational Research Journal53(5), 1411–1449.

Kyburg, R. M., Hertberg-Davis, H., & Callahan, C. M. (2007). Advanced Placement and International Baccalaureate programs: Optimal learning environments for talented minorities? Journal of Advanced Academics18(2), 172–215.

Ladson-Billings, G. (1995). Toward a theory of culturally relevant pedagogy. American Educational Research Journal32(3), 465–491.

Ladson-Billings, G. (2006). From the achievement gap to the education debt: Understanding achievement in US schools. Educational Researcher35(7), 3–12.

Ladson-Billings, G., & Tate, W. F. (1995). Toward a critical race theory of education. Teachers College Record97(1), 47–68.

Lee, O., & Buxton, C. (2011). Engaging culturally and linguistically diverse students in learning science. Theory Into Practice50(4), 277–284.

Lee, O., Deaktor, R., Enders, C., & Lambert, J. (2008). Impact of a multiyear professional development intervention on science achievement of culturally and linguistically diverse elementary students. Journal of Research in Science Teaching45(6), 726–747.

Lemke, J. K., & Sabelli, N. H. (2008). Complex systems and educational change: Towards a new research agenda. In M. Mason (Ed.), Complexity theory and the philosophy of education(pp. 112–123). Chichester, UK: Wiley-Blackwell.

Lichten, W. (2010). Whither Advanced Placement — now? In P. M. Sadler, G. Sonnert, R. H. Tai, & K. Klopfenstein (Eds.), AP: A critical examination of the Advanced Placement program(pp. 233–243). Cambridge, MA: Harvard Education Press.

Llosa, L., Lee, O., Jiang, F., Haas, A., O’Connor, C., Van Booven, C. D., & Kieffer, M. J. (2016). Impact of a large-scale science intervention focused on English Language Learners. American Educational Research Journal53(2), 395–424.

Marcotte, D. E., & Hansen, B. (2010). Time for school. Education Next10(1), 52–59.

May, H., & Supovitz, J. A. (2011). The scope of principal efforts to improve instruction. Educational Administration Quarterly47(2), 332–352.

Mikeska, J. N., Shattuck, T., Holtzman, S., McCaffrey, D. F., Duchesneau, N., Qi, Y., & Stickler, L. (2017). Understanding science teaching effectiveness: Examining how science-specific and generic instructional practices relate to student achievement in secondary science classrooms. International Journal of Science Education39(18), 2594–2623.

National Academy of Sciences, National Academy of Engineering, & Institute of Medicine. (2007). Rising above the gathering storm: Energizing and employing America for a brighter economic future. Washington, DC: National Academies Press.

National Research Council. (2000). How people learn: Brain, mind, experience, and school. Washington, DC: National Academies Press.

National Research Council. (2002). Learning and understanding: Improving advanced study of mathematics and science in U.S. high schools. Washington, DC: National Academies Press.

National Research Council. (2005). How students learn: History, mathematics, and science in the classroom. Washington, DC: National Academies Press.

National Research Council. (2006). America’s lab report: Investigations in high school science. Washington, DC: National Academies Press.

National Research Council. (2012a). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. Washington, DC: National Academies Press.

National Research Council. (2012b). Education for life and work: Developing transferable knowledge and skills in the 21st century. Washington, DC: National Academies Press.

National Student Clearninghouse. (2017). The role of community colleges in postsecondary success (No. 0117–0). Herndon, VA: National Student Clearninghouse Research Center.

NGSS Lead States. (2013). Next generation science standards: For states, by states. Washington, DC: Achieve, Inc. On behalf of the 26 states and partners that collaborated on the NGSS.

Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004). How large are teacher effects? Educational Evaluation and Policy Analysis26(3), 237–257.

OECD. (2013). Country note. Results from PISA 2012. United States. Paris, France: OECD Publishing.

Opfer, V. D., & Pedder, D. (2011). Conceptualizing teacher professional learning. Review of Educational Research81(3), 376–407.

Ost, B. (2014). How do teachers improve? The relative importance of specific and general human capital. American Economic Journal: Applied Economics6(2), 127–151.

Palardy, G. J. (2013). High school socioeconomic segregation and student attainment. American Educational Research Journal50(4), 714–754.

Papay, J. P., & Kraft, M. A. (2015). Productivity returns to experience in the teacher labor market: Methodological challenges and new evidence on long-term career improvement. Journal of Public Economics130, 105–119.

Paris, D. (2012). Culturally sustaining pedagogy: A needed change in stance, terminology, and practice. Educational Researcher41(3), 93–97.

Penuel, W. R., Fishman, B., Yamaguchi, R., & Gallagher, L. P. (2007). What makes professional development effective? Strategies that foster curriculum implementation. American Educational Research Journal44(4), 921–958.

Penuel, W. R., Gallagher, L. P., & Moorthy, S. (2011). Preparing teachers to design sequences of instruction in earth systems science: A comparison of three professional development programs. American Educational Research Journal48(4), 996–1025.

Price, A. C., & Chiu, A. (2018). An experimental study of a museum-based, science PD programme’s impact on teachers and their students. International Journal of Science Education40(9), 941–960.

Reardon, S. F. (2013). The widening income achievement gap. Educational Leadership70(8), 10–16.

Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools, and academic achievement. Econometrica73(2), 417–458.

Rockoff, J. E. (2004). The impact of individual teachers on student achievement: Evidence from panel data. The American Economic Review94(2), 247–252.

Roegman, R., & Hatch, T. (2016). The AP lever for boosting access, success, and equity. Phi Delta Kappan97(5), 20–25.

Ronfeldt, M., Loeb, S., & Wyckoff, J. (2013). How teacher turnover harms student achievement. American Educational Research Journal50(1), 4–36.

Roth, K. J., Garnier, H. E., Chen, C., Lemmens, M., Schwille, K., & Wickler, N. I. Z. (2011). Videobased lesson analysis: Effective science PD for teacher and student learning. Journal of Research in Science Teaching48(2), 117–148. 

Rothstein, J. M. (2004). College performance predictions and the SAT. Journal of Econometrics121(1–2), 297–317.

Rowan, B., Correnti, R., & Miller, R. J. (2002). What large-scale, survey research tells us about teacher effects on student achievement: Insights from the Prospects study of elementary schools (CPRE Research Report Series No. RR-051). Philadelphia, PA: Consortium for Policy Research in Education.

Sass, T. R., Hannaway, J., Xu, Z., Figlio, D. N., & Feng, L. (2012). Value added of teachers in high-poverty schools and lower poverty schools. Journal of Urban Economics72(2–3), 104–122.

Schneider, J. (2009). Privilege, equity, and the Advanced Placement program: Tug of war. Journal of Curriculum Studies41(6), 813–831.

Schuchardt, A. M., Tekkumru-Kisa, M., Schunn, C. D., Stein, M. K., & Reynolds, B. (2017). How much professional development is needed with educative curriculum materials? It depends upon the intended student learning outcomes. Science Education101(6), 1015–1033.

Secker, C. von. (2002). Effects of inquiry-based teacher practices on science excellence and equity. The Journal of Educational Research95(3), 151–160. 

Seidel, T., & Shavelson, R. J. (2007). Teaching effectiveness research in the past decade: The role of theory and research design in disentangling meta-analysis results. Review of Educational Research77(4), 454–499. 

Shih, T.-H., & Fan, X. (2009). Comparing response rates in e-mail and paper surveys: A meta-analysis. Educational Research Review4(1), 26–40. 

Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher15(2), 4–14. 

Skaalvik, E. M., & Skaalvik, S. (2010). Teacher self-efficacy and teacher burnout: A study of relations. Teaching and Teacher Education26(4), 1059–1069. 

Snijders, T. A. B., & Bosker, R. J. (2012). Multilevel analysis: An introduction to basic and applied multilevel analysis (2nd ed.). Thousand Oaks, CA: SAGE.

Snyder, T. D., de Brey, C., & Dillow, S. A. (2016). Digest of Education Statistics 2015 (NCES 2016-014). Washington, DC: National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education.

Solorzano, D. G., & Ornelas, A. (2004). A critical race analysis of Latina/o and African American Advanced Placement enrollment in public high schools. The High School Journal87(3), 15–26.

Supovitz, J. A., Sirinides, P., & May, H. (2010). How principals and peers influence teaching and learning. Educational Administration Quarterly46(1), 31–56. 

Supovitz, J. A., & Turner, H. M. (2000). The effects of professional development on science teaching

practices and classroom culture. Journal of Research in Science Teaching37(9), 963–980.

Taliaferro, J. D., & DeCuir-Gunby, J. T. (2008). African American educators’ perspectives on the Advanced Placement opportunity gap. The Urban Review40(2), 164–185.

The College Board. (2014). The 10th annual AP report to the nation. New York, NY: The College Board.

Tschannen-Moran, M., Hoy, A. W., & Hoy, W. K. (1998). Teacher efficacy: Its meaning and measure. Review of Educational Research68(2), 202–248.

van Aalderen-Smeets, S. I., & Walma van der Molen, J. H. (2015). Improving primary teachers’ attitudes toward science by attitude-focused professional development. Journal of Research in Science Teaching52(5), 710–734. 

van Geel, M., Keuning, T., Visscher, A. J., & Fox, J.-P. (2016). Assessing the effects of a school-wide data-based decision-making intervention on student achievement growth in primary schools. American Educational Research Journal53(2), 360–394. 

Veal, W. R., Riley Lloyd, M. E., Howell, M. R., & Peters, J. (2016). Normative beliefs, discursive claims, and implementation of reform-based science standards. Journal of Research in Science Teaching53(9), 1419–1443. 

Viechtbauer, W. (2005). Bias and efficiency of meta-analytic variance estimators in the random-effects model. Journal of Educational and Behavioral Statistics30(3), 261–293.

Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software36(3), 1–48.

Viechtbauer, W. (2016). Meta-analysis package for R [R]. Retrieved from http://www.metafor-project.org/

Warren, B., Ballenger, C., Ogonowski, M., Rosebery, A. S., & Hudicourt-Barnes, J. (2001). Rethinking diversity in learning science: The logic of everyday sense-making. Journal of Research in Science Teaching38(5), 529–552. 

Waters, T., Marzano, R. J., & McNulty, B. (2003). Balanced leadership: What 30 years of research tells us about the effect of leadership on student achievement. A working paper. Aurora, CO: Mid-Continent Regional Educational Lab.

Willms, J. D. (2010). School composition and contextual effects on student outcomes. Teachers College Record112(4), 1008–1037.

Woessmann, L. (2004). How equal are educational opportunities? Family background and student achievement in Europe and the US (CESIFO Working Paper No. 1162). Munich, Germany: Ifo Institute for Economic Research.

Wyatt, J. N., & Mattern, K. D. (2011). Low-SES students and college outcomes: The role of AP fee reductions. New York, NY: The College Board.

Zarate, M. E., & Pachon, H. P. (2006). Gaining or losing ground? Equity in offering Advanced Placement courses in California high schools 1997-2003. Los Angeles, CA: Tomas Rivera Policy Institute.

Zhang, X., Patel, P., & Ewing, M. (2014). AP® Potential predicted by PSAT/NMSQT® scores using logistic regression (Statistical Report No. 2014-1). New York, NY: The College Board.




APPENDIX


Appendix Table A1. Factor Structure of Composite Variables

 

Biology Year 2

Biology Year 3

Chemistry Year 1

Chemistry Year 2

Physics Year 1

Variable

FL

U

SC

FL

U

SC

FL

U

SC

FL

U

SC

FL

U

SC

 Administrative support

Principal understands students’ challenges

 0.804

0.354

 0.438

 0.782

0.389

 0.442

 0.808

0.347

 0.446

 0.777

0.396

 0.415

 0.802

 0.358

 0.438

Principal understands teachers’ challenges

 0.817

0.333

 0.474

 0.781

0.390

 0.440

 0.814

0.338

 0.461

 0.796

0.367

 0.459

 0.810

 0.343

 0.461

Principal supports PD

 0.559

0.688

 0.157

 0.547

0.701

 0.171

 0.562

0.685

 0.157

 0.559

0.688

 0.172

 0.554

 0.693

 0.156

Lighter teaching load for AP teachers

 0.312

0.902

 0.067

 0.352

0.876

 0.088

 0.342

0.883

 0.074

 0.390

0.848

 0.097

 0.388

 0.850

 0.089

Fewer out-of-class responsibilities

 0.268

0.928

 0.056

 0.297

0.912

 0.072

 0.274

0.925

 0.057

 0.337

0.887

 0.080

 0.348

 0.879

 0.077

Exclusive funding for the AP science course

 0.360

0.870

 0.080

 0.367

0.865

 0.093

 0.343

0.883

 0.074

 0.347

0.879

 0.084

 0.377

 0.858

 0.086

Availability of equipment to perform labs

 0.458

0.790

 0.112

 0.464

0.785

 0.130

 0.476

0.773

 0.118

 0.466

0.783

 0.126

 0.425

 0.819

 0.101

Availability of consumables to perform labs

 0.480

0.769

 0.120

 0.492

0.758

 0.143

 0.469

0.780

 0.115

 0.478

0.772

 0.131

 0.457

 0.791

 0.113

     Cronbach’s α (administrative support)

 0.727

 

 

 0.723

 

 

 0.729

 

 

 0.731

 

 

 0.737

 

 

 Enactment of AP redesign: Practices

Students work on lab investigations

 0.300

0.727

 0.161

 0.327

0.706

 0.163

 0.293

0.783

 0.129

 0.372

0.742

 0.083

 0.368

 0.718

 0.206

Use a science practice in class outside the lab

 0.337

0.880

 0.151

 0.587

0.702

 0.316

 0.428

0.829

 0.180

 0.322

0.844

 0.106

 0.626

 0.659

 0.406

Guidance on content integration questions

 0.722

0.503

 0.567

 0.558

0.631

 0.327

 0.719

0.497

 0.504

 0.813

0.380

 0.588

 0.386

 0.628

 0.244

Guidance on open/free response questions

 0.730

0.528

 0.546

 0.605

0.627

 0.360

 0.722

0.531

 0.475

 0.799

0.437

 0.500

 0.354

 0.710

 0.199

Students reporting lab findings to each other

 0.404

0.763

 0.208

 0.518

0.676

 0.284

 0.363

0.775

 0.162

 0.282

0.742

 0.108

 0.507

 0.648

 0.322

Students work on inquiry lab investigations

 n.i.s.

n.i.s.

 n.i.s.

 0.595

0.658

 0.339

 0.465

0.748

 0.216

 0.312

0.787

 0.112

 0.674

 0.575

 0.496

     Cronbach’s α (AP practices)

 0.666

 

 

 0.717

 

 

 0.676

 

 

 0.706

 

 

 0.711

 

 

 Enactment of AP redesign: Curriculum

Refer to the “Big Ideas” of physics

 0.702

0.485

 0.290

 0.719

0.467

 0.305

 0.748

0.424

 0.292

 0.781

0.417

 0.289

 0.773

 0.407

 0.343

Refer to how enduring understandings

  relate to “Big Ideas”

 0.731

0.401

 0.365

 0.738

0.377

 0.384

 0.733

0.372

 0.324

 0.774

0.373

 0.322

 0.765

 0.385

 0.356

Refer to learning objective from AP

  curriculum

 0.786

0.417

 0.379

 0.761

0.430

 0.353

 0.795

0.386

 0.342

 0.806

0.398

 0.312

 0.746

 0.464

 0.293

Refer to curriculum framework

 0.717

0.518

 0.279

 0.720

0.525

 0.276

 0.793

0.424

 0.311

 0.769

0.442

 0.268

 0.698

 0.540

 0.236

     Cronbach’s α (AP curriculum)

 0.833

 

 

 0.836

 

 

 0.859

 

 

 0.867

 

 

 0.882

 

 

 Self-efficacy

Students perform better because of my effort

 0.571

0.674

 0.566

 0.609

0.629

 0.534

 0.544

0.704

 0.538

 0.619

0.617

 0.614

 0.608

 0.630

 0.514

Scores improve because of my teaching

 0.633

0.599

 0.705

 0.664

0.559

 0.654

 0.614

0.623

 0.685

 0.615

0.622

 0.606

 0.662

 0.562

 0.627

Teaching can overcome student backgrounds

 0.429

0.816

 0.351

 0.471

0.778

 0.334

 0.438

0.808

 0.377

 0.462

0.787

 0.360

 0.516

 0.734

 0.375

Extra teaching effort            --0.326          

Produces little change

0.894

–0.243

-0.361

0.870

 –0.229

 –0.386

0.851

 –0.316

 –0.342

0.883

 –0.237

 –0.361

 0.870

 –0.221

     Cronbach’s α (self-efficacy)

 0.569

 

 

 0.617

 

 

 0.588

 

 

 0.602

 

 

 0.630

 

 

Note. FL = factor loading, U = uniqueness, SC = scoring coefficient, n.i.s. = not in survey





Appendix Table A2. Biology 2015: Multi-level Ordered Logistic Regression Analysis, Biology Year 3

 

Model 1

Model 2

Model 3

Model 4

 

Variable

OR

SE

z

OR

SE

z

OR

SE

z

OR

SE

z

 

Level 1

  


         

PSAT score

8.316***

0.186

94.88

8.086***

0.182

92.77

7.845***

0.177

91.24

7.846***

0.177

91.29

Ed level: Some post-secondary


 


1.101*

0.050

2.14

1.065

0.048

1.40

1.066

0.048

1.42

Ed level: Bachelor’s


 


1.291***

0.057

5.82

1.217***

0.053

4.46

1.218***

0.053

4.48

Ed level: Graduate degree


 


1.430***

0.068

7.57

1.341***

0.064

6.19

1.344***

0.064

6.24

Level 2


 



 



 



 


Funding < $200


 



 


1.135

0.079

1.81

1.156*

0.080

2.10

Funding > $300


 



 


0.905

0.084

–1.07

0.894

0.082

–1.22

% FRLP


 



 


0.200***

0.029

–11.05

0.232***

0.034

–10.04

Administrative support


 



 


1.050

0.034

1.51

1.040

0.033

1.25

Hours of AP instruction in 10 hours


 



 


1.028***

0.006

4.69

1.026***

0.006

4.41

Enactment: Practice elements


 



 


1.056

0.038

1.51

1.078*

0.039

2.09

Enactment: Curriculum elements


 



 


0.901**

0.031

–3.05

0.903**

0.031

–2.97

Int: Practice * Curriculum elements


 



 


1.055

0.030

1.89

1.054

0.030

1.87

Number of labs


 



 


1.009

0.006

1.37

1.006

0.006

0.91

Years AP teaching experience


 



 


1.017**

0.006

2.74

1.017**

0.006

2.76

Years AP redesign experience


 



 


1.049

0.056

0.89

1.015

0.055

0.28

Self-efficacy


 



 


1.024

0.032

0.74

1.033

0.032

1.04

Importance of PD for AP scores


 



 


1.005

0.034

0.15

0.998

0.034

–0.07

Number of above-the-line PD


 



 



 


0.978

0.034

–0.63

Number of below-the-line PD


 



 



 


1.023

0.020

1.16

PD includes active learning


 



 



 


0.816***

0.036

–4.54

PD has responsive agenda


 



 



 


1.133*

0.067

2.10

PD focuses on student work


 



 



 


0.880*

0.045

–2.52

PD models teaching


 



 



 


0.960

0.050

–0.80

PD helps relationship building


 



 



 


1.018

0.068

0.27

PD effectively supports instruction


 



 



 


1.159

0.092

1.87

Above-the-line PD duration in hours


 



 



 


1.001

0.001

1.11

    Cut point 1

–5.011***

0.057

–87.63

–4.825***

0.064

–75.30

–4.759***

0.164

–29.03

–4.839***

0.165

–29.25

    Cut point 2

–1.495***

0.039

–38.66

–1.312***

0.049

–26.81

–1.232***

0.159

–7.75

–1.310***

0.160

–8.18

    Cut point 3

1.560***

0.039

39.97

1.750***

0.050

35.21

1.830***

0.159

11.49

1.752***

0.160

10.92

    Cut point 4

4.565***

0.053

86.77

4.758***

0.061

78.12

4.828***

0.163

29.59

4.750***

0.164

28.90

Random Effects

Level 2 variance

1.078

 


1.034

  

0.819

 


0.781

 


Deviance Statistics

Deviance

41824.99

  

41756.00

 

41544.99

  

41502.30

  

AIC

41836.99

  

41774.00

 

41588.99

  

41564.30

  

BIC

41884.82

  

41845.75

  

41764.38

  

41811.45

  

χ2

11960.98

  

68.99

  

211.01

  

42.69

  

df

1

  

3

  

13

  

9

  

Sig

***

  

***

  

***

  

***

  

Δ% Var. accounted for by model

44.42

  

2.29

  

11.09

  

1.95

  

Level 2 ICC

0.25

  

0.24

  

0.20

  

0.19

  

Note. Ns = 21,429; Nt = 1,243. *** p < 0.001. ** p < 0.01. * p < 0.05. Null model level 2 variance = 1.940; null model ICC = 0.37; FRLP = free or reduced-priced lunch program, Int = Interaction term.



Appendix Table A3. Biology 2014: Multi-level Ordered Logistic Regression Analysis, Biology Year 2

 

Model 1

Model 2

Model 3

Model 4

 

Variable

OR

SE

z

OR

SE

z

OR

SE

z

OR

SE

z

 

Level 1

            

PSAT score

7.953***

0.163

101.42

7.770***

0.160

99.28

7.573***

0.157

97.95

7.572***

0.156

97.97

Ed level: Some post-secondary


  

1.056

0.044

1.32

1.027

0.043

0.64

1.027

0.043

0.65

Ed level: Bachelor’s


  

1.257***

0.051

5.63

1.200***

0.049

4.49

1.200***

0.049

4.49

Ed level: Graduate degree


  

1.310***

0.057

6.19

1.247***

0.054

5.07

1.247***

0.054

5.07

Level 2


  


 



 



 


Funding < $200


  


 


1.240**

0.089

2.99

1.237**

0.088

2.99

Funding > $300


  


 


0.994

0.093

–0.07

0.971

0.090

–0.32

% FRLP


  


 


0.148***

0.022

–12.77

0.164***

0.024

–12.11

Administrative support


  


 


1.061

0.034

1.83

1.073*

0.035

2.18

Hours of AP instruction in 10 hours


  


 


1.027***

0.006

4.45

1.026***

0.006

4.24

Enactment: Practice elements


  


 


1.061

0.038

1.68

1.062

0.037

1.70

Enactment: Curriculum elements


  


 


0.874***

0.031

–3.80

0.873***

0.031

–3.80

Int: Practice * Curriculum elements


  


 


1.009

0.032

0.29

1.007

0.031

0.22

Number of labs


  


 


1.022***

0.006

3.68

1.023***

0.006

3.74

Years AP teaching experience


  


 


1.008

0.006

1.19

1.006

0.006

0.96

Years AP redesign experience


  


 


1.314*

0.154

2.34

1.224

0.143

1.73

Self-efficacy


  


 


1.030

0.033

0.91

1.035

0.033

1.09

Importance of PD for AP scores


  


 


0.980

0.033

–0.61

0.971

0.033

–0.87

Number of above-the-line PD


  


 



 


1.057

0.036

1.60

Number of below-the-line PD


  


 



 


1.025

0.025

1.04

PD includes active learning


  


 



 


0.839***

0.035

–4.21

PD has responsive agenda


  


 



 


1.072

0.062

1.20

PD focuses on student work


  


 



 


0.912

0.044

–1.90

PD models teaching


  


 



 


0.948

0.046

–1.10

PD helps relationship building


  


 



 


1.018

0.060

0.31

PD effectively supports instruction


  


 



 


1.093

0.076

1.29

Above-the-line PD duration in hours


  


 



 


1.001

0.001

0.98

    Cut point 1

–4.832***

0.054

–89.54

–4.687***

0.060

–77.56

–4.191***

0.235

–17.82

–4.334***

0.235

–18.42

    Cut point 2

–1.494***

0.039

–38.08

–1.346***

0.048

–27.81

–0.831***

0.233

–3.57

–0.974***

0.233

–4.18

    Cut point 3

1.444***

0.039

36.88

1.596***

0.049

32.83

2.110***

0.233

9.05

1.967***

0.233

8.43

    Cut point 4

4.382***

0.049

88.60

4.534***

0.057

79.41

5.037***

0.235

21.41

4.894***

0.235

20.81

Random Effects

Level 2 variance

1.252

 


1.216

  

0.925

 


0.891

 


Deviance Statistics

Deviance

50051.24

  

49994.14

 

49727.51

  

49693.70

  

AIC

50063.24

  

50012.14

 

49771.51

  

49755.69

  

BIC

50112.02

  

50085.32

  

49950.39

  

50007.75

  

χ2

13449.05

  

57.10

  

266.63

  

33.82

  

df

1

  

3

  

13

  

9

  

Sig

***

  

***

  

***

  

***

  

Δ% Var. accounted for by model

41.27

  

1.68

  

13.66

  

1.59

  

Level 2 ICC

0.28

  

0.27

  

0.22

  

0.21

  

Note. Ns = 25,108; Nt = 1,265. *** p < 0.001. ** p < 0.01. * p < 0.05. Null model level 2 variance = 2.132; null model ICC = 0.39; FRLP = free or reduced-priced lunch program.




Appendix Table A4. Chemistry 2015: Multi-level Ordered Logistic Regression Analysis, Chemistry Year 2

 

Model 1

Model 2

Model 3

Model 4

 

Variable

OR

SE

z

OR

SE

z

OR

SE

z

OR

SE

z

 

Level 1

  


         

PSAT score

5.565***

0.113

84.18

5.452***

0.113

81.89

5.320***

0.110

80.81

5.323***

0.110

80.85

Ed level: Some post-secondary


  

1.056

0.050

1.13

1.026

0.049

0.54

1.026

0.049

0.53

Ed level: Bachelor’s


  

1.169**

0.053

3.45

1.109*

0.050

2.29

1.109*

0.050

2.29

Ed level: Graduate degree


  

1.259***

0.060

4.81

1.186***

0.057

3.57

1.185***

0.057

3.55

Level 2


  


 



 



 


Funding < $200


  


 


1.108

0.091

1.24

1.124

0.092

1.43

Funding > $300


  


 


1.127

0.116

1.16

1.145

0.118

1.32

% FRLP


  


 


0.085***

0.015

–13.55

0.095***

0.017

–12.84

Administrative support


  


 


0.999

0.036

–0.02

0.999

0.036

–0.03

Hours of AP instruction in 10 hours


  


 


1.030***

0.007

4.13

1.030***

0.007

4.14

Enactment: Practice elements


  


 


1.080

0.044

1.91

1.072

0.044

1.71

Enactment: Curriculum elements


  


 


0.821***

0.033

–4.85

0.823***

0.034

–4.75

Int: Practice * Curriculum elements


  


 


1.036

0.035

1.07

1.042

0.035

1.24

Number of labs


  


 


1.053***

0.007

7.36

1.053***

0.007

7.33

Years AP teaching experience


  


 


1.027***

0.007

4.03

1.026***

0.007

3.87

Years AP redesign experience


  


 


1.345*

0.171

2.33

1.271

0.163

1.87

Self-efficacy


  


 


1.120**

0.042

3.00

1.109**

0.042

2.76

Importance of PD for AP scores


  


 


0.890**

0.035

–2.95

0.884**

0.035

–3.10

Number of above-the-line PD


  


 



 


1.014

0.043

0.33

Number of below-the-line PD


  


 



 


1.006

0.024

0.26

PD includes active learning


  


 



 


0.824***

0.044

–3.65

PD has responsive agenda


  


 



 


1.077

0.073

1.10

PD focuses on student work


  


 



 


0.962

0.056

–0.67

PD models teaching


  


 



 


0.965

0.057

–0.59

PD helps relationship building


  


 



 


0.947

0.066

–0.78

PD effectively supports instruction


  


 



 


1.253**

0.092

3.07

Above-the-line PD duration in hours


  


 



 


1.002

0.002

1.19

    Cut point 1

–2.594***

0.050

–52.38

–2.471***

0.059

–41.56

–2.017***

0.252

–8.02

–2.116***

0.253

–8.36

    Cut point 2

–0.267***

0.046

–5.84

–0.143***

0.057

–2.52

0.319***

0.251

1.27

0.221***

0.253

0.87

    Cut point 3

2.086***

0.048

43.21

2.212***

0.059

37.54

2.672***

0.252

10.61

2.575***

0.253

10.17

    Cut point 4

4.112***

0.055

74.43

4.237***

0.065

65.55

4.694***

0.253

18.53

4.597***

0.255

18.05

Random Effects

Level 2 variance

1.900

 


1.860

  

1.229

 


1.200

 


Deviance Statistics

Deviance

48133.58

  

48104.57

 

47672.81

  

47645.20

  

AIC

48145.58

  

48122.57

 

47716.81

  

47707.21

  

BIC

48193.35

  

48194.22

  

47891.97

  

47954.03

  

χ2

8532.51

  

29.01

  

431.76

  

27.60

  

df

1

  

3

  

13

  

9

  

Sig

***

  

***

  

***

  

**

  

Δ% Var. accounted for by model

31.86

  

1.33

  

22.68

  

1.20

  

Level 2 ICC

0.37

  

0.36

  

0.27

  

0.27

  

Note. Ns = 21,204; Nt = 1,219. *** p < 0.001. ** p < 0.01. * p < 0.05. Null model level 2 variance = 2.784; null model ICC = 0.46; FRLP = free or reduced-priced lunch program.




Appendix Table A5. Chemistry 2014: Multi-level Ordered Logistic Regression Analysis, Chemistry Year 1

 

Model 1

Model 2

Model 3

Model 4

 

Variable

OR

SE

z

OR

SE

z

OR

SE

z

OR

SE

z

 

Level 1

            

PSAT score

5.230***

0.094

92.24

5.159***

0.094

90.17

5.046***

0.092

88.98

5.048***

0.092

89.03

Ed level: Some post-secondary


 


0.977

0.041

–0.55

0.946

0.040

–1.30

0.945

0.040

–1.33

Ed level: Bachelor’s


 


1.122**

0.045

2.86

1.067

0.043

1.62

1.066

0.043

1.59

Ed level: Graduate degree


 


1.143**

0.049

3.13

1.087

0.046

1.95

1.086

0.046

1.93

Level 2


 



 



 



 


Funding < $200


 



 


0.983

0.078

–0.21

1.018

0.080

0.23

Funding > $300


 



 


0.880

0.087

–1.30

0.910

0.089

–0.97

% FRLP


 



 


0.108***

0.019

–12.95

0.122***

0.021

–12.35

Administrative support


 



 


1.034

0.037

0.93

1.025

0.036

0.70

Hours of AP instruction in 10 hours


 



 


1.039***

0.007

5.72

1.037***

0.007

5.46

Enactment: Practice elements


 



 


1.011

0.039

0.28

1.023

0.039

0.60

Enactment: Curriculum elements


 



 


0.847***

0.032

–4.38

0.859***

0.033

–4.03

Int: Practice * Curriculum elements


 



 


0.990

0.032

–0.32

0.989

0.032

–0.33

Number of labs


 



 


1.047***

0.007

6.67

1.045***

0.007

6.43

Years AP teaching experience


 



 


1.033***

0.006

5.55

1.032***

0.006

5.50

Years AP redesign experience


 



 


     —

   —

     —

    —

  —

    —

Self-efficacy


 



 


1.106**

0.040

2.77

1.102**

0.039

2.71

Importance of PD for AP scores


 



 


0.953

0.034

–1.33

0.935

0.034

–1.86

Number of above-the-line PD


 



 



 


1.068

0.041

1.70

Number of below-the-line PD


 



 



 


1.000

0.026

–0.01

PD includes active learning


 



 



 


0.893*

0.042

–2.41

PD has responsive agenda


 



 



 


1.050

0.060

0.85

PD focuses on student work


 



 



 


0.889*

0.042

–2.50

PD models teaching


 



 



 


0.886*

0.045

–2.41

PD helps relationship building


 



 



 


0.918

0.053

–1.49

PD effectively supports instruction


 



 



 


1.348***

0.080

5.04

Above-the-line PD duration in hours


 



 



 


1.001

0.001

1.40

    Cut point 1

–2.696***

0.045

–59.31

–2.628***

0.054

–48.34

–2.842***

0.073

–38.94

–2.833***

0.072

–39.23

    Cut point 2

–0.286***

0.042

–6.83

–0.217***

0.052

–4.22

–0.427***

0.071

–6.06

–0.418***

0.070

–5.99

    Cut point 3

1.826***

0.043

41.99

1.896***

0.053

35.85

1.685***

0.071

23.62

1.694***

0.071

24.00

    Cut point 4

3.751***

0.049

77.33

3.821***

0.057

67.03

3.606***

0.074

48.52

3.616***

0.074

49.13

Random Effects

Level 2 variance

1.915

 


1.894

  

1.374

 


1.312

 


Deviance Statistics

Deviance

61774.89

  

61750.32

 

61365.07

  

61312.53

  

AIC

61786.99

  

61768.32

 

61407.07

  

61372.53

  

BIC

61835.99

  

61841.96

  

61578.91

  

61618.02

  

χ2

10115.90

  

24.57

  

385.25

  

52.53

  

df

1

  

3

  

12

  

9

  

Sig

***

  

***

  

***

  

***

  

Δ% Var. accounted for by model

28.30

  

0.79

  

19.47

  

2.33

  

Level 2 ICC

0.37

  

0.37

  

0.29

  

0.29

  

Note. Ns = 26,449; Nt = 1,443. *** p < 0.001. ** p < 0.01. * p < 0.05. Null model level 2 variance = 2.671; null model ICC = 0.45; FRLP = free or reduced-priced lunch program.




Appendix Table A6. Physics 2015: Multi-level Ordered Logistic Regression Analysis, Physics Year 1

 

Model 1

Model 2

Model 3

Model 4

 

Variable

OR

SE

z

OR

SE

z

OR

SE

z

OR

SE

z

 

Level 1

            

PSAT score

1.612***

0.021

76.75

4.942***

0.105

74.95

4.853***

0.103

74.13

4.849***

0.103

74.10

Ed level: Some post-secondary


 


1.048

0.052

0.96

1.017

0.050

0.34

1.016

0.050

0.32

Ed level: Bachelor’s


 


1.194***

0.056

3.80

1.139**

0.053

2.77

1.138**

0.053

2.76

Ed level: Graduate degree


 


1.176**

0.059

3.24

1.121*

0.056

2.28

1.122*

0.056

2.29

Level 2


 



 



 



 


Funding < $200


 



 


1.136

0.101

1.44

1.125

0.100

1.33

Funding > $300


 



 


1.038

0.118

0.33

1.033

0.117

0.29

% FRLP


 



 


0.155***

0.030

–9.75

0.160***

0.031

–9.60

Administrative support


 



 


1.020

0.041

0.50

1.022

0.041

0.56

Hours of AP instruction in 10 hours


 



 


1.027**

0.009

2.91

1.028**

0.009

2.95

Enactment: Practice elements


 



 


1.035

0.048

0.75

1.052

0.050

1.08

Enactment: Curriculum elements


 



 


0.860***

0.035

–3.68

0.871**

0.037

–3.28

Int: Practice * Curriculum elements


 



 


1.019

0.038

0.51

1.017

0.038

0.45

Number of labs


 



 


1.018*

0.007

2.43

1.017*

0.007

2.28

Years AP teaching experience


 



 


1.027***

0.007

4.16

1.027***

0.007

4.14

Years AP redesign experience


 



 


   —

  —

    —

   —

  —

     —

Self-efficacy


 



 


1.121**

0.045

2.84

1.128**

0.046

2.98

Importance of PD for AP scores


 



 


0.910*

0.037

–2.35

0.913*

0.037

–2.22

Number of above-the-line PD


 



 



 


0.949

0.053

–0.94

Number of below-the-line PD


 



 



 


1.002

0.019

0.10

PD includes active learning


 



 



 


1.022

0.063

0.35

PD has responsive agenda


 



 



 


1.039

0.077

0.52

PD focuses on student work


 



 



 


0.927

0.056

–1.24

PD models teaching


 



 



 


0.954

0.065

–0.69

PD helps relationship building


 



 



 


1.144

0.085

1.80

PD effectively supports instruction


 



 



 


0.940

0.083

–0.70

Above-the-line PD duration in hours


 



 



 


0.999

0.002

–0.33

    Cut point 1

–1.549***

0.047

–32.96

–1.441***

0.058

–24.91

–1.466***

0.079

–18.46

–1.470***

0.080

–18.44

    Cut point 2

0.745***

0.046

16.23

0.855***

0.057

14.95

0.833***

0.079

10.54

0.829***

0.079

10.45

    Cut point 3

2.498***

0.049

50.66

2.609***

0.060

43.50

2.583***

0.081

31.89

2.579***

0.081

31.73

    Cut point 4

4.677***

0.061

76.12

4.786***

0.070

68.23

4.755***

0.089

53.60

4.750***

0.089

53.39

Random Effects

Level 2 variance

1.325

 


1.302

  

0.982

 


0.968

 


Deviance Statistics

Deviance

42981.52

  

42961.44

 

42764.12

  

42753.58

  

AIC

42993.52

  

42979.44

 

42806.12

  

42813.58

  

BIC

43050.76

  

43050.30

  

42971.46

  

43049.79

  

χ2

7013.52

  

20.08

  

197.32

  

10.54

  

df

1

  

3

  

12

  

9

  

Sig

***

  

***

  

***

  

n.s.

  

Δ% Var. accounted for by model

36.73

  

1.09

  

15.30

  

0.64

  

Level 2 ICC

0.29

  

0.28

  

0.23

  

0.23

  

Note. Ns = 19,413; Nt = 876. *** p < 0.001. ** p < 0.01. * p < 0.05. Null model level 2 variance = 2.094; null model ICC = 0.39; FRLP = free or reduced-priced lunch program.






Cite This Article as: Teachers College Record Volume 122 Number 2, 2020, p. 1-64
https://www.tcrecord.org ID Number: 23024, Date Accessed: 12/4/2021 6:00:54 PM

Purchase Reprint Rights for this article or review
 
Article Tools
Related Articles

Related Discussion
 
Post a Comment | Read All

About the Author
  • Christian Fischer
    University of Tübingen
    E-mail Author
    CHRISTIAN FISCHER is an Assistant Professor in Educational Effectiveness at the Hector Research Institute of Education Sciences and Psychology at the University of Tübingen, Germany. His research examines pathways to improve STEM teaching and learning, in particular through the use of digital technologies. Recent publications include Fischer et al., “Investigating Relationships Between School Context, Teacher Professional Development, Teaching Practices, and Student Achievement in Response to a Nationwide Science Reform,” in Teaching and Teacher Education; and Fischer et al., “Adapting to the Large-Scale Advanced Placement Chemistry Reform: An Examination of Teachers’ Challenges and Instructional Practices,” in the Journal of Chemical Education.
  • Brandon Foster
    Education Development Center, Inc.
    E-mail Author
    BRANDON FOSTER is a Research Associate II at the Education Development Center, Inc. His areas of specialization include school readiness, culturally relevant measures for educational research, career and college readiness, and building researcher/practitioner partnerships centered on student data initiatives. He recently co-published “Culturally Embedded Measurement of Latino Caregivers’ Engagement in Head Start: A Tale of Two Forms of Engagement,” in Early Education and Development.
  • Ayana McCoy
    University of Massachusetts Boston
    E-mail Author
    AYANA MCCOY is an Associate Project Director in the Center of Science and Mathematics in Context (COSMIC) at the University of Massachusetts Boston. Her research interests include exploring the nexus of social justice and science education by addressing the disparity of students of color in the STEM K–20 pathway and the intersectionality of race, class, and gender within a science education context. She recently co-published “When Do Students in Low-SES Schools Perform Better-Than-Expected on a High-Stakes Test? Analyzing School, Teacher, Teaching, and Professional Development Characteristics,” in Urban Education.
  • Frances Lawrenz
    University of Minnesota
    E-mail Author
    FRANCES LAWRENZ is the Associate Vice President for Research for the University of Minnesota and Wallace Professor of Teaching and Learning in the Department of Educational Psychology in the College of Education and Human Development. Her research interest is STEM program evaluation. She recently co-published “Promoting Evaluation Capacity Building in a Complex Adaptive System,” in Evaluation and Program Planning.
  • Chris Dede
    Harvard University
    E-mail Author
    CHRIS DEDE is the Timothy E. Wirth Professor in Learning Technologies at Harvard’s Graduate School of Education. His fields of scholarship include emerging technologies, policy, and leadership. In 2007, he was honored by Harvard University as an outstanding teacher, and in 2011 he was named a Fellow of the American Educational Research Association. From 2014–2015, he was a Visiting Expert at the National Science Foundation Directorate of Education and Human Resources. His edited books include: Scaling Up Success: Lessons Learned from Technology-based Educational Improvement; Digital Teaching Platforms: Customizing Classroom Learning for Each Student; Teacher Learning in the Digital Age: Online Professional Development in STEM Education; Virtual, Augmented, and Mixed Realities in Education; and Education at Scale: Engineering Online Learning and Teaching.
  • Arthur Eisenkraft
    University of Massachusetts Boston
    E-mail Author
    ARTHUR EISENKRAFT is Distinguished Professor of Science Education, Professor of Physics, and Director of the Center of Science and Math in Context (COSMIC) at the University of Massachusetts Boston. In 2017, he was the sole recipient of the National Science Board Public Service Award for exemplary contributions to public understanding of science and engineering. His current scholarly work is focused on curricula innovation, professional development, and sustainable change in school districts. His recent edited books include Beyond the Egg Drop: Infusing Engineering into High School Physics and Teacher Learning in the Digital Age: Online PD in STEM Education. His project-based learning high school text, Active Physics, is now in its third edition.
  • Barry Fishman
    University of Michigan
    E-mail Author
    BARRY FISHMAN is Arthur F. Thurnau Professor of Learning Technologies in the University of Michigan School of Information and School of Education. His research focuses on games as models for learning environments, teacher learning, and the development of usable, scalable, and sustainable learning innovations through design-based implementation research. His recent publications include Fishman et al., “Investigating relationships between school context, teacher professional development, teaching practices, and student achievement in response to a nationwide science reform,” in Teaching and Teacher Education; and Penuel and Fishman, “Large-scale Science Education Intervention Research We Can Use,” in the Journal of Research in Science Teaching.
  • Kim Frumin

    E-mail Author
    KIM FRUMIN is a doctoral candidate at the Harvard Graduate School of Education. She studies research–practice partnerships and online teacher professional learning. Frumin recently co-published “Adapting to Large-Scale Changes in Advanced Placement Biology, Chemistry, and Physics: The Impact of Online Teacher Communities,” in the International Journal of Science Education.
  • Abigail Levy
    Education Development Center, Inc.
    E-mail Author
    ABIGAIL JURIST LEVY is Co-director of Science and Math Programs, Education Development Center, Waltham, Massachusetts. Her research focuses on the impact of science instruction on student outcomes, and policies and practices that advance the effectiveness of the teaching workforce. She recently co-published “Science Specialists or Classroom Teachers: Who Should Teach Elementary Science?” in Science Educator.
 
Member Center
In Print
This Month's Issue

Submit
EMAIL

Twitter

RSS