School Composition and Context Factors That Moderate and Predict 10th-Grade Science Proficiency
by Mark C. Hogrebe & William F. Tate IV - 2010
Background: Performance in high school science is a critical indicator of science literacy and regional competitiveness. Factors that influence science proficiency have been studied using national databases, but these do not answer all questions about variable relationships at the state level. School context factors and opportunities to learn science may vary geographically across states and interact with demographic composition variables.
Purpose: The purpose was to examine relationships between 10th-grade science proficiency and school context factors related to school environment, courses, and teachers. The moderating or interaction effects were examined for the school demographic composition variables of free/reduced lunch and minority percentages on variable relationships with science proficiency scores.
Population and Unit of Analysis: Data for this study consisted of all Missouri high schools in 2002 with a 10th-grade class size of at least 25 students (N = 423). Unit of analysis was the single school.
Research Design: This was a secondary data analysis study that used variables collected annually from all schools in Missouri. Multiple regression was used to examine relationships and moderating effects of school demographic composition. Predictor variables were grouped into three categories for school context: school environment, course related, and teacher related. The outcome variable was 10th-grade scientific attainment as measured by the Missouri state proficiency test in science.
Results: School context variables of higher dropout and mobility rates signaled greater risk factors, especially when moderated by free/reduced-price lunch percentage (FRL pct) and minority status. When FRL pct and Minority pct were higher, lower science proficiency scores were associated with elevated dropout rates. Similarly, greater mobility was related to lower science scores when school FRL pct was high. Some school-level variables interacted positively with FRL pct and minority status, which resulted in higher science scores. Schools with more FRL and minority students achieved higher science proficiency scores when they had a greater percentage of courses taught by highly qualified teachers and more teachers were regularly certified. Higher science scores were associated with greater percentages of master’s degree teachers in schools with a larger percentage of minority students. A surprising finding revealed a geographic influence and demonstrates why testing for interactions can lead to better understanding of the data.
Conclusions: The findings are consistent with the status attainment literature and the theoretical arguments associated with geography and educational attainment in that socioeconomic status and minority status are important predictive factors in Missouri. As an extension of previous research, this study demonstrates that the school composition variables of FRL pct and Minority pct are significantly related to science proficiency in the 10th grade. Not only are they predictive of science proficiency scores, but they also interact with each other and moderate the relationships between school context variables and 10th-grade science scores. This study suggests that teacher quality in high-poverty majority-minority school settings remains an important policy target for reform and improvement.
States across the nation have initiated efforts to highlight the importance of building capacity in science (Battelle Technology Partnership Practice & SSTI, 2006; Building Engineering and Science Talent, 2006). The rationale for capacity building in science is aligned with two long-standing national goals (Kamen & Benavot, 1991). First, states are seeking a competitive advantage in terms of human capital. Second, a high-quality science education is seen as foundational for building a literate citizenry who must be able to make both political and personal decisions on the basis of modern biomedical science, emerging technology, environmental science, and other areas of science impacting the human condition. Currently, science education in the United States must address two interrelated challenges: the limited quantity of science-literate citizens, and the quality of school science learning experiences. These two challenges are captured in the results of international comparisons of science achievement (Gonzales et al., 2004). According to the report, Rising Above the Gathering Storm: Energizing America for a Brighter Future, U.S. students are not performing at levels that generate the desired competitive advantage relative to other countries (Committee on Prospering in the Global Society of the 21st Century, 2007). Although the desired level of competitive advantage is rarely stated in reports of this sort, it is clear that in national assessments of science proficiency, performance grows worse in later grades (Berliner & Biddle, 1997; Grigg, Lauko, & Brockway, 2006). The international and national science trends are also a concern with respect to the goal of building a scientifically literate citizenry (Center for Science, Mathematics, and Engineering Education, 1998).
Numerous reports have been produced that call for reversing science education proficiency trends (Linn, 2007). Many of these reports argued that the nation is in need of improved standards-based K12 science programs and state the need for additional research to evaluate their fidelity and effectiveness (Weiss, Knapp, Hollweg, & Burrill, 2002). The 1994 reauthorization of the Elementary and Secondary Act made standards-based reform a central strategy to improve public education by mandating that states create challenging standards aligned with testing and accountability indicators (Massell, 2008). The No Child Left Behind Act of 2001 (Public Law 107-110), or NCLB, included provisions that required content standards and aligned assessments in mathematics and English/language arts, as well as standards for science in three grades. According to Hamilton (2003), in the 20012002 school year, 49 states and the District of Columbia were implementing statewide testing programs. Moreover, most of these testing programs were in place before the enactment of NCLB, but NCLB requirements mandate many of these efforts and in some cases suggest that expansion will occur.
Statewide assessment and accountability programs provide important repositories of information related to students science proficiency and school context measures, including school environment, course-related variables, and teacher characteristics, as well as school demographic composition. Typically, this information is gathered as part of isolated descriptive reports to fulfill government-related requirements. Yet, Weiss et al. (2002) argued that gaps in current research include the limited number of studies that examine the relationship between science achievement and particular elements of the system under investigation. The purpose of this study is to address a part of this gap in the literature. Specially, this study is an investigation of the relationship between 10th-grade science proficiency, and school composition and context factors in the state of Missouri. In addressing this relationship, the present research takes a different approach than other studies have. Instead of grouping demographic composition factors like socioeconomic status (SES) and race/ethnicity with the school context variables to see which account for unique variance, this study tests for interaction of the demographic characteristics with school context variables. It attempts to answer the question, Do the relationships between school context variables and science proficiency change as a result of differing percentages of minority students or those receiving free/reduced lunches?1
Why examine the relationship of school composition and context variables with science proficiency in the state of Missouri? In a comparison of National Assessment of Educational Progress (NAEP) results that included Missouri versus national average science scales for Grade 8 in 2000, the state of Missouri has a higher average scale than the nation (OSullivan, Lauko, Grigg, Qian, & Zhang, 2003). However, NAEP performance estimates were not adjusted to account for socioeconomic differences or demographic differences among states and jurisdictions. Thus, it is important to view the state versus national performance comparison with a level of caution. Within the state of Missouri, White students outperformed Black and Hispanic students. Consistent with the literature, students who were not eligible for free/reduced-price lunch outperformed students who were. Yet, it was not clear from the analysis what factors were influencing state-level science achievement as measured by NAEP.
Other scholars have examined the relative importance of predictors of science achievement and proficiency (Byrnes & Miller, 2007; Wood, Lawrenz, Huffman &, Schultz, 2006). The methodologies used in these two studies typify studies of science achievement. Byrnes and Millers study used data collected in the National Longitudinal Education Study (NELS:88), a nationally representative sample of over 12,000 adolescents followed for 12 years. The sample design is a unique strength of NELS:88. Wood et al. (2006) collected a nonrandom sample drawn from three different states. The study design provided an opportunity to examine each schools environment and aspects of school culture. However, it would be a challenge to use either data set to produce state-level predictors of science proficiency. Yet, there is a pressing need to better understand the factors influencing science achievement in the context of state-level standards, accountability, and education reform (Beatty, 2008). In addition, as Borman et al. (2004) demonstrated in their study of academic achievement in the state of Florida, it is vitally important to understand the predictors of school-level performance where consideration of the racial distribution of students across schools in a state is a consideration in the research design. Like the state of Florida, Missouri shares a unique history with other Southern states related to race, poverty, and educational opportunity. Morris and Monroe (2009) stated that Missouri shares a history and culture with the South that has had a tangible influence on segregation, inequality, and educational outcomes. They argued further that Missouri, as well as other states in the South, is uniquely important in any effort to understand race, poverty, and schooling in that it has a dual role of being perceived as offering unique economic and social opportunities particularly for African Americans while possessing some of the poorest regions in the country. In fact, the state of Missouri has embarked on an ambitious effort to expand the biotechnology and information technology industries in the region. It would be enormously insightful and impactful to understand what mechanisms are influencing the educational opportunity structures of students indigenous to the state and region (Tate, 2008). Tate demonstrated that there is often a misalignment among the economic goals in a region, opportunity structures, and the educational attainment of traditionally underserved students. As a result, building the capacity to provide an ongoing regional perspective focused on educational attainment is warranted. This study will include all public high schools in the state of Missouri with 10th-grade enrollment of 25 or more students. Per the Morris and Monroe (2009) and Tate (2008) recommendation, the intent of this study is to contribute to the geography of opportunity literature in science education.
In the following section, opportunity-to-learn theory in science education and related empirical findings are described. Next, a description of relevant theory and past research on the role of school composition and context factors in academic achievement is outlined. In the subsequent section, a description of the present analysis, including the variables used in this study to describe and explore the relationship between 10th-grade science proficiency and school context variables, is provided. The succeeding section describes the study findings, including a discussion of which school context variables accounted for unique variance in 10th-grade science proficiency. The discussion of findings includes how some of the school context variables were moderated by school demographic composition variables. The interactions indicated that the relationships of the school context variables to science proficiency scores varied across the moderator variables. The final remarks conclude with a discussion of the implications of the findings for theory building, future research, and public policy.
OPPORTUNITY TO LEARN: SCIENCE EDUCATION
Moses and Cobb (2001) argued that the most urgent social issue affecting poor people and people of color is economic access. Their argument linked economic access and full citizenship to mathematics and science literacy. Moses and Cobb described the opportunity to learn mathematics and science as similar to the 1950s and 1960s struggle to secure political access in the form of voting rights. The reasoning is straightforward: Many of the vestiges of segregation are less prominent or of a different form than in the past. Moreover, although the political process is more open today, economic accessin particular, professional and entrepreneurial opportunities in the information economyis restricted according to individual knowledge, skills, and understandings associated with science and technology (Tate, 2004).
Some scholars have theorized about the relevant school, classroom, and student factors influencing opportunity to learn mathematics and science (see, e.g., Tate, 1995, 2001). Many of the factors described in these theoretical discussions have been tested empirically. For example, Byrnes and Miller (2007) examined the relative importance of three sets of factorsopportunity, propensity, and distalas predictors of math and science achievement. Opportunity factors, or opportunity to learn, was described as the culturally specific contexts in which a student is presented with content to learn or provided opportunities to practice skills. Thus, variables related to exposure (e.g., coursework, homework, and instructional emphasis) or teaching quality (teacher background, classroom practice, and so on) would be classified as opportunity factors. In contrast, propensity factors are related to the ability or willingness to learn once a student has been presented with content. Examples of propensity factors include cognitive attributes (e.g., intelligence, aptitude, and preexisting skills) and motivational attributes (e.g., self-efficacy, values, and competence perceptions). The Byrnes and Miller model assumes that when the opportunity and propensity conditions are fulfilled in a student, higher achievement will be the product. Opportunity and propensity factors are viewed as proximal causes of achievement. According to Byrnes and Miller, another set of factors are properly classified as factors that make possible or account for the emergence of opportunities to learn or propensities. They argued that factors such as SES, parental educational expectations for their children, students expectations, and prior educational experiences explain the emergence of opportunities and propensities. Referred to as distal factors, they operate in a time prior to the manifestation of opportunities and propensities. Byrnes and Miller hypothesized that achievement differences among demographic subgroups largely reflect the fact that gender and race/ethnicity are confounded with distal, opportunity, and propensity factors. It follows from this hypothesis that gender and race/ethnicity will not explain a substantial amount of variance in achievement outcomes if distal, opportunity, and propensity factors are controlled. Testing this model, Byrnes and Miller found that the total amounts of explained variance were 80.6% for 10th-grade math and 76.6% for 12th-grade math. The corresponding calculations for 10th- and 12th-grade science were about 20% points lower60.8% and 57.7%, respectively. According to Byrnes and Miller, the difference in total explained variance is largely due to additional explained variance for math than for science with respect to distal factors (on average, 42% vs. 29%) and opportunity factors (11% vs. 1%). In this model, demographic factors accounted for less explained variance for math than for science (less than 1% vs. 2%3%). A key conclusion of the Byrnes and Miller study is that the combination of opportunity and propensity factors is an important predictor of science and mathematics achievement. However, the 10th-grade science achievement model, although possessing some predictive value, did not fully account for the total variance.
There are at least two possible reasons that the variance is not fully accounted for in the Byrnes and Miller (2007) study. One reason the variance may not be fully accounted for involves the challenge of measurement. Byrnes and Miller acknowledged that other ways to improve on their efforts would be to unpack or refine the measurement of certain factors. They suggested that it would be useful to unpack the variance accounted for by race/ethnicity and to gain a better understanding of the mechanisms that link SES and science achievement. A second reason the variance in this study may not be fully accounted for is that the modeling strategy ignores the structural factors that are at least hypothetically associated with science achievement, such as school composition. Secada (1991) warned that the psychological tradition stressed individual and mental constructs, as opposed to the objective and social context. Although clearly important, the distal, propensity, and opportunity model does not examine the relationship between science achievement and school composition. In fact, Byrnes and Miller (2007) called for additional science achievement studies that examine school composition factors such as teacher quality. Konstantopoulos (2006) examined the school effects on student achievement in multiple national surveys. He found that substantial variation in student achievement lies within schools, not between schools. Yet, there is also considerable between-school variation in achievement, which becomes larger over time. There remains a clear need to better understand the relationships among school context factors, school composition factors, and science proficiency.
BACKGROUND ON SCHOOL VARIABLES
There is a long and extensive literature that examines the effects of school and classroom composition on educational outcomes (e.g., Finn & Voelkl, 1993; Konstantopoulos, 2006; Peterson & Mayer, 1999; St. John, 1970). Educational outcomes that are typically measured include student engagement, graduate rates, and academic achievement. There is a substantial body of research that has investigated the influence of socioeconomic composition of schools and classrooms on academic achievement. Studies have demonstrated that the SES of students (and therefore their schools and classrooms) is an important indicator associated with student achievement. SES is usually a reference to some combination of family income, education, and employment. Two meta-analytic reviews of SES and achievement strongly suggested that the impact of SES on school achievement is much higher when the focus is on organizational units, such as schools, rather than individuals (Sirin, 2005; White, 1982). These reviews are part of a literature that is sometimes referred to as status attainment research (Tajalli & Opheim, 2004). A consistent finding in this literature is that schools with a higher percentage of low-SES students tend to demonstrate relatively lower academic achievement on average. A typical measure of low SES at the school level is the percentage of students eligible for free and reduced-price lunch. Conversely, schools with a higher percentage of middle- and high-SES students tend to perform better, on average, on measures of achievement than schools with less affluent student bodies. The relationship between achievement as measured by test scores and SES is widely replicated in the social science literature (Sirin, 2005). Miller (1995) contended that the differences in academic achievement patterns between high- and low-SES children are due in large measure to the fact that differences in education-relevant family resources between these groups are too large for schools to overcome under existing conditions.
The relationship between SES and achievement should be interpreted with population trends in mind. Historically, poverty has been more severely concentrated among African American and Hispanic students than among Whites (Procter & Dalacker, 2003; United States General Accounting Office [U.S. GAO], 1993). Miller (1995) found that a major reason that African Americans, Hispanics, and Native Americans tend to do less well academically, as measured by achievement tests, is that these groups are overrepresented among low-SES families and underrepresented among high-SES families. However, Millers analyses demonstrated that variations among social classes do not fully account for racial/ethnic differences in academic achievement patterns among groups. There are also large within-social-class differences in academic achievement patterns among groups. According to Miller, one possible explanation for these differences is that standard methods of measuring social class tend to overestimate the education-relevant resources available to many non-Asian minority students and their families. The focus on variables that approximate income, such as parents level of education, do not fully capture the role that wealth plays in a familys class standing, social status, home ownership, and residential community, and school quality and composition (Shapiro, 2004). Despite this conceptual challenge, the interaction of minority status and SES is an important relationship in status attainment research.
School environmental factors also can influence academic achievement. Several factors are potentially related to school composition and science achievement. High student mobility rates within a school over the academic term are viewed as a potential impediment to student learning, and ultimately achievement. According to the U.S. GAO (1994), school officials have reported that high numbers of mobile students can negatively influence teachers ability to organize and deliver instruction. Other instructional challenges potentially associated with high student mobility include assessing the needs of new students, determining their past education experiences, and providing instruction that builds on student experience. Highly mobile students are more likely to be low achievers, repeat a grade, exhibit frequently occurring behavioral and emotional problems, or drop out (Jelleyman & Spencer, 2008; U. S. GAO, 1994; Wood, Halfon, Scarlata, Newacheck, & Nessim, 1993). According to South, Haynie, and Bose (2007), adolescent residential and school mobility is linked to an increased risk of dropping out. Their study found an increased risk of dropping out among mobile and nonmobile students attending schools with high rates of mobility. This finding was partially attributable to lower levels of school attachment and weaker academic achievement in high-mobility schools.
High dropout rates may signal a level of dysfunction in the school environment (Balfanz, 2000). More specifically, a dropout problem in a school setting may be an indirect measure of a weak collective response to a difficult challenge. Relatively high dropout rates in a school are a direct threat to the advancement of learning and achievement. If a student is not in school, he or she cannot participate in science coursework. In addition, if students are not in school, they cannot participate in interventions aligned with achieving learning goals. The nations overall graduation rate is about 70% (Graduation Profiles, 2007). The method used to calculate these rates has been subject to debate; the trend, nevertheless, is a serious threat to science achievement and attainment across the nation.
Another school environment indicator that may influence achievement is the discipline incident rate. The rationale for school discipline is to maintain an environment that supports learning and academic achievement. However, Raffaele Mendez (2003) found that increased use of school suspension and expulsion is associated with increased risk of student dropout and lower achievement. Skiba and Rausch (2004) examined the relationships among achievement, discipline, and race. Findings from this study suggested that a combination of factors, including poverty, race, and school rate of out-of-school-suspensions, could predict school passing rates on the math and English/language arts section of the Indiana State Test of Educational Progress (ISTEP). In addition, regardless of demographic factors, schools with higher rates of out-of-school suspension had lower percentages of students who passed ISTEP.
Teacher/pupil ratio is a variable that gives an indicator of the average size of a class in a school setting. It has been argued that class size is an important school environment indicator because it captures the potential availability of teachers to interact with students (Okpala, Smith, & Jones, 2000). However, class size research has produced inconsistent results where the focus has been on the relationship between class size and academic achievement. Konstantopoulos (2008) examined the effects of small classes on mathematics and reading achievement using data from a 4-year large-scale randomized experiment. Findings from meta-analysis and quantile regression methods consistently demonstrated that higher achieving students benefited more from being in small classes in early grades than other students. The results also suggested that, although all types of students benefited from being in small classes, reductions in class size did not reduce the achievement gap between low and high achievers. Finn and Achilles (1990) also found that a significant benefit accrued to students in reduced-sized classes in mathematics and reading (see also Nyhan & Alkadry, 1999). However, Hanusheks (1997) meta-analytic study demonstrated that when family inputs are taken into consideration, student achievement was unaffected by class size. Irrespective of the conflicting findings, researchers continue to explore the relationship between class size and achievement. Moreover, policy makers continue to debate the merits of class size reduction as a policy option to support the improvement of academic performance.
The research on the effect of school enrollment on student achievement has yielded inconclusive results. Fowler and Walberg (1991) examined the relationship between student achievement and school size using data from 293 public secondary schools. They concluded that enrollment was the next most consistent, influential, and negatively related variable to school outcomes (after district SES and percentage of students from low-income families). The findings suggested that smaller school districts and schools are more effective at advancing students educational outcomes than larger schools. In contrast, Hanushek (1997) argued that school size has no influence on student achievement.
Education attainment discussions include a renewed focus on science, including an emphasis on mandatory coursework and more rigorous standards (Building Engineering and Science Talent, 2006). One response by researchers has been greater attention to the quality, quantity, and distribution of college-preparatory and Advanced Placement (AP) science courses (Gollub, Bertenthal, Labov, & Curtis, 2002). Two of the most important predictors of school science achievement in large-scale assessments of science have been (a) increased time on task in high-level science and (b) the number of college-preparatory science courses taken (OSullivan et al., 2003; Perkins, Kleiner, Roey, and Brown, 2004). ACT research suggests that science reasoning, as measured by the ACT Assessment, is related to high school coursework (Harmston & Pliska, 2001). In this ACT study, students with a college preparatory science background outperformed those who lacked the core courses, on average. This finding is consistent with the literature in science achievement.
State science standards for students also directly apply to what their teachers must know, understand, and be able to teach. Research has indicated that how much teachers know about their subject has a positive influence on student achievement (Darling-Hammond, Berry, & Thoreson, 2001; Ferguson & Brown, 2000). Some states have defined highly qualified teacher as an individual who has the appropriate certification for an assigned teaching assignment. Darling-Hammonds (2000) study using data from the NAEP found that various measures of teacher education, such as teacher preparation in education and certification, have strong positive relationships with academic achievement. She argued that state policy regarding teacher education could potentially influence the quality of teachers and their ability to support learning. Other research has described a statistically significant positive link between teachers degree and academic achievement (Goldhaber & Brewer, 1996). Wright, Horn, and Saunders (1997), building on the Tennessee Value-Added Assessment System, examined the relative magnitude of teacher effects on student achievement while analyzing the influences of intraclassroom heterogeneity, student achievement level, and class size on academic growth. The findings indicated that teacher effects are dominant factors affecting student academic growth, whereas the classroom context variables of heterogeneity among students and class size demonstrated minimal influence on academic growth. Heck (2007) examined the relationship between teacher quality as an organizational property of schools, and student achievement as part of a longitudinal cohort study consisting of more than 14,000 students nested in a random sample of 197 elementary schools. Teacher quality was measured using the percentage of teachers at each school who were fully certified, who passed content knowledge examinations, and who met state performance standards. He discovered that as a school resource, collective teacher quality was positively related to school achievement levels in reading and mathematics. In addition, within schools, higher teacher quality was associated with reduced gaps in student learning rates by social class and race/ethnicity.
Two research syntheses suggest that the literature on teacher characteristics and academic achievement does not produce consistent results. Both Hanushek (1997) and Wayne and Youngs (2003) concluded that there is no systematic relationship between teacher characteristics and student achievement. Wayne and Youngs review focused on existing studies that examined the relationship between student achievement and four categories of teacher characteristics: college ratings, test scores, degrees coursework, and certification status. They found inconclusive evidence regarding the relationship between teacher characteristics and student achievement. Although some of the studies in their review found relationships between student achievement and teachers college rating, test scores, degrees, course work, and certification status, others showed no such relationships across the different grade levels and across different subjects.
Research examining the relationship between teachers salaries and student achievement suggests that a positive relationship exists. Sanders (1993) examined the relationship between teachers salaries and several measures of education attainment, such as ACT scores, high school graduation rates, and the percentage in high school planning to attend college. The study found that teacher salaries were correlated with all three outcome measures. Findings from a study conducted by Smith (2004) also suggest that there is an association between teachers salaries and increases in students test scores in mathematics.
Because many of the findings from research undertaken to date have shown mixed results relating the effects of school environment variables, course-related variables, and teacher variables on achievement, it is critical to examine the extent to which these factors are likely to influence achievement. The literature very rarely has examined effects of these school and classroom factors on science achievement.
The purpose of this study was to determine which school composition and context factors were related to the average science MAP (SciMAP) index scores for all 423 regular Missouri public high schools with 10th-grade enrollment of 25 or more students. The school context factors under study were conceptualized as school environment, course-related, and teacher-related variables. To better understand these relationships, the influences of two demographic composition moderator variables were explored: each schools free/reduced-price lunch percentage and minority percentage. Including the demographic composition variables in the model as predictors, along with the school, course, and teacher variables, disguises the predictive effects of these other variables. Instead, the demographic composition variables need to be examined as to how they moderate or influence the relationship of school, course, and teacher variables with science scores. This interaction question adds significant understanding to the nature of these relationships.
Research Question 1: Which of the school, course, and teacher variables are related to the average SciMAP index scores while controlling for the other variables?
Research Question 2: Of the school, course, and teacher variables that account for unique variance in average science MAP index scores, which ones interact with (are moderated by) the schools percentage of students receiving free/reduced-price lunches and percentage of minority students?
Research Question 3: What is the nature of the interaction, and how do the demographic moderator variables influence the relationships between school, course, and teacher variables, and science MAP index scores?
DATA AND UNIT OF ANALYSIS
The data set for this study consisted of 423 Missouri public high schools in 2002 with a 10th-grade class size of at least 25 students. The focus was on high school because it is proficiency at this level that is critical for success in future high school and college science courses. In Missouri, the state high school proficiency test in science was administered only at the 10th-grade level. Data from 2002 were used because it was the last year that the administration of the science proficiency test was mandatory across the state (until reinstatement in 2008). Only high schools with at least 25 10th-grade students total were included to limit the very small high schools that may operate differently than those with larger enrollments and class sizes. In addition, not included were schools that were entirely special education or alternative schools. Thus, the data consisted of the population of 423 regular Missouri public high schools with a 10th-grade enrollment of 25 or more students.
The unit of analysis was the single school. Data points were the aggregate variable characteristic for the school, such as the schools average science proficiency index or percentage of teachers with masters degrees. There were four major predictor variable groupings, and each comprised variables at the school level. The analysis examined the relationship between school-level characteristics and the school-level science outcome.
The predictor variables were grouped into four categories: school environment, course related, teacher related, and student demographics. The outcome variable was 10th-grade scientific attainment as measured by the Missouri state proficiency test in science. This 10th-grade index of science proficiency was the only standardized measure of science across all high schools in the state. These variables were collected from all schools by the Missouri Department of Elementary and Secondary Education (DESE; Missouri DESE, 2008a).
The DESE variable definitions are as follows:
School environment variables
Mobility rate for the academic year. For Grades 912, the number of students who transfer in plus the number who transfer out, divided by the total enrollment.
High school dropout rate. For Grades 912, the number of dropouts divided by the total of September enrollment, plus transfers in, minus transfers out, minus dropouts, added to September enrollment, then divided by 2. Disaggregated data were provided only for those groups that had 30 or more students enrolled in Grades 912.
Number of Dropouts ÷ ((Total September Enrollment + (Transfers In Transfers Out Dropouts)) ÷ 2)
Discipline incident rate. Number of incidences reported divided by total enrollment in Grades 912. An incident was reported when a student was removed from the traditional classroom setting for 10 or more consecutive days. Multiple short sessions (cumulative removals adding up to 10 days) were not included.
Students-to-classroom teacher ratio. The ratio of students in Grades 912 to regular classroom teachers, excluding special education, remedial reading, Title I, and vocational teachers.
Student enrollment in Grades 912. The total student enrollment is a measure of the size of the school in which the 10th grades used in this study reside.
Highly qualified courses percentage. Percentage of courses taught by highly qualified teachers. A highly qualified teacher was an individual who had the appropriate certification for his or her teaching assignment. A highly qualified teacher was one who had at least a bachelors degree; who had demonstrated content expertise by passing a state-approved test or completing an academic major or coursework equivalent to a major; and who held full certification for his or her current teaching assignment. This was a school composition variable, so the percentage of courses taught by highly qualified teachers included all subjects, not just science.
Credits earned in advanced courses percentage. Enrollment in each designated advanced course was multiplied by the unit of credit for the course. These advanced enrollment credits were totaled. The number of class periods in a school day (credits possible in a school day) was multiplied by the total number of juniors and seniors (11th- and 12th-grade students). The total of all advanced enrollment credits was divided by the total number of all enrollment credits available to juniors and seniors. Credits earned in advanced course percentage = advanced enrollment credits / total possible enrollment credits for juniors and seniors. As with highly qualified courses, this was a school composition variable, so the percentage of credits earned in advanced courses included all subjects, not just science.
High school teacher average salary. The average regular-term salary of high school teachers in the school. Fringe benefits were not included.
Teacher average years experience. Average years of teaching experience for teachers in a school.
Teacher masters degree percentage. Percentage of teachers in a school with a masters degree in any field.
Teacher regular certificate percentage. Percentage of teachers in a school with regular teaching certificates, as opposed to no certificate. As a school composition variable, it included all teaching certificates, not just science certificates.
Student demographic variables
Free/reduced-price lunch percentage. Percentage of the high school enrollment receiving free/reduced-price lunch (FRL pct).
Minority student percentage. Percentage of the high school enrollment consisting of the total number of students in the following minority groups (Minority pct): African American (14.7%, n = 38,803); Hispanic (1.6%, n = 4,091); Asian (1.2%, n = 3,300); and American Indian (.3%, n = 769). The sum of these percentages for minority students across all schools in the present study was 17.8%. The racial demographic percentage breakdown of the minority students within this sum of the percentages was 82.7% African American students, 8.7% Hispanic, 7.0% Asian, and 1.6% American Indian. These four categories were combined to create the Minority pct variable that allowed for testing continuous interactions with the other predictors. A limitation of the data set is that we could not differentiate among minority groups within the Minority pct variable. However, Missouri demographics are not like those of many Sunbelt states. The major minority group is African American. The state has not had the influx of other minority groups as experienced by Texas, Florida, Georgia, and other Southern states. Thus, the Minority pct category represents a reasonable reflection of the state demographic. The construct largely consists of African Americans. Because our study is focused on school composition, it is important to recognize that in the state of Missouri, very few high schools are majority-minority in which Hispanics, Asians, or American Indians are the predominant racial group. This is a very important fact for interpretative purposes.
Science proficiency outcome variable
The outcome of 10th-grade science proficiency was a measure from the Missouri Assessment Program (MAP; Missouri DESE, 2008b). The MAP testing program was used to assess annual yearly progress in Grades 3, 7, and 10 in 2002 and to comply with the assessment directives of the state legislature and No Child Left Behind. The MAP science test measured students progress relative to the Missouri Show-Me standards (Missouri DESE, 2008b). The science test assessed eight content areas, or strands: matter and energy, force and motion, living organisms, ecology, earth processes, universe, scientific inquiry, and science and technology.
Test items included three types: multiple-choice items from the TerraNova, a nationally normed test; constructed response items; and performance event items, which involve longer and more demanding tasks that require students to work through problems or experiments. The MAP tests were scored by CTB/ McGraw-Hill and reported as MAP Scale Scores based on students correct responses and points earned. The Scale Scores were used to indicate five achievement levels: Step 1, Progressing, Nearing Proficient, Proficient, and Advanced.
The score for each school used in this analysis was the science MAP (SciMAP) index calculated by DESE using the five achievement levels:
MAP Index =
(pct STEP 1*1) + (pct Progressing*1.5) + (pct Nearing Proficient*2) + (pct Proficient*2.5) + (pct Advanced*3)
Rationale for variables
All the predictor variables were based on data from Grades 912 and were not available from DESE as disaggregated by grade level. A potential limitation of the data is that the predictor variables were based on Grades 912, whereas the criterion variable was Grade 10 only. For a number of reasons, we do not believe this is a limitation that misrepresents the variable relationships. First, Grade 10 data were a part of the Grades 912 data and would be highly correlated with the total school data. Second, variables such as mobility, dropout, and discipline incident rates for Grades 912 reflect the climate and context of the school that 10th graders attend. We wanted to know how the school context, as measured by these variables, was related to science proficiency (as measured by SciMAP in Grade 10). Third, the teacher variables, like percentage of teachers with masters degrees and regular teaching certificates, were also for Grades 912. Most teachers are responsible for courses at more than one grade level, so the 10th-grade students were likely being taught by some of the same individuals teaching Grades 9, 11, and 12. The 912 teacher variables represented the teacher resources in the school that were not confined to one grade level. With the school as the unit of analysis, the teacher and course variables portray the context of the school that produced the mean SciMAP scores. Finally, because these were school-level data, the assumption was that the mean 10th-grade SciMAP scores for schools would be highly correlated with the mean SciMAP scores of 11th- and 12th-grade students at the same school (if they were available). Schools with higher mean SciMAP scores in 10th grade would most likely be the schools with the higher mean scores in Grades 11 and 12. Therefore, we assume that the mean SciMAP scores in science for 10th grade represent the science proficiency of the school, relative to all other schools.
Multiple regression was used to determine which of the school, course, and teacher variables were related to the SciMAP index across the 423 individual high schools.
To get a clear picture of which school, course, and teacher variables were significantly related to the SciMAP index, these 11 variables were entered into a regression equation without the student demographic variables of FRL pct and Minority pct. Subsequently, only the variables that accounted for significant SciMAP index variance were retained in a final regression model.
To better understand the influences of FRL pct and Minority pct on the variables from the final model, the interaction of each predictor variable with each demographic variable was examined separately in predicting SciMAP index. FRL pct and Minority pct were treated as moderator variables (Ghiselli, Campbell, & Zedeck, 1981; Warner, 2008). This approach was incorporated to gain a better understanding of how the demographic variables interact with important school, course, and teacher variables and moderate their relationship with SciMAP index.
The means and standard deviations for all variables are reported in Table 1. Pearson correlation coefficients are reported in Table 2.
Table 1. Means and Standard Deviations (n = 423)
Table 2. Correlation Matrix for All Variables (n = 423)
Note. HS = high school. *p < .05 (two-tailed). **p < .01 (two-tailed).
RESEARCH QUESTION 1
Which of the school environment, course-related, and teacher-related variables are related to the average SciMAP index scores while controlling for the other variables? This question was addressed by including all these variables in the full regression model reported in Table 3. The R2 for this full model was .344 and reported in Table 5 along with the R2s for all the other models to be discussed later. In the full model that accounted for 34.4% of the variance in school average SciMAP index scores, 7 of the 11 variables accounted for unique variance: mobility rate, dropout rate, discipline incident rate, percentage of courses taught by highly qualified teachers, teacher average salary, percentage of teachers with masters degrees, and percentage of teachers with regular certificates. A higher significance level of p < .10 was used as the criterion for keeping variables for further analysis in order to reduce the probability of making a Type II error in specifying the final model.
Table 3. Full Regression Model With All Variables (R2 = .344) (n = 423)
Note. Dependent variable was average 10th-grade Science MAP Index for each high school. Variables in bold were included in the reduced regression model (Table 4). Significance level set higher (p < .10) for initial variable screening to reduce the probability of a Type II error in model specification. HS = high school.
Next, the seven variables from the full model that accounted for unique variance in SciMAP index scores were used to make another reduced or final regression model. This final model accounted for 34.3% of average SciMAP index scores and is reported in Table 4. Six of the seven variables in the final model accounted for significant unique variance. Teacher average salary was the only variable that did not account for unique variance at the p < .05 significance level.
Table 4. Reduced Regression Model With Only Significant Variables (p < .10) From Full Model (R2 = .343) (n = 423)
Note. Dependent variable was average 10th-grade Science MAP Index for each high school. With significance level set at p < .05, teacher average HS salary was not statistically significant. HS = high school.
The regression coefficients show the relationships of each variable to SciMAP index scores with the effects of the other variables held constant. For example, the coefficient for dropout rate was -1.345, which indicates that for every percentage point increase in the dropout rate, the average SciMAP index score decreases by 1.345 points. The discipline incident rate has a similar negative effect (-.775) on SciMAP scores. The mobility rate also has a negative relationship with SciMAP scores when controlling for the other variables. For every 1-point increase in the mobility rate, average SciMAP scores decrease by -.204 points. Increases in teachers with masters degrees, courses taught by highly qualified teachers, and teachers with regular certificates were all associated with increases in average SciMAP scores.
It is important to note that the science MAP index was a continuous dependent variable that had an approximate normal distribution. The standardized residuals for the final model predicting SciMAP were normally distributed. Because some of the predictors were proportion variables and positively skewed, an arcsine transformation was applied to them in the final model (Cohen, Cohen, West, & Aiken, 2003). This secondary analysis produced very similar results to using the data in raw form; therefore, the data reported here were left in raw score form to make interpretation of findings more transparent.
RESEARCH QUESTION 2
Of the school environment, course-related, and teacher-related variables that account for unique variance in average SciMAP index scores, which ones interact with (are moderated by) the schools percentage of students receiving free/reduced-price lunches and percentage of minority students?
Table 5 reports the results of tests for interactions of percentage receiving free/reduced-price lunches (FRL pct) by each of the six variables in the final regression model (Table 4) that accounted for significant unique variance in SciMAP index scores. Each interaction model has three predictors: two variables and their interaction. For the interaction models, each variable was centered in respect to its mean (Cohen et. al., 2003). Cohen and associates recommend centering continuous predictors and their interactions to clarify the interpretation of regression coefficients and eliminate nonessential multicollinearity.
Table 5. Tests for Interactions With Free/Reduced Lunch Percentage (n = 423)
Note. Each variable was centered with respect to its mean. Dependent variable was average 10th-grade Science MAP Index for each high school. HS = high school.
Out of the six models tested for FRL pct interaction, five contained significant interactions: FRL pct by mobility rate, by dropout rate, by courses with highly qualified teachers, by masters degrees, and by teachers with regular certificates. The only interaction that was not significant was FRL pct by discipline incident rate. All five models with significant interactions accounted for at least 25% of SciMAP variance. Even though the FRL pct by discipline incident rate model did not contain a significant interaction, the model still accounted for 24% of SciMAP score variance, with discipline incident rate accounting for unique variance over and above FRL pct. In the five models with significant interactions, the relationship between FRL pct and SciMAP scores was negative while controlling for the other variable and interaction. For example, in the FRL pct by dropout rate model, when dropout rate is at the mean, then every increase of 1 point in the FRL pct predicts a -.348 decrease in average SciMAP scores.
The final interaction examined for free/reduced lunch was FRL pct by Minority pct. It was important to see if the moderator variables, when used in combination, interacted to produce different levels of SciMAP score performance. The model containing these variables and their interaction accounted for 44.8% of the variance in average SciMAP index scores. Both variables and their interaction accounted for significant unique variance (see Table 5).
The next set of interactions for minority percentage by the six variables in the final regression model (Table 4) is reported in Table 6. Out of the six models tested for Minority pct interaction, four contained significant interactions: Minority pct by dropout rate, by courses taught by highly qualified teachers, by masters degrees, and by teachers with regular certificates. All four models with significant interactions accounted for at least 39% of SciMAP variance. In the four models with significant interactions, the relationship between Minority pct and SciMAP scores was negative while controlling for the other variable and interaction.
Table 6. Tests for Interactions With Minority Percentage (n = 423)
Note. Each variable was centered with respect to its mean. Dependent variable was average 10th-grade Science MAP Index for each high school.
RESEARCH QUESTION 3
What is the nature of the interaction, and how do the demographic moderator variables influence the relationships of school, course, and teacher variables with SciMAP index scores?
After determining which interactions with the demographic moderator variables were significant, the interactions were plotted to better understand the nature of the relationships. The significant interactions for FRL pct are graphed in Figure 1 and for Minority pct in Figure 2.
Using FRL pct as the moderator variable, the interaction model was graphed at FRL percentages of 15% (-1 SD) and 50% (+1 SD) represented by the two lines in each graph. The lines were graphed by setting the values of the other variable at +1 SD and -1 SD. The graphs convey a variety of useful information for interpreting the interactions. For example, in Figure 1(a), the top line represents schools with a low FRL rate of 15%, whereas the bottom line is for schools with a high FRL rate of 50% (+1 SD). The significant interaction can be seen in the two unparallel lines. The top line shows that schools with low FRL pct tended to have higher SciMAP index scores and that the score difference between low FRL pct, low-mobility schools and low FRL pct, high-mobility schools was only 2 SciMAP index points. In contrast, the bottom line shows that schools with higher FRL pct tended to have similar SciMAP index scores in low mobility schools, but lower SciMAP index scores when mobility was high. For high-mobility schools, the percentage of students receiving FRL was associated with an 8-point difference in school average SciMAP performance. Other interaction patterns can be observed in Figures 1(b), 1(c), and 1(e). Schools with a higher percentage of FRL students had lower average SciMAP scores when the dropout rate was higher (1b), when the percentage of courses taught by highly qualified teachers was lower (1c), and when the percentage of teachers with regular certificates was lower (1e).
An unexpected result occurred in Figure 1(d) for the interaction of FRL pct by masters degrees. In graph 1(d), schools with high FRL (50%) had lower SciMAP scores when teachers with masters degrees was 23%, but other high FRL schools had even lower SciMAP scores when their percentage of masters degree teachers was higher, at 55%. It may seem counterintuitive that schools with more masters degrees had lower average SciMAP index scores. This result will be examined later in the article.
Figure 1. Interactions with free/reduced-price lunch percentage (n = 423)
The four significant interaction patterns for the minority moderator variable can be observed in Figure 2 (a), (b), (c), and (d). For schools with almost no minority students, SciMAP index scores were higher compared with schools with a greater percentage of minority students (33%). The difference in SciMAP scores was even lower for schools with 33% minority students when the dropout rate was higher (2a), when percentage of courses taught by highly qualified teachers was lower (2b), when the percentage of teachers with masters degrees was lower (2c), and when the percentage of teachers with regular certificates was lower (2d).
None of the significant interactions with the minority moderator variable resulted in a counterintuitive result, as occurred with the FRL moderator. In particular, Figure 2(c) demonstrates that schools with a higher percentage of teachers with masters degrees scored higher on the SciMAP index, and schools with fewer masters degree teachers scored lower. This result occurred for both low- and high-minority schools, although high-minority schools had a greater decrease in SciMAP scores when there were fewer teachers with masters degrees (23%). So what happened with the FRL by masters degree interaction where higher masters degree percentages were associated with lower scores?
Figure 2. Interactions with minority percentage (n = 423)
Explaining the surprising interaction
To better understand why a higher masters degrees percentage was related to lower SciMAP scores when interacting with FRL pct, the 423 schools were divided into two groups using a mean split based on the FRL pct mean of 32. Correlations were computed for masters degrees with SciMAP scores in the groups above and below the FRL pct mean. The correlation for masters degrees with SciMAP scores was negative (r = -.232) when the FRL pct was greater than 32, but positive (r = .228) for the group with less than 32% FRL. The significant negative correlation indicates that the relationship for masters degrees with SciMAP scores differs across schools with higher FRL pct. Among schools with a high FRL pct, those with a higher masters degree percentage tended to have lower average SciMAP scores. This pattern did not exist for schools with FRL pct below the mean, nor did it emerge for the correlations based on Minority pct.
The next step was to examine the group of schools with high FRL pct and determine where they were located. This was done by using the locale code assigned by the National Center for Education Statistics (NCES; 2002) to each high school. There were eight locale codes used by NCES in 2002: large city, mid-size city, large town, small town, urban fringe large city, urban fringe mid-size city, rural inside a metropolitan statistical area (MSA), and rural outside of a MSA. Further investigation of the schools by locale revealed that there were only two locales that had schools with FRL pct above the mean of 32, and these were large city (mean FRL pct = 48.59%) and rural outside MSA (mean FRL pct = 40.36%). Forty-six percent of the high schools (n = 196) were located within these two locales.
Figure 3 clearly shows the source of the negative relationship between masters degrees pct and SciMAP scores. Schools in large cities had the third highest average masters degrees percentage but the lowest average SciMAP score by 29 points. In contrast, schools in rural areas outside MSAs had the lowest percentage of masters degrees but an average SciMAP score only 7 points below schools in the locale with the highest average SciMAP score. This additional analysis demonstrates that there was a real difference in variable relationships for schools with high percentages (greater than 40%) of students receiving FRL and that the differences were manifested in the unexpected lower SciMAP scores for schools with a higher percentage of teachers with masters degrees. The bottom line in the FRL interaction graph Figure 1(d) shows this negative relationship.
Figure 3. NCES locales plotted by science MAP index scores and average masters degree percentage (n = 423)
The first research question attempted to discover which prominent school environment, course-related, and teacher-related variables could account for unique variance in 10th-grade science proficiency as measured by the average school science MAP index scores. The full regression model accounted for 34% of school-level SciMAP score variance, and 7 of the 11 variables accounted for unique variance representing each of the three major categories. For school environment, the significant variables were mobility rate, dropout rate, and discipline incident rate. For the course-related category, the significant variable was percentage of courses taught by highly qualified teachers. And finally, the teacher-related category variables that accounted for significant unique variance were teacher average salary, percentage of teachers with masters degrees, and percentage of teachers with regular certificates. When these variables were included in the final regression model, it accounted for a statistically and practically significant 34% of the variance in SciMAP index scores. In accounting for 34% of the SciMAP variance, these school environment, course-related, and teacher-related variables tell us something about their relationship to the average 10th-grade science proficiency at the school-level, as discussed next.
SCHOOL ENVIRONMENT VARIABLES
The mobility rate had a negative relationship with SciMAP scores when accounting for the other variables. Schools with higher mobility rates tended to have lower SciMAP scores. A high percentage of students moving into and out of a school each year are detrimental to learning, especially in science education. Elevated thinking processes come into play when students are required to focus on science concepts, related investigations, and connections among these concepts and learning experiences. High mobility disrupts a students thinking processes and negatively influences sustained opportunity to learn. Changing schools is likely disruptive to a students ability to grasp the coherence and progression of science concepts. Because the science curriculum and sequence varies across districts and from school to school, mobile students are unlikely to continue science instruction exactly at the point where they left off in prior school experiences. In addition, some students have social problems related to adjusting to a new environment. The adjustment period may distract a students attention and concentration from a focus on science learning. In addition, a mobile and changing student body may present challenges to teachers and administrators that divert energy and resources from academic focus.
Dropout and discipline rates
The dropout and discipline rates were each negatively related to SciMAP scores when accounting for the other variables. Schools with higher dropout and discipline rates tended to have lower SciMAP scores. The negative relationship with science proficiency indicated that schools with higher dropout and discipline rates were dealing with problems that were associated with, and possibly contributing to, poorer school-level academic performance. Both rates give an indication of the amount of disruption that occurred at a school. Even though the range of the discipline rate was relatively low, an actual incident was indicative of a rather serious violation in that it resulted in removal from the traditional classroom for 10 or more consecutive days. Multiple short suspensions of fewer than 10 days were not included in the discipline rate, so the rate is probably an underestimate of disruption.
The one course-related variable that accounted for statistically significant unique variance in predicting SciMAP was the percentage of courses taught by highly qualified teachers who had appropriate certification for their teaching assignment. More courses taught by highly qualified teachers were related to higher SciMAP scores while controlling for the other variables. Because this was a summary variable for all courses, it did not delineate the percentage of science courses taught by teachers certified in science content areas. Therefore, we must assume that schools with most of their courses taught by highly qualified teachers would have science courses taught by teachers certified in science, and likewise for other subjects. The finding that this composite variable for courses taught by highly qualified teachers did account for statistically significant unique variance lends support to this assumption. In addition, this finding was consistent with Hecks (2007) discovery that as a school resource, collective teacher quality in terms of certification status was positively related to school achievement. In this study, the organizational context was high school and the outcome measure was science proficiency score, whereas Hecks research involved the elementary school setting with reading and mathematics outcome measures. It appears that collective teacher quality is positively related to academic achievement in both elementary school and high school.
The two teacher variables that were positively related to SciMAP scores while controlling for the other variables were the percentage of teachers with masters degrees, and percentage with regular certification. Both of these variables indicated that schools with more masters degree teachers and certified teachers tended to have higher SciMAP scores. Having close to 100% of teachers certified and more teachers with advanced degrees was characteristic of schools with higher SciMAP scores. These results support the mentioned finding for the percentage of courses taught by highly qualified teachers: namely, that teacher qualifications are important in creating a school learning environment that leads to higher performance on the state science proficiency test. Schools that have more teachers with advanced training leading to masters degrees and who teach subjects in which they are highly qualified, along with faculty who are certified, have higher average SciMAP scores. These variables included all teachers, not just those with masters degrees and certification in science. Again, this finding was consistent with Hecks (2007) discovery that collective teacher qualification as a school resource was positively related to student achievement. The current studys focus on 10th-grade science proficiency differed from Hecks research, in which the outcome measures are elementary mathematics and reading achievement.
Because all the variables selected for the full model had some support from the literature that they were associated with academic performance, why did four of the variables fail to show a relationship with SciMAP scores? The first variable that did not account for unique variance in average SciMAP index scores was the student per classroom teacher ratio. In fact, the zero-order correlation showed that there was no relationship between SciMAP scores and classroom size (r = -.050). The mean for students per classroom teacher was 18.29 (SD = 3.63), with a maximum of 29. An average class size of 18 students, plus or minus about 7 students for 95% of the schools, was likely manageable in most situations. School size as measured by enrollment in Grades 912 showed no relationship to average science MAP scores (r = -.015) even though enrollment ranged from 67 to 2,534 (M = 5,930, SD = 553).
The percentage of credits earned in advanced courses did not account for statistically significant variance over the other variables in predicting SciMAP scores. In addition, the zero-order correlation with SciMAP scores was low (r = .099). It was anticipated that credits earned in advanced courses would account for significant variance in SciMAP scores while controlling for the other variables because more participation in advanced courses should include more students taking upper-level science courses, thus leading to higher science test scores. This failure of credits earned in advanced courses to account for significant variance may be due to the concept of course credit inflation (Dougherty, 2008), in which the course level or title indicating advanced may not reflect higher level instructional content and/or subject mastery. Students may be taking an advanced course in name only, without mastering the content. Second, the lack of significance may be due to a more basic limitation of this study in that science proficiency was only tested by the state in Grade 10; therefore, 10th graders would not have the opportunity to earn many credits in advanced courses until their junior and senior years.
Teacher average salary failed to account for statistically significant unique variance at p < .05, probably because of the high correlation with percentage of teachers with masters degrees (r = .698). Schools with more masters degree teachers were paying higher salaries accompanying the advanced degrees, so these two variables were likely measuring a similar construct.
MODERATOR VARIABLE EFFECTS
The following sections discuss the statistically significant interactions of FRL pct and Minority pct with the school, course, and teacher variables in predicting SciMAP scores. These significant interactions show how FRL pct and Minority pct moderate the relationships and increase the amount of variance accounted for in SciMAP scores.
FRL pct interacted with or moderated the relationship of five variables in predicting SciMAP scores: FRL pct by mobility rate, by dropout rate, by courses with highly qualified teachers, by masters degrees, and by teachers with regular certificates. Minority pct interacted with all the same variables except for mobility. Each FRL pct interaction model accounted for at least 25% of SciMAP score variance. Although this is a statistically and practically significant amount of variance, the Minority pct interaction models accounted for even moreat least 44% of the variance. Evidently, the Minority pct variable was acting as a more powerful moderator than FRL pct in the relationship between the school, course, and teacher variables with SciMAP scores.
In the FRL by mobility interaction, the SciMAP score difference for schools with low versus high mobility was only 2 points for schools with fewer students receiving free or reduced-price lunches (top line, Figure 1a). There was little SciMAP score difference between low and high FRL pct schools when the mobility rate was low; however, the difference was 8 SciMAP points when the mobility rate was high (bottom line, Figure 1a). High mobility was associated with lower SciMAP scores when the school FRL pct was high. Minority pct did not interact with mobility, but schools with high mobility had lower SciMAP scores than schools with low mobility for both high and low Minority pct schools.
The interaction patterns for dropout pct with FRL pct and Minority pct were similar. There was little difference in SciMAP scores between schools with high or low dropout rates when the moderator variable percentages were low (top line, Figure 1b and Figure 2a). But there were significant SciMAP score differences when dropout rates were at least 6% for schools with higher FRL pct or Minority pct (bottom line, Figure 1b and Figure 2a). FRL pct and Minority pct moderated the relationship between dropout rate and SciMAP scores, resulting in lower scores as their percentages increased. A high dropout rate was associated lower SciMAP scores when the school FRL pct or Minority pct were high.
High-quality course pct interactions
When FRL pct or Minority pct was high, SciMAP scores were better when most all of the courses were taught by teachers certified to teach in their content areas. SciMAP scores were lower for schools in which only 89% or fewer of the courses were taught by highly qualified teachers (bottom line, Figure 1c and Figure 2b). A larger percentage of courses taught by highly qualified teachers was associated with greater SciMAP scores when the school FRL pct or Minority pct was high.
Teachers with regular certificates pct interactions
The interaction patterns for regular certificates with FRL pct and Minority pct were similar. There was little difference in SciMAP scores between schools with a high or low percentage of teachers with regular certificates when the moderator variable percentages were low (top line, Figure 1e and Figure 2d). However, there were significant SciMAP score differences when the percentage of regular certificates was lower than 93% for schools with higher FRL pct or Minority pct (bottom line, Figure 1e and Figure 2d). For example, the SciMAP score difference between high and low FRL schools was only 5 points when 100% of the teachers had regular certificates but increased to 16 points when only 93% had regular certificates (Figure 1e). A larger percentage of teachers with regular certificates was associated with higher SciMAP scores when the school FRL pct or Minority pct was high.
Teachers with masters degrees pct interactions
The interaction of masters degree pct and FRL pct had an unexpected outcome and was further analyzed and discussed in the Results section. Specifically, FRL pct had a moderating effect on the relationship between masters degree pct and SciMAP scores that was heavily influenced by the large number of rural schools with rather high percentages of FRL students but low number of teachers with masters degrees. In contrast, large city schools also had high percentages of FRL students but higher percentages of masters degree teachers. This combination produced a moderating effect of FRL pct that was opposite of the expected direction (bottom line, Figure 1d). A high percentage of teachers with masters degrees was associated with lower SciMAP scores when the school FRL pct was high. This finding is likely a function of the geographical distribution of masters degree teachers rather than descriptive of the true nature of the relationship between advanced degrees and student achievement. The true nature of the relationship is probably more similar to that described next for Minority pct.
The interaction of Minority pct with masters degree pct was in the expected direction. When Minority pct was high, SciMAP scores were greater if more of the teachers had masters degrees. SciMAP scores were lower in schools that had fewer masters degree teachers (bottom line, Figure 2c). A larger percentage of teachers with masters degrees was associated with higher SciMAP scores when Minority pct was high.
Interaction of the two moderator variables
The interaction model between the moderator variables of FRL pct and Minority pct accounted for almost 45% of the SciMAP variance. The significant interaction of the moderator variables showed that SciMAP scores varied across different levels of FRL pct and Minority pct. There was a large SciMAP score difference of 19 points between schools with low FRL pct, low Minority pct, and high FRL pct, high Minority pct (Figure 1f). The interaction is depicted in the greater negative slope of the high FRL pct bottom line of Figure 1f. The SciMAP score difference between low and high FRL pct schools was only 4 points when the Minority pct was low but increased to 13 points when the Minority pct was high. A higher percentage of Minority students was associated with lower SciMAP scores in schools with a higher percentage of students receiving free-reduced price lunches.
This secondary analysis study used data defined and collected by the state of Missouri from its high schools. The methodology section described the variables in detail, along with their limitations. The authors had no control over the types of data collected, nor the data management. This research investigated variable relationships and their interactions but was not designed to make causal inferences. Furthermore, the unit of analysis was the school, so significant variable relationships at the school level may not be the same at the individual level.
The present findings are consistent with the status attainment literature and the theoretical arguments associated with geography and educational attainment discussed earlier, in that SES and minority status are important factors in Missouri. As an extension of previous research, this study demonstrates that the school composition variables of FRL pct and Minority pct are significantly related to science proficiency in the 10th grade. Not only are they predictive of science proficiency scores, but they also interact with each other and moderate the relationships between other school context variables and 10th-grade science scores. The presence of these interactions and moderator effects may help explain why previous research has not been entirely consistent in the findings about how some of school and teacher variables relate to student achievement. Note, for example, that Byrnes and Miller (2007) concluded that race accounted for very little of the explained variance in their study of science achievement. Instead, distal, propensity, and opportunity factors demonstrated greater predictive value. However, the model being tested largely focused on psychological constructs such as propensity and opportunity factors (proximal causes), as well as factors that might account for the emergence of opportunity (distal factors). The study reported here suggests that theories of science achievement should include relevant school composition factors and how both race and SES interact and moderate science proficiency. Going forward, future research focused on science achievement should consider testing models that include potential proximal predictors, distal factors, and school composition and context factors.
Specifically, the findings presented in this study demonstrated that selected school context variables for the school environment, courses, and teachers were related to 10th-grade science proficiency scores in Missouri high schools. Some of these school variables were moderated by the percentages of free/reduced-price lunch students and/or minority students. These interactions mean that the relationships of the school context variables to science proficiency scores differ and depend on the percentages of FRL and/or minority students in the school. Higher percentages of FRL and/or minority students present greater risk factors, especially when combined with less favorable school context variables. Future research using school context variables should seriously consider testing for interactions with demographic composition factors rather than simply including them as additional predictors in the model.
The school context variables of higher dropout rates and mobility rates signaled greater risk factors, especially when moderated by SES and minority status. When the FRL pct and Minority pct were higher, lower 10th-grade science proficiency scores were associated with elevated dropout rates. Similarly, greater mobility was related to lower SciMAP scores when the school FRL pct was high.
Some school context variables for courses and teachers interacted positively with SES and minority status, which resulted in higher science scores. Schools with more FRL and minority students achieved higher science proficiency scores when they had a greater percentage of courses taught by highly qualified teachers and more of their teachers were regularly certified. In addition, higher SciMAP scores were associated with a greater percentage of masters degree teachers in schools with a larger percentage of minority students.
Although this is not a study that infers causality, would it be possible for schools with high percentages of FRL and minority students to increase science proficiency by having more science courses taught by faculty trained in science content areas, more teachers who are regularly certified, and more teachers who have masters degrees? The data in the present study suggest that schools with greater percentages of FRL and minority students have higher 10th-grade science proficiency scores when these course and teacher factors are present. Moreover, the data suggest that collective teacher quality is an important school resource. Heck (2007) had a similar finding in a study of the relationship between teacher quality as an organizational construct, and achievement in elementary school. While teacher quality is generally acknowledged in public debates as important, there remain some skeptics who view state certification as non-essential, as well as the lack of capacity and dedicated resources to address this significant equity challenge in science education.
This article is based on research and development funded by the National Science Foundation under award No. ESI-0227619. The funding supports the St. Louis Center for Inquiry in Science Teaching and Learning (CISTL). Any opinions, findings, and conclusions or recommendations expressed here are those of the author and do not necessarily reflect the views of the National Science Foundation.
1. The term science proficiency is being used in this article to capture the intent of the state of Missouri rather than any agreed-on notion in the research literature.
Balfanz, R. (2000). Why do so many urban public school students demonstrate so little academic achievement? In M. G. Sanders (Ed.), Schooling students placed at risk: Research, policy, and practice in the education of poor and minority adolescents (pp. 3762). Mahwah, NJ: Erlbaum.
Battelle Technology Partnership Practice and SSTI. (2006, April). Growing the nations bioscience sector: State bioscience initiatives 2006. Retrieved December 26, 2008, from http://www.bio.org/local/battelle2006/
Beatty, A. (2008). Common standards for K-12 education? Considering the evidence. Washington, DC: National Research Council.
Berliner, D. C., & Biddle, B. J. (1997). The manufactured crisis: Myths, fraud, and attack on Americas public schools. White Plains, NY: Longman.
Borman, K. M., Eitle, T. M., Michael, D., Eitle, D. J., Lee, R., Johnson, L., et al. (2004). Accountability in a postdesegregation era: The continuing significance of racial segregation in Floridas schools. American Educational Research Journal, 41, 605631.
Building Engineering and Science Talent. (2006). Making Missouri a national leader in mathematics, engineering, technology and science (A data book for Governor Blunts summit). San Diego, CA: Author. Retrieved December 26, 2008, from http://governor.mo.gov/mets/metsleader.pdf
Byrnes, J. P., & Miller, D. C. (2007). The relative importance of predictors of math and science achievement: An opportunity-propensity analysis. Contemporary Educational Psychology, 32, 599629.
Center for Science, Mathematics, and Engineering Education. (1998). Every child a scientist: Achieving scientific literacy for all. Washington, DC: National Academy Press.
Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Erlbaum.
Committee on Prospering in the Global Economy of the 21st Century. (2007). Rising above the gathering storm: Energizing and employing America for a brighter economic future. Washington, DC: National Academy Press.
Darling-Hammond, L. (2000). Teacher quality and student achievement: A review of state policy evidence. Journal of Educational Policy Analysis, 8, 88114.
Darling-Hammond, L., Berry, B., & Thoreson, A. (2001). Does teacher certification matter? Evaluating the evidence. Educational Evaluation and Policy Analysis, 23, 5777.
Dougherty, C. D. (2008). Do college preparatory courses live up to their labels? Using statewide longitudinal data systems to assess the rigor of academic courses. Retrieved December 8, 2008, from Data Quality Campaign Web site: http://www.dataqualitycampaign.org/files/publications-dqc_college_prep_courses-101308.pdf
Graduation profiles. (2007, June). Retrieved January 3, 2009, from Education Week Web site: http://www.edweek.org/ew/articles/2007/06/12/40gradprofiles.h26.html
Ferguson, R. F., & Brown, J. (2000). Certification test scores, teacher quality, and student achievement. In D. W. Grissmer & J. M. Ross, Analytical issues in the assessment of student achievement (pp. 133156, NCES 2000-050). Washington, DC: U.S. Department of Education, National Center for Education Statistics.
Finn, J. D., & Achilles, C. M. (1990). Answers and questions about class size: A statewide experiment. American Educational Research Journal, 27, 557577.
Finn, J. D., & Voelkl, K. E. (1993). School characteristics related to student engagement. Journal of Negro Education, 62, 249268.
Fowler, W. J., & Walberg, H. J. (1991). School size, characteristics, and outcomes. Educational Evaluation and Policy Analysis, 13, 189202.
Ghiselli, E. E., Campbell, J. P., & Zedeck, S. (1981). Measurement theory for the behavioral sciences. San Francisco: W. H. Freeman and Company.
Goldhaber, D. D., & Brewer, D. J. (1996). Evaluating the effect of teacher degree level on educational performance. Rockville, MD: Westat. (ERIC Document Reproduction Service No. ED406400)
Gonzales, P., Guzmán, J. C., Partelow, L., Pahlke, E., Jocelyn, L., Kastberg, D., et al. (2004). Highlights from the trends in international mathematics and science study (TIMMS) 2003 (NCES 2005-005). U.S. Department of Education, Institute of Education Sciences. Washington, DC: U.S. Government Printing Office.
Gollub, J. P., Bertenthal, M. W., Labov, J. B., & Curtis, P. C. (Eds.) (2002). Learning and understanding: Improving advanced study of mathematics and science in U. S. high schools. Washington, DC: National Academy Press.
Grigg, W. S., Lauko, M. A., & Brockway, D. M. (2006). The nations science report card: Science 2005 (NCES 2006-466). U.S. Department of Education, National Center for Education Statistics. Washington, DC: U. S. Government Printing Office.
Hamilton, L. (2003). Assessment as a policy tool. In R. E. Floden (Ed.), Review of research in education (pp. 2568). Washington, DC: American Educational Research Association.
Hanushek, E. (1997). Assessing the effects of school resources on student performance: An update. Educational Evaluation and Policy Analysis, 19, 129146.
Harmston, M. T., & Pliska, A. (2001). Trends in ACT mathematics and science reasoning achievement, curricular choice, and intent for college major: 1995-2000 (ACT Research Report Series 2001-3). Iowa City, IA: ACT.
Heck, R. H. (2007). Examining the relationship between teacher quality as an organizational property of schools and students achievement and growth rates. Educational Administration Quarterly, 43, 399432.
Jelleyman, T., & Spencer, N. (2008). Residential mobility in childhood and health outcomes: A synthetic review. Journal of Epidemiology and Community Health, 62, 584592.
Kamen, D. H., & Benavot, A. (1991). Elite knowledge for the masses: The origins and spread of mathematics and science education in national curricula. American Journal of Education, 99, 137180.
Konstantopoulous, S. (2006). Trends of school effects on student achievement: Evidence from NLS: 72, HSB: 82, and NELS: 92. Teachers College Record, 108, 25502582.
Konstantopoulos, S. (2008). Do small classes reduce the achievement gap between low and high achievers? Evidence from Project Star. Elementary School Journal, 108, 275291.
Linn, M. C. (2007). Science education that makes sense. Research Points, 5(1). Retrieved December 26, 2008, from http://www.aera.net/uploadedFiles/Journals_and_Publications/Research_Points/RP_Summer07.pdf
Massell, D. (2008, January). The current status and role of standards based reform in the states. Paper prepared for the National Research Council Workshop on Assessing the Role of K12 Academic Standards in States, Washington, DC. Retrieved December 27, 2008, from http://www7.nationalacademies.org/cfe/Massell%20State%20Standards%20Paper.pdf
Miller, L. S. (1995). An American imperative: Accelerating minority educational advancement. New Haven, CT: Yale University Press.
Morris, J. E., & Monroe, C. R. (2009). Why study the U.S. South? The nexus of race and place in investigating Black student achievement. Educational Researcher, 38(1), 2136.
Moses, R. P., & Cobb, C. E. (2001). Radical equations: Math literacy and civil rights. Boston: Beacon Press.
Missouri Department of Elementary and Secondary Education. (2008a). About school core data. Retrieved December 8, 2008, from http://dese.mo.gov/divimprove/coredata/about.html
Missouri Department of Elementary and Secondary Education. (2008b). Missouri Assessment Program: Guide to interpreting results. Retrieved December 8, 2008, from http://dese.mo.gov/divimprove/assess/documents/girmanual.pdf
National Center for Education Statistics, U.S. Department of Education. (2002). School locale codes 19872000 (NCES 2002-02) by Nancy Speicher. Arnold A. Goldstein, project officer. Retrieved December 8, 2008, from http://nces.ed.gov/pubs2002/200202.pdf
No Child Left Behind Act of 2001, Pub. L. No. 107-110, 115, Stat. 1425 (2002).
Nyhan, R. C., & Alkadry, M. G. (1999). The impact of school resources on student achievement test scores. Journal of Education Finance, 27, 885907.
Okpala, C. O., Smith, F., & Jones, E. B. (2000). A clear link between school and teacher characteristics, student demographics, and student achievement. Education, 120, 487494.
OSullivan, C. Y., Lauko, M. A., Grigg, W. S., Qian, J., & Zhang, J. (2003). The nations report card science 2000 (NCES 2003-453). Washington, DC: U.S. Department of Education, Institute of Education Sciences.
Perkins, R., Kleiner, B., Roey, S., & Brown, J. (2004). The high school transcript study: A decade of change in curricula and achievement 1990-2000 (NCES 2004-455). Washington, DC: U.S. Department of Education, Institute of Education Sciences.
Peterson, P., & Mayer, S. (Eds.) (1999). Earning and learning: How schools matter. Washington, DC: Brookings Institution Press.
Procter, B. D., & Dalacker, J. (2003). Poverty in the United States: 2002 (U.S. Census, Current Population Reports, P60-222). Washington, DC: United States Government Printing Office. Retrieved November 29, 2008, from http://www.census.gov/prod/2003pubs/p60-222.pdf
Raffaele Mendez, L. M. (2003). Predictors of suspension and negative school outcomes: A longitudinal investigation. New Directions for Youth Development, 99, 1733.
St. John, N. H. (1970). Desegregation and minority group performance. Review of Educational Research, 40, 111133.
Sanders, W. H. (1993). Expenditures and student achievement in Illinois: New evidence. Journal of Public Economics, 52, 403416.
Secada, W. (1991). Diversity, equity, and cognitivist research. In E. Fennema, T. Carpenter, & S. Lamon (Eds.), Integrating research on teaching and learning mathematics (pp. 1753). Albany: State University of New York Press.
Shapiro, T. M. (2004). The hidden cost of being African American: How wealth perpetuates inequality. Oxford, England: Oxford University Press.
Sirin, S. (2005). Socio-economic status and academic achievement: A meta-analytic review of research. Review of Educational Research, 75, 417453.
Skiba, R., & Rausch, M. K. (2004). The relationship between achievement, discipline, and race: An analysis of factors predicting ISTEP scores (Children Left Behind Policy Briefs Supplementary Analysis 2-D). Bloomington: Indiana University, Center for Evaluation and Education Policy. Retrieved December 2, 2008, from http://ceep.indiana.edu/ChildrenLeftBehind/pdf/2D.pdf
Smith, M. O. (2004). How the apportionment of district money is associated with differences in the Grade 8 mathematics and reading achievement scores for Minnesota students. Journal of Education Finance, 29, 299314.
South, S. J., Haynie, D. L., & Bose, S. (2007). Student mobility and school dropout. Social Science Research, 36, 6894.
Tajalli, H., & Opheim, C. (2004). Strategies for closing the gap: Predicting student performance in economically disadvantaged schools. Educational Research Quarterly, 28, 4454.
Tate, W. F. (1995). School mathematics and African American students: Thinking seriously about opportunity-to-learn standards. Educational Administration Quarterly, 31, 424448.
Tate, W. F. (2001). Science education as a civil right: Urban schools and opportunity-to-learn considerations. Journal for Research in Science Teaching, 38, 10151028.
Tate, W. F. (2004). Brown, political economy, and the scientific education of African Americans. In R. E. Flooden (Ed.), Review of research in education (pp. 147184). Washington, DC: American Educational Research Association.
Tate, W. F. (2008). Geography of opportunity: Poverty, place, and educational outcomes. Educational Researcher, 37(7), 397-411.
United States General Accounting Office. (1993). School age demographics: Recent trends pose new educational challenges (HRD-93-105BR). Washington, DC: Author. Retrieved November 29, 2008, from http://archive.gao.gov/t2pbat5/149755.pdf
United States General Accounting Office. (1994). Elementary school children: Many change schools frequently, harming their education (Report to the Honorable Marcy Kaptur, House of Representatives, GAO/HEH- S94-95). Washington, DC: Author. Retrieved November 30, 2008, from http://archive.gao.gov/t2pbat4/150724.pdf
Warner, R. M. (2008). Applied statistics: From bivariate through multivariate techniques. Los Angeles: Sage.
Wayne, A. J., & Youngs, P. (2003). Teacher characteristics and student achievement gains: A review. Review of Educational Research, 73, 89112.
Weiss, I. R., Knapp, M. S., Hollweg, K. S., & Burrill, G. (Eds.) (2002). Investigating the influence of standards: A framework for research in mathematics, science, and technology education. Washington, DC: National Academy Press.
White, K. R. (1982). The relationship between socioeconomic status and academic achievement. Psychological Bulletin, 91, 461481.
Wood, D., Halfon, N., Scarlata, D., Newacheck, P., & Nessim, S. (1993). Impact of family relocation on childrens growth, development, school function, and behavior. Journal of the American Medical Association, 270, 11341138.
Wood, N., Lawrenz, F., Huffman, D., & Schultz, M. (2006). Viewing the school environment through multiple lenses: In search of school-level variables tied to student achievement, Journal of Research in Science Teaching, 43, 237254.
Wright, S. P., Horn, S. P., & Sanders, W. L. (1997). Teacher and classroom context effects on student achievement: Implications for teacher evaluation. Journal of Personnel Evaluation in Education, 11, 5767.