Home Articles Reader Opinion Editorial Book Reviews Discussion Writers Guide About TCRecord
transparent 13
Topics
Discussion
Announcements
 

Similar Students, Different Results


by Trish Williams & Michael W. Kirst - January 25, 2006

Why do some California elementary schools serving largely low-income students score as much as 250 points higher on the state's academic performance index (API) than other schools with very similar students?

Why do some California elementary schools serving largely low-income students score as much as 250 points higher on the state's academic performance index (API) than other schools with very similar students?


That’s the research question asked by a new, large-scale EdSource-led study that surveyed principals and teachers in 257 such schools across the state.  What we learned is that the higher performing schools tend to have four interrelated practices at the core of their operation—prioritizing student achievement; implementing a coherent, standards-based curriculum and instructional program; analyzing student-assessment data from multiple sources; and ensuring availability of instructional resources.


Many studies have examined successful schools as a group, in an effort to understand their methods or best practices. This study—conducted by EdSource and researchers from Stanford University, the University of California, Berkeley, and the American Institutes for Research—took a different tack. Rather than looking at a specific performance zone, we examined elementary schools within a specific, fairly narrow socioeconomic and demographic band but across the full range of school performance.


Specifically, we garnered the participation of 257 out of the 550 elementary schools that fall within the 25–35th percentile band of California’s School Characteristics Index (SCI).  This band of schools has high levels of students from low-income families, or from ethnic minorities, and/or who are learning English as a second language. Our dependent variable was the school’s Academic Performance Index (API), which is based annually for these schools on student scores in English Language Arts and math on the California Standards Tests in grades 2–5. We surveyed all the principals and 80 % of the K–5 classroom teachers at the schools in our sample, giving us 257 principal surveys and over 5,500 teacher surveys.


Our surveys included over 300 items each covering seven broad domains often associated with effective schools.  The questions were updated to reflect California’s current standards based reform environment, asking specifically what principals and teachers were doing, and how frequently. Once the data was collected, we used a weighted analysis to assure that results were statistically representative of all public, non-charter elementary schools in the 25–35th SCI band. We then used several types of regression analyses to determine which school practices differentiated the highest performing schools from the lowest performing ones.


WHAT SCHOOLS DO CLEARLY MAKES A DIFFERENCE


We’ve heard people say that you can guess a school’s Academic Performance Index (API) score by the zip code of its students. But the 250-point difference in API between schools serving similarly disadvantaged students shows that’s not true.  What do the higher performing schools do differently? From the school district through to the classroom, they focus on improving student achievement against the state’s academic standards and align all their practices around that goal. Of the seven domains we studied, these four had the biggest impact on a school’s API:


Prioritizing Student Achievement. Where teacher and principal answers to multiple survey questions indicated higher expectations for students, their schools had, on average, higher API scores than schools whose staffs indicated lower expectations. In more successful schools, teachers and principals alike reported that their schools have well defined plans for instructional improvement and that they put priority on meeting the state’s API goals and the federal adequate yearly progress goals. Teachers and principals also report that their schools set measurable goals for exceeding the mandated API student subgroup growth targets for improved achievement.


Implementing a Coherent, Standards-based Curriculum and Instructional Program. Teachers who report the following were more likely to be in higher performing schools: instructional consistency within grades; curricular alignment from grade-to-grade; classroom instruction guided by state academic standards; curriculum materials in math and language arts aligned with the state’s standards; in a district that addresses the instructional needs of English learners at their school. Principals were more likely to be in higher performing schools if they reported that the district has clear expectations for student performance aligned with the district’s adopted curriculum and that the district evaluates the principal based on the extent to which instruction in the school aligns with the curriculum.


Using Assessment Data to Improve Student Achievement and Instruction. Strongly correlated with a higher school API was the extensive use of student assessment data by the district and the principal in an effort to improve instruction and student learning. For example, principals more often reported that they and the district use assessment data from multiple sources (including frequent curriculum program tests and annual California Standards Tests) to evaluate teachers’ practices and to identify teachers who need instructional improvement. Principals report using this data to develop strategies to follow up on the progress of selected students and help them reach academic goals. According to these principals, the district expects all of its schools to improve achievement, evaluates principals based on student achievement, and provides support for site-level planning related to improving achievement.


Ensuring Availability of Instructional Resources. Where more teachers reported having regular or standard certification for teaching in California, schools had, on average, higher API scores. The same was true of schools where principals more often reported that their districts provide sufficient and up-to-date instructional materials as well as support for supplementary instruction for struggling students and for facilities management. Teachers with at least five years of full time teaching experience were more likely, on average, to be from schools with higher APIs. Principal experience was also correlated with higher school achievement.


The survey also included questions related to schools’ efforts to involve parents, teacher collaboration and development, and the enforcement of high expectations for student behavior. Although each of these types of practices made some contribution to a school’s API score, and were likely practiced by most of the schools in the sample, they were not nearly as strongly correlated with higher school performance as were the four key interactive, interdependent school improvement practices described above.


WHAT THE STUDY FINDS ABOUT TEACHING AND LEARNING


Our study underscores that there is no single silver bullet for school success; multiple factors and practices are at work. Yet it appears that in the context of standards based, accountability-driven reform, at least for elementary schools with large numbers of low-income students, a major implication is that the caliber of teaching and learning appears directly related to the level of system-wide coherence. The clearer the strategic through line from district to school to classroom, the more likely it is that higher achievement will result. Put another way, the more all the requisite pieces are aligned, seen as interdependent parts of a whole, and overtly structured to support the work of teachers in the classroom, the stronger the benefits to learning.

The study provided some glimpses of how this looks in action:

District practices include setting clear expectations that schools meet API and AYP growth targets, including for subgroups, as well as providing schools with achievement data and evaluating principal performance and teacher practices based on that data. They also include ensuring that math and language arts curricula are aligned with state standards, that instruction is focused on achievement and that schools have adequate facilities and textbooks as well as resources for struggling students.


Principal leadership is redefined to focus on effective management of the school improvement process. In general, API scores were higher in schools with principals whose responses indicate that they act as proactive managers of school improvement, driving the reform process, cultivating the school vision, and extensively and personally using student assessment data for a wide variety of school improvement areas of focus, including evaluation of teacher practice and assistance to struggling students.


An effective teaching staff appears to be characterized by certain ingredients. In higher achieving schools, principals reported that a higher proportion of their teaching staff had, in order of priority: a demonstrated ability to raise student achievement, strong content knowledge, characteristics that made them a good fit with the school culture, training in curriculum programs, and the ability to map curriculum standards to instruction. These teachers were also supportive of colleagues’ learning and improvement, able to use data from student assessments, familiar with the school community, excited about teaching, and familiar with state standards.


ADDITIONAL RESEARCH QUESTIONS


An interrelated question we plan to follow up on is the relationship between school-wide use of specific curriculum packages, other curricular and instructional school practices, and school performance. In addition, our teacher survey included a robust set of questions around EL instruction that have yet to be analyzed.


Reference


Williams, T., Kirst, M., Haertel, E., et al. (2005). Similar students, different results: Why do some schools do better? A large-scale survey of California elementary schools serving low-income students. Mountain View, CA: EdSource.


The initial findings report, Similar Schools, Different Results: Why Do Some Schools Do Better?, including appendices with more details on the research methodology and the demographics of the school and student sample, can be found at www.edsource.org




Cite This Article as: Teachers College Record, Date Published: January 25, 2006
https://www.tcrecord.org ID Number: 12299, Date Accessed: 5/20/2022 11:29:20 PM

Purchase Reprint Rights for this article or review
 
Article Tools
Related Articles

Related Discussion
 
Post a Comment | Read All

About the Author
  • Trish Williams
    Executive Director, EdSource
    E-mail Author
    TRISH WILLIAMS, study project director, has been Executive Director of EdSource since 1992. Under her leadership, EdSource has expanded its research role, broadened the education policy topics it researches and reports on, diversified and significantly increased its audience reach within California and nationally, and established a reputation as a premier resource for high quality and impartial information, research, data, and analysis. Besides Similar Schools, Different Results, EdSource has conducted other surveys, including one the past two years with Californiaís 500+ charter schools, and participated on major research or evaluation teams, including California's Class Size Reduction Research Consortium and the state funded II/USP evaluation. Before coming to EdSource, Williams had served as a Presidential Management Intern and then a management analyst for three years in Washington D.C. at the U.S. Department of Health and Human Services. She subsequently served eight years as a program and policy consultant to the Oklahoma Commission on Children and Youth, a statewide agency with oversight powers over the state departments serving children. Williams's two sons are graduates of the Cupertino public schools where Williams served as an active volunteer at the elementary, middle, high school, and district levels. Williams holds a bachelors degree in English literature and a masterís degree in urban studies/public policy from her hometown school, the University of Tulsa.
  • Michael Kirst
    Stanford University
    E-mail Author
    MICHAEL W. KIRST, principal investigator for the study, has been Professor of Education and Business Administration at Stanford University since 1969. He is a faculty affiliate with the Department of Political Science, and has a courtesy appointment with the Graduate School of Business. Kirst was a member of the California State Board of Education (1975-1982) and its president from 1977 to 1981. Before joining the Stanford faculty, he held positions with the federal government including Staff Director of the U.S. Senate Subcommittee on Manpower, Employment and Poverty, and Director of Program Planning and Evaluation for the Bureau of Elementary and Secondary Education in the U.S. Office of Education. He has been a fellow at the Center for Advanced Study in Behavioral Sciences; a member of the National Academy of Education since 1979; Vice-President of the American Educational Research Association; and commissioner of the Education Commission of the States. He co-founded Policy Analysis for California Education (PACE) and is on the management/research staff of the Consortium for Policy Research in Education His numerous articles address school finance politics, curriculum politics, intergovernmental relations, and education reform policies. Recent books include The Political Dynamics of American Education (2005) and From High School to College (2004). Kirst holds a bachelor's degree in economics from Dartmouth College, an M.P.A. in government and economics from Harvard University, and a Ph.D. in political economy and government from Harvard.
 
Member Center
In Print
This Month's Issue

Submit
EMAIL

Twitter

RSS