Scaling Up STEM Academies Statewide: Implementation, Network Supports, and Early Outcomes

by Viki Young, Ann House, David Sherer, Corinne Singleton, Haiwen Wang & Kristin Klopfenstein - 2016

This article presents a case study of scaling up the T-STEM initiative in Texas. Data come from the four-year longitudinal evaluation of the Texas High School Project (THSP). The evaluation studied the implementation and impact of T-STEM and the other THSP reforms using a mixed-methods design, including qualitative case studies; principal, teacher, and student surveys; and a quasi-experimental approach to examining the effects of the programs on student achievement and achievement-related behaviors. The article presents a brief description of T-STEM characteristics and early outcomes on math and science achievement. It then examines variation in the local interpretation and enactment of the blueprint designed to guide T-STEM academy development. A multidimensional infrastructure supported T-STEM implementation and provided needed technical assistance. Yet T-STEM academies’ primary affiliations through their district or external support network provided the strongest context for implementation, shaping priorities and affording resources and know-how. The article offers key lessons for policymakers and practitioners engaged in supporting STEM efforts in particular and new school models in general.

INTRODUCTION

Mathematics and sciencelong the acknowledged domain of the academically giftedlie at the crux of the knowledge economy, now and for the foreseeable future. For policymakers and reformers, however, endorsing a small, educated elite with strong academic training in science, technology, engineering, and mathematics (STEM), while a large proportion of the population remains ill-fitted to the new economy, is untenable (National Research Council, 2005; PCAST, 2010). Inclusive STEM schools are predicated on the dual premises that math and science competencies can be developed, and that students from traditionally underrepresented subpopulations need access to opportunities to develop these competencies to become full participants in areas of economic growth and prosperity. Inclusive STEM schools do not screen prospective students on the basis of strong prior academic achievement. Rather, they build in supports to engage students in STEM and provide them with opportunities to master STEM content and related skills. Although inclusive STEM programs can exist in a wide variety of school contexts, this chapter focuses specifically on standalone, whole STEM schools or schools-within-schools that operate as autonomous units.

The chapter sets the stage with a description of the early T-STEM initiative (a large-scale inclusive STEM school initiative in Texas), presents early results on T-STEM effects on a range of student outcomes, and highlights factors that facilitated and constrained early implementation at the T-STEM academies. It culminates in key lessons taken from this statewide STEM scale-up. Data come from the four-year longitudinal evaluation of the Texas High School Project (THSP).¹ The evaluation studied the implementation and impact of T-STEM and the other THSP reforms using a mixed-methods design, including qualitative case studies; principal, teacher, and student surveys; and a quasi-experimental approach to examining the effects of the programs on student achievement and achievement-related behaviors. ²

THE T-STEM INITIATIVE

With an investment of approximately $120 million in 51 academies and seven T-STEM technical assistance centers (as of 20092010), the T-STEM initiative in Texas was the largest investment in inclusive STEM high schools in the United States at that time. The first T-STEM schools were funded in 20062007. In addition, seven regional T-STEM centers formed a statewide technical assistance infrastructure, intended to support T-STEM academies specifically and to improve math and science education statewide.

A relatively detailed T-STEM blueprint³ guided school leaders planning and implementation of T-STEM academies. The blueprint articulated central tenets for T-STEM academies such as providing a rigorous academic curriculum, instruction relevant to real-world problems and careers, accelerated access to STEM coursework, and personalized learning supports for students. The blueprint described school design features organized into seven general areas: mission-driven leadership; T-STEM culture; student outreach, recruitment, and retention; teacher selection, development, and retention; curriculum, instruction, and assessment; strategic alliances; and academy advancement and sustainability. Within each of these seven areas, the blueprint provided two to five design statements. For example, the blueprint directed academies to regularly offer advisory periods, provide common planning time for teachers, and host parent seminars on college readiness. Central to T-STEM academy design, the blueprint required that teachers organize instruction around project-based and problem-based learning, that students earn 12 to 30 college credit hours by graduation, and that they complete an internship or senior capstone project.

By design, T-STEM academies were also small schools serving approximately 100 students per grade, run by the local school district or a charter management organization (CMO). The blueprint stipulated that T-STEM academies must be nonselective; they could not select students based on prior performance and must have a student population that is more than 50% economically disadvantaged or more than 50% from ethnic/racial minority groups. The T-STEM academies were typically located in high-need areas, mainly the inner cities of the major metropolises, the Rio Grande Valley, and rural East Texas. T-STEM academies ranged in school structure, as standalone high schools and as schools within schools. Most academies were startups; existing schools converted into T-STEM academies were the minority. Figure 1 illustrates the characteristics of students attending T-STEM academies and other THSP schools that were in operation from 20092010, compared with students in non-THSP schools. In keeping with the blueprint, a larger proportion of students in T-STEM schools was economically disadvantaged and drawn from racial/ethnic minorities than in non-THSP high schools.

Figure 1. Selected student characteristics of T-STEM and non-THSP schools, 20092010

Note. The number of schools is shown in parentheses after each school category. Non-THSP schools refers to all non-THSP schools in the state serving grades 9, 10, 11, or 12. From the Academic Excellence Indicator System (AEIS) 20092010 academic year.

SUPPORTS FOR SCALE-UP: T-STEM CENTERS, COACHES, AND PRIMARY AFFILIATIONS

In addition to the guidance provided by the T-STEM blueprint, T-STEM academies drew on a wide variety of supporting organizations as the initiative matured and scaled up. Figure 2 shows the different sources of support and influence that were available to each T-STEM academy. The left side of the diagram shows sources of support that were directly tied to the T-STEM initiative and were thus provided to all T-STEM academies. The right side of the diagram presents sources of support that were local in nature and thus varied across T-STEM academies.

Some supports were provided to T-STEM academies at the level of the T-STEM network. These network-level supports included an online portal that offered vignettes and videos showcasing best practices among T-STEM academies, and regional and statewide conferences for T-STEM academies to share successes and challenges in implementing their respective programs.

The seven T-STEM centers located at universities and regional education centers throughout the state also offered supports to T-STEM academies. These centers were designed to serve as statewide resources for T-STEM academies (and for other schools requesting assistance regarding STEM education). The centers provided guidance, resources, and professional development related to T-STEM blueprint implementation, pedagogy (particularly project-based learning), STEM content, and community partnerships. All seven T-STEM centers provided coaching for school leaders and professional development for teachers. Individual centers provided supports reflecting both the unique resources of each center as well as the specific needs of each academy. The centers also worked together to provide some coordinated services, including a foundational project-based learning (PBL) workshop for teachers and an annual T-STEM Best Practices Conference.

In addition to T-STEM centers, T-STEM program officers and coaches unaffiliated with T-STEM centers worked directly with academies to advise principals on implementing the blueprint elements, administrative issues, and instructional leadership (e.g., by co-designing a classroom observation tool). As T-STEM academies matured, meetings between T-STEM coaches and principals provided ongoing opportunities for coaches to discuss the extent to which the academies were meeting the expectations of the T-STEM blueprint, and to guide principals as they sought to implement more blueprint elements as their respective schools matured. In addition to being advocates for and communicators of the blueprint, coaches also responded to the T-STEM academy leaders individual needs and requests. Many coaches developed strong relationships with T-STEM academy leaders over time, which in turn allowed the coaches some influence over how to increase the alignment between the schools current practices and the T-STEM blueprint.

In addition to the centralized and regional supports that all T-STEM academies were provided as part of their membership in the T-STEM network, each individual T-STEM academy was supported locally by its primary affiliationthe organization upon which the academy depended most for school design and ongoing support, whether its district, CMO, or external support network. As will be discussed later in this chapter, each primary affiliation had a different set of priorities and capacities related to T-STEM blueprint implementation that reinforced or weakened the academys blueprint implementation in different domains.

Taken together, the centralized and regional supports provided to all T-STEM academies (e.g., the T-STEM blueprint, coaches, and centers) and the local influence of each academys primary affiliation shaped the implementation of T-STEM on the ground as the T-STEM initiative scaled up.

Figure 2. T-STEM Academies network of supports

T-STEM ACADEMIES EARLY IN IMPLEMENTATION

In the early years covered by the THSP evaluation, the T-STEM academies were primarily focused on establishing the elements specified in the T-STEM blueprint, instructional and otherwise. As many of them were new schools, they also needed to attend to the typical start-up pressuresfinding and retrofitting facilities, recruiting and training teachers, recruiting students, establishing procedures for an expanding school, and so on. T-STEM academies experienced the predictable pains of conversion and start-up described in prior studies of small high schools (cf. AIR/SRI, 2005; Kahne, Sporte, & de la Torre, 2006; Young et al., 2009). Moreover, the schools confronted the pressing needs of ninth-graders, who often entered the schools academically ill-prepared. The T-STEMs concentrated on bringing them up to grade level in one year and then sustaining their achievement to avoid falling under Texass accountability sanctions. The T-STEM academies aims to improve high school achievement, graduate their students, and get them accepted to college is best understood within this context.

Beyond these start-up demands at the school level, T-STEM academy teachers were still developing their understanding and practice in PBL. Although many T-STEM academies were schools of choice and therefore could hire teachers who expressed commitment to PBL, teachers nonetheless needed time to master a complex, nontraditional instructional approach.

EARLY OUTCOMES OF THE T-STEM INITIATIVE

Not surprisingly, the achievement outcomes that T-STEM academies commonly pursued were determined largely by the broader state accountability context. Although T-STEM academies have attained acceptable, recognized, or exemplary ratings in the Texas accountability systemthus escaping the turnaround pressures at underperforming schoolsthey nonetheless monitored student performance closely throughout the year to ensure that students meet or exceed the annual Texas Assessment of Knowledge and Skills (TAKS)⁴ proficiency standards. These outcomes are the most heavily emphasized measurable outcomes in the state. Success on TAKS was essential to the prospects of any given T-STEM academy, not only because of its importance for students, but also in terms of building a reputation for academic excellence that would attract future students.

The THSP evaluation tracked cohorts of students beginning in the ninth grade, using TAKS achievement results and other measures of academic progression for ninth-, 10th-, and 11th-graders served by T-STEM academies. The last year of results under the THSP evaluationfor outcomes from the 20092010 school yearcombined the effects for T-STEM academies that began operations in 20072008, 20082009, or 20092010.⁵ To estimate the effect of T-STEM and the other THSP programs, we matched comparison schools outside the THSP programs to each THSP school (including T-STEM academies) using a combined exact matching and propensity score matching method.⁶ Our approach took into account a wide range of observable school-level characteristics that included student demographics, prior achievement, accountability rating, teacher experience, and teacher demographics. The effects for each of the THSP programs, including T-STEM, were estimated together in the same hierarchical models to maximize the precision of the estimates, controlling for student-level demographics and prior achievement and school-level characteristics. (Detailed methods are described in Young et al., 2011.)

Overall, T-STEM academies demonstrated some impact in math and science achievement and pro-academic behaviors; however, the T-STEM advantage during early implementation appeared to be subject-specific and inconsistent across grade levels. In 20092010, T-STEM academy students scored slightly higher than matched comparison school peers on 10th-grade TAKS-Math.⁷ The effect size is relatively small at 0.08 standard deviations, but it is positive and in one of the core STEM areas. In addition, 10th graders in T-STEM schools had a higher likelihood (1.5 times) of meeting or exceeding TAKS in all four core subjects tested in that grade (a combined measure). Students in T-STEM academies had a higher likelihood (1.4 times) of passing Algebra I by ninth grade, compared with peers in comparison schools. Tenth-grade students in T-STEM academies also had a lower likelihood (82%) of being absent from school than did students in the matched comparison schools. However, T-STEM academy students achieved similar scores to their matched comparison school peers on each of ninth-grade TAKS-Reading and TAKS-Math, 10th-grade TAKS-English, TAKS-Social Studies, and TAKS-Science, and 11th-grade TAKS-Math, TAKS-English, TAKS- Social Studies, and TAKS-Science. Table 1 tabulates the results for all of the outcomes analyzed through the THSP evaluation.

Table 1. T-STEM Effect on Ninth-, Tenth-, and Eleventh-Grade Outcomes in 20092010

Table 1 (continued). T-STEM Effect on Ninth-, Tenth-, and Eleventh-Grade Outcomes in 20092010

UNDERLYING EARLY OUTCOMES: VARIATION IN T-STEM ACADEMIES

The expectations for T-STEM academies development, as laid out in the T-STEM blueprint, were multidimensional and ambitious. Evaluation findings during the early stages established that T-STEM academies could not implement the many blueprint elements all at once, and that they deliberately staged how they established different elements, postponing those involving upper-year students for later implementation. Program officers subsequently revised the T-STEM blueprint in 2010 to acknowledge such developmental phases and to provide guidance on priorities during planning, the first year, and the second year of operation. The early phases of T-STEM implementation posed their own trials amid certain successes, and T-STEM blueprint implementation varied in important ways across the schools, which in turn might have accounted for the inconsistent impact on student outcomes.

BLUEPRINT INTERPRETATION AND IMPLEMENTATION

The T-STEM academies began implementing different aspects of the blueprint, began at different readiness levels, and achieved different levels of depth based on their particular organizational contexts and capacities. These interpretations and local adaptations produced T-STEM academies that varied in the extent to which they adhered to the design as expressed in the blueprint. Drawing on both site visit and survey data from the THSP evaluation, we describe below the variation in curriculum and instruction, student support strategies, and partnerships with higher education and business, all key blueprint elements.

Developing an Ambitious Vision of Curriculum and Instruction

The T-STEM blueprint called for an approach to curriculum and instruction that was distinct from traditional approaches, including providing students with accelerated access to STEM content and integrating technology into classroom teaching and learning. The most far-reaching instructional component of the T-STEM blueprint was the vision of project-based learning (PBL). The blueprint required that T-STEM academies organize instructional expectations around problem-based and project-based learning (T-STEM Academy Design Blueprint, 2010, p. 8) and defined PBL as an inquiry-based instructional approach, in a real-world context, where students generate the pathways and products that meet defined, standards-based outcomes . . . (T-STEM Academy Design Blueprint, 2010, p. 40). When implemented well, PBL can infuse a curriculum with both rigor and relevance, challenging students to use their skills in an immersive and meaningful setting (Boaler, 1997, 1998; Gallagher, Stepien, & Rosenthal, 1992; Penuel, Means, & Simkins, 2000).

On the whole, T-STEM teachers implemented PBL more so than their peers at other THSP schools, based on teacher surveys of THSP schools.^{8, 9} Nonetheless, site visit data from spring 2010 revealed that PBL implementation varied substantially, both across and within T-STEM academies. All of the visited T-STEM schools implemented PBL in some form and to some degree, but implementation varied from widespread use of PBL strategies by most of the teachers in a school, to very infrequent use of PBL strategies by just a few teachers. For example, at one school, many teachers implemented long-term, standards-based projects that asked students to apply their knowledge in a real-world setting.¹⁰ At this school, the leadership strongly promoted PBL, teachers received common planning time to develop projects, and all teachers participated in in-depth training in PBL strategies. In contrast, at another school, only some teachers participated in a voluntary one-time PBL training and they struggled to incorporate PBL into their classroom instruction. At this school, a handful of teachers sporadically implemented PBL, but the practice had yet to permeate the instructional norms of the school. Several staff members attributed the low implementation to PBL training that was not sustained or embedded and to inconsistent participation across the faculty. These contrasting cases support earlier findings suggesting that schools that trained the entire staff in PBL strategies and scheduled regular time for planning project-based units had greater success in implementing PBL schoolwide (see Young et al., 2010).

Individual teachers further explained the difficulties in implementing PBL in the classroom. For example, an ELA teacher at one T-STEM school emphasized the transition time necessary for teachers to learn the more student-centered approach entailed in PBL: I understand that it takes two to three years for instructors to feel really comfortable [with PBL], where students are driving the learning, where instructors quietly guide. At another academy, a science teacher was reluctant to try PBL because she felt students did not have the prerequisite skills to access project-based lessons. She reflected, [The students] need to be taught how to work in groups and how to focus, and then they also need some basic coursework in order to get them to the level to where they can use their math and science to solve problems. These two examples point to the central role that students play in their own learning within a project-based curriculum and to the difficulty that some teachers had in shifting from traditional teacher-focused instruction. Overall, then, the following factors accounted for much of the variation in the extent to which instruction at the T-STEM academies reflected the PBL ideal expressed in the blueprint: (1) student readiness and teachers perceptions of student readiness, (2) teachers understanding of and supports to learn PBL, and (3) time to experiment with, adapt, and refine instructional practices to be more student-centered and project-based.

Supporting Student Success

Strategies to support students were an essential characteristic of the T-STEM academies and were expressed in the blueprint in terms of tiered supports such as outreach, early intervention strategies, mentoring, tutoring, counseling, and other supports for academic and socio-emotional growth (T-STEM Academy Design Blueprint, 2010, p. 6), as well as regular advisory periods for all students. Because T-STEM schools fostered a climate of high academic expectations and required students to take more advanced math and science courses, entering studentsmany coming from lower-performing schoolsfaced real risks of not meeting these expectations.

T-STEM academies offered different types of supports to try to mitigate that risk. In addition to the extensive tutoring typical in Texas high schools, T-STEM academies also established student advisories, as required by the blueprint. Advisories provided teachers with dedicated time to support students in small class settings outside of regular courses.¹¹ The advisories as implemented in T-STEM academies differed in purpose and frequency. For example, one school focused advisory on fostering relationships between teachers and students, building character through readings and discussions, and supporting academic success through regular check-ins about courses, homework, grades, and attendance. Another school focused advisory on preparatory skills, such as practicing for the SAT and preparing college materials like resumes, personal statements, and financial aid applications. At T-STEM academies that used advisories less, staff relied on the small-school structure to ensure that each student felt connected to the school community.

Across the majority of T-STEMs, teachers and students reported that the small size of their schools supported student success in and of itself because it facilitated strong relationships. Indeed, T-STEM staff articulated the criticality of all students having teachers who know them as learners and as individuals, in whom the students can confide about the personal worries they bring to school that affect their concentration and engagement. Insofar as all T-STEM academies adhered to a small-school requirement, the role of a small-school community fostering positive teacherstudent relationships was fairly consistent across the academies.

CMO and district did vary in whether and how they prepared younger students for a demanding high school STEM curriculum. Some CMOs and, to a lesser extent, districts, turned their attention to middle schools. One CMO funded under T-STEM served middle and high school students and strove for vertical alignment, particularly in math and science, to help middle school teachers increase the rigor of their courses. In another instance, a district promulgated project-based instructional strategies and provided corresponding training to its middle and elementary school teachers as a strategy to help younger students develop the skills they would need to succeed at the T-STEM academy in grades nine through 12. Other districts, however, placed less emphasis on STEM preparation at the earlier grades. These differences in district and school support strategies for students further contributed to the range and variation in T-STEM academies.

Offering College and Work-based Experiences

In line with T-STEMs college- and career-readiness mission and blueprint specifications, T-STEM academies began developing dual-credit programs (where students earn high school and college credit simultaneously) and work internship opportunities. Because the majority of T-STEM academies included in the evaluation had served only one to two 11th-grade cohorts at the time of data collection, this aspect of the model was generally still under development.

To offer dual-credit courses, the majority of T-STEM academies had IHE partnerships in place; however, fewer than half (44%) of the 11th-grade T-STEM students surveyed reported enrolling in college courses, whether offered on a college campus, online, or at the high school campus. These results reflected the limited college course offerings as the T-STEM academies developed their upper-year programs, as well as student challenges in accessing college courses. Students typically faced difficulties in passing the prerequisite college placement exam. Supports to prepare for that exam differed, with 62% of 11th-grade T-STEM students surveyed reporting receiving assistance. Although certain dual-credit courses do not require passing the placement exam, almost 10% of the 11th-graders in T-STEM schools reported that failing the college placement exam prevented them from taking college credit, the most frequently cited barrier.

During the early implementation years, which coincided with an economic downturn, T-STEM leaders also reported that securing sufficient partnerships with local businesses to provide many students with internships was elusive. T-STEM students were less likely to complete internships compared to other postsecondary supports, with 25% of 11th-grade students reporting participation. Approximately 20% also reported job shadowing or observations at work sites as part of their T-STEM experiences.

The T-STEM academies were clearly still putting in place the college- and career-related opportunities specified in the blueprint during the years of the evaluation. At that early stage of development, T-STEM students experiences with college- and career-related opportunities differed across schools and did not yet meet the standards laid out in the blueprint.

LESSON 1: COMMUNICATING A VISION AND SPECIFYING SCHOOL REFORM REQUIREMENTS MAY BE AN IMPORTANT FIRST STEP, BUT SCHOOLS WILL INTERPRET AND IMPLEMENT KEY ELEMENTS OF A BLUEPRINT IN VERY DIFFERENT WAYS.

For the T-STEM initiative, the school vision and requirements were embodied in the T-STEM blueprint, which served as the guiding design document for the T-STEM academies. Contrary to the image of big-box stores opened according to a blueprint, however, the academies were by no means identical. Rather, the academies enacted the T-STEM blueprint ranging in the depth of implementation, strategies for implementation, and whether the elements were enacted at all.

As discussed, T-STEM academies varied in their attempts to implement key elements of the T-STEM blueprint. They differed in their interpretations of project-based learning in particular, with some academies consistently providing experiential learning opportunities for all students, and other academies used project-based strategies little or not at all. T-STEM academies also implemented student supports in very different ways despite the blueprint requirements. Advisory periods provide the clearest examplesome T-STEM schools focused advisory periods on promoting teacher and student relationships, others focused advisory periods on skill-building and college readiness activities, and other schools did not create formal advisory structures. Finally, T-STEM schools varied in the extent to which they provided college- and career-readiness opportunities, such as internships or college course credit. Ultimately, the role of the blueprint was to communicate a vision of the T-STEM academy and to guide implementation. The T-STEM initiative could not require adherence to the blueprint and needed to demonstrate flexibility in response to varied local contexts.

While variation in school-level implementation is not surprising, the T-STEM scale-up highlights the need for local actors to understand the principles at the core of a particular reform so that adaptations can nonetheless stay true to the intended policy goal (McLaughlin & Mitra, 2001). For example, one T-STEM academy offered advisory only monthly because weekly meetings were too difficult to schedule. However, as the school culture was very collaborative and tight-knit, teachers and students were well aware of who their advisors and advisees were and checked in informally on a daily basis. The goal of creating meaningful relationships among teachers and students to keep students engaged seemed to be accomplished, even though local conditions prevented advisory from being implemented as described in the blueprint.

In part, varying implementation across T-STEM academies was a function of specificity in the blueprint language and the complexity of the practices espoused in the blueprint. Different blueprint elements were articulated with differing levels of precision and concreteness, which in turn influenced implementation. For example, the blueprint included direction to use a lottery system for enrollment when student demand exceeded availability. This policy was relatively simple and the lottery process was commonly understood across the T-STEM schools. However, the requirement that T-STEM academies implement an advisory period was not as clear. While advisory can be broadly understood as a support system for students, its content, frequency, staffing, and philosophy of what supports to provide and how to provide them led to very different approaches, as described previously.

T-STEM academies were also less likely to implement more complex aspects of the blueprint, at least early on in their development. They more easily implemented structural elements of the model (e.g., school size requirements, mandates regarding admissions policy, specific required social support structures like advisory) than instructional components (e.g., using PBL as the primary instructional strategy in classrooms). In large part, the instructional components were complex in terms of what teachers needed to learn, understand, and master in the classroom. Teachers needed to shift to a student-centered approach to teaching, integrate skills mastery within real-world projects, and assess student learning through project work. Moreover, for most schools, creating a school structure that would support PBL was a complex undertaking that entailed, for instance: providing teachers with a safe space to experiment with PBL despite the high-stakes accountability environment in Texas, structuring the day to include time for teachers to work across disciplines, finding or creating a rigorous curriculum through projects, and scheduling to allow students enough time to work on projects during a single period, to name just a few challenges. The complexity of the principles underlying PBL and the actual practices, as well as the complexity of the process to move a whole school in that direction, yielded a wide range in PBL implementation. Although all T-STEM schools assigned their students projects, very few delivered a majority of instruction in a project-based format.

Although the T-STEM blueprint was a necessary description of the essential characteristics of a mature T-STEM, it alone could not communicate what the practices might look like on the ground and within the specific school context. The T-STEM academies interpreted blueprint elements according to their local capacity and understandings, resulting in varying degrees of implementation fidelity to the T-STEM design. The T-STEM initiative used a network strategy to support schools in implementing the T-STEM model more reflective of the blueprint.

T-STEM NETWORK: SUPPORTING IMPLEMENTATION AT SCALE

The successes and challenges regarding the T-STEM implementation described above underscore the schools need for assistance in translating the T-STEM vision into their daily operations. Ideally, such assistance would help each academy understand the T-STEM academy design (i.e., the blueprint) and the plan for implementation within its local context, and address any roadblocks to implementation. As described earlier, aside from the supports provided through their respective districts, CMOs, and/or external support providers, T-STEM academies also received supports from the T-STEM networknamely, from the T-STEM centers and from T-STEM initiative coaches. These supports from within the T-STEM network were important because they provided direct guidance and built school capacity to implement the T-STEM model itself.

To begin, the reach of the T-STEM initiative within the district influenced the kind of support the T-STEM network was able to provide. In some cases, the T-STEM academy was part of a district-wide initiative, with active support and involvement starting with district leadership and reaching broadly throughout the district. In other cases, the T-STEM academy was working largely on its own, whether as a standalone school or a school within a school. The T-STEM network was able to have a larger impact in places where the T-STEM endeavor was part of a district-wide effort, although in many ways the network support was arguably even more critical for academies operating in isolation. For example, in one district, the superintendent was an early champion for a variety of THSP reforms, including a district-wide early college high school initiative and turning one school into a T-STEM academy. In this district, the relationship between THSP and the district grew to be a widespread collaboration, where THSP had significant involvement both at the district level and with individual schools.

For academies operating in relative isolation within their district, the extent to which the school had the latitude to pursue its own agenda versus the extent to which the academy had to follow district requirements also influenced the relationship between the T-STEM network and the academy. On the one hand, the T-STEM network supports could potentially do morethat is, support more extensive reformsat schools with more flexibility to implement programs different from those of the district as a whole. On the other hand, we found that the T-STEM network became a more important resource for schools that had less latitude to operate outside of district programs and strategies. In these cases, the T-STEM network provided valuable guidance on how to implement T-STEM within the district constraints. One school leader described their coach as a sounding board, giving direction, clearing hurdles, running interference for us to do what we need to do. In another instance, the T-STEM coach helped a principal conduct classroom walkthroughs using a consistent protocol to look for elements of high-quality instruction, even though its CMO and the school did not have the same expectations for classroom-based PBL as those described in the blueprint.

Not surprisingly, the extent to which the academys existing school model matched the T-STEM blueprint also influenced the role that the T-STEM network played in supporting that school. For academies following a school model (e.g., from the CMO, district, or an external support provider) that already mapped closely to the core T-STEM blueprint elements, the T-STEM network supports were less critical. Academies whose school model either did not address or differed from certain aspects of the T-STEM blueprint had further to go in terms of T-STEM implementation, and the T-STEM network supports provided important guidance. In such cases, the T-STEM network sometimes also found itself playing a role that tended more toward monitoring and enforcementi.e., ensuring that baseline T-STEM characteristics were implemented at the school. This role had to be balanced with providing constructive feedback and support.

In each case, the T-STEM academy context influenced not only how much support and how much value the T-STEM network was able to provide, but also the kind of work that the T-STEM network was able to do with the academy. In cases where the T-STEM academy had broad support within the district and the latitude to truly implement its own program, and where other existing models were compatible, the T-STEM network was able to focus on core T-STEM principles, like instructional reform toward PBL. In contrast, at academies where contextual and design-based factors limited the T-STEM implementation, the T-STEM network was more likely to support more tangential reformslike establishing a robotics clubthat furthered the T-STEM model without transforming everyday classroom instruction.  

As the T-STEM network matured, the T-STEM centers became an increasingly prominent and valuable support for the academies over the course of the THSP initiative. Although the T-STEM centers and coaches initially provided incidental support for academies, often on an ad-hoc basis, these relationships grew and deepened over time. The centers gradually shifted to building coordinated and collaborative relationships with one another, and cultivating relationships with area partners such as universities and businesses. As centers increasingly collaborated with one another and expanded their outreach to T-STEM academies, the T-STEM network itself strengthened to become a more coordinated statewide network that benefitted T-STEM academies and STEM education more broadly.

LESSON 2: EVEN WITH A BLUEPRINT, TECHNICAL ASSISTANCE IS STILL REQUIRED TO HELP INTERPRET AND ENACT ITWITHIN CONTEXTUAL CONSTRAINTS.

Given multiple influences on the T-STEM academiesfrom district context to competing school modelssimply sharing a blueprint was not enough to scale a model statewide. In the case of the T-STEM initiative, the T-STEM centers and coaches provided necessary capacity building. Their technical assistance helped, to varying extents, to move each school in its growth toward a mature academy. They interpreted and localized the blueprint requirements, and provided guidance to school leaders as they met with various implementation barriers. Although they were a unifying force, this set of technical assistance providers was forced to address challenges unique to each school, its resources and interests, including the work of negotiating the academys vision for itself with the T-STEM definitions and expectations.

The reach and influence of an external network to provide technical assistance, however, was conditioned by the relative latitude the school had to incorporate changes that might have differed from its districts or CMOs priorities and strategies. Thus, for technical assistance providers to support schools in implementation effectively, understanding these constraints and identifying the rapidity with which the school is able to implement any of the core elements will help develop realistic goals and timelines and shape professional development needs, especially on instructional elements that are more complex to understand and change.

As we discuss more in the next section, the districts, CMOs, and external support providers had their own understandings of the blueprint elements; each entity prioritized different elements of the model (e.g., PBL, advisory, business and higher education partnerships) and had different visions for how those elements should operate in practice and what they should accomplish. The network of T-STEM centers and coaches brought a more centralized and unified understanding of the blueprint elements to the initiative, helping to address these differing interpretations on the ground.

PRIORITIES BASED ON T-STEM ACADEMIES PRIMARY AFFILIATIONS

The primary affiliation of the T-STEM academyi.e., the organization upon which an academy depends most for school design and ongoing supportconstituted one of the largest influences on the schools instructional vision, approach, and capacity. The T-STEM academies included in the THSP evaluation had one of three types of primary affiliations: district, CMO, or external support provider. These affiliations emphasized different features of the T-STEM blueprint and differed in their understanding of specific blueprint elements. The T-STEM academies negotiated among blueprint elements and the priorities of their district, CMO, or external support provider in ways that were in potential tension with the T-STEM model, which designed or reinforced key features of their school that were consonant with the blueprint.

The contrast in implementing PBLwidely varied as we discussed earlieroffers a clear example of the role of the academies primary affiliation. For example, T-STEM academies belonging to the New Tech Network, an external support provider, pursued PBL with team-teaching across subject areas, week in and week out. The New Tech school model had already established proof points that demonstrated how schools could organize to provide interdisciplinary team-teachinga standard method for teaching students how to collaborate on project teamsand rubrics and exemplars for teacher-created, content-rich, interdisciplinary projects. T-STEM academies belonging to the New Tech Network had a stronger orientation toward and more consistent daily implementation of PBL than other T-STEM academies because the New Tech T-STEMs actively chose the New Tech vision of PBL and received substantive and deep professional development for teachers to learn how to structure PBL effectively.

Other T-STEM academies followed different expectations for PBL as understood and expressed by their respective district or CMO leaders. For example, at one CMO, all students were required to participate in a science fair that consumed their extracurricular time for about four months of the school year. Although teachers worked with the students after school and on weekends to complete their science fair projects, teachers consistently expressed that they were not expected to follow through with the science fair project during class time and that it was not necessary to explicitly integrate PBL in any other way during regular classes. As a result, STEM instruction at this CMO was very traditional.

Other priorities merged into T-STEM academies because CMOs and districts allocated instructional coaches, had literacy initiatives, or promoted family education strategies at their T-STEM academies, based on their strategic priorities. Because districts and CMO leaders have line authority over any T-STEM academies in their respective systems, falling in line with the broader strategic priorities is understandable and unavoidable. On the ground, the implemented T-STEM model incorporated both the T-STEM blueprint elements and the local efforts, and could look markedly different across T-STEM academies.

Affiliations with districts, CMOs, and external support providers also built capacity at the respective T-STEM academies by providing personnel, resources, and expertise for their priorities (e.g., literacy across the curriculum or team-teaching). In addition to the New Tech example of training teachers in a systematic approach to scaffolding PBL and student collaboration, CMOs with solid replication strategies and experience provided some or many of the start-up supports to new campuses opened under T-STEM, and they often transferred teacher leaders from schools already up and running to launch their new sites. Such influences were not unique to T-STEM, but were integral to the initiatives actual implementation and to the program that students actually experienced.

LESSON 3: DISTRICTS, CMOS, AND EXTERNAL SUPPORT NETWORKS CAN PROVIDE EXPERTISE AND CAPACITY IN SCALING UP RAPIDLY; THEIR INFLUENCE SHAPES THE EXTENT TO WHICH IMPLEMENTATION REFLECTS THE ENVISIONED SCHOOL MODEL.

As the T-STEM scale-up illustrates, leveraging existing resourcesreplication expertise, established school models and educational programming, and relational networkscan accelerate growth plans. The districts, CMOs, and external support providers respective roles arguably allowed the T-STEM program overall to achieve statewide scale more quickly than it otherwise would have by, for example, recruiting individual operators. Leveraging external support providers, districts, and CMOs allowed the T-STEM initiative to open and operate a cohort of T-STEM academies at scale within a target timeframe set by policymakers. The CMOs and external support providers in particular were able to bring school designs and start-up procedures to use in opening new T-STEM academies.

Yet the CMOs had gained experience in replicating their own specific school models, and because they were the dominant influence on their schools, the T-STEM academies they opened were amalgams of the model envisioned in the blueprint and their own school model. To the extent that the school model was fairly well aligned with the blueprint elementse.g., the principles of interdisciplinary learning as applied to math and science subjects, or PBL embedded in the core curriculumthe schools were closer to the ideal T-STEM academy. Indeed, they helped their academies reach, relatively quickly, mature and role model levels of implementation, as defined by the blueprint. Where the school model diverged from the blueprint, leveraging existing CMOs capacity to start new schools required more negotiation and time to bring the school model closer to the T-STEM vision. An initiative leveraging such partnerships could assess and refine how district, CMO, or external support provider priorities reinforce initiative goals and strategies as schools mature and integrate different influences to offer variations on the intended program model.

DRAWING LESSONS FROM T-STEMS EARLY IMPLEMENTATION AT SCALE

The T-STEM initiative represented a very large investment in a statewide scale-up effort, creating 51 inclusive STEM academies over its first five years. While the data described here were collected during early implementation phases for some academies, evaluation results showed a few positive student outcomes, specifically for higher 10th-grade mathematics scores, higher likelihood of passing Algebra I by ninth grade, and higher attendance rates than students in matched comparison schools.

Nonetheless, overall student outcomes were mixed. These mixed findings are, in part, attributable to (1) the variation in how and the extent to which T-STEM academies implemented blueprint elements; (2) the complexity of implementing the ambitious school model, even while the T-STEM network grew increasingly influential; and (3) the relatively strong influences exerted by districts, CMOs, and external support providers that forged different T-STEM models on the ground and experienced by students. From these findings, we draw three lessons regarding this statewide initiative.

"

Lesson 1: Communicating a vision and specifying school reform requirements may be an important first step, but schools will interpret and implement key elements of a blueprint in very different ways.

"

Lesson 2: Even with a blueprint, technical assistance is still required to help interpret and enact itwithin contextual constraints.

"

Lesson 3: Districts, CMOs, and external support networks can provide expertise and capacity in scaling up rapidly; their influence shapes the extent to which implementation reflects the envisioned school model.

The analysis and lessons drawn from the T-STEM initiative provide insight into a unique and ambitious model of statewide school scale-up. While the T-STEM blueprint and T-STEM centers and coaches offered a core vision and supports for reaching that vision, academies varied widely in the elements they prioritized in implementation, their capacity to make the more complex changes associated with instruction, and how they organized key structures such as student supports, dual-credit offerings, and career development.

Among T-STEM academies, replicating many of the structures, such as small school size and application and lottery processes, was generally successful. Yet the T-STEM effort targeted more than just structuresit also aimed to provide a setting for innovative instruction, deep STEM learning, and school cultures fostering student ambition and success. This level of innovation, interaction, and culture was more varied, with some academies demonstrating more progress than others.

While the experiences of the initial T-STEM academies led project officers to collaborate with coaches and many others in the T-STEM network to provide more specificity and clarity in the blueprint, this specificity was not sufficient in facilitating a scale-up effort. The additional elements that moved T-STEM academies from simple structural replication to implementation of the culture and spirit of the model relied on the coaches and technical support providers to develop a shared understanding of the T-STEM vision and elements, and work overtime to communicate this understanding to each T-STEM academy, tailored somewhat to the academys own context. Other influential professional development providers bolstered school staffs knowledge of and skills with PBL by focusing intensely on instruction. These experiences point to the power and necessity of school-by-school capacity building even as an initiative attempts to scale up quickly.

All of these factorsschools initial capacity and understanding, additional structures and resources such as technical assistance and coaching to build school capacity and facilitate change, and the direct influence of districts, CMOs, and external support providersled to adaptations in T-STEM models as implemented. Some adaptations were planned, such as relying on the small school environment to build strong studentteacher relationships rather than using advisory periods. Some emerged unintentionally, such as the variation in how PBL was incorporated into the curriculum. Adaptations in reform designs are necessary simply because student learning, teacher learning, local resources, reform history, and organizational norms all converge to both facilitate and constrain designed reforms. Rather than indicating whether such adaptations are positive or negative, the early years of the T-STEM initiative illustrate the need to attend to multiple perspectives, understandings, and capacities for changea complex negotiation between design and feasibility that is continually in flux as local contexts evolve.

Acknowledgments

A large team supported the research included in this paper. The authors wish to thank Nancy Adelman and Barbara Means, Co-Principal Investigators of the Evaluation of the Texas High School Project; Lauren Cassidy, Reina Fujii, Teresa Lara-Meloy, Christine Padilla, and Kaily Yee at SRI International, and Angela Luck and Rachel Howell at Copia Consulting for their contributions to T-STEM data collection and analysis; and Priyanka Singh at University of Texas-Dallas for supporting the student outcomes analysis. The authors are grateful to the Texas Education Agency (TEA) for funding the Evaluation of the Texas High School Project. The views presented here are solely those of the authors and do not necessarily represent those of TEA.

Notes

1. T‑STEM was one of multiple high school reform initiatives under the Texas High School Project, formed by an alliance of state public agencies and private foundations. The alliance included the Texas Education Agency (TEA), Office of the Governor, Texas Legislature, Texas Higher Education Coordinating Board (THECB), Bill & Melinda Gates Foundation (BMGF), Michael & Susan Dell Foundation, Communities Foundation of Texas (CFT), National Instruments, Wallace Foundation, Greater Texas Foundation, and Meadows Foundation. THSP included the following initiatives: T‑STEM, Early College High School, New School/Charter Schools, and various comprehensive high school reform programs—High Schools That Work, High School Redesign and Restructuring, and High School Redesign, and District Engagement [IS THIS ALL ONE PROGRAM? IF NOT, THE “AND” BEFORE “HIGH SCHOOL” SHOULD BE DELETED].

2. See the third comprehensive annual report of the evaluation of THSP for full methods details (Young et al., 2011).

3. T‑STEM Design Blueprint, Rubric, and Glossary (2010). Updated version available at http://www.tstemblueprint.org/

4. The study period of the THSP evaluation preceded the change to end-of-course exams in the Texas state testing system. At the time of data collection, all students in grades nine through 11 took the TAKS.

5. See the third comprehensive annual report of the THSP evaluation (Young et al., 2011) for analysis details. T‑STEM academies funded to begin as middle schools were not included in the THSP evaluation until they year they began serving ninth-graders.

6. THSP schools were matched within specified ranges on key school-level characteristics affecting student achievement, including grad [GRADE?] span, campus accountability rating, TAKS math and TAKS reading passing rates for the prior year, urbanicity, enrollment, Title I status, and percentage of Black and Hispanic students. Where more than six comparison schools met these criteria, the six schools closest in propensity score to the THSP school were retained as the comparison schools. See Young et al. (2011) for further details.

7. All results statistically significant at p < 0.05 unless otherwise specified.

8. All THSP schools receiving grant funding in 20062007, 20072008, 20082009, or 20092010 were asked to respond to principal, teacher, and student surveys in spring 2010. Survey data pertain only to THSP schools, as non-THSP schools were not included in the survey sample.

9. Level of PBL is a composite of multiple survey items. The mean for T‑STEM teachers is .39 and for teachers at other THSP schools is .29, p < .05, where 1 = Teacher asked students to complete projects over an extended period of time, aligned with state and district content standards, used technology, and addressed real-world problems once or twice a month or more, and 0 = Teacher engaged in these practices a few times this year or less.

10. For example, English, math, and social studies teachers collaborated to design a project that involved the Great Wall of China. In math class, students produced comprehensive measurements of the Wall. In English class, students read literature that connected to the Wall. And in history class, students studied the historical conditions that precipitated the Walls construction.

11. The T‑STEM blueprint defines an advisory as a time “that is regularly scheduled . . . and focuses on personalizing the student experience (builds relationships with students and parents, develops character,

and fosters global literacy)” (THSP, 2010, p. 5).

References

American Institutes for Research/SRI International. (2005). Rigor, relevance, and results in new and conventional high schools. Menlo Park, CA: SRI International.

Boaler, J. (1997). Experiencing school mathematics; Teaching styles, sex, and settings. Buckingham, England: Open University Press.

Boaler, J. (1998). Open and closed mathematics: Student experiences and understandings. Journal for  Research in Mathematics Education, 29(1), 4162.

Coburn, C. (2005). Shaping teacher sensemaking: School leaders and the enactment of reading policy.  Educational Policy, 19(3), 476509.

Gallagher, S. A., Stepien, W. J., & Rosenthal, H. (1992). The effects of problem-based learning on

problem solving. Gifted Child Quarterly, 36(4), 195200.

Kahne, J., Sporte, S., & de la Torre, M. (2006). Small schools on a larger scale: The first three years of the Chicago High School Redesign Initiative. Chicago, IL: Consortium on Chicago School Research.

McLaughlin, M. (1987). Learning from experience: Lessons from policy implementation. Educational Evaluation and Policy Analysis, 9(2), 171178.

McLaughlin, M. W., & Mitra, D. (2001). Theory-based change and change-based theory: Going broader and going deeper. Journal of Educational Change, 2, 301323.

National Research Council. (2005). Rising above the gathering storm. Washington, DC: National

Academies Press.

Penuel, W. R., Means, B., & Simkins, M. B. (2000). The multimedia challenge. Educational Leadership, 58(2), 3438. 

Presidents Council of Advisors in Science and Technology (PCAST). (2010). Prepare and inspire: K12 education in science, technology, engineering, and math (STEM) for Americas future. Washington, DC: White House Office of Science and Technology Policy.

Spillane, J. (2000). Cognition and policy implementation: District policymakers and the reform of

mathematics education. Cognition and Instruction, 18(2), 141179.

T‑STEM Academy Design Blueprint. (2010). Retrieved from http://nt-stem.tamu.edu/Academies/blueprint.pdf

Young, V., Adelman, N., Bier, N., Cassidy, L., Keating, K., Padilla, C., . . . Yee, K. (2010). Evaluation of the Texas High School Project. Second comprehensive annual report. Austin, TX: Texas Education Agency.

Young, V., Adelman, N., Cassidy, L., Goss, K., House, A., Keating, K., . . . Yee, K. (2011). Evaluation of the Texas High School Project. Third comprehensive annual report. Austin, TX: Texas Education Agency.

Young, V., Humphrey, D., Wang, H., Bosetti, K., Cassidy, L., Wechsler, M., . . . Schanzenbach, D. (2009, April). Renaissance Schools Fund-supported schools: Early outcomes, challenges, and opportunities. Menlo Park, CA and Chicago, IL: SRI International and Consortium on Chicago Schools Research.

Cite This Article as: Teachers College Record Volume 118 Number 13, 2016, p. 1-26 https://www.tcrecord.org ID Number: 20601, Date Accessed: 11/27/2021 5:59:54 PM Purchase Reprint Rights for this article or review