Post-secondary Outcomes of Innovative High Schools: The Big Picture Longitudinal Study
by Karen D. Arnold & Georgiana Mihut - 2020
Context: Educational reform efforts have taken the form of different school models intended to reduce educational inequality. Personalized, interest-based schools and academically focused, “No Excuses” schools are two leading small-school designs with sharply contrasting approaches to innovation. Given mixed research findings about the successes and challenges of school reform models in the United States, it is imperative to understand how educational outcomes of students relate to the philosophy and distinguishing characteristics of particular school models such as these. At the same time, evaluating social mobility effects of high school education across educational reform models requires examination of common metrics such as high school graduation rate and college entrance and degree attainment.
Purpose: This study sought to establish whether and how a personalized, interest-based secondary school reform model is associated with graduates’ characteristics and post-secondary outcomes—and to place these findings in relation to student outcomes reported by a leading No Excuses school network.
Setting: Big Picture Learning is a network of innovative small schools that serves primarily low-income and minoritized students through an individualized, relational, real-world-based high school experience. The Big Picture educational model features individualized learning plans connected to extensive internships, independent learning organized around student interests, authentic assessments, and close, informal relationships between students and adults.
Research Design: The Big Picture Longitudinal Study tracked 1900 graduates from six graduating high school classes. Data sources included student and school advisor surveys, National Student Clearinghouse college enrollment data, and interviews with graduates’ former advisors. Published outcomes data for KIPP No Excuses schools provided comparative information. Analyses comprised descriptive statistics of survey data and multivariate regression analyses connecting high school exit data to college outcomes.
Findings: The Big Picture Learning model is extremely successful in meeting its stated goals of fostering positive relationships, helping students discover and pursue their interests, and promoting high school graduation and college entrance. Results for academic subject achievement and college persistence are mixed, however. Big Picture graduates have similar college matriculation rates but somewhat lower six-year graduation rates than alumni from the KIPP No Excuses school network. Alumni from both networks show high rates of college attrition.
Conclusion: When taken alone and in context of other innovative school models, the Big Picture results point to the difficulty of sustaining secondary school gains in the post-high school lives of low-income students and highlight shortcomings of traditional colleges in serving this population.
Big Picture Learning is a network of innovative high schools that has emerged over the past 20 years as a leading example of the small school movement. Big Picture began the network with the mission of reducing social inequality through a fundamental redesign of education that provides opportunities for social mobility and self-realization for low-income youth, primarily urban students of color who aspire to be the first in their families to attend college. Initiated with funding from the Lumina and Irvine foundations, the Big Picture Longitudinal Study (BPLS) tracks 1900 students who were members of six graduating classes in 23 Big Picture schools to study whether and how this personalized, interest-based secondary school approach is associated with improved post-secondary outcomes for low-income students. The study includes data about high school experiences, college plans, and college and career outcomes collected from multiple data sources: student surveys in the final semester of high school and the first fall after graduation, advisor surveys on college readiness for each graduating high school senior, advisor interviews about students career and job status two to three years after high school, and college enrollment status provided by the National Student Clearinghouse StudentTracker. The paper reports on descriptive demographic and high school-era survey findings and presents logistic regression results connecting these baseline measures to subsequent college outcomes.
The study was directed by a university-based researcher with no formal role or previous connection to Big Picture Learning or any of the networks schools.1 The longitudinal research was designed to document outcomes of the Big Picture high school design, to understand the conditions for successful transitions to college and careers, and to inform school improvement.
This paper seeks to identify the elements of the innovative Big Picture Learning approach to high school education for low-income students that are associated with positive outcomes for post-high school life. The focus of the research is how the Big Picture Learning design shapes its students and how ready its graduates are for college and life beyond the Big Picture. Results are framed in a broader view of small school reforms by contrasting the Big Picture Learning model with reported postsecondary outcomes for graduates of a leading No Excuses high school network that serves the same population.
BIG PICTURE LEARNING HIGH SCHOOLS
The Big Picture Longitudinal Study follows six years of graduates from Big Picture Learning schools, a growing high school network that has gained national and international attention for success in graduating urban low-income students of color and working in partnership with students to assure their admission to college (Bradley & Hernández, 2019; Levine, 2002; Littky, 2004; McDonald, 2005). The first Big Picture high school opened in Providence, Rhode Island, in 1996, graduating its first class four years later with a 96% high school graduation rate and 98% college acceptance record (www.bigpicture.org/big-picture-history/). With significant support from the Bill and Melinda Gates Foundation, Big Picture began spreading the design, which now includes more than 60 schools in the United States and associated schools abroad. Most Big Picture Learning schools are public schools in urban school districts; some started from scratch and others converted from existing schools. A minority of schools are charter schools or schools-within-a-school divisions of large high schools. Admission at the schools covered in this study is open; no achievement screening or lottery is required to attend. Big Picture Learning high schools are intentionally small, with an average enrollment of 120 students.
Big Picture Learning high schools are distinguished by several unique characteristics designed to offer students an individualized, relational, real-world-based high school experience that supports them in aspiring to higher education, preparing for the workforce, and gaining admission to college. The advisory is the core of the Big Picture experience in these small schools. Students enter high school in an advisory of 1325 students and remain with the same advisor (teacher) and group of fellow students until graduation. Advisors customize learning activities for each student, established at individualized learning plan meetings held each term with the student, a parent/guardian, and a field placement mentor. Advisors use the learning plans to coordinate each students learning across independent studies, internship projects, andin some schoolsgroup classes. They assist students in implementing their learning plans, including completing projects and preparing assessment presentations. The culture of Big Picture schools is informal and relational: students move freely around the school during most of the day, address their advisors and other adult staff by first name, and participate in discussions and activities that invite them to share their personal lives and outside interests.
This curriculum is personalized to incorporate another hallmark of Big Picture Learning schools: Learning Through Internships (LTIs). LTIs entail an extensive field placement related to each students self-identified area of interest. Spending two full days a week of immersion in real-world employment sites is intended to help students identify their passions, develop workplace proficiency, plan future career goals, and practice academic skills in settings beyond the classroom. Each academic term, students develop and document a learning plan in a series of meetings with their advisor, internship supervisor, and a parent or guardian. This individualized plan maps internship projects and tasks to academic competencies. A few Big Picture schools hold some formal classes, where students learn in groups during in-school days. More commonly, students spend the bulk of the school week working at their internship sites, meeting in one-on-one conferences with advisors, and doing independent work for their individualized projects. Depending on their interests and academic needs, students may take online or community college courses during the school day.
Assessment is in the form of exhibitions, in which students give public presentations on their learning and demonstrate the results of projects that connect their internship learning to academic skills. For exhibitions, students prepare writing and other materials that document their learning. Grades and standardized testing are absent or deemphasized when required by state and local regulations.
Every Big Picture student is expected to take college entrance examinations, apply to at least one post-secondary institution, and complete financial aid applications. With slight variations across schools, 95100% of students are accepted into college with financial aid (Arnold et al., 2009; Washor et al., 2008). The Big Picture Learning model institutionalizes roles for students parents and guardians. Advisors are expected to get to know the families of their advisory inside and outside school. A parent or guardian attends student learning plan meetings and exhibitions, which are arranged to accommodate parents schedules. Parents are also included in the schools extensive college information and admission support activities. In sum, the student-centered Big Picture Learning consists of individualized learning around student interests; extensive experiential learning in internships; assessment by exhibition; and close informal relationships among students, school advisors, workplace internship mentors, peers, and parents.
BIG PICTURE LEARNING IN CONTEXT
American high schools have been the focus of national attention for decades because of low graduation ratesparticularly among students of colorand student performance that lags behind international peers (Bloom & Unterman, 2014). Low-income students are concentrated at high schools in high-poverty areas, and these graduates are less likely than higher-income peers to begin higher education and to obtain a degree when they do enroll (Chetty et al., 2017). Among the main models of education reform aimed at closing these K16 gaps, Big Picture Learning is an example of the small school, highly personalistic approach. This model is substantially different from so-called No Excuses schools, another leading small school reform model that serves the same population with a philosophy centered around intensive academic classroom study, extended instructional time, and heavy student homework requirements (Cheng et al., 2017; Thernstrom & Thernstrom, 2004).
THE SMALL SCHOOL MOVEMENT
Amid widespread calls for education reform, the creation of small schools was a popular approach that received broad support in the early 2000s (Kafka, 2008). The Bill and Melinda Gates Foundation had invested over $1.5 billion by 2006 in a variety of schools nationwide with different approaches to reform. The Gates initiative followed two basic strategies: the creation of brand-new small schools and the conversion of existing schools into smaller entities (Semel & Sadovnik, 2008). Generally speaking, these small schools possess a shared school-design model and belong to an organization or network, such as the Big Picture Learning network, that provides implementation resources as well as professional development and learning (Ravitz, 2010).
Theodore Sizer and Deborah Meier were among the leading education reformers who catalyzed and influenced the modern small school movement by creating smaller, more personal schools in the 1980s, based on the belief that the large, impersonal bureaucratic comprehensive high school was part of the reason for low achievement and high dropout rates in urban schools (Semel & Sadovnik, 2008, p. 1745). Educational reformers Dennis Littky and Elliot Washor founded Big Picture Learning in 1995 from this perspective; in fact, they incubated the idea at Sizers Annenberg Institute for School Reform at Brown University.
In stressing personalization, individualization, and experiential education, this school model deliberately moved away from a standardized, mechanical view of curriculum and toward critical engagement, interactive meaning-making, and self-realization in the context of real-world experiences (Ravitz, 2010, p. 291). Semel and Sadovnik (2008) described the tradition of early 20th century child-centered progressive schools with several elements that are featured in the Big Picture approach, including the role of teacher as facilitator rather than the authoritarian figure from which all knowledge flows (p. 1748), the preference for individual or small group work over formal classroom instruction, flexible physical space, an emphasis on independent study and project work, and a changing curriculum that adjusts to students interests and needs.
SMALL SCHOOL REFORM RESEARCH FINDINGS
Research findings about the successes and challenges of school reform models in the United States have been mixed. Regardless of whether these variable findings about school effects are artifacts of different reform approachesfor example, the differing contexts of new startup schools contrasted with conversion schoolsor the result of different methodologies, it has been difficult for researchers to come to a consensus about the impact of small schools on student outcomes. Some research claimed that small schools have achieved superior outcomes in comparison to larger counterparts by creating a sense of belonging, improving interpersonal relations, strengthening teacher commitment, increasing student participation, developing greater program coherence, and modestly increasing student expectations (Wyse, Keesler, & Schneider, 2008, p. 1881). Small schools purportedly have better attendance and graduation rates, smaller classes, more cohesive academic curricula (Bloom & Unterman, 2012, 2014), and increased college application rates (Darling-Hammond et al., 2002; Wyse et al., 2008, p. 1895). Some of this work has looked specifically at subgroups such as low-income students of color, finding that small school reform can markedly improve graduation rates (Bloom & Unterman, 2014; Darling-Hammond et al., 2002).
However, for every study demonstrating gains, there has been other research suggesting caution and presenting discouraging results from small school environments (Wyse et al., 2008, p. 1894). Mathematics is still the most intractable problem: Gates Foundation small schools students performed worse in mathematics and only slightly better in English and reading than their peers from similar backgrounds who attended larger schools (Shear & Means, 2008; Wyse et al., 2008, p. 1881). Phillips (1997, p. 676) uncovered a link between communitarian values and lowered mathematics achievement: In schools where teacher caring was high, mathematics test scores were low, suggesting that teachers in some schools may be more concerned with maintaining affective relations with students than with imparting skills. This indicator is of particular concern given that mathematics performance serves a gatekeeping function in college degree programs (Douglas & Attewell, 2017). Some researchers also reported conflicting findings about students post-secondary expectations of education, including number and type of colleges to which students had applied (Bloom & Unterman, 2014).
Weighing small schools in the balance, it seems clear that there are some areas in which these reform model schools excel and others in which more progress is needed. Using indicators of school climate as the measure, the data indicate that converting from a large high school to a small school model generally translates into higher expectations, greater personalization, and more respect and responsibility (Shear & Means, 2008, p. 2023). However, moving beyond metrics related to sense of community and considering small schools in terms of their effects on increasing achievement, it would appear that this reform may not be working as effectively for the population for which it was designed (Wyse et al., 2008, p. 1882). Even this conclusion needs to be taken as provisional, however, given the conflicting ways that researchers operationalize achievement and the contradictory results across studies.
The student-centered, experiential reform model of Big Picture Learning schools can be usefully contrasted to the prominent No Excuses school model that serves the same population. No Excuses is the popular label associated with urban charter schools founded on the philosophy that students need intensive teacher- and subject-centered work in traditional college preparation subjects along with highly structured expectations for student behavior (Macey et al., 2009). These academically focused schools typically expand instructional time through a longer school day and school year. Other features include frequent testing as part of data-driven instruction, an emphasis on hard work, and intensive teacher selection and development processes (Macey et al., 2009). The Knowledge Is Power Program (KIPP) school network is the best-known and best-researched example of the No Excuses approach. Research results on KIPP schools show that attendance at KIPP schools is associated with higher mathematics and English Language Arts scores (Tuttle et al., 2015; U.S. Department of Education, 2018) and with high levels of college matriculation (Tuttle et al., 2015). However, critics have questioned whether the combination of heavy workload, authoritarian role of teachers, and strict disciplinary practices at No Excuses schools are overly harsh and whether graduates are prepared to be self-directed in college (Ellison, 2012; Golann, 2015; Goodman, 2013; Lack, 2009). And although KIPP graduates far outperform general public-school demographic peers in college matriculation and graduation, less than half of their graduates earn a bachelors degree, according to the KIPP networks own reports (KIPP Foundation, 2011).
Most research about school reform effects concerns high school graduation rates and academic achievement. When included at all, college outcomes are typically restricted to matriculation rather than persistence and degree attainment. For instance, Martinez and Klopott, (2005) reviewed the literature on school reform, college access, and the predictors of college-going behavior but not degree attainment. (The review excluded school networks.) They concluded that the strongest predictors of college-going behavior across the designs were high schools academic rigor and strong social and academic support. The Big Picture Learning and KIPP network philosophies and designs vary sharply in their approach to these elements. It seems imperative, given the tangle in the empirical research, that innovative schools like Big Picture Learning be evaluated on effectiveness in achieving their own expressed goals. At the same time, evaluating social mobility effects of high school education across educational reform models requires examination of common metrics such as high school graduation rate and college entrance and degree attainment.
The Big Picture Longitudinal Study triangulated multiple longitudinal data sources to trace student outcomes. Table 1 shows the study instruments and their timing: web-based surveys of students and advisors designed for the project, student college enrollment data available through the National Student Clearinghouse (NSC) Student Tracker, and alumni information provided by Big Picture Longitudinal Study adults two to three years after high school graduation.
Between 43% and 94% of Big Picture Longitudinal Study graduates from the Classes of 2006 to 2011 from 23 schools (n=1,916) completed baseline high school exit surveys designed for the study about their high school experiences, college admissions, and post-secondary plans in the month before graduating from their high school. The survey instruments included a web-based Student Transition Survey and an Advisor Survey that was given to students and advisors, respectively, in the month before high school graduation. Seniors advisors provided surveys about each graduating student (n=1,830) in roughly the same percentages (54%90% response rate). The advisor surveys asked advisors to assess each of their students level of school engagement, relationships with adult and peers, college preparedness, personal development, and parent/guardian involvement in high school and postsecondary planning. The First Fall Update surveyed new alumni (n=695), with another web-based survey in the October following high school graduation. This First Fall Update survey covered students post-secondary activities and college experiences but with response rates that were significantly lower than the baseline exit surveys (20%33%). A single attempt at a planned all-alumni survey in 2007 yielded fewer than 50 responses, so this data collection was discontinued. Instead, study data about college outcomes come from National Student Clearinghouse (NSC) StudentTracker enrollment data reported by colleges. College Enrollment Tracker data were obtained from the NSC in the winter of 2013 to reflect student enrollment patterns at that point in time.2 More than 3,600 U.S. higher education institutions report their enrollments to the NSC, including community colleges and certificate programs that cover approximately 98% of U.S. post-secondary students (NSC, 2018).
Data about personal, vocational, and civic outcomes of all alumni, including those not enrolled in college, were collected through Big Picture Learning staff members who were identified by school principals as likely to be in touch with their former students two to three years after high school. In most cases, these adults were the students advisors who had worked with the alumni in the Big Picture high school advisories. In a few cases, the identified connector was a college counselor or school administrator who kept track of this information for the school. Doctoral students conducted half-hour phone interviews in which they asked these connectors about every student from their advisory who had completed a baseline survey as high school seniors for the Big Picture Longitudinal Study. Interviewers asked about each graduates educational, jobs, personal, and civic activities since high school. They also invited connectors to provide context for students post-high school academic and personal outcomes in light of that students Big Picture experiences and personal circumstances. The 32 connectors from the Big Picture classes of 20062011 who participated in this data collection were able to give first- or second-hand information about the post-college educational, vocational, or personal status of 95% (n=918) of their former students. The six-person research team coded the interview transcripts with a numerical coding rubric that demonstrated robust interrater reliability (>80%).
Study participants are members of the Classes of 2006 to 2011 from 23 Big Picture Schools. All schools were included that had been open for at least four years by the time of data collection. The majority of the 1,916 respondents are classified as low-income, as indicated by eligibility for free/reduced lunch during high school (62%74% across schools and graduation cohorts); 18% reported special needs qualifying them for Individualized Education Plans. Students of color made up 75% of the sample, and 56% are native speakers of a language other than English, including an unknown number of undocumented immigrants. Eighty percent of the longitudinal study participants would be the first in their family to earn a college degree. A majority of student participants across all years are female (56%), resulting in an overrepresentation of women among survey respondents. According to their advisors, the fathers of 36% of BPLS students are absent from their students lives. In high school, about 5% of students reported caring for dependent children, although the Student Transition Survey did not ask directly if students were parents.
Big Picture Learning students come from communities with high levels of academic underachievement, geographic transition, and high school dropout (Bailey & Dynarski, 2011; Walpole, 2007). There are no academic selection criteria to enter a Big Picture high school. Some motivated students are attracted to the educational approach, but many others come to Big Picture schools after being disengaged or struggling academically in their local (traditional) schools (Frishman, 2014, personal communication). These students resemble their demographic peers school mobility to a certain extent: a quarter of the graduating seniors in the study had spent less than four years in their Big Picture high school. However, 12% of seniors had remained in the school for more than four years. Among many other features of the Big Picture Learning design, the willingness to continue working with students until they were ready to graduate contributes to a graduation rate of 95% across the network (www.bigpicture.org/schools/).3
Does the Big Picture Learning approach work? The Big Picture Longitudinal Study attempted to answer this question in two ways: 1) by examining the fit between Big Picture school distinguishers and accounts by students and advisors and 2) by assessing the outcomes of Big Picture graduates in college and careers. These two approaches enable an evaluation of the design both in light of the networks own goals and in terms of the goal of increasing college access and success among low-income students.4
FIT WITH BIG PICTURE LEARNING DISTINGUISHERS
Big Picture Learning high schools feature individualized learning plans that engage students through exploring and following their own interests and passions. As described earlier, distinguishers of the design also include strong, sustained relationships with adults, real-world engagement through extensive internships (LTIs), and authentic assessment in the form of products and oral exhibitions of learning. Finally, Big Picture schools aim to prepare all students for lives of self-realization and social mobility, including readiness for college and careers.
Seniors responses on the Student Transition Survey were a close match with Big Picture Learnings student-centered curricular characteristics and learning goals. Specifically, students identified LTIs, advisors, exhibitions, assistance with the college application process, and the opportunity to take college courses as their most valuable preparation for the future. Students and advisors reported that cultural characteristics of Big Picture schoolssuch as a supportive community, the expectation that all students apply to college, and encouragement to explore interests and develop personal qualitieswere highly significant in preparing them for the future. Students and advisors testified to the value of independent studies, networking, and relational skill building.
Real-world Learning and Self-knowledge
Across graduation years, seniors ranked What high school taught best in this order: 1) knowing my own strengths and weaknesses, 2) naming my own interests and passions, 3) preparation for success in college, 4) public speaking, 5) learning on my own, 6) being tolerant, and 7) knowing how to make choices and decisions.
Advisors ranked each students ability to name and follow [his or her own] passions as the single greatest strength across their students, indicating that this quality was strong for 86% of their students. Students concurred, with 97% reporting that their Big Picture high school did an excellent job of enabling them to be able to name and follow my passions. Self-knowledge and the ability to define and follow ones interests appear to be strongly realized components of the Big Picture Learning philosophy.
Even more important than school-based self-reflection projects, the ability to name and follow ones passions is cultivated in the process of deciding upon, carrying out, and publicly presenting the products of student Learning Through Internships experiences. The semi-weekly LTIs came up repeatedly in survey questions as the top or second most important preparation for the future. Students mentioned the skills and attitudes they learned, the interests they discovered, and the networks they built. Students attributed a variety of skills to their LTI, including navigating and succeeding in a professional environment, interacting and working with adults, being independent and responsible for their work, and gaining hard and soft skills specific to jobs in their interest areas.
Along with real-world engagement, study findings demonstrated strong relationships between students and adults, the Big Picture distinguisher associated with the advisory system and related to the schools small size and relational culture. When asked how successful various aspects of the high school experience had been for preparing them for life after graduation, students placed their advisor at the top of the list (Internship (LTI) experiences and LTI mentor were rated as the second and third most important factors, respectively). Eighty-one percent of the students said they had another adult, other than a parent or guardian, who is supportive of you, and who you can turn to. The most-mentioned person in this category was the students advisor. Students reported in the Student Transition Survey that their advisors were the most important non-parental influence in their decision to go to college. Perhaps most tellingly, students stayed in touch with their advisors. Eighty-five percent of alumni survey respondents had been in in touch with their advisors in the five months after they graduated, and 20% had been in contact at least six times. Connector Study data show that advisors were able to give information about 95% of their former advisees two years after graduation (Arnold et al., 2016).
The relationships in Big Picture Learning schools go beyond advisor and student. Besides frequent mention of the LTI mentors, students also cited principals, internship coordinators, and staff members as influential adults and supporters. Advisors reported that 90% of their students had multiple supportive relationships with adults and that 87% had multiple supportive relationships with peers.
High average student response ratings on most measures pointed to strong parent/guardian involvement in their students education. Across study cohorts, 77% of students said their parent or guardian was at the school at least once a month and 89% said their parent and advisor communicated once a month or more. Perhaps influenced by their parents systematic inclusion in substantive school activities, more than 90% of seniors said they talked with their parents about their school progress, post-high school plans, career goals, and future aspirations.
Advisors reported that their students each had at least one parent or guardian who was somewhat involved (48%) or extremely involved (40%) as an active member of the students learning team, for instance by active participation in learning plan meetings and exhibitions, attending college preparation events, or communicating with them about their student. Overall, parents were less involved in college planning activities, however. The majority of parents were involved in their students college search and admission tasks. According to students, however, a sizeable minority of parents were not at all involved in helping students research colleges (31%), making sure the student met application and financial aid deadlines (28%), or actively participating in filling out financial aid applications (23%). Only about half of parents/guardians were knowledgeable enough about the college admission process to be of practical assistance to their students, according to advisors. This suggests that between one quarter and one half of students relied on their Big Picture school as their primary or only support in navigating the college search, application, and financial aid process.
Independence, Resourcefulness, and Personal Skills
Students and advisors indicated that experiences of seeking and holding internships and presenting exhibitions resulted in significant personal skills development. Students reported that schools were particularly effective in developing public speaking skills and the ability to interact with adults. In general, students were highly confident in their schools contributions to their personal development, oral communication, interpersonal skills, and ability to interact with those from diverse backgrounds. Students also credited Big Picture schools with preparing them well for work and life after high school.
Advisors rated their graduating seniors as well prepared in most areas of personal development, especially in terms of the ability to articulate and pursue their interests. They rated their students more variably in organizational skills, motivation, and time management skills, which carry implications for students success in college. Advisors frequently expressed concerns about providing the right amount of support to help their students succeed while allowing them to become self-reliant enough to persist in college without the individualized accountability and wrap-around support provided by the advisory system.
Although Big Picture high schools do not measure success solely in terms of college acceptance, they do set a strong college-going culture, including the requirement that all students take college entrance examinations and complete admission and financial aid applications. Senior year exhibitions feature post-secondary planning. School staff members take students on college visits, invite alumni and other guests to schools to discuss college, and provide extensive support to parents in admission and financial aid processes. College-related outcomes of the Big Picture Learning design can be evaluated through the longitudinal study in terms of academic and personal readiness for college, post-secondary aspirations, and college attendance and persistence.
In keeping with their high school immersion in independent interest-based study and their practice in exhibitions, students felt particularly well prepared in the areas of public speaking and learning on their own (Table 2). Self-assessment in writing, critical thinking, and reading were more variable, but very few students said they were poorly prepared in these areas. However, only 29% of study participants rated their schools as excellent in mathematics preparation, and one in five seniors rated this area as poor. Like the students themselves, advisors were confident of their seniors speaking abilities, more mixed in their assessment of student writing, and concerned about mathematics proficiency. College readiness in science emerged as an area of weakness for a substantial group of graduating students, according to advisors. Table 3 shows the aggregated advisor results across the study years.
College Courses During High School
Advisors helped students enroll in community college courses to fulfill content requirements for their individual learning plans. Encouraged at all of the schools, community college coursework was particularly convenient for students whose Big Picture high schools were nearby or even adjacent to community college campuses. Approximately three-quarters of survey respondents (78%) took college courses while in high school. Of students who enrolled in college courses while in high school, 46% took three or more courses.
College Aspirations and Enrollment Plans
Like low-income students nationally (Berzin, 2010; Lee, Hill, & Hawkins, 2012), Big Picture students held high college aspirations. As graduating seniors, at least 80% of each cohort reported that they expected to attain a bachelors or advanced degree. Expressed enrollment plans are a more realistic metric, especially given that the longitudinal study participants were high school seniors who had each completed applications and been accepted to at least one college. In their last semester of high school, 8086% of study respondents reported that they expected to attend college in the fall following graduation. Among the remaining students, half planned eventual enrollment. (Actual college enrollment patterns are detailed later.)
Nearly half (44%) of seniors decided to attend college while at their Big Picture high school. Even though the remaining students, before entering high school, said they intended to go to college, close to two-thirds of the entire group (71%) reported that their Big Picture experiences influenced their decisions to go to college. Students understanding of the career benefits of a college degree was the most influential criterion in the decision to attend college, reflecting the highly vocational emphasis of most first-generation students (Chen & Carroll, 2005). The next most frequently cited reasons included access to college and admissions information, and family and advisor encouragement. Other influences varied but primarily reflected family relationships, such as a perceived family obligation to become the first to attend college or to serve as a role model to younger siblings.
The availability of a particular career-related program or major, the type of institution (2-year or 4-year), and the institutions distance from home were students top priorities in choosing colleges. These considerations remained consistent over study cohorts. Only 27% of students identified cost as one of their top three considerations. Even though institutional graduation rate and campus climate strongly influence the likelihood that a student will persist in college until graduating (Titus, 2004), very few seniors included aspects of campus graduation rates or institutional diversity among their top three considerations in choosing a college.
Advisors agreed that high school seniors were highly committed to beginning college, saying that half of their students were extremely strongly committed and 30% were somewhat strongly committed to beginning college, earning a degree, and pursuing a particular interest area within higher education. This assessment matched students responses: half of students reported that they were absolutely certain they would finish college. If they left college, the remaining students reported, it would be because it cost more than their family could afford (20%) or to accept a good job (14%). Less than 5% of students reported that they might need to leave college because of insufficient academic ability (4%) or insufficient reading or study skills (2%). The small group of graduating seniors who were not planning immediate enrollment was more concerned about their academic preparation for college. The top three reasons this group reported for delaying or planning never to attend college were: Cant afford it (40%), Do not feel academically prepared (26%), and Want to make money (23%).
CAREER PLANS AND OUTCOMES
The majority of students who were not planning to attend college in the fall after high school graduation1520% of their cohortintended to pursue employment. Four percent of seniors planned to enter the military, and 79% said they were headed into volunteer programs, training/vocational programs, or internships. It is important to note that nearly all of these students reported that they planned eventual college enrollment.
Vocational and personal outcomes for Big Picture alumni were gathered primarily from the Connector Study5 in which advisors were interviewed by phone about their former advisory members (Arnold et al., 2016). As described, connectors were able to provide interviewers with information about 95% of their former students, including graduates who were conventionally successful and those who were struggling or not in college. In addition to yielding more information than alumni response rates provided, a significant benefit of this approach was the ability to contextualize student success. For instance, a connector characterized a young father who was not in college as a significant success because this graduate had defied his earlier path by staying alive and withdrawing from gang membership. Connectors deep histories with the students allowed them to provide perspectives and commentaries that were unlikely to emerge from surveying only the students.
Connector Study data show that at least 19% of seniors in four class cohorts did not enroll in college but pursued other post-secondary plans. These students trained as EMTs and Certified Nursing Assistants (CNA), joined Year Up and City Year, and enrolled in vocational or technical schools. Over half of all college-going and non-attending graduates (66%) were working at paid jobs. Some students without career-related paid jobs or post-secondary credentials were pursuing side businesses or involvement in arts and civic domains. Connectors pointed out that many students were following their interests and pursuing their passions after high school, whether or not they attended college.
According to connectors, internship experience that students gained through the LTI component of the Big Picture design provided many non-college attendees with connections and skills that they applied to future careers. Among this group were solid career achievers whose internships led them to well-paying jobs that did not require college degrees. Advisors reported that at least 46% of the alumni were studying or working in fields related to the internships they had at Big Picture schools. For example, one student who interned as a veterinary technician during her senior year of high school was still working at the same office. Though she had dropped out of the college where she was enrolled in a pre-vet program, she had held a steady position as a vet tech for five years. Her advisor insisted, The skills from the LTI got her the job, not her studying in college.
Connectors were able to contextualize the achievement levels of students who never enrolled in college by recalling their progress: I got her as a ninth grader with a second-grade reading and math level but I consider her one of the big successes because she did graduate [high school] in four years and when she graduated I also got her to finish a CNA program, and she did pass a state exam. Connectors reminded interviewers that it was an achievement to get some students to high school graduation, including many of the fifth-year seniors. Big Pictures high graduation rates produce alumni who might have dropped out of other schools. Former advisors counted success in terms of where students began and did not assume that college was right for everyone. In this light, a student with autism spectrum disorder who became a forest ranger, a former gang member who was gainfully employed, and an involved father are appropriately considered success stories according to Big Picture Learning goals, though neither ever enrolled in college.
In contrast to descriptive advisor and student survey data and connector interviews, college outcomes results were obtained by statistical analysis of integrated dataset of surveys and college outcome data. Surveys collected between 2006 and 2011 through the High School Exit Surveys and the Advisor Surveys are included as baseline measures (n=2057). The college outcome variables used as part of this study were extracted from the National Student Clearinghouse (NSC) Student Tracker data reported by postsecondary institutions in 2013 (n=1037) and in 2015 (n=604). The two sets of NSC data were integrated at the student level using student identification numbers (n=1266). Multiple college outcome variables, listed in Table 6, were used as part of the regression analyses employed as part of this research.
Nearly all U.S. higher education institutions report their enrollments to the NSC, including community colleges and certificate programs. However, NCS does not directly report college outcome variables. As such, multiple computations were performed on the original NSC data to generate reliable outcome variables. Most significantly, creating the persistence variables required multiple data transformations. This variable was calculated by creating categorical variables that estimated the term in which a student withdrew from college, using the number of days in college reported by NSC as a proxy. A student who completed two consecutive terms of at least 100 days each was considered to have completed their first year. A student who started a third consecutive term after completing two consecutive terms of at least 100 days was considered to have persisted to their second year.
Factor analyses procedures were used to compute select independent variables. The factor analysis procedures were conceptually driven, and included all the survey items measuring the same constructs. For example, all items that measured student self-assessment were included in one varimax rotation factor analysis. This analysis yielded two distinct factors: (1) student assessment of personal characteristics (explaining 32% of variance in self-assessment indicators) and (2) student self-assessment of academic characteristics (explaining 18% of variance in self-assessment indicators). Reliability analyses were conducted on all emerging factors, with Cronbach Alpha values between α = .66 and α = .935. The independent variables employed as part of this research, their definitions, associated descriptive statistics, as well as construct reliability information (measured using Cronbach Alpha) are included in Table 6.
The peril of longitudinal studies is missing data (Hedeker & Gibbons, 2006; Thompson & Holland, 2003; Young et al., 2006). For each predictor variable, data are missing for between 18.5%–53.2% of cases. For each outcome variable, data are missing for 38.5% of cases. The data are not missing completely at random. For example, the outcome variable data are missing by high schools and years, due to school record keeping. Within the survey instruments, data are missing at random. For the purpose of the logistic regressions performed as part of this research, unconditional mean substitution has been performed for predictor variables. No data imputations were performed on the outcome variables.
Graduates of the Big Picture network enroll in college in high numbers, but they register mixed success in completing key college milestones. While 87.68% of students in the BPLS for whom NSC data is available enrolled in college, only 53.55% persisted to the second year, and only 36.02% completed their sophomore year. Table 4 includes a breakdown of college persistence rates of students in the Big Picture Longitudinal Study.
As Table 5 shows, college enrollment rates for Big Picture students in the focal cohorts are similar toor higher thanfigures at the leading No Excuses school network (Big Picture enrollment 88%; KIPP 78%). Notably, the 6-year college graduation rate of Big Picture students is twice as high as the national average for peer demographic groups from the lowest income group. However, a smaller fraction of students in the BPLS graduate from college than students at KIPP high schools (Big Picture 6-year graduation rate 29%; KIPP 36%). Even fewer Big Picture students graduate after four years (16%) or five years (20%).
*Graduation rates are six years post-high school (22% of Big Picture student sample). Big Picture 4-year graduation rate is 16%; 5-year graduation rate is 20%). See KIPP (2017) and NCES (2016, 2017a, 2017b) for full references. See note 6 for definitions of the measure pertaining to low income students.
Given the dichotomous nature of the outcome variables, logistic regressions were performed to test the relation between high-school related predictor variables and college outcomes. Logistic regression analysis assumptions were tested prior to completing the regression procedures. Most significantly, Variance Inflation Factor (VIF) was used to test for multicollinearity among the predictor variables. Because student self-perception variables correlate between themselves (VIF > 10, tolerance < .2), we maintain only one of the student self-perception variables in the final regression model.
The effects of demographic variables, parental involvement in high school and college, advisor assessment of academic and personal aptitudes of students, as well as students self-assessment of academic characteristics were used as predictors in a model aiming to test what explains students enrollment in college (Table 8), completion of first year of college (Table 9), persistence to second year (Table 10), and graduation from college (Table 11). Regression tables 8-11 appear at the end of the article.
Results of the binary logistic regression indicated that there was a significant association only between ESL status and college enrollment (χ2(10) = 35.473, p < .001), with ELL students more likely to enroll in college. It is notable that students whose first language was not English were more likely than native English speakers to attend college. Similarly noteworthy is the finding that graduates’ college outcomes were the same across racial groups. Advisor assessment of students’ personal characteristics and sex predicted completion of first year in college (χ2(10) = 21.422, p < .05), with females more likely to complete their first year in college. The advisor assessment of students’ personal characteristics was the only variable to act as a statistically significant predictor of persistence to second year (χ2(10) = 22.821, p < .05). Last, the logistic analysis suggested a significant relation between sex, parental involvement in high school, and parental involvement in college on one hand and graduation from college on the other hand (χ2(10) = 28.292, p < .005). All regression models, while statistically significant, explain a small fraction of variability in the outcome variables (Negelkerke R2 <0.051). Regression tables are included at the end of the article. This indicates that high-school related variables collected throughout the Big Picture Longitudinal study were generally unsuccessful at predicting college outcomes. It is likely that high-school related variables do little to capture the complex ecology faced by the students of Big Picture. Factors outside college readiness and academic skills apparently significantly affect their college outcomes.
The longitudinal analysis on college outcomes has a number of limitations. First, NSC data is missing for 38.5% of cases. The data are missing for several schools in the BPLS, thus raising difficulties for generalizations to the whole network. Second, the NSC data has a number of limitations, including underrepresentation of community colleges, which likely contributes to the underestimation of college participation among BPLS graduates. The study includes the cohorts between 2006 and 2011. However, for most students NSC data is available only up to 2013, meaning that a significant proportion of respondents did not meet the 6-year mark for graduation from college. This poses reliability challenges for the college graduation outcome variable. Second, NSC data poses challenges for the calculation of persistence variables due to issues such as varying terms across institutions in the dataset, double enrollment for students, and data collection errors. Some of these issues may impact the reliability of the persistence data reported in this paper. Last, self-reported data has a limited ability to reliably measure the constructs of interest. While reported data from advisors mitigates some of these concerns, Big Picture schools do not collect standardized measure of college readiness that allow for comparisons across high schools.
The Big Picture Longitudinal Study was designed to discover what happens to students after graduating from Big Picture Learning network high schools and how their post-graduate lives were influenced by their innovative high school education. Big Picture schools differ sharply from nearly all public and private high schools in the United States. Deliberately intended to be a disruptive innovation to urban secondary education (Christensen et al., 2008), these high schools are not organized around prescribed, subject-specific classes with shifting groups of peers, different teachers, and traditional testing. Instead, Big Picture Learning offers small schools featuring advisories, an individualized interest-based curriculum, extensive internships, and authentic assessment. Does this radical approach work? By what criteria should it be judged?
A note of caution needs to accompany the answer to these questions. The longitudinal study cannot definitively answer the question of how Big Picture student outcomes relate to high school program design. From a methodological standpoint, the impossibility of a randomized control/treatment experimental design prevents causal conclusions. Much of the study information is self-reported. Low response rates prohibited longitudinal data collection from representative groups of older alumni. College graduation rates continue to change as the sample cohort gets older. At the point represented by NSC data, only two of the six represented classes had been out of high school for at least the typical six-year time span for measuring degree attainment. The extensive connector information about former students is second-hand. Finally, generalizations across the network need to be interpreted cautiously because not all Big Picture schools employ exactly the same practices and structures. In fact, the Big Picture Learning design is deliberately flexible, encouraging constant experimentation and school variability according to local practices and state regulations.
These methodological shortfalls are common in studying innovative school networks. Study designs demonstrating causation are difficult or impossible in schools without lottery systems or with small enrollments. This study could have been improved by better school record keeping, school-based integration of data sets, and routine collection by all schools of National Student Clearinghouse data. These practices are recommended for any school network; however, resources and capacity for conducting longitudinal studies is a major challenge for decentralized small school networks without permanent research staffs. Studying multiple cohorts is one way that small-school networks can increase the reliability and validity of results.
Despite these limitations, study findings indicate clear patterns across six student cohorts in the relationship between a Big Picture education and students readiness for successful adult lives (Table 7). From the establishment of the first Big Picture high school in 1996, founders Dennis Littky and Eliot Washor defined student success in terms of adult self-fulfillment, meaningful work, financial security and upward mobility, healthy relationships, and civic engagement (Levine, 2002; Littky, 2004). The Big Picture Learning school network considers high school graduation and college degrees as the means to these ends, rather than the final goal. Littky and Washor invited an outside researcher to conduct a longitudinal study of alumni precisely because their long-term vision of high school outcomes required assessment of graduates adult lives.7
CRITERIA FOR SUCCESS
The correspondence between Big Picture Learning distinguishers and the outcomes reported by students and advisors points to Big Pictures high degree of success in facilitating student engagement, social capital, personal and vocational development, and high school completion among low-income, urban students. The innovative aspects of the Big Picture Learning design relate directly to its outcomes. The cornerstone philosophical tenets of personalization, interest-based real-world relevance, close relationships, and preparation for college and careers emerge as the most important influences on students. Relationships and relevance are two pillars of the Big Picture model that study findings indicate are extremely successful. These school distinguishers continue to function into early adulthood. The elements of their Big Picture education that study respondents identify as best preparing students for the future are a close match with the large literature on positive youth development (Damon, 2004; Lerner et al., 2005).
The second central criterion for success is the measure of graduates college and career attainment. The Big Picture high school experience appears to have a strong positive influence on students aspirations for higher education. Nearly all of the students begin college, either immediately after high school graduation or after short delays. In comparison with their demographic peers nationally, Big Picture Learning alumni attend college in significantly higher percentages. However, they graduate at lower rates than the overall U.S. average and at somewhat lower rates than published results from the KIPP No Excuses model school network.
Study results suggest that high school factors influencing college success are primarily academic. Both advisors and students identified weakness in mathematics and science preparation, a particularly worrisome finding given the well-known connection between post-secondary remedial coursework and college attrition (Long & Boatman, 2013). Personal qualities associated with college success, such as organization, time management, and independent learning skills are also necessary for college success (Conley, 2005, 2010). These were generally but not uniformly positive for Big Picture learning graduates.
Both the Big Picture Learning (95%) and KIPP (92%) networks have higher high school graduation rates than the national average for all students (84%) and the national average for economically disadvantaged students (77%) (KIPP, 2017; NCES, 2017a). One of the possible interpretations for why small schools in general achieve these results is through the framework of the Hawthorne effect (Adair, 1984; McCambridge et al., 2014; McCarney et al., 2007). Traditionally, the Hawthorne effect or the observer effect is understood as the change of behavior by observed participants due to their awareness of being observed or researched. The close attention that teachers and staff give students in small schools may act in a similar way, and permanently incentivize students to perform their best. There are no studies that confirm the sustained persistence of the Hawthorne effect over a long stretch of time, and as such the effect is unlikely to explain the full effect of the small school movement on high-school achievement. Nor does the Hawthorne effect explain why there might be different outcomes across small school models.
In sum, the Big Picture Longitudinal Study suggests that the Big Picture Learning design works extremely well in achieving its goals for students high school graduation and for student development in personal, interpersonal, and vocational development. On the criteria of college academic readiness and degree attainment, the outcomes are mixed.
Interpreting College Outcomes
High school contexts. As described, Big Pictures highly student-centered approach has resulted in modest college graduation rates. College completion for the network lags behind published rates for No Excuses schools. And even the best-documented No Excuses model, the KIPP school network, has not managed to exceed the average of high-income students for college completion (KIPP, 2011; 2017). Tensions exist for both kinds of schools in balancing what students want to know with what colleges require them to know. The two types of schools also face tensions in balancing student self-direction with organizational compliance. Big Picture Learning and No Excuses schools have positioned themselves quite differently in regard to these tensions.
Big Picture has responded to study findings about shortfalls in college readiness and completion in two ways. First, the network has put considerable effort and resources into academic skills development in ways that fit its educational model. Secondand in keeping with national trendsBig Picture has moved away from a college for all message and is actively promoting a joint college and career readiness approach for all students. (Andrew Frishman, 2017, personal communication; Symonds et al., 2011). School networks with similar student-centered philosophies can adopt these strategies. However, it is unclear whether high schools that serve low-income students can simultaneously center education around student interests, offer an experiential curriculum that is relevant to their lives, and ensure advanced proficiency in core college subjects.
Non-school contexts. High schools can make progress in identifying and resolving these tensions. However, students still experience the effects of contexts outside of the school in the form of poor prior schooling, family unfamiliarity with higher education, and problems associated with poverty, racism, and linguistic background. The majority of advisor comments about problematic personal readiness for college frequently referred to difficult circumstances in students lives outside of school.
Becoming the first in the family to earn a college degree requires access to financial and emotional support, faith in ones ability to succeed, and the knowledge and ability to negotiate the bureaucratic and academic tasks of college enrollment and persistence. The careful alignment of these factors provided by the Big Picture high school can break down under the weight of students post-high school immediate settings (Castleman et al., 2012). During students high school years, the small size, advisory structure, and required family involvement of Big Picture schools optimizes the chance that they will have at least one adult who knows them deeply over a period of years. While in high school, many Big Picture students benefit from the time and effort on the part of staff to bring the many factors in their surroundings into balance and congruence. This type of support, however, is generally unavailable in settings beyond high school.
Big Picture students experience challenges emanating from non-school institutions and structures. Financial aid policies and the timing of college loan applications in the summer, for instance, disadvantage students with few family financial resources and limited knowledge of complicated banking processes (Castleman & Page, 2014). Because these students come from marginalized social groups, their expectations are formed in light of messages from the larger culture about the likelihood of success for people like them. In short, even the best high school cannot singlehandedly overcome the conditions that produce and sustain systems of social inequality. In this light, Big Picture Learning is successful as measured by graduates extremely high college-going rates and their dogged, if often inefficient, persistence toward degrees.
The U.S. higher education system. A different interpretation for the varied higher education outcomes is that the norms of U.S. higher education are at least partially to blame for the inefficient college trajectory of many Big Picture graduates. In sharp contrast to their high school experience, Big Picture alumni in college encounter prescribed requirements and largely non-experiential learning. The expectation for proficiency in Algebra II, for example, might be more of an arbitrary requirement than a valid competency for students who are not specializing in math or science (Hacker, 2012). For the vocationally motivated majority of first-generation college students, the career relevance of decontextualized material can be off-putting and might cause them to be less engaged in learning. The relationships with educators that sustained students in high school are less accessible in higher education: approximately half of Big Picture college students who responded to the First Fall Update after graduation reported lacking even one personal connection with an adult on campus. Without prioritizing graduation rates or student diversity during the college search, Big Picture alumni can flounder in college because of high student turnover on a particular campus or marginalization in predominantly White institutions. There is evidence of these experiences in connector interviews.
It is also possible, of course, that Big Picture provided too much support for students during high school or failed to prepare them academically. It could be that Big Picture alumni cannot succeed without more assistance than any college can be expected to provide. Much more research on Big Picture students actual college experiences and access to their transcripts would be required to begin evaluating this possibility. Big Picture Learning co-founder Dennis Littky has come to believe that it is colleges that need to change in order to value and engage Big Picture alumni, specifically, and low-income students in general. He has therefore begun a network of colleges, whose inaugural institution, College Unbound, enrolled its first entering class in 2009 in Providence, Rhode Island. (http://collegeunbound.org/) Network co-founder Elliot Washor has continued to expand the experiential, work-related education that appears to function so successfully in the Big Picture Learning design (Washor & Mojkowski, 2013). The growing number of competency-based education programs is a move in this direction, but such models are not yet widespread in higher education (Nodine, 2016; Voorhees, 2001).
Educational rhetoric calls for all students to be college and career ready (Conley & McGaughy, 2012) and conceptualizes a quality education as preparing students for both higher education and occupations. By the time they graduate after spending two full days in internships during every academic term, Big Picture seniors have considerable experience in closely supervised, extensive workplace internships related to their career interests. Both students and advisors report a high level of confidence in seniors readiness for paid work roles. The Big Picture model component of real-world relevance therefore appears to be effective, although the study did not include direct evidence about post-graduation job performance and satisfaction. Findings did include direct evidence that the network is effective in helping students aspire to higher education and complete the college admission process. Given these high school outcomes, the uneven subsequent progress of Big Picture students in higher education supports the contention that colleges require change in order to create the conditions for success among underrepresented students by taking seriously the overlap between career and college readiness. Movement toward competency-based college education is one emerging trend that fits these conclusions. There are isolated examples of higher education programs, such as College Unbound and BrownConnect, that target low-income students with experiential, for-credit programs aiming to prepare those without the financial or social capital to obtain unpaid internship opportunities for post-graduate careers (Brown University, 2014). However, most of the post-secondary innovations that target low-income students, such as distance learning and for-profit colleges, are even less capable than traditional higher education of offering the interest-based, experiential, relationally rich experiences that worked so well for Big Picture high school students.
The Big Picture Longitudinal Study fits into the larger debate about post-secondary education in the United States: how can we prepare students simultaneously for careers and college? The system of Big Picture Learning schools calls into question what college could or should be. The network of small, student-centered schools does an excellent job of graduating high school students and keeping them in the post-secondary pipeline longer than their demographic peers. The Big Picture design of relationships and relevance does an outstanding job of engaging students and making them feel cared for, supported, and connected to adults. Big Picture alumni have meaningful experience in work environments. Unfortunately, these accomplishments still fall short of solving the articulation problem between K12 and higher education for students with non-dominant cultural capital and limited financial resources. Although Big Picture alumni are more likely to persist in college and earn degrees than students from equivalent backgrounds nationally, initial evidence indicates that the percentage of Big Picture alumni college degree attainment still lags behind the national average for all U.S. high school graduates. The persistent challenges in mathematics preparation for the majority of students, and in college reading and writing for subgroups of students, are certainly related to Big Picture graduates college outcomes. Trade-offs between the relational, experiential small school culture and college readiness in math appear as an issue for Big Picture and across the small school movement (Wyse et al., 2008). In any case, academic preparation is only one of the interacting factors affecting alumni outcomes. The interacting effects of family, community, culture, finances, and college systems also destabilize graduates progression to higher education.
The results of the Big Picture Longitudinal Study indicate that it is possible to support students through the high school graduation and college application process. Big Picture Learning schools and similar small-school networks continue to work on approaches to advanced literacy and numeracy that students find engaging and relevant. High schools can use the example of Big Picture to replicate the aspects of the model that empower students and prepare them for careers and college entrance. It is possible that these should be combined with some aspects of the No Excuses school approach, a challenging task given the divergent philosophy and design of the two models.
Researchers need to develop alternative metrics for following students longitudinally with funding that is sufficient and accommodates the organizational realities of small school networks. The limitations of school data availability and integration across data sources are particularly important issues to address, including how to assess and compare non-traditional transcripts and non-standardized assessments. Research across different school networks using common metrics would be challenging but particularly valuable. The difficulties of conducting research with schools that serve low-income urban youth need to be overcome to the extent possible, but policy will almost certainly continue to be reliant on data sets that are incomplete and non-standardized. This study attempted to overcome some of the limitations of the available data by triangulating among different sources. By gathering data from students, advisors, and the National Student Clearinghouse, this study was able to provide a more nuanced characterization of the Big Picture Learning network. While time- and resource-intensive, such approaches can serve to complement existing data metrics.
Low levels of college persistence point to a need for further work within the higher education community to retain the students who enroll. And the current discourse of college access and success needs to move beyond the assumption that underrepresented studentsnot collegesneed to change. The Big Picture Longitudinal Study suggests that higher education might experiment with adopting some of the core principals of the Big Picture Learning high school design in order to increase degree attainment among first-generation, low-income students. Allowing students to pursue individual interests in their first two years of college, connecting academic study to real-world workplaces, and facilitating relationships between students and faculty/staff might be particularly effective higher education strategies for retaining students from the Big Picture and similar school models. Finally, even radical reforms of secondary education and higher education are insufficient for overcoming social inequality, given the ways that inequality is institutionalized in law, policy, the economy, and culture (Arnold et al., 2012). In the end, equalizing the life chances of low-income students will require the common will to acknowledge, understand, and interrupt this interacting web of influences.
The first author designed and conducted the Big Picture Longitudinal study in response to an invitation by network leaders to carry out a study of their graduates similar to her longitudinal study of high school valedictorians (Arnold, 1995). Methodological decisions, instrument design, and analyses were the sole responsibility of the researcher. During the study, she received assistance from Big Picture Learning staff in implementing data collection in schools and presented periodic reports on interim results to principals for the purpose of institutional improvement. The research was partially funded by Big Picture Learning, primarily through grants received by Big Picture. Graduate student support was provided by the researchers university. No restrictions were placed by the network or any of its staff on what was studied or on the dissemination of findings.
Although National Student Clearinghouse Student Tracker data is the most comprehensive and widely used college-reported enrollment source, the accuracy of the data has been questioned by some school officials (www.lohud.com/story/news/education/2014/11/20/local-superintendents-say-state-data-wrong/70026968/).
Depending on the year and set of Big Picture Learning Schools included, the on-time graduation rate ranges from 87% to 95%.
Inevitably, results of the Big Picture Longitudinal Study include variability across individual schools and study years. Although there were minor year-to-year changes in some items, the cross-network findings reported here appeared across the years of the study and at the same orders of magnitude. Small class sizes and changes at constantly innovating schools preclude statistically sound school-level analyses.
Direct information from students who were not enrolled in college was otherwise unobtainable due to an overrepresentation of 4-year college attenders in the sample of respondents to the First Fall Update Survey after graduation and a very low response rate in what was intended to be the first annual all-alumni survey.
The high-school graduation rate for low-income students pertains to economically disadvantaged students or students who are eligible for free or reduced-price meals under the National School Lunch and Child Nutrition Program. The college enrollment rate for low-income students refers to the bottom 20 percent of all family incomes. The national average graduation rate for low-income students is a composite score on parental education and occupations, and family income in 2002. The low socio-economic status group is the lowest quartile.
Because of the focus on Big Picture Distinguishers and college enrollment, the Longitudinal Study researchers did not investigate externally mandated accountability measures, such as state tests. See a study by Sara Suchman (2012) for an investigation of this topic in Big Picture Learning schools.
Adair, J. G. (1984). The Hawthorne effect: A reconsideration of the methodological artifact. Journal of Applied Psychology, 69(2), 334−345.
Aikin, W. M. (1942). Adventure in education: The story of the eight-year study with conclusions and recommendations (Vol. 1). Harper & Brothers.
Arnold, K. D. (1995). Lives of promise: What becomes of high school valedictorians. Jossey Bass.
Arnold, K. D., Fleming, S., DeAnda, M., Castleman, B., & Wartman, K. L. (2009). The summer flood: The invisible gap among low-income students. Thought & Action, 25, 23-34.
Arnold, K. D., Lu, E. C., & Armstrong, K. J. (2012). The ecology of college readiness. ASHE Higher Education Report, 38(5). Jossey Bass.
Arnold, K. D., *Wartman, K. L., *Brown, P. G., *Gismondi, A. N., *Pesce, J. R., & *Stanfield, D. A. (2016). The Connector Study: A strategy for collecting post-graduation data about low-income students. Journal of Education for Students Placed at Risk, 21(3), 174-189. DOI: 10.1080/10824669.2016.1172230
Bailey, M. J., & Dynarski, S. M. (2011). Gains and gaps: Changing inequality in U.S. college entry and completion. Population Studies Center Research Report 11−746. Ann Arbor, Michigan.
Berzin, S. C., (2010). Educational aspirations among low-income youths: Examining multiple conceptual models. Children & Schools, 32(2), 112-124.
Bloom, H. S., & Unterman, R. (2012). Sustained positive effects on graduation rates: Produced by New York Citys small public high schools of choice. MDRC Policy Brief.
Bloom, H. S., & Unterman, R. (2014). Can small high schools of choice improve educational prospects for disadvantaged students? Journal of Policy Analysis and Management, 33(2), 290319. doi:10.1002/pam.21748.
Bradley, K., & Hernández, L. E., (2019). Big Picture Learning: Spreading relationships, relevance, and rigor one student at a time. Learning Policy Institute.
Brown University. (2014). BrownConnect enables undergraduates to follow their own paths beyond the classroom. Retrieved from https://brownconnect.brown.edu/static/about
Castleman, B. L., & Page, L. C. (2014). A trickle or a torrent? Understanding the extent of summer melt among college-intending high school graduates. Social Science Quarterly, 95(1), 202−220.
Castleman, B. L., Arnold, K. D., & Wartman, K. L. (2012). Stemming the tide of summer melt: An experimental study of the effects of post-high school summer intervention on college enrollment. The Journal of Research on Educational Effectiveness, 5(1), 118.
Castleman, B. L., Page, L. C., & Schooley, K. (2014). The forgotten summer: Does the offer of college counseling after high school mitigate summer melt among college-intending, low-income high school graduates? Journal of Policy Analysis and Management, 32(2), 320−344.
Chen, X., & Carroll, C. D. (2005). First-generation students in postsecondary education: A look at their college transcripts (NCES 2001−126). Washington, DC: U.S. Department of Education National Center for Education Statistics.
Cheng, A., Hitt, C., Kisida, B., & Mills, J. N. (2017). “No Excuses” Charter Schools: A Meta-Analysis of the Experimental Evidence on Student Achievement. Journal of School Choice, 11(2), 209−238.
Chetty, R., Friedman, J. N., Saez, E., Turner, N., & Yagan, D. (2017). Mobility report cards: The role of colleges in intergenerational mobility. (NBER Working Paper No. w23618). National Bureau of Economic Research.
Christensen, C. M., Horn, M. B., & Johnson, C. W. (2008). Disrupting class: How disruptive innovation will change the way the world learns. McGraw-Hill.
Conley, D. T. (2005). College knowledge: What it really takes for students to succeed and what we can do to get them ready. Jossey-Bass.
Conley, D. T. (2010). College and career ready: Helping all students succeed beyond high school. Jossey-Bass.
Conley, D. T., & McGaughy, C., (2012). College and career readiness: Same or different? Educational Leadership, 69(7), 29-34.
Damon, William (January 2004). What is positive youth development? Annals of the American Academy of Political and Social Science, 591, 1324.
Darling-Hammond, L., Ancess, J., & Ort, S. W. (2002). Reinventing high school: Outcomes of the coalition campus schools project. American Educational Research Journal, 39(3), 639−673.
Douglas, D., & Attewell, P. (2017). School mathematics as gatekeeper. The Sociological Quarterly, 58(4), 648−669. DOI: 10.1080/00380253.2017.1354733
Ellison, S. (2012). It’s in the name: A synthetic inquiry of the Knowledge is Power Program (KIPP). Educational Studies, 48(6), 550-575.
Golann, J. W. (2015). The paradox of success at a no-excuses school. Sociology of Education, 88(2), 103119. https://doi.org/10.1177/0038040714567866
Goodman, J. F. (2013). Charter management organizations and the regulated environment: Is it worth the price? Educational Researcher, 42(2), 89-96.
Hacker, A. (2012, July 28). Is algebra necessary? New York Times. http://www.nytimes.com/2012/07/29/opinion/sunday/is-algebra-necessary.html?_r=0
Hedeker, D., & Gibbons, R. D. (2006). Longitudinal data analysis. Wiley-Interscience.
Kafka, J. (2008). Thinking big about getting small: An ideological genealogy of small-school reform. Teachers College Record, 110(9), 18021836.
KIPP Foundation (2011). The promise of college completion: KIPPs early successes and challenges. San Francisco, CA: KIPP Foundation. http://www.kipp.org/wp-content/uploads/2016/09/CollegeCompletionReport.pdf
KIPP Foundation (2017). How we measure success? Are our students climbing the mountain to and through college? Retrieved from https://www.kipp.org/results/national/#question-4:-are-our-students-climbing-the-mountain-to-and-through-college
Lack, B. (2009). No excuses: A critique of the Knowledge is Power Program (KIPP) within charter schools in the USA. Journal for Critical Education Policy Studies, 7(2), 126-153.
Lee, J. O., Hill, K. G., & Hawkins, J. D. (2012). The role of educational aspirations and expectations in the discontinuity of intergenerational low-income status. Social Work Research, 36(2), 141−151. doi: 10.1093/swr/svs025
Lerner, R. M.; Almerigi, J. B.; Theokas, C.; Lerner, J. V. (2005). Positive youth development. Journal of Early Adolescence, 25(1), 10–16.
Levine, E. (2002). One kid at a time: Big lessons from a small school. Teachers College Press.
Littky, D. (2004). The Big Picture: Education is everyones business. ASCD.
Long, B. T., & Boatman, A. (2013). The role of remedial and developmental courses in access and persistence. In L. W. Perna & A. Jones (Eds.), The state of college access and completion: Improving college success for students from underrepresented groups (pp. 77−95). Routledge.
Macey, E., Decker, J., & Eckes, S. (2009). The Knowledge is Power Program (KIPP): An analysis of one model's efforts to promote achievement in underserved communities. Journal of School Choice, 3(3), 212-241.
Martinez, M., & Klopott, S. (2005). The link between high school reform and college access and success for low-income and minority youth. American Youth Policy Forum.
McCambridge, J., Witton, J., & Elbourne, D. R. (2014). Systematic review of the Hawthorne effect: New concepts are needed to study research participation effects. Journal of Clinical Epidemiology, 67(3), 267−277.
McCarney, R., Warner, J., Iliffe, S., Van Haselen, R., Griffin, M., & Fisher, P. (2007). The Hawthorne Effect: A randomized, controlled trial. BMC Medical Research Methodology, 7(1), 30.
McDonald, J. P. (2005). Scaling up the Big Picture: Summary of findings. Washington, D.C.: Institute for Education and Social Policy. (ERIC Document Reproduction Service No. ED486213).
National Student Clearinghouse (NSC). Student tracker. Retrieved from www.studentclearinghouse.org/colleges/studenttracker/
NCES. (2016). Table 104.92. Retrieved from https://nces.ed.gov/pubs2016/2016006.pdf
NCES. (2017a). Common core of data: America’s public schools. Retrieved from https://nces.ed.gov/ccd/tables/ACGR_RE_and_characteristics_2015−16.asp
NCES. (2017b). Table 302.30. Retrieved from https://nces.ed.gov/programs/digest/d17/tables/dt17_302.30.asp
Nodine, T. R. (2016). How did we get here? A brief history of competency-based higher education in the United States. The Journal of Competency-Based Education, 1(1), 5−11.
Phillips, M. (1997). What makes schools effective? A comparison of the relationships of communitarian climate and academic climate to mathematics achievement and attendance during middle school. American Educational Research Journal, 34(4), 633- 662.
Ravitz, J. (2010). Beyond changing culture in small high schools: Reform models and changing instruction with project-based learning. Peabody Journal of Education, 85(3), 290312. doi: 10.1080/0161956X.2010.491432
Semel, S. F., & Sadovnik, A. R. (2008). The contemporary small-school movement: Lessons from the history of progressive education. Teachers College Record, 110(9), 17441771.
Shear, L., & Means, B. (2008). Contrasting paths to small-school reform: Results of a 5-year evaluation of the Bill & Melinda Gates Foundations National High Schools Initiative. Teachers College Record, 110(9), 19862039.
Suchman, S. P. (2012). Negotiating dual accountability systems: Strategic responses of big picture schools to state-mandated standards and assessment (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses. (1293021636).
Symonds, W. C., Schwartz, R., & Ferguson, R. F. (2011). Pathways to prosperity: Meeting the challenge of preparing young Americans for the 21st century. Pathways to Prosperity Project, Harvard University Graduate School of Education.
Thernstrom, A., & Thernstrom, S. (2004). No excuses: Closing the racial gap in learning. Simon and Schuster.
Thompson, R., & Holland, J. (2003). Hindsight, foresight and insight: The challenges of longitudinal qualitative research. International Journal of Social Research Methodology, 6(3), 233−244.
Titus, M. (2004). An examination of the influence of institutional context on student persistence at 4-year colleges and universities: A multilevel approach. Research in Higher Education, 45(7), 673−699.
Tuttle, C. C., Gleason, P., Knechtel, V., Nichols-Barrer, I., Booker, K., Chojnacki, G., Coen, T., & Goble, L. (2015). Understanding the effect of KIPP as it scales: Volume 1, impacts on achievement and other outcomes. Mathematica Policy Research.
Unterman, R. (2014, October). Headed to college: The effects of New York Citys small high schools of choice on postsecondary enrollment. (Policy brief). Retrieved from http://www.mdrc.org/publication/headed-college.
U.S. Department of Education (2018, January). Knowledge is power program (KIPP). What Works Clearinghouse intervention report. Washington, DC: U.S. Department of Education [Institute of Education Sciences], National Center for Education Evaluation and Regional Assistance, What Works Clearinghouse.
Voorhees, R. A. (2001) Competency-based learning models: A necessary future. New Directions for Institutional Research (Vol. 110, pp. 5−13). Jossey-Bass.
Walpole, M. (2007). Economically and educationally challenged students in higher education: Access to outcomes. ASHE Higher Education Report, 33(3). Jossey-Bass.
Washor, E., & Mojkowski, C. (2013). Leaving to learn: How out-of-school learning increases student engagement and reduces dropout rates. Heinemann.
Washor, E., Arnold, K. D., & Mojkowski, C. (2008). Taking the long view of success: Big Picture schools track graduates successesand strugglesin life beyond high school. Educational Leadership, 66(4), 60−64.
Wyse, A. E., Keesler, V., & Schneider, B. (2008). Assessing the effects of small school size on mathematics achievement: A propensity score-matching approach. Teachers College Record, 110(9), 1879–1900.
Young, A. F., Powers, J. R., & Bell, S. L. (2006). Attrition in longitudinal studies: Who do you lose? Australian and New Zealand Journal of Public Health, 30(4), 353-361.
χ2(10) = 35.473, p < .001
χ2(10) = 21.422, p < .05
χ2(10) = 22.821, p < .05
χ2(10) = 28.292, p < .005
There are no related articles to display