Impeded Attainment? The Role of State Exit Examination-Alternative Route Policy Combinations

by Anne Traynor & Allison E. Chapman - 2015

Background: To allay public concerns that state graduation examination mandates might unfairly hinder some students’ educational attainment prospects, most states with exit exam requirements offer alternative routes to earning a regular high school diploma. In spite of poor public documentation of alternative route usage rates, some exit exam states’ alternative credentialing policies have been linked to their relatively high reported graduation rates. However, there is little empirical evidence that these alternative route policies blunt the reported negative effects of exit exam requirements on diploma attainment in the general student population.

Purpose: We investigate the consequences of several distinct state exit exam-alternative route graduation policy combinations on the subsequent educational attainment of tenth graders in the graduating class of 2004.

Research Design: Data for our study are drawn from the cohort of U.S. tenth graders (N = 13,636) sampled by the Education Longitudinal Study of 2002–2006. We use logistic regression models to estimate the relationships between state exam difficulty-alternative route policy combinations and two educational attainment outcomes: high school diploma acquisition and postsecondary school enrollment.

Conclusions: While we find no relationship between exit exam policies and students’ subsequent postsecondary school enrollment, we conclude that students subject to relatively difficult state exit exams are less likely to earn a regular high school diploma than those not subject to an exam requirement. Estimating our models in the subsample of English language learner students as a model plausibility check, we observe marginally significant, but sizable, negative effects of both minimum-competency and more-difficult exit exams on diploma attainment. Our results suggest that alternative route options neither eliminate, nor appreciably attenuate, more-difficult exit exams’ negative impact on diploma attainment in the general student population or among English language learners.

Since the 1970s, many U.S. states have enacted legislation specifying that regular high school diplomas will be issued only to students who obtain marks above a certain passing score on a graduation-qualifying examination, i.e., exit exam, in one or more academic subjects, most commonly mathematics and reading or language arts. Supporters of exit exam mandates maintain that the laws should increase student achievement by motivating learning, as well as raise the signaling value of state high school diplomas to employers. Opponents of these policies contend that they hinder diploma attainment among the most disadvantaged youth, and may directly cause school dropout.

In a recent literature synthesis of forty-six empirical exit exam studies, Holme et al. (2010) determined there was no consistent, defensible evidence of association between state exit exam policies and high school students achievement. Further, only a single study in Texas concluded that employers may interpret exit exam results as relevant to employee qualifications (Martorell 2004); others have found no indication of the intended signaling effect (Warren et al. 2008). While the intended positive consequences of graduation exam policies for college- and career-readiness have seldom occurred, such policies may limit attainment, particularly among youth attending underresourced, low-performing schools (Holme 2013; Perna and Thomas 2009). To allay public concerns that state exit exam mandates might hinder educational attainment, most states with exam requirements offer alternative routes to graduation for all students. During the 20112012 school year, nineteen of the twenty-five states with exit exams offered all students one or more alternative routes, relying on a myriad of possible substitute tests or performance assessments, to acquire a regular high school diploma (McIntosh 2012, 1718). However, alternate route policies effectiveness in fostering educational attainment is not well understood.

The purpose of our study is to examine the relationship between state exam difficulty-alternative route policy combinations and two educational attainment outcomeshigh school diploma acquisition and postsecondary school enrollmentof tenth-graders in the graduating class of 2004. We draw data from the cohort of U.S. tenth graders sampled by the Education Longitudinal Study of 200206 (ELS). After reviewing previous research, we describe the ELS data and our attainment models, emphasizing our operational definitions of regular diploma and alternative route. Using logistic regression modeling, we find no relationship between exit exam policies and students subsequent postsecondary school enrollment, but that students subject to relatively difficult state exit exams are less likely to earn a regular high school diploma than those not subject to an exam requirement. Alternative route options neither eliminate nor appreciably attenuate more-difficult exit exams negative impact on diploma attainment in the general student population.

EXIT EXAM POLICIES AND EDUCATIONAL ATTAINMENT

Exit exams have been posited to influence terminal educational attainment through several mechanisms. Exit exam policies, perhaps mediated by student scores on initial testing, could cause dropout episodes if students who fail feel discouraged (Heilig 2011; Reardon et al. 2010) and disengage from school, or if educators neglect instruction for students perceived to have a low probability of passing the exam (Holme 2013) or encourage them to leave school (Heilig and Darling-Hammond 2008; Sipple, Killeen, and Monk 2004). Unless any such dropouts later return to school, it would be expected to diminish their learning outcomes and attainment. Alternatively, students who repeatedly fail an exit exam might tend to remain in school until they fulfill course credit requirements, perhaps even beyond their class graduation, but eventually leave without a diploma or with a substandard secondary credential, which may limit their terminal attainment, but not their high school learning outcomes. However, increased diploma attainment could be a potential result of exit exam policies if initial failure motivated students toward improved school engagement and academic performance, or prompted their schools to better address their learning needs (Jacob 2001; Muller and Schiller 2000).

Early exit exam investigations analyzing National Educational Longitudinal Study of 1988 data, collected when most state exit exams were minimum competency testsexams assessing proficiency in knowledge and skills introduced prior to high schoolfound no relationship between exam policies and later dropout, except among the lowest-achieving eighth graders (Jacob 2001). However, these findings cannot be readily generalized to the subsequent decades during which many states have transitioned from basic skills exams to possibly more rigorous, curriculum-aligned comprehensive or end-of-course exams. More current studies comparing student subgroups scoring just above and below particular states exit exam passing scores, who were assumed to be equivalent on both observed and unobserved traits, have concluded that exam failure increased dropout among students near the passing mark in New Jersey (Ou 2010), but not in Texas (Martorell 2004), California (Reardon et al. 2010) or Massachusetts (Papay et al. 2010). Although exit exam failure impact on dropout clearly varies by state, when it occurs, it tends to affect disadvantaged students: low-income urban students (Papay et al. 2010), racial minorities (Ou 2010), or English language learners (ELLs; Ou 2010). Recently, Hemelt and Marcotte (2013) used district-level panel data by grade spanning the 19972008 school years from the Common Core of Data (CCD) to estimate the effect of state exit exams on dropout rates. They found that exit exams increased Grade 12 dropout rates, especially among African-American males and Hispanic students.

Others have investigated the relationship between exam policies and high school diploma attainment, which has well-understood implications for adults wages, health, and other life outcomes (Levin and Belfield 2007), unlike dropout episodes. Utilizing multiple U.S. Census data cross-sections from the 1980s to identify exit exam effects, Dee (2003) found no association between state exit exam policies and individual secondary school attainment. In Census data from the 1980s and 90s, when more states were instituting exit exams or modifying existing exams to increase their difficulty, Dee and Jacob (2007) observed negative effects of both less difficult exams, which assessed elementary- or middle-school level knowledge, and more difficult exams, which included some content typically not introduced until Grade 9 or later, on secondary school attainment, which were larger among Black students than in the general student population. Due to data limitations, their analyses did not distinguish high school diploma recipients from equivalency diploma holders, and omitted many personal and school characteristics potentially confounded with state exit exam policies in determining graduation outcomes, for example, individual academic achievement.

Although minimum competency exit exam policies appear to incite school dropout or at least hinder diploma acquisition among low-performing students, and more difficult exam requirements seem to appreciably limit attainment in the entire adult population, there is little evidence that exit exam policies affect postsecondary school enrollment or attainment, on average. Bishop and Mane (2001) asserted that students in exam states, apprised of their academic deficiencies early in high school, might tend to undertake remedial study, fostering increased college readiness, enrollment and retention. They reported weak, marginally significant positive relationships between exit exam policies and later college enrollment of eighth graders. However, subsequent work concluded that minimum competency exam policies have no effect on college attendance (Dee 2003; Dee and Jacob 2007; Warren et al. 2008). Dee and Jacob (2007) revealed a positive effect of more difficult exit exam requirements on college attendance among Hispanic females, but Warren et al. (2008) reported a null relationship between more difficult exams and college attendance, with no significant heterogeneity by ethnicity. Discerning a slight negative effect of barely failing Texas exit exam on college enrollment, but none on college attainment, Martorell (2004) concluded that the consequences of exam failure tend to affect high school students who have a low probability of finishing college.

Contemporary state exit exams apparent negative impact on high school attainment has been publicized by policy reports, as well as in the academic research record. Perhaps in response, since 2004, policymakers in many exit exam states have increased the number of distinct alternative routes to graduation available (Thurlow et al. 2009). Zhang (2009) reported that during the 20072008 school year, six exit exam states awarded regular diplomas to more than 1,000, or greater than 1%, of their enrolled secondary students utilizing alternative procedures. While state legislators, governors, and department of education officials may perceive these policies as having important implications for the fairness of state exit exam systems, frequently intervening to expand or block particular alternative route options (Krentz et al. 2005), alternative routes consequences for attainment have seldom been evaluated.

ALTERNATIVE ROUTE POLICIES AND EDUCATIONAL ATTAINMENT

We define an alternative route to graduation as any process permitting students in an exit exam state to obtain a regular high school diploma (Thurlow et al. 2009) without achieving exam results above the passing score on all test sections. Although, as terminal educational credentials, regular and nonstandard high school diplomas, including equivalency diplomas and attendance certificates, indicate similarly constrained employment prospects, regular diploma awardees tend to earn higher wages and enjoy better health (e.g., Levin and Belfield 2007), while attendance certificate recipients are the least likely to be hired (Hartwig and Sitlington 2008) or admitted to postsecondary education (Erickson and Morningstar 2009). In addition to the documented advantages of regular diploma receipt, current federal law impels states to report graduation rates based on receipt of regular diplomas, only; thus, we treat only processes permitting students to obtain regular diplomas as alternative routes. We emphasize, also, that our alternative route definition focuses on diploma attainment methods potentially accessible to all students. Procedures restricted to certain subgroups, for example, students with disabilities or ELLs, are not covered by our definition. (During the time period covered by our data, no exam states offered alternate graduation procedures specifically for ELLs, although all states permitted them to test with accommodations, Gayler et al. 2004.)

The rationale for policies offering alternative routes to graduation hinges on the widespread belief that some students are not able to fully demonstrate their knowledge on traditional standardized tests, composed primarily of multiple-choice items (e.g., Thurlow et al. 2010, iv). Professional testing standards advise that when diplomas will be awarded on the basis of exam scores, students should be provided with retest opportunities or construct-equivalent testing alternatives of equal difficulty to demonstrate the skills or knowledge (AERA, APA, and NCME 1999, 146). To address objections to exit exam policy introduction or passing score increases, legislatures in many exam states have compelled their departments of education to develop alternative means by which students academic competency can be tested.

By the early 2000s, most exam states had authorized one or more alternative routes to graduation.  Gayler et al. (2004) and Krentz et al. (2005) detail alternative processes offered in 2004, while Thurlow et al. (2010) describe routes available more recently. Exam states alternative route options compose a diverse collection of substitute assessments that can broadly be categorized as traditional tests, performance assessments, or waivers. Alternative routes based on traditional tests typically require an applicant to score either above a reduced passing mark on the exit exam (usually permitted for no more than one test in the exam battery), or above a concordant passing score on a different standardized test (e.g., an Advanced Placement test, the ACT). Alternative processes based on performance assessments vary in degree of standardization, but often involve submission of portfolios of performance evidence, including course grades or teacher recommendation letters, to a review board, or demonstrating competency through standardized performance tasks, scored according to a rubric by trained judges. Alternative routes comprising the third major category, hardship waivers, are offered by a few states to senior students whose unusual extenuating circumstances (e.g., death of a parent) may prevent them from passing an exit exam. Existing alternative route regulations illustrate varying opinions regarding the degree to which results from other assessments are informative about the academic competencies assessed by state exit exams. The most conservative policies, from a psychometric perspective, treat the state exam as the only adequate measure of secondary school academic proficiency; no alternative routes are authorized. If a states exit exam is adequately aligned to the curriculum and otherwise technically sound, these policies assure trait equivalence, at least on average, of the graduation-qualifying scores for all examinees. Less conservative policies permit substitution of scores from other assessments that are assumed to be sufficiently aligned to the given states curriculum, and of comparable difficulty to the states exam. The most liberal policies allow submission of concordant scores from one of many other high school-level standardized tests, regardless of the extent to which the instruments correspond to the states curriculum, or of unstandardized collections of performance evidence that cannot be translated to achievement levels on the exit exam scale. Krentz et al. (2005) characterized about 70% of available alternative routes as intended to elicit degrees of knowledge and skills comparable to corresponding state exit exam requirements, although these equivalent routes were not necessarily aligned to state academic standards.  

In spite of poor public documentation of alternative route usage rates (Thurlow et al. 2010), some exit exam states alternative credentialing policies have been linked to their relatively high reported graduation rates. Darling-Hammond et al. (2005) contended that states using multiple measures for graduation, including certain exit exam-alternative route policy combinations, had higher graduation rates during the late 1990s and early 2000s than exit exam states with no alternative routes and only a single diploma option, which tended to have low, and declining, graduation rates. However, only two empirical studies have examined the relationship between alternative route policies and students educational outcomes. Thurlow et al. (2009) found that the correlation between a states graduation rate and the number of alternate routes offered was miniscule (and not statistically significant). To support the validity of state exit exam systems, they suggested more comprehensive evaluation of the consequences of various alternative route policies.

Recently, Hemelt and Marcotte (2013) found that between 1997 and 2008, increases in dropout rates following introduction of a new exit exam were lower in states offering alternative routes to graduation than in states that required all students to pass the exit exam, although the availability of alternative routes did little to mitigate the increases in dropout rates among African-American males and Hispanic students. Alternative routes that entailed successful completion of substitute standardized tests appeared to be least effective means of preventing dropout rate increases; in states with substitute-test-based routes, the increases in Grade 12 dropout rates after exam introduction, while not statistically significant, were sizable, and significant increases in dropout rates among Grade 10 and 11 students occurred. It should be noted that, in contrast to the characterization of nonstandard high school diplomas in this paper, the authors classified certain types of nonstandard credentials issued by states as alternative routes. They concluded that pairing exit exams with alternative routes, particularly routes other than substitute standardized tests, may lessen the exams impact on Grade 12 dropout.  

The extant literature suggests specific hypotheses regarding the impact of exit exam policies on high school diploma attainment and postsecondary enrollment. Dee and Jacob (2007) concluded that state exit exam policies have negative effects on individual diploma attainment in the general population, the magnitude of which are greater in states with more-difficult exit exams than in states with minimum-competency exams (Hypothesis 1), but Warren et al. (2008) concluded that these policies have no association with individual postsecondary enrollment, regardless of exam difficulty (Hypothesis 2). We pursue alternative route policy evaluation by addressing the research question: Does accounting for alternative route policies eliminate any observed negative effects of exit exam requirements on diploma attainment? This study uniquely contributes to the existing literature by (a) analyzing a recent data source that permits us to account for prior achievement, coursework requirements, and numerous additional personal and school factors potentially confounded with exit exam policies in determining graduation outcomes, (b) separating general education students from certain students with disabilities who are often subject to different graduation requirements and (c) avoiding reliance on the questionable assumption (Jacob 2001) that data on all variables, including the graduation outcome, is missing completely at random (see, e.g., Shuster 2012) by implementing multiple imputation.  

METHODS

DATA

The ELS obtained a nationally-representative probability sample of individual and school characteristics data (Ingels et al. 2007) from 16,019 Grade 10 students attending 750 U.S. high schools in Spring 2002. Follow-up surveys in 2004 and 2006 monitored the educational progress of sample members. The ELS data allows final secondary school attainment status to be determined for nearly the entire student population, including the sizable proportion of students who remain in high school for more than four years or are temporary dropouts at the end of Grade 12. While ELS design is not ideal for investigating the school dropout process, in general, since decisions about dropout may occur as early as middle school (Rumberger 2011), results of Martorells (2004) investigation in Texas suggest that any permanent dropout driven by exit exam requirements is most likely to occur in Grades 11 and 12, a time period captured by the ELS data (see also Hemelt and Marcotte 2013).

Because many exit exam states have specific legislative provisions waiving the exam requirement, or permitting substantial modification of the tests items or passing score for students with individualized education or Section 504 plans (Krentz et al. 2005), we apply a filter to exclude 1,734 students with disabilities from the sample. 649 eligible students who declined to respond during the base-year survey are not covered by the sampling weights that render the sample nationally representative, so are also dropped, resulting in a final analytic sample of 13,636 students representing all fifty states and the District of Columbia. Importantly, students whose final graduation status in 2006 was unknown are retained in the sample, as are students deemed unable to respond to the survey questionnaire in the base year. To address missing values, we use multiple imputation, a principled method for population inference in the presence of missing data (Schafer and Graham 2002), as will be described in our analytic strategy.

OUTCOME VARIABLES

To capture final secondary school attainment status for as many students as possible, we take attainment of a regular high school diploma by Spring 2006 as an outcome variable. Because most states offer several diploma types, each contingent on different requirements, to certify high school leavers, we follow Krentz et al.s (2005) classification of the available credentials as regular or nonstandard. Regular diplomas include each states most frequently-issued secondary qualification, as well as honors or endorsed diplomas signifying higher levels of achievement. Students who had not earned regular diplomas, and were no longer enrolled in school, had attained GED certificates or certificates of attendance, were enrolled in GED programs, or were still attending high school at the time of the 2006 survey, are categorized as non-graduates. The binary diploma attainment variable is a composite of data drawn primarily from Spring 2006 school records, but secondarily from student interviews (Ingels et al. 2007). Our postsecondary enrollment indicator denotes students enrolled at any postsecondary institution during Spring 2006, whether their enrollment was immediate after secondary school completion or delayed for one or more semesters. We designate students who were enrolled in postsecondary school for some duration following high school completion, but were no longer enrolled at the time of the second follow-up interview, as non-enrollees.

EXIT EXAM AND ALTERNATIVE ROUTE POLICY INDICATORS

We utilize two independently-conducted education policy reports to identify states that had exit exam mandates applicable to students in the Class of 2004, and to determine which test states offered any alternative routes to graduation that were open to all students (Gayler et al. 2004; Krentz et al. 2005). For Class of 2004 students, 20 states required exit exams. Five of these states, Florida, Louisiana, Massachusetts, Nevada, and New Jersey, had introduced new or more challenging exams effective for students in the Class of 2003, while Alaskas new and Virginias redeveloped exams were first required for students in the Class of 2004 (Gayler et al. 2004). Currently all graduation criteria based on state exit exams are conjunctive (e.g., van Rijn et al. 2012)students must pass each exam in the test battery, although not necessarily during the same test administration, to earn a diploma.

Although previous exit exam studies have either assigned exam policies to all state residents, or dropped private school students from nationally-representative samples, from ELS administrator reports of state exam policies, we identify four states, Indiana, New York, North Carolina, and Ohio, that required private school students to pass exit exams. Students are assigned the exam policy applicable to their base-year school. We then further classify state exit exam requirements for the Class of 2004 by difficulty following Dee and Jacob (2007, 194), with the exceptions that Alaska is identified as having a minimum-competency (i.e., less difficult) state exam, and Arkansas is classed as not requiring an exit exam (Gayler et al. 2004; Krentz et al. 2005). Table 1 categorizes exit exam states by exam difficulty level, and displays their projected graduation rates for the Class of 2004, calculated by Swanson (2004). To account for additional local exit exam requirements, we use administrator questionnaire responses to mark students subject to local (i.e., district- or school-mandated) exit exams, which are of unknown difficulty.

Table 1. State Alternative Route Policies by Exit Exam Difficulty for Class of 2004
	Alternative Routes
Exam difficulty	Test-based	Other
Low	Minnesota (78.9%)	Alaska (64.2%)
		Mississippi (58.0%)
		New Mexico (61.2%)
High	Florida (53.0%)	Georgia (55.5%)
	Massachusetts (71.0%)	Indiana (72.4%)
	New York (61.4%)	New Jersey (86.3%)
	Ohio (70.7%)
	Virginia (73.8%)
Sources. Projected graduation rate, given in parentheses, from Swanson (2004).

To determine which exit exam states offered alternative routes to graduation for the Class of 2004, we refer to Gayler et al. (2004) and Krentz et al. (2005). Alternative routes required students to score above a set passing score on a performance assessment, or one of several nationwide standardized tests; to score above a reduced exit exam cut score in one tested subject after passing other subjects; or to present an acceptable portfolio of student work, among other methods of demonstrating academic competence. One state, Florida, permitted issuance of regular high school diplomas to GED recipients who met other criteria (Florida Admin. Code 1998). Alternative route policies of schools administering local exit exams are not represented in our attainment models, as they could not be determined.

Thurlow et al. (2010) recommended designing alternative routes based on performance assessment, rather than traditional tests, so we delineate exam states alternative route policies as test-based, or other. We define test-based alternative routes as graduation options that require students to exceed a passing score on a traditional standardized test composed primarily of multiple-choice questions. The routes we designate as other are either performance assessments, varying in degree of standardization, or waiver policies with unspecified application requirements. Because many states supply multiple alternative routes, we first label states with performance or waiver options as providing other routes, regardless of whether they also offer test-based routes. The remaining route policies, which permitted only various standardized test scores to be substituted for exit exam scores, are classified as test-based. Among states alternative route options, Massachusetts and Ohios policies (Krentz et al. 2005) fit least easily into our classification scheme. Massachusetts seniors were permitted to substitute subject-area course grades or a portfolio of work for one of four required exit exam scores, but only after attaining a slightly-reduced cut score on that subject test. Ohio students who passed four out of five exit tests and met other specific performance criteria were eligible for standard diplomas. Since these states performance assessment routes could not be attempted unless a student met score requirements on most components of the states test battery, we describe Massachusetts and Ohios policies as test-based. Table 1 groups exit exam states by type of alternative route to graduation.

COVARIATES

All personal and family variables are derived from the 2002 student, parent or teacher questionnaires, ELS achievement test results, or students high school transcripts. We control for students gender, race/ethnicity, and age, measured in years. Their base-year achievement levels are represented by two covariates, ninth-grade GPA in academic courses from student transcripts, and a factor score computed from three variables: standardized reading test score, standardized mathematics test score, and a sum score from the English teacher questionnaire assessing student writing ability. To measure household socioeconomic status (SES), we generate a factor score from five variables: mother and fathers education levels and occupations, and family annual income for all students who had observed values on at least two of these variables. Both the SES and achievement factor scores are centered to have means of zero, and have observed values ranging between -3 and 3. We designate students as language minority youth if they indicated English was not their native language, their parent was a nonnative English speaker who reported communicating with their children in their native language at least sometimes, or they were otherwise recognized as ELLs. Students are classified as ELLs if transcript notations or course codes indicate they were enrolled in English as a Second Language courses at any time during high school. In addition, since transcripts are unavailable for a small proportion of students, linguistic minority students reported by their English or Mathematics teacher to be behind in coursework due to limited English proficiency are identified as ELLs.  

Other established predictors of school dropout (Rumberger 2011) that could be associated with state exam policies are incorporated into the model. From parent survey data, we devise two binary indicators: one for parental involvement, specifically, whether the parent ever attends school functions, and the second marking students who had repeated any grade between Kindergarten and Grade 9. The remaining individual covariates are drawn from student self-reports. The weekly time a student spends outside of school hours completing homework is used as a covariate, along with an indicator for students with a sibling who had previously dropped out of school, a strong predictor of attrition (e.g., Jacob 2001). Other predictors include a college-preparatory coursework track enrollment indicator, and numbers of previous suspensions from school, and absences, during the prior school year. In our model for postsecondary enrollment, we also add a binary indicator marking students who expect to earn a Bachelors degree.

Features of students schools influence their attainment outcomes (Balfanz and Legters 2004). Our model incorporates covariates representing each schools mean SES and achievement among ELS-sampled students, as well as a count of required science and mathematics course units, which states often adjust concurrently with graduation exam policies. We use information from the CCD to devise a set of indicators for school locale and public/private status, and continuous measures of Grade 10 enrollment and minority student percentage. To characterize state economic climates, which might encourage or discourage high school completion and postsecondary enrollment, we control for the states 2002 unemployment rate, reported by the U.S. Bureau of Labor Statistics, as well as the wage ratio of high school graduates to dropouts or Bachelors degree recipients to high school graduates (Shuster 2012), in the diploma acquisition and postsecondary enrollment models, respectively, drawn from US Census 2000 data. We also distinguish statewide attainment levels by the percentage of residents over age 25 that held a Bachelors degree in March 2002, from the Census Current Population Survey.

DESCRIPTIVE STATISTICS

Weighted descriptive statistics for the sample data are displayed in Table 2, and proportions of missing values are reported in Table 3. While the unconditional probability of diploma attainment appears to be slightly lower for tenth-graders subject to state exit exams, whether more-difficult or minimum-competency, than for other students, the groups probabilities of postsecondary education enrollment seem to be similar. Comparing tenth-graders subject to state exam exams to others in their cohort, we observe that those compelled to pass exams are more likely to be Black, to have repeated previous grades, and to have lower ninth-grade GPAs, but also are more likely to be enrolled in college-preparatory course sequences and to expect to earn a Bachelors degree. We note also that students subject to minimum-competency state exams have slightly higher SES, and students with more-difficult exam requirements slightly lower SES, than the sample mean. As would be anticipated since many state exit exam mandates apply only to public schools, students with state exam requirements are less likely than others to attend private schools. The school environments of students subject to state exit exams are distinctive: their schools tend to be relatively large, with high-minority student body compositions. Considering alternative route coverage, we observe that students subject to minimum-competency state exams are less likely than those subject to more-difficult exams to have any alternative route to graduation available, and that the routes provided for those subject to more-difficult exams tend to be test-based.

Table 2. Weighted Sample Descriptive Statistics, by State Exit Exam Type
	Minimum-competency (n = 1,381)			More-difficult (n = 4,997)			Total (n = 13,636)a
	M	SD		M	SD		M	SD
Outcomes:
Regular high school diploma (0, 1)	0.89			0.90			0.91
Postsecondary enrollment (0, 1)	0.60			0.59			0.60
Individual characteristics:
Female (0, 1)	0.54			0.52			0.51
Age	16.31	0.56		16.35	0.58		16.31	0.53
Race/Ethnicity (0, 1)
Asian	0.03			0.03			0.04
Black	0.23			0.19			0.14
Hispanic	0.06			0.16			0.16
Other	0.08			0.04			0.05
White	0.59			0.58			0.61
Language minority (0, 1)	0.11			0.17			0.17
SES factor score	0.06	0.87		-0.06	0.85		0.00	0.86
Sibling dropped out (0, 1)	0.12			0.14			0.13
Parent attends school functions (0, 1)	0.87			0.81			0.83
Composite achievement score	-0.01	0.88		0.01	0.89		0.00	0.91
Ninth-grade academic GPA (0 4)	2.55	0.94		2.55	0.90		2.61	0.91
English language learner (0, 1)	0.03			0.05			0.05
Homework hours per week	5.89	5.72		5.47	5.58		5.90	5.80
Absences in 2001-02 (1 5)	2.64	1.07		2.57	1.06		2.59	1.07
Suspensions from school (1 10)	2.29	0.78		2.25	0.76		2.23	0.74
Repeated K-9 grade(s) (0, 1)	0.11			0.12			0.10
College preparatory track (0, 1)	0.56			0.55			0.53
Expects Bachelors degree (0, 1)	0.85			0.84			0.83
School characteristics:
Private (0, 1)	0.03			0.04			0.08
Locale (0, 1)
Urban	0.26			0.31			0.30
Urban fringe large city	0.19			0.30			0.30
Urban fringe medium city	0.10			0.09			0.09
Town	0.30			0.21			0.22
Rural	0.14			0.08			0.10
Grade 10 enrollment, divided by ten	39.66	23.63		38.77	23.59		37.46	23.93
Mean SES factor score	0.05	0.42		-0.06	0.45		-0.01	0.48
Mean composite achievement score	0.01	0.40		0.01	0.45		0.01	0.48
Percentage minority students	37.40	26.22		38.73	31.27		34.57	30.74
Required  math & science          coursework (8 14)	11.86	1.12		12.00	1.02		11.58	1.17
State characteristics:
Alternative route policy (0, 1)
Test-based route(s)	0.22			0.49			0.21
Other route	0.23			0.21			0.11
None	0.55			0.30			0.17
Graduate:dropout earnings ratio	1.21	0.03		1.26	0.06		1.26	0.08
Percentage college graduates	26.27	5.39		26.90	3.59		26.70	3.89
Unemployment rate	5.74	0.96		5.69	0.59		5.79	0.83
Sources.Earnings ratio from US Census 2000 PUMS; Unemployment rate from US Bureau of Labor Statistics; State attainment from Snyder et al., 2004; Other variables from  Education Longitudinal Study Base Year and Second Follow-Up (Restricted Use). Note.a. Includes students subject to local exit exams (n = 1,197) or no exam (n = 5,959), or with unknown exam requirement status (n = 102).

Table 3. Weighted Missing Value Proportions
	p
Outcome:
Regular high school diploma	0.04
Postsecondary enrollment	0.12
Individual characteristics:
Female	0.01
Age	0.01
Race/Ethnicity	0
Language minority	0.01
SES factor score	0.02
Sibling dropped out	0.20
Parent attends school functions	0.18
Composite achievement score	0.02
Ninth-grade academic GPA	0.12
English language learner	0.05
Homework hours per week	0.03
Absences in 2001-02	0.06
Suspensions from school	0.04
Repeated K-9 grade(s)	0.18
College preparatory track	0.06
Expects Bachelors degree	0.12
School characteristics:
Private	0
Locale	0
Grade 10 enrollment, divided by ten	0
Mean SES factor score	0
Mean composite achievement score	0.01
Percentage minority students	0.02
Required math & science coursework	0.08
Graduation exam policy
More-difficult state test	0
Minimum-competency state test	0
Local test	0.01
No test	0.01
State characteristics:
High school graduate:dropout earnings ratio	0
Percentage college graduates	0
Unemployment rate	0

ANALYTIC STRATEGY

We impute missing data on all variables for twenty replications, equal to the maximum missingness proportion observed for any model variable (Bodner 2008), using the bootstrapping-based EM algorithm implemented by the Amelia II program under the assumption that the complete data are jointly multivariate-normally distributed (Honaker et al. 2012). The imputation model is comprised of the outcome variables, all predictors, and six auxiliary variables drawn from the 2004 and 2006 follow-up surveys, listed in Table 4. We selected auxiliary variables likely to be strongly predictive of missing values on the model variables, particularly the attainment outcomes (Graham 2009), for example, an indicator of Spring 2004 school enrollment status. The continuous imputed values are rounded to integers 0 or 1 for the outcome variables so that logistic regression models can be estimated, but otherwise are not manipulated (Graham 2009, 563). Amelias overimputation procedure permits limited evaluation of the imputation models performance, plotting confidence intervals around values that would have been imputed for observed cases, had those cases been missing (Honaker et al. 2012). We utilize the software Mplus (Muthén and Muthén 2010) to estimate logistic regression models for each attainment indicator. Since our inference population is all tenth-graders (excluding certain students with disabilities) enrolled in U.S. high schools during the Spring 2002 sampling window, the data was weighted using the base-year student weight, and standard errors were adjusted to account for the stratified sampling design of the study, and clustering of attainment outcomes by school (Balfanz and Legters 2004). We report and interpret odds ratiosexponentiated coefficients from the logistic regression modelsas effect size measures. We also report partially standardized, or X-standardized, regression coefficients, transformed raw regression coefficients that have been scaled by the estimated standard deviations (prior to imputation) of their corresponding predictor variables. It is important to note that all our analyses focus on obtaining national results for tenth graders; we do not conduct separate analyses for each state subpopulation since the ELS sampling design does not support such analyses (Ingels et al. 2007), and the variability in state-level policies that can only be observed in a nationwide sample is central to our research questions.

Table 4. Imputation Model Auxiliary Variables
Variable Name	Value Range	Year Measured	Survey	Weighted Proportion Missing
Grade 10 academic GPA	04	2004	transcript	0.10
Highest mathematics course    taken	16	2004	transcript	0.09
Plans postsecondary education enrollment	0, 1	2004	student questionnaire	0.13
Dropout at first follow-up	0, 1	2004	composite	0.02
Dropout episode during study	0, 1	2006	composite	0.00
Grade level at first follow-up	1012	2004	transcript	0.12

To test Hypothesis 1, we first estimate a model, Model 1, representing state exit exam policies only by the exams difficulty, a policy delineation supported by previous research (Dee and Jacob 2007), and marking students subject to local exit exams. Students in the no exam condition serve as a meaningful reference group for statistical tests of the exam variable set. The estimated coefficient for minimum-competency state exit exams, displayed in Table 5, is not even marginally significant in Model 1, although negative as hypothesized. Because we cannot conceive of any likely mechanism by which alternative routes could affect graduation for students subject to minimum competency exams if these exam policies are not associated with diploma attainment, to answer our research question, in Model 2 we replace only the more-difficult state exam variable with its three corresponding alternative route-exam difficulty combination indicators.

Table 5. Logistic Regression Predicting Attainment of Regular High School Diploma
	Model 1					Model 2
	B	SE	OR	X-Std B	B		SE
Individual characteristics:
Female	0.01	0.084			0.01		0.086
Age	-0.46***	0.082	0.63	-0.25	-0.47***		0.085
Race/Ethnicity (ref group = White)
Asian	0.21	0.231			0.22		0.226
Black	0.32*	0.155	1.38	0.11	0.33*		0.155
Hispanic	0.13	0.187			0.13		0.185
Other	-0.33	0.174			-0.30		0.168
Language minority	0.25	0.182			0.25		0.189
SES factor score	0.27***	0.070	1.31	0.24	0.27***		0.066
Sibling dropped out	-0.46***	0.122	0.63	-0.15	-0.42***		0.116
Parent attends school functions	0.38**	0.121	1.47	0.14	0.40**		0.119
Composite achievement score	0.22**	0.073	1.24	0.20	0.23**		0.072
Ninth-grade academic GPA	1.01***	0.069	2.74	0.90	0.99***		0.074
English language learner	-0.48*	0.197	0.62	-0.11	-0.49*		0.203
Homework hours per week	0.01	0.009			0.01		0.009
Absences in 2001-02	-0.34***	0.039	0.71	-0.36	-0.34***		0.041
Suspensions from school	-0.22***	0.041	0.80	-0.16	-0.23***		0.043
Repeated K-9 grade(s)	-0.46**	0.138	0.63	-0.14	-0.44**		0.145
College preparatory track	0.15	0.094			0.15		0.094
School characteristics:
Private	0.20	0.213			0.23		0.215
Locale (ref = Town)
Urban	0.15	0.145			0.14		0.144
Urban fringe large city	0.30*	0.140	1.35	0.14	0.31*		0.142
Urban fringe medium city	0.17	0.185			0.18		0.185
Rural	0.24	0.175			0.23		0.176
Grade 10 enrollment, divided by ten	-0.01*	0.002	0.99	-0.12	-0.01*		0.002
Mean SES factor score	0.13	0.182			0.13		0.180
Mean composite achievement score	0.17	0.180			0.17		0.178
Percentage minority students	0.00	0.002			0.00		0.002
Required math & science coursework	0.01	0.046			0.00		0.046
Graduation exam policy      (ref = No test)
More-difficult state test	-0.24*	0.114	0.79	-0.12
Minimum-competency state test	-0.17	0.161			-0.16		0.162
Local test	-0.24	0.172			-0.24		0.177
State characteristics:
High school graduate:dropout      earnings ratio	1.73	0.890			1.69		0.890
Percentage college graduates	-0.01	0.014			-0.01		0.014
Unemployment rate	-0.09	0.088			-0.12		0.088
More-difficult state test with test-      based alternative route(s)					-0.23		0.136
More-difficult state test with other      alternative route					-0.40*		0.164
More-difficult state test with no      alternative route					-0.11		0.157
Sources.Earnings ratio from US Census 2000 PUMS; Unemployment rate from US Bureau of Labor Statistics; State attainment from Snyder et al., 2004; Other variables from  Education Longitudinal Study Base Year and Second Follow-Up (Restricted Use). Note.OR = odds ratio; * p < .05; ** p < .01; *** p < .001.

Subsequently, since we anticipated heterogeneity in exit exam effects on tenth-graders educational outcomes (e.g., Dee and Jacob 2007), to contribute to model validation, we apply our models to data from the subsample of 739 ELL students. As posited by Abedi and Gándara (2006), previous research in New Jersey (Ou 2010), New York (Menken 2008), and Texas (Heilig 2011) suggests that exit exams pose a greater hindrance to diploma attainment among ELLs than among other students. If our models are reasonable, we would expect to observe larger negative effects of exit exam policies on diploma attainment, and perhaps postsecondary enrollment, in the ELL subpopulation than in the general population.

RESULTS

DIPLOMA ATTAINMENT

Table 5 displays results from our two alternative models for high school diploma attainment. To evaluate relative model fit, we compare mean Bayesian information criterion (BIC) values for the models. A BIC difference of greater than 10 (Raftery 1995) suggests that the model with the larger BIC has lower odds of being the true population model. Model 2, which contains indicators representing alternative route by more-difficult exam policy combinations, has a mean BIC estimate of 6,624 (SD = 63.9), while Model 1, accounting for only exam difficulty policies, has a mean BIC of 6,586 (SD = 57.9). Although not strictly applicable to mean BIC values, employing Rafterys criterion, Model 1 was preferred. In addition, the estimated coefficients for the exam difficulty-alternative route indicators in Model 2, which are negative and at least marginally significant for more difficult exams under both test-based (p = .086) and other (p = .016) alternative route conditions, but nonsignificant for more difficult exams with no alternative route (p = .480), suggested that our difficulty by alternative route classification scheme is capturing other exam system features. Specifically, we suspect alternative route availability may be operating as a proxy for exam difficulty, denoting the most difficult among the more difficult state exit exams. Alternative route policies clearly fail to eliminate the observed negative relationships between more-difficult state exit exams and diploma attainment outcomes. Since both substantive considerations and fit results favor the more parsimonious Model 1, we will interpret estimates from Model 1.

The overall pattern of statistically significant, and nonsignificant, variables evident in Table 5 suggests students graduation outcomes are predominantly determined by personal and family background characteristics, as would be expected based on theory (e.g., Rumberger 2011). Controlling for factors that may signal later perseverance toward high school completion, tenth graders subject to more-difficult state exit exams have 0.79 times the odds of attaining a diploma of, or 27% lower odds of attaining a diploma than {[(1/0.79 1) × 100]%}, than students who are not required to pass an exit exam. The effect of more-difficult state exit exams on individual diploma attainment is less negative than the effects of repeating a previous grade level, or of having a sibling who dropped out of school, two factors recognized to markedly influence students graduation prospects (Rumberger, 2011), either of which reduces a tenth graders odds of diploma attainment by about 59% (by calculations analogous to that given above using the odds ratio). The effect of being subject to a more-difficult state exit exam is nearly equal to the reduction in odds of diploma attainment expected per additional school suspension during the previous year. However, we find no statistically significant relationship between either minimum-competency state exam, or local exit exam policies, and individual graduation outcomes, although both coefficients are negative.  

POSTSECONDARY ENROLLMENT

Next addressing Hypothesis 2, we estimate the associations between state and local exit exam policies and individual postsecondary education enrollment. Because alternative routes to graduation do not appear to increase students odds of diploma attainment in states with difficult exit exams, and we cannot conceive of any plausible means by which alternative routes could affect postsecondary enrollment except through changes in diploma attainment, we do not analyze any models accounting for alternate route policies. Controlling for salient predictors, there is no evidence that exit exam policies are related to tenth-graders postsecondary enrollmentas shown in Table 6, the estimated coefficients for more-difficult state, minimum-competency state, and local exit exams are quite small, and none exceeds its standard error.

Table 6. Logistic Regression Predicting Enrollment in Postsecondary Education
	B	SE	OR	X-Std B
Individual characteristics:
Female	0.18**	0.059	1.20	0.09
Age	-0.22**	0.074	0.80	-0.12
Race/Ethnicity (ref group = White)
Asian	0.31*	0.134	1.37	0.09
Black	0.10	0.100
Hispanic	-0.13	0.123
Other	-0.46***	0.130	0.63	-0.10
Language minority	0.31**	0.118	1.36	0.13
SES factor score	0.43***	0.044	1.54	0.38
Sibling dropped out	-0.34**	0.099	0.71	-0.11
Parent attends school functions	0.24**	0.083	1.27	0.09
Composite achievement score	0.19***	0.047	1.20	0.17
Ninth-grade academic GPA	0.66***	0.047	1.93	0.59
English language learner	-0.26	0.146
Homework hours per week	0.02***	0.006	1.02	0.12
Absences in 2001-02	-0.16***	0.030	0.85	-0.17
Suspensions from school	-0.10*	0.050	0.90	-0.07
Repeated K-9 grade(s)	-0.19	0.116
College preparatory track	0.22***	0.056	1.24	0.11
Expects Bachelors degree	0.74***	0.080	2.09	0.27
School characteristics:
Private	0.33**	0.112	1.38	0.14
Locale (ref group = Town)
Urban	0.11	0.105
Urban fringe large city	0.16	0.101
Urban fringe medium city	0.05	0.102
Rural	0.15	0.108
Grade 10 enrollment, divided by ten	0.00	0.002
Mean SES factor score	0.29**	0.111	1.33	0.15
Mean composite achievement score	0.26*	0.124	1.30	0.13
Percentage minority students	0.01*	0.002	1.01	0.12
Required math & science coursework	0.02	0.033
Graduation exam policy (ref group = No test)
More-difficult state test	-0.01	0.079
Minimum-competency state test	0.03	0.120
Local test	0.09	0.108
State characteristics:
4-year college:high school graduate earnings      ratio	0.49	0.368
Percentage college graduates	0.00	0.010
Unemployment rate	-0.09*	0.044	0.91	-0.07

Table 7. Logistic Regression Predicting Attainment of Regular High School Diploma for English Language Learners
	Model 1				Model 2
	B	SE	OR	X-Std B	B	SE
Individual characteristics:
Female	0.09	0.301			0.08	0.295
Age	-0.41*	0.198	0.66	-0.33	-0.40*	0.197
Race/Ethnicity (ref group = White)
Asian	-0.20	0.622			-0.23	0.613
Black	-0.03	0.701			0.11	0.760
Hispanic	-0.59	0.600			-0.50	0.628
Other	-0.31	1.051			-0.29	1.068
SES factor score	0.20	0.238			0.20	0.235
Sibling dropped out	-0.58	0.398			-0.63	0.393
Parent attends school functions	0.42	0.341			0.39	0.353
Composite achievement score	0.02	0.259			0.04	0.253
Ninth-grade academic GPA	0.94***	0.198	2.56	0.85	0.93***	0.204
Homework hours per week	0.06	0.035			0.06	0.032
Absences in 2001-02	-0.14	0.147			-0.13	0.141
Suspensions from school	-0.27*	0.136	0.76	-0.26	-0.25	0.143
Repeated K-9 grade(s)	-0.31	0.446			-0.22	0.438
College preparatory track	0.19	0.306			0.17	0.291
School characteristics:
Private	-0.30	0.952			-0.27	0.862
Locale (ref = Town)
Urban	-0.22	0.484			-0.15	0.492
Urban fringe large city	0.07	0.461			0.07	0.459
Urban fringe medium city	-1.32	0.723			-1.21	0.743
Rural	0.44	1.085			0.64	1.068
Grade 10 enrollment, divided by ten	-0.01	0.007			-0.01	0.007
Mean SES factor score	0.48	0.515			0.42	0.509
Mean composite achievement score	0.17	0.516			0.23	0.497
Percentage minority students	0.01	0.008			0.01	0.008
Required math & science coursework	-0.08	0.156			-0.05	0.157
Graduation exam policy      (ref = No test)
More-difficult state test	-0.71	0.409
Minimum-competency state test	-1.13	0.675			-1.13	0.673
Local test	-0.06	0.427			-0.13	0.417
State characteristics:
High school graduate:dropout      earnings ratio	-0.09	2.796			-0.40	2.752
Percentage college graduates	-0.08	0.052			-0.08	0.057
Unemployment rate	0.07	0.281			0.11	0.282
More-difficult state test with test-      based alternative route(s)					-0.85	0.477
More-difficult state test with other      alternative route					-0.28	0.679
More-difficult state test with no      alternative route					-0.84	0.562
Sources. Earnings ratio from US Census 2000 PUMS; Unemployment rate from US Bureau of Labor Statistics; State attainment from Snyder et al., 2004; Other variables from  Education Longitudinal Study Base Year and Second Follow-Up (Restricted Use). Note. OR = odds ratio; p < .10; * p < .05; ** p < .01; *** p < .001.

EXIT EXAM EFFECTS FOR ENGLISH LANGUAGE LEARNERS

Estimates for the diploma attainment and postsecondary enrollment models in the ELL subpopulation are displayed in Tables 7 and 8. Comparison of average BIC values for the two diploma attainment models suggests that as in the general student population, Model 1, which accounts for only exam difficulty (mean BIC = 637.3, SD BIC = 14.51) should be preferred over Model 2, which also accounts for alternative route policy type (mean BIC = 647.4; SD BIC = 12.22). Although the Model 1 coefficients in Table 7 for both more-difficult and minimum-competency exam policies are negative and substantial in magnitude, suggesting severely decreased odds of high school diploma attainment for ELLs subject to state exit exam policies, even after controlling for many other related factors that could plausibly impede their graduation, the estimates are only marginally statistically significant. As among tenth-graders in the general population, Table 8 shows no evidence that exit exam policies inhibit postsecondary educational enrollment by ELL students.

Table 8. Logistic Regression Predicting Enrollment in Postsecondary Education for English Language Learners
	B	SE	OR	X-Std B
Individual characteristics:
Female	0.51*	0.252	1.67	0.26
Age	-0.43*	0.171	0.65	-0.35
Race/Ethnicity (ref group = White)
Asian	0.33	0.432
Black	0.36	0.634
Hispanic	-0.21	0.430
Other	-0.05	0.927
SES factor score	0.26	0.186
Sibling dropped out	-0.65	0.398
Parent attends school functions	0.02	0.288
Composite achievement score	0.36	0.233
Ninth-grade academic GPA	0.75***	0.206	2.12	0.68
Homework hours per week	0.04	0.021
Absences in 2001-02	-0.13	0.118
Suspensions from school	-0.10	0.187
Repeated K-9 grade(s)	0.03	0.391
College preparatory track	0.02	0.262
Expects Bachelors degree	0.99**	0.371	2.69	0.45
School characteristics:
Private	0.00	0.691
Locale (ref group = Town)
Urban	0.17	0.552
Urban fringe large city	0.11	0.523
Urban fringe medium city	-0.46	0.748
Rural	0.87	0.935
Grade 10 enrollment, divided by ten	-0.00	0.006
Mean SES factor score	0.90*	0.444	2.46	0.48
Mean composite achievement score	-0.51	0.428
Percentage minority students	0.00	0.007
Required math & science coursework	0.17	0.176
Graduation exam policy (ref group = No test)
More-difficult state test	-0.23	0.366
Minimum-competency state test	-0.72	0.555
Local test	0.30	0.334
State characteristics:
4-year college:high school graduate earnings ratio	0.57	1.694
Percentage college graduates	-0.01	0.044
Unemployment rate	-0.13	0.220
Sources.Earnings ratio from US Census 2000 PUMS; Unemployment rate from US Bureau of Labor Statistics; State attainment from Snyder et al., 2004; Other variables from  Education Longitudinal Study Base Year and Second Follow-Up (Restricted Use). Note.OR = odds ratio; p < .10; * p < .05; ** p < .01; *** p < .001.

DISCUSSION

To evaluate alternative route policies efficacy in facilitating educational attainment among high school students in exit exam states, we tested statistical models of two individual attainment outcomes: diploma acquisition and postsecondary enrollment. Our analytic method overcame some limitations of similar studies by accounting for possible cross-state differences in students prior achievement and dropout tendency, and implementing multiple imputation to reduce any estimation bias due to non-randomly missing (i.e., NMAR) data. We conclude that more difficult state graduation exam policies have substantial negative associations with regular diploma attainment by general-education, school-enrolled tenth graders, consistent with Dee and Jacobs (2007) findings from Census data, although neither local nor minimum-competency state exams were significantly related to graduation outcomes. Both more-difficult and minimum-competency exams appear to have particularly large negative effects on diploma attainment in the ELL subpopulation, consistent with findings from previous single-state regression analyses (Ou 2010) and mixed-methods studies (Heilig 2011; Menken 2008), although because the regression coefficients are only marginally statistically significant, we interpret them as merely suggestive. As pointed out by others (e.g., Jacob 2001), the negative effect of exit exam policies on diploma attainment could be perceived as an intended outcome of policy implementation. Although estimated coefficients for state and local exit exams in the postsecondary enrollment models were negative, we found no significant relationship between any of the exit exam policies and college-going, as Warren et al. (2008) concluded. Addressing our research question, we ascertained that, at least among tenth-graders in the Class of 2004, policies offering alternative routes to graduation failed to eliminate the observed negative effects of relatively difficult exit exam requirements on diploma attainment, regardless of whether these states alternative route systems were solely test-based, or included performance-based or hardship waiver options, which was partially consistent with Hemelt and Marcottes (2013) findings regarding district-level Grade 12 dropout rates, as will be discussed further below.

LIMITATIONS

We emphasize that our results apply strictly to exit exam effects on diploma attainment; although a students failure to graduate could be attributable either to diploma withholding or school dropout, our choice of diploma acquisition as an outcome variable does not permit distinction between these two mechanisms. Because this study, like all exit exam policy investigations, is based on observational datastudents cannot be randomly assigned to exit exam conditionsthe possibility remains that important variables correlated with both exam policy type and individual diploma attainment status have been omitted from our attainment models. A further limitation of our study is that we cannot readily ascertain the generalizability of our findings regarding alternative route efficacy to present alternative route systems due to poor public documentation of state alternative route usage rates over time.

EXIT EXAM POLICIES

While the average effect of minimum-competency exams on high school diploma acquisition appears to be negligible (see also Jacob 2001), since 2004, exit exam states have completely transitioned to more difficult exams, which can have substantial negative effects on diploma attainment. By school year 20112012, all state exit exams, whether comprehensive or aligned to particular courses, presented at least some material first introduced at high school level (McIntosh 2012). Because six states in addition to those with exit exam requirements for the ELS cohort have since introduced or re-introduced exit exams (Arizona, Arkansas, California, Idaho, Oklahoma, Oregon, and Washington; McIntosh 2012), while only two states (North Carolina and Tennessee) have eliminated their exam requirements, and the trend has been toward increasingly difficult exit exams, the negative direction of our estimated association between more difficult exit exams and diploma attainment seems likely to be generalizable to students subject to state exit exams developed or revised since 2004. However, it is possible that the exams effective difficulty has been reduced over time if instruction targeting exam content or superficial exam features has increased (e.g., Holme 2013; Koretz 2005). In the future, the administration of shared high school summative assessments across states, aligned to the Common Core standards in language arts and mathematics, is likely to influence exit exam policies and systems, but not necessarily lead to their standardization. Although we lacked information to model local exit exam difficulty levels in the present study, we suppose that our conclusions relating more-difficult state exit exams to decreased odds of individual diploma attainment would also extend, to some degree, to district exit exams of comparable difficulty, since students could only avoid a district exam by changing their place of residence or, in some districts, enrolling in private schools, either of which would be costly.

ALTERNATIVE ROUTE POLICIES

Although we examined only policies applicable to the graduating Class of 2004, consistent with previous findings on Grade 12 district dropout rates following introduction of rigorous exit exams in some states over the period from 19972008 (Hemelt and Marcotte 2013), our findings indicate that alternative routes do not eliminate more-difficult exit exams negative effects on diploma attainment. Hemelt and Marcotte concluded that non-test-based alternative routes might be a promising means to prevent diploma attainment declines in exam states, but we found that when difficult exit exams were paired with non-test-based routes, students odds of diploma attainment were still considerably lower in those states than in states that did not have exit exams. It is, of course, possible that the availability of alternative routes discourages students at risk of failure from dropping out of school through Grade 12, but permits few of these students to actually attain diplomas. Hemelt and Marcotte noted that their study design did not permit the effects of alternative route or exit exam policies on dropout to be distinguished from the effects of other policies applied contemporaneously to the same cohort. Our results highlight the difficulty of isolating the distinct impacts of states alternative route policies and exam features (e.g., difficulty, passing score). Rigorous exit exams seemingly more negative consequences for diploma acquisition in states with alternative routes than in states without alternative routes could occur because states with the most difficult exams are likely to devise alternative routes to graduation intended to buffer the exams impact (Zhang 2009). Our results suggest that Class of 2004 students earning diplomas by alternative routes tended to be those predicted to graduate regardless of alternative route availability, that the proportion of general-education students graduating by alternative routes was quite low, or both.

The generalizability of our results regarding alternative routes available to the Class of 2004 to recent graduating classes attainment outcomes is difficult to assess. For current alternative route options to more effectively reduce exit exams negative effect on diploma attainment, a greater proportion of students who would not otherwise graduate (e.g., through retesting) would have to know about, qualify for, access, and complete currently available routes, sufficient to offset any increases in exam difficulty. Our results would not necessarily extend to states that have introduced new alternative routes requiring less-restrictive qualifications, novel types of performance, or lower passing criteria (see Thurlow et al. 2010). Also, we cannot dismiss the possibility that existing alternative routes have become more effective in some states. However, nearly all alternative routes added to new or existing exit exam programs for more recent graduating classes are similar to routes previously available in other states, suggesting that current alternative route options are unlikely to erase exit exams likely negative effects on graduation outcomes. While our studys design does not permit us to identify elements of alternative route policies or implementation associated with the policies effectiveness, we suggest further investigation of students knowledge of, access to, and qualifications for or performance on, currently available routes.  

For an exit exam system to be legally defensible, state education officials must publicize information about the exam to students, parents, educators, and the public (Ward 2000). Although Krentz et al. (2005) and Thurlow et al. (2009) reported that information regarding alternate route policies tended to be difficult to find on state education agency websites, which often lacked even basic explanation of the process to pursue particular routes, the extent to which students and their parents are knowledgeable about alternative routes to graduation has never been assessed. During 20032004, some states department of education staff had great difficulty verifying their states alternative route options, suggesting that public knowledge of alternative routes to high school graduation for students in the Class of 2004 was likely to have been limited (Krentz et al. 2005). More recently, Ou (2010) conjectured that schools may not be providing students with adequate information about alternative routes to graduation and retesting opportunities. Also, focus groups with ELLs from five Arizona high schools, and their parents, revealed that few families, either parents or students, knew of the main alternative route to graduation offered by the state (Minnici et al. 2007). Educators in these students schools reported that administrators discouraged them from apprising students of the states alternative route because they perceived the policy as potentially in flux, and were concerned that knowing about it would reduce students effort to pass the exit exam, which was also used for federal accountability reporting. For a states graduation requirements to be reasonable, information about alternative routes must not only be publicized, but also be known to parents and students, with no serious discrepancies in awareness of the routes across socioeconomic strata.

Even if all students had equal knowledge of alternative routes to graduation, the likelihood that a qualified student will attempt an alternative route may depend on family and school characteristics, as well as the design of the policy. A recent survey by Thurlow et al. (2010) found that two states, Minnesota and Mississippi, permitted either students or school personnel to initiate alternative route processes, but other states restricted application for an alternative route to only students or their parents, or only school administrators. Because both the assessment instruments and personnel time required to implement alternative routes can be costly, students financial resources, as well as policy features, may influence equitable access. Responsibility for the administrative cost of specific alternative route processes may be assigned entirely to school districts or a state, or shared with the student applicant. In particular, student applicants may bear part of the cost of alternative routes based on scores from nationwide standardized tests, although some test retailers offer fee waivers. Evidence that all qualified students have access to any alternative routes offered is necessary to support the fairness of high school graduation requirements in exit exam states.

Given that the more rigorous exit exams currently administered in many states appear to pose a barrier to diploma acquisition, and that alternative route policies do not negate these exams impact on attainment, policymakers should reassess existing alternative routes utility and fairness, particularly to disadvantaged students (Gayler et al. 2004, 110111; Thurlow et al. 2010, 3638). While our findings suggest the need for local educators to ensure that all available processes to earn a regular high school diploma are known and accessible to qualified students, further research in exit exam states is required to document the extent that students are aware of existing alternative routes, and to assess whether awareness levels influence the routes efficacy in fostering attainment.

Acknowledgments

During manuscript preparation, the first author was supported by a University Distinguished Fellowship from the Graduate School at Michigan State University. This papers content may not reflect the views of Michigan State University. We are grateful to Phoebe Winter for helpful comments on an earlier version of this paper.

References

Abedi, J., & Gándara, P. (2006). Performance of English language learners as a subgroup in large-scale assessment: Interaction of research and policy. Educational Measurement: Issues and Practices 26(5), 3646.

American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (1999). Standards for educational and psychological testing. Washington, DC: American Psychological Association.

Balfanz, R., & Legters, N. (2004). Locating the dropout crisis: Which high schools produce the nation's dropouts? In G Orfield (Ed.) Dropouts in America: Confronting the graduation rate crisis, (pp. 5784). Cambridge, MA: Harvard Education Press.

Bishop, J. H., & Mane, F. (2001). The impacts of minimum competency exam graduation requirements on high school graduation, college attendance and early labor market success. Labour Economics 8(2), 203222.

Bodner, T. E. (2008). What improves with increased missing data imputations? Structural Equation Modeling 15(4), 65175.

Darling-Hammond, L., Rustique-Forrester, E., & Pecheone, R. L. (2005). Multiple measures approaches to high school graduation. Stanford, CA: Stanford University. Retrieved from https://edpolicy.stanford.edu/sites/default/files/publications/multiple-measures-approaches-high-school-graduation.pdf.

Dee, T. S. (2003). The first wave of accountability in No Child Left Behind? In

P. E. Peterson & M. R. West (Eds.), No Child Left Behind? The politics and practice of school accountability (pp. 215241). Washington, DC: Brookings Institution.

Dee, T. S., & Jacob, B. A. (2007). Do high school exit exams influence educational attainment or labor market performance? In A. Gamoran (Ed.), Standards-Based Reform and the Poverty Gap (pp. 154197). Washington, DC: Brookings Institution.

Erickson, A. S., & Morningstar, M. E. (2009). The impact of alternate high school exit. Certificates on access to postsecondary education. Exceptionality 17(3),150163.

Florida Admin. Code. 6A-6.021 (1998). https://www.flrules.org/gateway/ruleNo.asp?id=6A-

6.021.

Gayler, K., Chudowsky, N., Hamilton, M., Kober, N., & Margery Y. (2004). State high school exit exams: A maturing reform. Washington, DC: Center on Education Policy.

Graham, J. W. (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology 60, 549576.

Hartwig, R., & Sitlington, P. L., (2008). Employer perspectives on high school diploma options for adolescents with disabilities. Journal of Disability Policy Studies 19(1), 514.

Heilig, J. V. (2011). Understanding the interaction between high-stakes graduation tests and English learners. Teachers College Record 113(12), 26332669.

Heilig, J. V., & Darling-Hammond, L. (2008). Accountability Texas-style: The progress and learning of urban minority students in a high-stakes testing context. Educational Evaluation and Policy Analysis 30(2), 75110.

Hemelt, S. W., & Marcotte, D. E. (2013). High school exit exams and dropout in an era of increased accountability. Journal of Policy Analysis and Management 32(2), 323349.

Holme, J. J. (2013). Exit strategies: How low-performing high schools respond to high school exit exam requirements. Teachers College Record 115(1), 123.

Holme, J. J., Richards, M. P., Jimerson, J. B., & Cohen, R. W. (2010). Assessing the effects of high school exit examinations. Review of Educational Research 80(4), 476526.

Honaker, J., King, G., & Blackwell, M., (2012). AMELIA II: A Program for missing data. [Blog post]. Retrieved from http://gking.harvard.edu/amelia

Ingels, S. J., Pratt, D. J., Wilson, D., Burns, L. J., Currivan, D., Rogers, J. E., & Hubbard-Bednasz, S. (2007). Education longitudinal study of 2002: Base-year to second follow-up data file documentation. Washington, DC: National Center for Education Statistics, US Department of Education.

Jacob, B. A. (2001). Getting tough? The impact of high school graduation exams. Educational Evaluation and Policy Analysis 23(2), 99121.

Koretz, D. (2005). Alignment, high stakes, and the inflation of test scores. In J. Herman & E. Haertel (Eds.), Uses and Misuses of Data for Educational Accountability and Improvement (pp. 99118). Malden, MA: Wiley-Blackwell.

Krentz, J., Thurlow, M., Shyyan, V., & Scott, D. (2005). Alternative routes to the standard diploma. Synthesis Report 54. Minneapolis, MN: National Center on Educational Outcomes, University of Minnesota. Retrieved from http://education.umn.edu/NCEO/OnlinePubs/Synthesis54.html.

Levin, H. M., & Belfield, C. R. (2007). Educational interventions to raise high school graduation rates. In C. Belfield & H. Levin (Eds.), The price we pay: Economic and social consequences of inadequate education (pp. 177199). Washington, DC: Brookings Institution.

Martorell, F. (2004). Do high school graduation exams matter? A regression discontinuity approach. Dallas, TX: Texas Schools Project, University of Texas at Dallas. Retrieved from http://www.utdallas.edu/research/tsp-erc/pdf/wp_martorell_2004_high_school_graduation_exams.pdf.

McIntosh, S. (2012). State high school exit exams: A policy in transition. Washington, DC: Center on Education Policy.

Menken, K. (2008). English learners left behind: Standardized testing as language policy. Clevedon, UK: Multilingual Matters.

Minnici, A., Zabala, D., & Bartley, A. P., (2007). Caught in the middle: Arizonas English language learners and the high school exit exam. Washington, DC: Center on Education Policy.

Muller, C., & Schiller, K. S., 2000. Leveling the playing field? Students educational attainment and states performance testing.Sociology of Education 73(3), 196218.

Muthén, L. K., & Muthén, B.O., (2010). Mplus users guide. 6th ed. Los Angeles: Muthén and Muthén.

Ou,D. (2010). To leave or not to leave? A regression discontinuity analysis of the  impact of failing the high school exit exam. Economics of Education Review 29(2), 171186.

Papay, J. P., Murnane, R. J., & Willett, J. B. (2010). The consequences of high school exit examinations for low-performing urban students: Evidence from Massachusetts. Educational Evaluation and Policy Analysis 32(1), 523.

Perna, L. W., & Thomas, S. L. (2009). Barriers to college opportunity: The unintended consequences of state-mandated testing. Educational Policy 23(3), 451479.

Raftery, A. E. (1995). Bayesian model selection in social research. Sociological Methodology 25, 111163.

Reardon, S. F., Arshan, N., Atteberry, A., & Kurlaender, M.. (2010). Effects of failing a high school exit exam on course taking, achievement, persistence, and graduation. Educational Evaluation and Policy Analysis 32(4), 498520.

Rumberger, R. W. (2011). Dropping out: Why students drop out of high school and what can be done about it. Cambridge, MA: Harvard University Press.

Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods 7(2), 147177.

Shuster, C. (2012). Re-examining exit exams: New findings from the education longitudinal study of 2002. Education Policy Analysis Archives 20(3). Retrieved from http://epaa.asu.edu/ojs/article/view/797.

Snyder, T. D., Tan, A. G., & Hoffman, C. M. 2004. Digest of education statistics 2003 (NCES 2005025). Washington, DC: US Department of Education, National Center for Education Statistics.

Sipple, J. W., Killeen, K., & Monk, D. H. (2004). Adoption and adaptation: School district responses to state imposed learning and graduation requirements. Educational Evaluation and Policy Analysis 26(2), 143168.

Swanson, C. B. (2004). Projections of 2003-04 high school graduates: Supplemental analyses based on findings from Who Graduates? Who Doesnt? Washington, DC: The Urban Institute. Retrieved from http://www.urban.org/publications/411019.html.

Thurlow, M. L., Cormier, D. C., & Vang, M. (2009). Alternative routes to earning a standard high school diploma. Exceptionality 17(3), 135149.

Thurlow, M., Vang, M., & Cormier, D. (2010). Earning a high school diploma through alternative routes. Synthesis Report 76. Minneapolis, MN: National Center on Educational Outcomes, University of Minnesota. Retreived from http://www.cehd.umn.edu/NCEO/onlinepubs/Synthesis76/Synthesis76.pdf.

van Rijn, P. W., Béguin, A. A., & Verstralen, H. H. (2012). Educational measurement issues and implications of high stakes decision making in final examinations in secondary education in the Netherlands. Assessment in Education: Principles, Policy and Practice 19(1), 117136

Ward, C. A. (2000). GI forum v. Texas Education Agency: Implications for state assessment programs. Applied Measurement in Education 13(4), 419426.

Warren, J. R., Grodsky, E., & Lee, J. C. (2008). State high school exit examinations and postsecondary labor market outcomes. Sociology of Education 81(1), 77107.

Zhang, Y. (2009). State high school exit exams: Trends in test programs, alternate pathways, and pass rates. Washington, DC: Center on Education Policy.

Cite This Article as: Teachers College Record Volume 117 Number 9, 2015, p. 1-34 https://www.tcrecord.org ID Number: 18071, Date Accessed: 10/17/2021 2:12:38 PM Purchase Reprint Rights for this article or review