Examining First-Time-in-College Student Enrollment Patterns and Performance in Select Courses Following Developmental Education Redesign in Florida

by Toby J Park, Chenoa S. Woods, David A. Tandberg, Keith Richard, Oguzcan Cig, Shouping Hu & Tamara Bertrand Jones - October 26, 2016

In this research note, we compare enrollment and passing rates of developmental courses and gateway (introductory college-level) courses for entering cohorts of first-time-in-college (FTIC) students before and after a major developmental education reform was implemented in Florida. Overall, we find that more FTIC students are taking and passing gateway courses in the post-reform era than before.

The year 2014 may well have been a watershed year in developmental education (DE) policy in Florida and beyond. In most states, first-time-in-college (FTIC) students identified as academically underprepared via placement tests are required to take developmental, or remedial, courses prior to enrolling in college-level (credit-bearing) coursework. Once seen as a necessary way to provide academic support to underprepared FTIC students, DE has come under scrutiny in recent years due to the large number of FTIC students identified as needing DE, its high cost, questions over the ability of placement tests to accurately assign FTIC students to courses, and disturbingly low college completion rates for FTIC students in DE courses. In response, a number of states have been exploring policies that allow for more flexibility in course placement and/or provide other ways to help accelerate FTIC students through DE, such as redesigned DE courses. In Florida, however, policymakers took a more drastic approach.

Through Senate Bill 1720 (2013), which was originally passed in 2013 and went into effect in 2014, the Florida legislature made DE optional for many FTIC students enrolling in the 28 Florida colleges in the Florida College System (FCS, formerly the Florida Community College System), and mandated that DE be offered through a variety of delivery modalities. Under the new legislation, FTIC students who entered 9th grade in a Florida public school in the 2003-04 school year and beyond are considered college ready, provided they earned a standard high school diploma, and are exempt from taking DE. As such, the law prohibits requiring placement testing or DE courses for these FTIC students. The law also exempts active duty members of the military from placement testing and developmental coursework. Further, the second part of the law requires that colleges offer instruction in ways that are thought to move FTIC students quickly into college credit courses, including co-requisite, accelerated, modularized, and contextualized delivery modes, as well as enhanced advising and other forms of supplemental support.

Recent estimates indicate that over half of all students across the country seeking an associates degree require at least one developmental course (Bailey, Jeong, & Cho, 2010; Complete College America, 2012), and the National Center for Education Statistics, in 2010, found that 1.7 million beginning students required at least one DE course each year. At the same time, over $3 billion are spent each year providing DE (Alliance for Excellent Education, 2011). In Florida alone, 70% of FTIC community college students are enrolled in at least one developmental course, which cost $154 million during the 2009-10 academic year (Underhill, 2013). However, only a small fraction of students who have taken DE actually go on to earn college-level credentials (Complete College America, 2012). In addition, studies have shown that approximately one-fourth of students assessed via placement tests are mis-assigned, mainly by being unnecessarily placed into DE courses, which could delay students educational progress (Scott-Clayton, Crosta, & Belfield, 2014). Thus, many argue that the Florida DE reform could improve overall student success rates by making DE optional and altering the structure of DE courses, while others have concerns over the extent to which students will be able to make accurate enrollment decisions that best prepare them for academic success. Our earlier work has shown that the law has changed both DE programs and practices across the FCS institutions (Park, Tandberg, Hu, & Hankerson, 2016) and student course choices (Hu et al., 2016; Park et al., 2016); now, in this paper, we utilize student records to explore student outcomes following the legislation.

In this research note, we focus on the first part of the Florida DE reform: how, for many FTIC students in 2014, DE was optional regardless of prior academic preparation. As FTIC students can now bypass DE, we compared enrollment and passing rates of DE courses and gateway courses for entering cohorts of FTIC students before and after the reform was implemented. Studying changes in these metrics provides insight into how the policy is related to key measures of student success. This is important as the issue of DE reform is not a phenomenon unique to Florida. State legislatures across the country are struggling to find ways to promote student attainment of educational credentials. Our findings, then, may be useful for both state policy makers and college leaders when they consider DE reforms.

RESEARCH DESIGN

To understand whether developmental education reform has improved FTIC student success in FCS colleges, we compared enrollment rates in both developmental education and gateway courses before and after the reform, as well as pass rates in both kinds of courses. As the legislation only gave exempt FTIC students the option to exercise additional choice in their course enrollment options, we compared only those FTIC students identified as exempt in 2014 to those FTIC students in 2013 who would have likely been exempt had the legislation been implemented in 2013. Data provided to us from the Florida Department of Education allowed us to make this comparison and provides additional indicators for student background characteristics and measures of prior high school academic preparation.

ANALYTIC SAMPLE CONSTRUCTION

We utilized data from the Florida Education Data Warehouse (FL-EDW), which provides longitudinal data on individual students from public K-12 schools as well as the 28 institutions in the FCS. Our data consist of two cohorts of first time in college (FTIC) students who initially enrolled in the FCS in fall 2013 or fall 2014.  We began with 68,440 FTIC students for fall 2013 and 68,315 FTIC students for fall 2014. In order to compare only those FTIC students with the option to bypass DE (exempt FTIC students) in fall 2014 to FTIC students who would have been exempt in fall 2013 had the legislation been implemented earlier, we turned to high school transcript data.

For fall 2013, likely exempt FTIC students were coded as those FTIC students who graduated from a Florida public high school in the spring of 2006 or later. The legislation defined exempt status in 2014 as any student who entered ninth grade in 2003-2004 and graduated from a Florida public school. For fall 2013, we assumed on-time graduation from high school and backed up the ninth grade year to 2002-2003, thus arriving at our spring 2006 or later high school graduation as our simulated exempt group. For fall 2014, a variable indicating exempt status was part of the FCS data; we used all FTIC students that were not coded as exempt-no by the FCS data. Thus, our analytic sample consisted of 44,240 FTIC students for fall 2013 and 46,123 FTIC students for fall 2014, or roughly 64.6 percent and 67.5 percent of the total FTIC cohort for fall 2013 and fall 2014, respectively.

DEPENDENT VARIABLES

We constructed dichotomous indicators of whether FTIC students (a) enrolled in DE courses (modeling mathematics, reading, and writing separately), (b) passed DE courses with a C- or better (modeling the three DE subjects separately), (c) enrolled in gateway courses (modeling ENC 1101: English Composition I and MAT 1033: Intermediate Algebra separately¹), and (d) passed gateway courses (modeling the two gateway courses separately). Descriptive statistics for these enrollment and pass rates, disaggregated by cohort, are provided in Table 1. With regard to gateway courses in particular, we note that the FCS has common course numbering across all institutions and that the gateway English course across the system is English Composition I (ENC 1101), and the traditional gateway mathematics course is Intermediate Algebra (MAT1033). There are, however, additional mathematics and statistics courses that can fulfill the gateway requirement, depending on a students intended major. Investigating outcomes in these courses will be covered in a future analysis.

Table 1. Descriptive Statistics, by Cohort
	Fall 2013		Fall 2014
	n	%	n	%
Student Characteristics (S)
Race/Ethnicity
Black	9,454	21.4	10,023	21.7
Hispanic	14,651	33.1	16,080	34.9
White	17,448	39.4	17,427	37.8
Other	2,687	6.1	2,593	5.6
Female	22,795	51.5	24,037	52.1
Free or Reduced Lunch Eligibility	20,980	47.4	23,571	51.1
High School Academic Preparation (HS)
Took Algebra 2	36,559	82.6	36,610	79.4
Took Trigonometry	2,050	4.6	2,339	5.1
Took Other Advanced Math	10,148	22.9	9,429	20.4
Earned AP English credit	6,263	14.2	5,641	12.2
Earned Honors English credit	24,571	55.5	24,616	53.4
N	44,240		46,123

 

INDEPENDENT VARIABLES

Our independent variable of interest was a simple indicator for whether a student was in the fall 2014 cohort. To account for any differences in the composition of the two cohorts, we controlled for a number of student background characteristics (S) and high school academic preparation (HS) factors that have also been shown to affect educational outcomes (Coleman, Hoffer, & Kilgore, 1982; Hearn, 1988; Kuh, Kinzie, Buckley, Bridges, & Hayek, 2007; Sewell, Haller, & Ohlendorf, 1970). In terms of student background characteristics, we included indicators for race/ethnicity, gender, and economic capacity. White was used as the reference category for race/ethnicity with indicators representing FTIC students from Black, Hispanic, and other racial/ethnic backgrounds. We collapsed Asian American, Pacific Islander, multiracial, and unknown races into the other category due to the relatively small representation of these groups in our analytic sample. The reference category for gender was male and we used free/reduced lunch eligibility in high school as a proxy for economic capacity. High school academic preparation includes the following dummy variables: whether a student earned credit in algebra 2, trigonometry, another advanced math course, advanced placement English, and/or honors English. Descriptive statistics for these student background characteristics and measures of high school academic preparation, disaggregated by cohort, are provided in Table 2.

Table 2. Enrollment Rates and Passing Rates
	Fall 2013		Fall 2014
	n	%	n	%
Developmental Education Enrollment and Success
Took Developmental Math	16,220	36.7	8,252	17.9
Took Developmental Reading	8,504	19.2	3,233	7.0
Took Developmental Writing	7,230	16.3	4,222	9.2
Passed Developmental Math	9,671	59.6	4,764	57.7
Passed Developmental Reading	6,733	79.2	2,392	74.0
Passed Developmental Writing	5,545	76.7	3,072	72.8
Gateway Education Enrollment and Success
Took Gateway ENC 1101	23,685	53.5	29,811	64.6
Took Gateway MAT 1033	10,020	22.7	16,614	36.0
Passed Gateway ENC 1101	17,781	75.1	20,841	69.9
Passed Gateway MAT 1033	6,277	62.6	8,444	50.8

 

MODEL SPECIFICATION

Across our different outcome measures, we utilized the following logistic regression equation:

logit(y_ijt) = α + β(2014_t) + θ(S_ijt) + γ(HS_ijt) + δ_j  

Under this specification, y_ijt represents one of outcome measures (taking/passing DE and gateway courses, by subject) for student i, at FCS institution j, in cohort t. Our independent variable is the dichotomous indicator of being in the post-implementation cohort (2014); S and HS are vectors of student characteristics and high school academic preparation indicators as outlined above. Finally, we included FCS institutional fixed effects (δ_j) to account for unobserved heterogeneity across institutions.

RESULTS

We present our results in several sections, one for each of our outcomes. For ease of interpretation, we present our results in Table 3 in terms of changes in predicted probability of our outcome measure in the case of a statistically significant difference between fall 2013 and fall 2014. Full model results, including point estimates, are available from the authors.

Table 3. Predicted Probabilities of Enrollment and Passing Rates
		Enrollment Rates				Passing Rates
		2013	2014	Difference		2013	2014	Difference
Developmental Education (DE)
	Mathematics	34.4%	15.2%	-19.2***		60.1%	58.1%	-1.9
	Reading	16.2%	5.5%	-10.8***		80.4%	75.4%	-5.1***
	Writing	12.8%	6.7%	-6.0***		78.0%	74.1%	-3.9***
Gateway
	ENC1101	53.7%	65.6%	11.9***		76.0%	70.8%	-5.2***
	MAT 1033	21.9%	35.4%	13.5***		63.3%	50.8%	-12.5***
Notes: Predicted probabilities based on models that include the full array of student characteristics (S) and high school academic preparation (HS). Statistically significant differences are indicated by *p<.05, **p<0.01, ***p<0.001.

DEVELOPMENT EDUCATION COURSE ENROLLMENT RATES

The predicted probability of enrolling in DE mathematics, reading, and writing all decreased in 2014: 19.2 percentage points for mathematics, 10.7 percentage points for reading, and 6.1 percentage points for writing.

DEVELOPMENT EDUCATION COURSE-BASED PASSING RATES

Among FTIC students enrolled in developmental courses, the predicted probability of successfully passing DE decreased slightly for reading (5.1 percentage points) and writing (3.9 percentage points), while the difference for DE math was not statistically significant.

GATEWAY COURSE ENROLLMENT RATES

The predicted probability for enrolling in gateway courses increased for both ENC 1101 (11.9 percentage points) and MAT 1033 (13.5 percentage points).

GATEWAY COURSE-BASED PASSING RATES

Among FTIC students enrolled, the predicted probability for gateway pass rates, however, decreased in both ENC 1101 (5.2 percentage points) and MAT 1033 (12.5 percentage points).

GATEWAY COHORT-BASED PASSING RATES

Another way to examine the gateway course passing rates is to examine the same outcome, but for all FTIC students in each cohort, not just the FTIC students enrolled in gateway courses. This cohort-based gateway completion rate can tell us whether, overall, more FTIC students are passing gateway courses following the reform. In terms of cohort-based gateway passing rates, both subjects show increases: 5.3 percentage points for ENC 1101 and 3.7 percentage points for MAT 1033.

Table 4. Predicted Probabilities of Gateway Cohort-Based Passing Rates
		2013	2014	Difference
	ENC 1101	39.8%	44.6%	5.4***
	MAT 1033	12.8%	16.5%	3.8***
Notes: Predicted probabilities based on models that include the full array of student characteristics (S) and high school academic preparation (HS). Statistically significant differences are indicated by: *p<.05, **p<0.01, ***p<0.001.

SUMMARY AND DISCUSSION

Given the option to bypass DE regardless of academic preparation, many FTIC students choose to do so, often in favor of enrolling directly in gateway courses. These changes in enrollment pathways are statistically significant, even after accounting for student characteristics and academic preparation. At the same time, there is a mixed message regarding overall FTIC student success. While DE course-based passing rates declined slightly, the declines in gateway course-based pass rates are larger, and both are statistically significant. However, when examining cohort-based gateway pass rates, a different story emerges: more FTIC students are taking and passing gateway courses in the post-reform era than before.  Put differently, even though course-based passing rates have declined, there are enough FTIC students electing to take gateway courses and being successful that the overall cohort-based pass rates have increased. Further, while not presented here, descriptive statistics disaggregated by FTIC student characteristics largely show a similar story: declines in DE course enrollment, increases in gateway course enrollment, declines in course-based passing rates for gateway courses, yet increases in cohort-based passing rates (see www.centerforpostsecondarysuccess.org for more details).

These findings have several possible explanations and implications. The fact that DE passing rates have declined slightly could be a result of FTIC students who recognize that they need DE in order to be successful, electing to take DE even when it is optional. Thus, the FTIC students in DE courses in 2014 are those who are likely less academically prepared on measures beyond what our data are able to capture. As such, it is not surprising that the DE course-based pass rates have declined slightly. In fact, one could argue that it is surprising that DE pass rates have not declined more. At the same time, the fact that gateway course passing rates have declined raises questions over whether FTIC students are able to self-place into courses accurately. In other words, if FTIC students taking gateway courses in fall 2014 are those who see themselves as academically prepared for such course work, we should expect to see no difference in passing rates. Instead, we see sharp declines, even after controlling for the measures of academic preparation that our data provide. This raises concerns that others have voiced over whether FTIC students are able to accurately assess their ability and take courses where they will be successful without preparatory DE courses. Overall, however, it is encouraging that more FTIC students, on the whole, are taking and passing gateway courses following the DE reform, giving credence to the notion that by giving FTIC students the option to self-place, we will see more cohort-based success in gateway courses.

In disaggregating the results by subject area, a more nuanced story appears. More FTIC students are bypassing developmental math than reading or writing, yet the changes in pass rates across all developmental courses are fairly similar. At the same time, increases in gateway course enrollments are slightly higher for math, but the decrease in pass rates for gateway math is twice the size of the decrease in gateway English. One possible explanation for this is that FTIC students are less able to gauge their ability in math than in English and are rushing into math gateway courses unprepared, only to fail the gateway course. This could also explain why the net gain in the total percentage of the FTIC students passing gateway math, while positive, is still smaller than gateway English. In future analyses, we intend to disaggregate the results further, by student demographic characteristics and by ability level, in order to determine if the reform is differentially affecting certain student subgroups. Descriptive statistics for course enrollment and passing rates for different student groups can be found online at www.centerforpostsecondarysuccess.org. In general, we find declines in enrollment in DE courses, increases in gateway course enrollment, declines in passing rates for gateway courses, yet increases in cohort-based passing rates for all student subgroups.

Despite having access to nearly 100,000 student records, these analyses are limited to the availability of specific data points. One thing we do not know is the advice given to FTIC students from their advisor on which courses to take. We plan to conduct future analyses using data from specific institutions in the FCS that collect these data in order to provide a more complete picture of the effect of the policy. In addition, we are also actively conducting site visits at a number of FCS institutions where we interview administrators, faculty, advisors, and FTIC students to better understand how FTIC students make decisions in an environment of increased choice. Understanding the mechanism behind student choice, the why a student bypasses DE, is paramount to a full analysis of the reform. Finally, future analyses are planned to explore the second part of the law, offering DE through a variety of instructional strategies, and how this is related to student success. The Florida legislature has drastically changed how DE is offered and for whom it is required. While early analyses provide a cautiously optimistic outlook, only through additional studies will we be able to more fully understand the impact of making DE optional.

For more information about our overall project investigating the Florida DE Reform, please visit the authors website: www.centerforpostsecondarysuccess.org.

Note

1. The FCS has common course numbering across all institutions. The gateway English course across the system is English Composition I (ENC 1101), and the traditional gateway mathematics course is Intermediate Algebra (MAT1033). There are, however, additional mathematics and statistics courses that can fulfill the gateway requirement, depending on a students intended major. Investigating outcomes in these courses will be covered in a future analysis.

References

Alliance for Excellent Education (May 2011). Saving now and saving later: How high school reform can reduce the nations wasted remediation dollars. Washington, DC. Retrieved from: http://eric.ed.gov/?id=ED520329

Bailey, T., Jeong, D. W., & Cho, S. W. (2010). Referral, enrollment, and completion in developmental education sequences in community colleges. Economics of Education Review, 29(2), 255270.

Coleman, J., Hoffer, T., & Kilgore, S. (1982). Cognitive outcomes in public and private schools. Sociology of Education, 55(2), 6576.

Complete College America. (2012). Remediation: Higher educations bridge to nowhere. Washington, DC. Retrieved from: http://www.completecollege.org/docs/CCA-Remediation-final.pdf

S. 1720, 113th Cong. (2013).

Hearn, J. C. (1988). Attendance at higher-cost colleges: Ascribed, socioeconomic, and academic influences on student enrollment patterns. Economics of Education Review, 7(1), 6576.

Hu, S., Park, T., Woods, C., Richard, K., Tandberg, D. A., & Bertrand Jones, T. (2016). Probability of success: Evaluation of Floridas developmental education redesign based on cohorts of first-time-in-college students from 2009-10 to 2014-15. Tallahassee, FL: Center for Postsecondary Success.

Kuh, G. D., Kinzie, J., Buckley, J., Bridges, B., & Hayek, J. (2007). Piecing together the student success puzzle: Research, propositions, and recommendations. ASHE Higher Education Report, 32(5). San Francisco, CA: Jossey-Bass.

National Center for Educational Statistics (2010). Digest of education statistics: Table 241. Washington, D.C.: U.S. Department of Education. Retrieved from https://nces.ed.gov/programs/digest/d10/tables/dt10_241.asp

Park, T., Tandberg, D., Hu, S., & Hankerson, D. (2016). One policy, disparate reactions: Institutional responses in Florida's developmental education reform. Community College Journal of Research and Practice, 40(10), 824837.

Park, T., Woods, C., Tandberg, D., Hu, S., Bertrand Jones, T., & Richard, K. (2016). When developmental education is optional, what will students do? Analysis of survey data on student course enrollment decisions in an environment of increased choice. Innovative Higher Education, 41(3), 221236.

Scott-Clayton, J., Crosta, P. M., & Belfield, C. R. (2014). Improving the targeting of treatment evidence from college remediation. Educational Evaluation and Policy Analysis, 36(3), 371393.

Sewell, W. H., Haller, A. O., & Ohlendorf, G. W. (1970). The educational and early occupational status attainment process: Replication and revision. American Sociological Review, 35(6), 10141027.

Underhill, B. (2013, Feb. 20). College remediation. Presentation at the Florida Senate the 2013 Regular Session Appropriations Subcommittee on Education Meeting, Tallahassee, FL. Retrieved from http://www.flsenate.gov/PublishedContent/Committees/2012-2014/AED/MeetingRecords/MeetingPacket_2056.pdf

Cite This Article as: Teachers College Record, Date Published: October 26, 2016 https://www.tcrecord.org ID Number: 21699, Date Accessed: 11/2/2021 12:09:42 PM Purchase Reprint Rights for this article or review