Home Articles Reader Opinion Editorial Book Reviews Discussion Writers Guide About TCRecord
transparent 13
Topics
Discussion
Announcements
 

The Impact of Full Time Enrollment in the First Semester on Community College Transfer Rates: New Evidence from Texas With Pre-College Determinants


by Toby J Park - 2015

Background/Context: Recent developments in state-level policy have begun to require, incentivize, and/or encourage students at community colleges to enroll full time in an effort to increase the likelihood that students will persist and transfer to four-year institution where they will be able to complete their bachelor’s degree. Often, these policies are predicated on the idea that full-time status is associated with greater engagement on behalf of the student, a concept that has been widely studied in higher education as it relates to student persistence and degree attainment.

Purpose: Building upon theory and observational studies, I seek to empirically test whether enrolling full time at a community college has a discernible effect on transferring to a four-year university.

Research Design: I follow four cohorts of first-time traditionally aged college students who graduated from a public high school in Texas in the years 2000–2003 and employ a propensity score matching procedure designed to reduce sample selection bias.

Findings: I find that enrolling full time increases overall transfer rates by at least 12%. These results are robust to the inclusion of many pre-college factors as well as to a sensitivity analysis, across four separate cohorts..

Conclusions/Recommendations: This study provides evidence in support of a key policy lever for increase transfer rates already in place in a handful of states: encouraging incentivizing, or requiring full time enrollment. The key, however, will be to develop policy that results in more students enrolling full time while also maintaining the open access mission of community colleges. While requiring students to enroll full time may not be appropriate in all contexts, states should seriously consider other ways to incentivize or, at a minimum, support and encourage full-time enrollment, particularly for first-time traditionally aged students.

Community colleges represent a vital segment of American higher education, enrolling 42% of all undergraduate students nationally. Yet, while nearly 80% of first-time students who enroll at community colleges intend to obtain a bachelor’s degree, only 23% do so within six years (U.S. Department of Education, 2005). At the state level, we see even more compelling figures. In Texas, a state that enrolls nearly 9% of community college students nationally (a share exceeded only by California), over 50% of first-time undergraduates are enrolled in a community college, yet only 14% of students who begin in a community college earn a bachelor’s degree within six years (Texas Higher Education Coordinating Board, 2010; U.S. Department of Education, 2010).1


How can we increase the baccalaureate degree attainment rate for first-time traditionally aged students who begin at a community college? A new wave of state-level policy has begun to focus on a key intermediary step: transferring from a community college to a four-year institution, and the factors that mediate this process. Specifically, certain state-level policies have focused efforts on encouraging, incentivizing, and/or requiring full-time enrollment at the community college for those students intending to transfer. In 2001, the Connecticut Community Colleges system began using a state longitudinal financial aid database to determine how different enrollment patterns can influence financial aid, persistence, and transfer. With this system, students in Connecticut are now able to make more informed enrollment decisions, including enrollment part time, by reviewing different enrollment and financial aid scenarios with community college personnel (Connecticut Community College System, 2011). Evidence from Connecticut demonstrates the soaring cost of enrolling part time, both in terms of financial burden on the student and the overall impact on eventual degree attainment (College Board Advocacy & Policy Center, 2010). Building on this model, California has begun an initiative to inform students of the financial benefit of enrolling full time: Financial aid officers first ask students the number of credits they intend to take and then compute different simulations demonstrating the amount of money underutilized by enrolling part time (Moltz, 2011).


Alongside these initiatives is that of the City University of New York’s (CUNY) New Community College (NCC), that, when originally proposed in 2008, presented a new concept to the community college sector. This new institution within the CUNY system was to require students to enroll full time with the intent to expedite degree attainment (The City University of New York, 2008). Though some were initially critical of this initiative (see Moltz, 2009), in a letter dated September 20, 2011, to Education Commissioner John B. King, Jr., New York Governor Andrew Cuomo approved the creation of the New Community College (City University of New York, 2011). Through the creation of this new community college, CUNY stands at the forefront of policy initiatives geared directly at improving transfer rates from community colleges to four-year institutions by focusing on full time enrollment.


The value of these credit load policies, however, is contingent upon whether enrolling full time has a positive impact on transfer. As such, the purpose of this paper is to better understand if enrolling full time in the first semester at a community college has a discernable effect on the likelihood of transferring to a four-year institution. Previous research has indicated that students who are more engaged in postsecondary education are more likely to persist and graduate (Pascarella & Terenzini, 2005). I seek to determine if full-time enrollment—one possible measure of engagement—increases the likelihood of transferring from a community college to a four-year institution. What follows is a review of the literature on student engagement, persistence and degree completion, and, finally, a handful of existing studies that have begun to address the relationship between full-time enrollment and transfer.


LITERATURE REVIEW


Student engagement has been broadly defined as the time and the energy put forth by students in educational activities as well as the institutional effort to use effective educational strategies (Kuh, 2001). Several theoretical perspectives on persistence in higher education have stressed the important of engagement, though in slightly different ways (Astin, 1985; Pace, 1984; Tinto, 1993). For example, Astin’s (1985) theory of involvement suggests that academic performance is directly related to student involvement, as defined by an investment of psychosocial and physical energy. Pace’s (1984) work stresses a “quality of effort” theory in that college student learning is a function of the student and the institution to engage students. In addition, Tinto’s (1993) model stresses the idea of integration—both academically and socially. Tinto stressed that students who persist are those who are committed both to a particular career goal and the individual institution at which they are enrolled, with the vehicle for this integration being engagement both with peers as well as the institution.  Indeed, Tinto (1999) has continued to stress involvement as crucial to persistence in higher education. Though these perspectives differ in some ways, they share a common theme: Increased student engagement is directly related to student persistence and degree completion in postsecondary education.


In testing these theories, scholars have explored various measures of engagement and their relationship to student persistence. For example, studies have examined the role of class discussions (Braxton, Milem, & Sullivan, 2000), out-of-class experiences (Terenzini, Springer, Pascarella, & Nora, 1995), student-faculty interactions (Kuh & Hu, 2001), and contact through student support services such as tutoring centers (Chaney, Muraskin, Cahalan, & Goodwin, 1998). Many of these studies, however, have often focused on a single institution and have not always been able incorporate information on student background characteristics (Pascarella & Terenzini, 2005). Recently, though, studies have begun to use large-scale data sets from multiple institutions (e.g. Kuh, Cruce, Shoup, Kinzie, & Gonyea, 2008). For example, correlational findings from the National Survey of Student Engagement (Kuh et al., 2005) suggests that students who are more engaged—through feeling academically challenged, participating in an active and collaborative learning environment, having more contact faculty members, enjoying enriching educational experiences and supportive campus environments, and spending more time on campus—are more likely to succeed in college.


In addition to expanding beyond a single institution, research has also begun to explore student engagement specifically for community college students (Pascarella, 1997; Schuetz, 2008). Engagement for community college students presents significant challenges as community college students tend to spend little out-of-class time on campus, as more than half of these students work more than 20 hours per week and nearly one-third commit 11 hours or more to caring for family members (Powers, 2007). Still, descriptive reports based on the Community College Survey of Student Engagement (CSSE) show that community college students enrolled in more credits are typically present on campus in much higher frequencies and are more engaged both inside and outside of the classroom (Center for Community College Student Engagement, 2009). Thus, one potential measure for engagement is the number of credit hours in which students are enrolled, though the existing literature is limited. Indeed, many have pointed to a need for additional research to focus on student engagement and its relationship on persistence and degree attainment for community college students (Marti, 2009).


Given the need for additional research, a handful of empirical studies have emerged in recent years using sophisticated quasi-experimental techniques to better understand persistence and degree completion for community college students. These studies, however, are often limited in terms of the data elements they are able to capture and the number of students in the sample. For example, Melguizo and Dowd (2009) make use of NCES’ National Educational Longitudinal Survey Class of 1992 (NELS: 88/2000) when investigating persistence and completion for community college transfer students compared to rising juniors at four-year institutions. While taken from a nationally representative dataset, their subset provides the researchers with only 1,034 viable observations (247 community college transfer students, and 787 four-year students). In addition, while recent work from Ohio by Long and Kurleander (2009) incorporates pre-college factors, their analysis did not include a valuable data item available that I am able to access in Texas: employment information. More specifically, I am able to capture whether a student worked during his or her senior year of high school, another measure of economic capacity and a factor that has received much attention in the literature academic preparedness of high school students (e.g. Rothstein, 2007).


In addition, a number of studies have focused on factors that may moderate the persistence and degree attainment for community college students. Specifically, college student persistence and degree completion have investigated differential outcomes along such lines as (1) sex (Surette, 2001) and race (Lee & Frank, 1990), (2) pre-college academic preparation (Bound, Lovenheim, & Turner, 2009), (3) economic status (Melguizo & Dowd, 2009), and (4) high school context (Fletcher & Tienda, 2010), yet very few studies have focused on the role of full-time enrollment. Thus, this study seeks to better understand the factors associated with transfer for community college, with focus on the role of full-time enrollment. I turn now to studies that have begun to explore this relationship.


In a descriptive study using data from the National Center for Education Statistics (NCES), Bradburn and Hurst (2001) find that transfer rates are significantly higher for students who (1) initially enroll in an academic degree program, (2) intend to complete an undergraduate degree, and (3) remain continuously enrolled during the first two years of their postsecondary education. Research by Adelman (1999, 2004, 2006) also points to the influence of academic intensity in the first year of enrollment at a community college. One definition of academic intensity Adelman tests is the number of credits earned during the first year of study, finding that students who earn fewer than 20 credits in the first year of coursework are at a significant disadvantage for eventual degree attainment (Adelman, 2006).


Aside from these correlational studies on increased credit hours and the relationship to persistence, a few studies have empirically explored this relationship. For instance, using data from a single community college, Mohammadi (1994) finds a statistically significant relationship between increased credit hours and persistence. Fike and Fike (2008) support this finding using similar, though larger and more recent data. While both of these studies model retention as a function increased credit hours, additional research has begun to empirically explore the relationship between credit hours and transfer. Doyle (2009) explored the relationship between full-time enrollment (12 or credits taken during a student’s first semester at the community college) and transfer to the four-year sector, finding that enrolling full time increases the probability for transfer by at least 11%.  Doyle’s study, however, is unable to account for a number of important pre-college factors, including academic preparation from high school as well as high school and community context.


While acknowledging that student success can take many forms (Braxton, 2006; Kuh, Kinzie, Buckley, Bridges, & Hayek, 2007), I look specifically at the role of full-time enrollment and its impact on transfer for community college students. I uniquely contribute to the literature by examining this impact in a multi-institutional fashion using a number of pre-college individual characteristics. I argue that full-time enrollment will increase community college student engagement and thereby positively impact successful transfer to a four-year institution for those students intending to earn a baccalaureate degree. The correlational findings from Adelman (1994, 2004, 2006) and previous findings using matching techniques from Doyle (2009) suggest an empirical link between enrollment status in the first semester and eventual transfer. I extend this body of knowledge by applying a quasi-experimental technique in a new setting and using a richer dataset, asking: “For first-time traditionally aged students beginning at a community college, what is the effect of full time initial enrollment on successful transfer to a four-year institution?” What follows is a presentation of the methods utilized in this study, followed by a results section, a discussion, specific limitations, and directions for future research.  


METHOD


This method section is organized into four sections. First, I provide a discussion of the counterfactual framework used to guide my approach. Next, I describe the specific technique I use to approximate an appropriate counterfactual. Third, I describe a sensitivity analysis designed to assess the validity of the model. Finally, I provide an overview and descriptive portrait of the dataset used in this analysis.

 

COUNTERFACTUAL FRAMEWORK

A naïve estimator of mean comparisons between those students who do and those who do not enroll full time demonstrates that those students who enroll full time are more likely to transfer than those who enroll in only a few courses (Adelman, 1999, 2004, 2006). While this could be the result of a causal pathway, there may be other factors influencing student enrollment as well as success. For instance, a student who enrolls full time may be better prepared academically, have a stronger financial situation, and have access to other forms of capital that those taking fewer courses simply do not possess. In addition, these same factors that may influence a student to enroll full time are also likely to influence whether the student is more apt to transfer. In essence, a simple comparison between those students who did and did not enroll full time would overestimate the impact of full-time enrollment on successful transfer. Thus, it becomes difficult to capture a “true” impact of full time enrollment on transfer rates. From a policy standpoint, this presents a significant problem. If measures are to be taken to improve student success, unbiased estimates and pathways must first be understood.


In this paper, I establish a counterfactual: a group of students who are similar their propensity to enroll full time based on observable characteristics, except some do and others do not enroll full time (Reynolds & DesJardins, 2009; Rubin, 1974, 1976). First, I establish a simple treatment situation. The variable yi1 represents the transfer outcome for those students who enrolled full time (12 of more credits) and the variable yi0 represents those students who did not.2 Thus, the impact of full-time enrollment for any student (Δ) is given by:


Δ = y1y0


The difficulty, however, is that I cannot observe outcomes for any student who simultaneously does and does not enroll full time (Holland, 1986). Instead, I am able to observe outcomes for two groups of students: those who did enroll full time and those who did not. As such, I establish z = 1 as those students who enrolled full time and z = 0 represent those who did not. As discussed, there may be other factors beyond credit hours that may explain the transfer outcomes for these students; I denote these factors as a vector of student characteristics, x. In the social sciences literature, the mean impact of the average treatment on the treated (ATT) estimates the effect of the treatment for those receiving the treatment (in this case enrolling, full time) with respect to what the outcome would have been for the same individual students if they had not received the treatment (Smith & Todd, 2001), expressed as:


ATT =

E(Δ | x, z=1) = E(y1y0 | x, z=1) =

E(y1 | x, z=1) – E(y0 | x, z=1)


In this analysis, data is available for the mean outcomes among the treated [E(y1 | x , z=1)], what is not known, however, is information about the counterfactual outcome [E(y0 | x, z=1)]. In randomized studies, data on the counterfactual is provided in the control group, provided that the groups were randomized along the characteristics x (Heckman, 1979). In observational studies, however, economists have turned to such modeling approaches as instrumental variable as well as semiparametric and nonparametric approaches including regression discontinuity designs (Lemieux & Milligan, 2006). For the purpose of this study, however, I turn to the nonparametric approach known as propensity score matching (Rubin, 1974, 1976). This sort of matching is designed to reduce selection by creating a counterfactual group more similar to the treatment group based on observable characteristics, and has become increasingly more popular in the field of education research (Agodini & Dynarski, 2004; Doyle, 2008). What follows is a brief discussion of this method.


Matching Technique


Essentially, I seek to match students on a number of key covariates and then test the impact of full-time enrollment on only those matched students in order to obtain a “true” effect of credit hours. First, I define the matching estimator: α. This estimator is identified by comparing the outcomes for the “treatment” group (those enrolling full time) with the “control” group (those not enrolling full time) conditioning on a common probability for selecting into the “treatment” group, p (Smith & Todd, 2001):


Let α =

E(y1y0 | z=1) =

E(y1 | z=1) – Ep|z=1 Ey (y | z=1, p) =

 

E(y1 | z=1) – Ep|z=1Ey(y | z=0, p)


In addition, the probability p is defined as (Smith & Todd, 2001):


Pr(z=1 | x) <1 for all x


Assuming this condition holds, I define the matching estimator for α as:


αM = [39_18153.htm_g/00002.jpg]


In this case, [39_18153.htm_g/00004.jpg] represents the matched outcome and can also be written as [39_18153.htm_g/00006.jpg]. In this case, I1 is the treatment group (those who enrolled full time) and I0 is the control group (those who did not). Sp is the region of common support between the two groups and n1 is the number of individuals in in the set [39_18153.htm_g/00008.jpg]. The match, then, for each treatment individual [39_18153.htm_g/00010.jpg] is often composed as a weighted average of all of the control individuals, where the weights, W(i, j), depend on the distance between pi and pj. In this analysis, however, I utilize a one-to-one matching technique known as nearest neighbor matching within a caliper. First, the data are randomly sorted and then treated students i are matched with a single non-treated student j such that the matched is defined by minj = ||pi – pj || within a caliper of .10σp. In other words, I take a treated student and find a non-treated match who has the minimum difference in propensity to receive treatment within a caliper width of .10 standard deviations of the propensity score. If a match cannot be found, the observation is eliminated from the analysis. This kind of matching, often termed nearest neighbor matching within a caliper, has become popular in the literature in instances of large sample sizes where multivariate analyses are used. Additionally, a caliper of .25σp is often used; however, in this study, I use a smaller caliper due to the large size of my sample.3


More intuitively, this procedure finds an equivalent student j for every treated unit i whose propensity to receive treatment (enrolling full time) is nearly identical based on observable characteristics. In essence, I have created equivalent comparison groups between which I can gauge a less biased impact of full time enrollment. Any remaining bias in this procedure depends on whether there is a critical mass of information contained in x (Heckman et al., 1998). In order to overcome this bias, I must have confidence that the information contained in x is sufficient to establish independence between the desired outcome and the treatment. Given the richness of the dataset large number of covariates upon which I am able to match, it is likely that I have satisfied this condition and that, to the extent that the array of covariates x captures all factors of bias towards enrolling full time, I will establish a unbiased relationship between full time enrollment and community college transfer.   


Sensitivity Analysis


As a robustness check, I undertake a sensitivity analysis designed to assess the bias of the ATT estimate outlined above. This type of sensitivity analysis is recommended to accompany any propensity score matching technique (Ichino et al., 2008) and assesses the sensitivity of the ATT estimate in a simulation where there may be an unobserved factor determining placement into either the treatment or control group. A potential threat to the matching technique would be an unobserved covariate, say, motivation, that is correlated with both the selection into treatment (enrolling full time) and successful transfer. Although I have established equivalent treatment and control groups based on the observables, by failing to account for this factor, any estimate of the impact of enrolling full time would be biased. It would appear as though enrolling full time had a stronger impact on transfer than would have occurred if I were able to control for a covariate such as motivation. While failing to account for unobservable factors such as motivation has plagued educational research, bringing many results into question, the sensitivity analysis proposed by Ichino, Mealli, and Nannicini (2008) seeks to identify how severe this factor would need to be in order to bias results.


First, I simulate an unobserved covariate that mimics the set of covariates I already have (i.e. race, sex, and high school preparation in its magnitude and presence in the sample, yet is uncorrelated with observable characteristics. If, after accounting for this unobserved variable, I remain able to obtain statistically significant, positive results, I can safely conclude that to the extent to which I have accounted the selection bias, there exists a true relationship between full time enrollment and transfer. Second,, I push the sensitivity analysis one step further as suggested by Ichino, Mealli, and Nannicini (2008) to determine so-called “killer” confounders—parameterizations of u such that the ATT estimate is driven to zero. A more complete description of this approach is in the appendix.


DATA


To complete this analysis, I make extensive use of a relatively underutilized, though rich, confidential, dataset: the Texas Schools Microdata Panel (TSMP). The TSMP is a restricted use administrative dataset that includes information on secondary school records from the Texas Education Administration (TEA) and post-secondary education outcomes from the Texas Higher Education Coordinating Board (THECB) from 1992 through 2010. Central to this project is providing a comprehensive and contemporary view of the role of the community college. The depth and breadth of information contained in the TSMP data system provides the opportunity to explore longitudinal studies with a rich set of covariates never before used to explain student success via the community college.


For the purpose of this analysis, I construct a sample of those students graduating from high school and immediately enrolling in the community college sector. In addition, I further limit the sample by including only those students who indicated their intent to complete a four-year degree, as recorded in the THECB database.4 I further condition on whether a student enrolls in an academic track as defined by the state. I follow four cohorts of students: those graduating from high school in 2000, 2001, 2002, and 2003. All of the analyses are performed for each cohort, separately, to provide a robustness check across cohorts.


I use a rich set of student characteristics to ascertain the propensity to enroll full time and categorize these factors revealed in the literature into four broad categories: (1) race and sex, (2) pre-college academic preparation, (3), economic capacity, and (4) high school context.  Indicators for race (Lee & Frank, 1990) and sex (Surrette, 2001) are included as important covariates as suggested by extensive previous work.


Pre-college preparation is operationalized as: enrollment in Advanced Placement (AP) coursework (Klopfenstien & Thomas, 2005) or International Baccalaureate (IB) coursework (Bailey & Karp, 2003), the completion of a trigonometry course (Adelman, 1999; Checkley, 2001; Long, Iatarola, & Conger, 2009; Tierney, Colyar, & Corwin, 2003), performance on the state math exam (Bound et al., 2009), and whether a student participated in a dual-enrollment program by earning college-level credits while still in high school (McCauley, 2007).


In terms of economic capacity, three measures are included: the state-defined free or reduced lunch indicator that the student was given in high school (Melguizo & Dowd, 2009), the wages earned by a student during his or her final (spring) semester in high school (Rothstein, 2007), and the county-level unemployment rate where the student attended high school (Niu & Tienda, 2011).  


High school context variables are also included: the pupil to teacher ratio (Lee & Smith, 1997), the overall enrollment (Lee & Smith, 1997), the percent minority (Black & Hispanic) (Fletcher & Tienda, 2010), and the high school urbanicity (as defined by the U.S. Census) (Fletcher & Tienda, 2010). Descriptive statistics are provided in Table 1.

[39_18153.htm_g/00012.jpg]





INDEPENDENT VARIABLE OF INTEREST


The key independent variable is that of enrolling full time in the first semester at the community college. This variable is coded as a dichotomy with zero for those students enrolling in 11 or less credits and one for those students enrolling in 12 or more. This definition of full-time status stems from the federal government in that undergraduate students become eligible to receive federal financial aid in the form of Stafford loans by enrolling in 12 or more academic credits per semester. As an example of how credit-taking behavior varies by observed covariates, Figure 1 is a kernel density plot (what can be viewed as a smoothed histogram) of credit-taking behavior in 2002, by race. The left “tails” of the distribution are larger for the Hispanic and Black students than the White students, illustrating how Hispanic and Black students are less likely, based on descriptive statistics, to enroll full time. The matching technique I use is designed to account for nonuniform distributions of the covariates in order to account for selection bias.


Figure 1. Density plot, by race, for credit taking behavior for those community college students intending to earn a bachelor’s degree, 2002


[39_18153.htm_g/00014.jpg]


DEPENDENT VARIABLE

The dependent variable of interest is that of eventual transfer to a four-year institution. Using unique identification codes, students are tracked through the higher education system in Texas. Eventual transfer is coded as a dichotomy with 0 for those students who never appear in the enrollment files at a Texas four-year institution and 1 for those students who enroll in three or more credit hours in at least one fall or spring semester at any point in the six-year span following initial enrollment in the community college sector. For instance, those students beginning their studies at a community college are coded as having transferred if found in any four-year enrollment file up to, and including, Spring 2006. Mean transfer rates, by covariate, are provided in Table 2.


[39_18153.htm_g/00016.jpg]


Taken collectively, Figure 1 and Table 2 demonstrate the need for the use of a counterfactual framework—there exists observable differences shown to be related to both a student’s likelihood to enroll full time and successfully transfer. Figure 1 illustrates how enrollment behavior is not uniform by race, with Hispanic and Black students enrolling full time at lower percentages than White students. In Table 2, this same characteristic is related to eventual transfer, with Hispanic and Black students transferring at lower rates. Thus, any comparison not taking into account these important factors would be subject to overreporting the impact of full-time enrollment on eventual transfer. By including information about race as well as sex, pre-college academic preparation, economic capacity, and high school context, I seek to provide an unbiased estimate to inform both policy and the research community. What follows are results from the propensity score analysis.


RESULTS


MATCHING RESULTS


The naïve estimator would compare the mean transfer rates between those who enrolled full time and those who did not. Instead, however, I seek to establish a counterfactual group more similar to the treatment group, based on observables, using a matching technique. Important in the matching technique is the initial analysis to determine each individual’s propensity to be a member of the treated group. In order to determine an individual’s propensity to enroll full-time, I conducted a probit with the selection variables. Table 3 shows the results from this regression. As originally suggested in the density plots, and as confirmed by the probit analysis, Hispanic students are less likely to enroll full time. In addition, I find that students who are better academically prepared from high school are more likely to enroll full time.



[39_18153.htm_g/00018.jpg]

Table 3 shows the sign and relative magnitude of the influence of the covariates as well as their statistical significance. It is difficult, however, to capture an easily comprehensible effect of the covariates without computing a few predicted probabilities. To better understand the effect of the selection variables, I create profiles of students and use the estimated coefficients to determine a predicted probability of taking enrolling full time. For instance, in 2000, a Hispanic male from an economically disadvantaged family background and lacking advanced academic preparation in high school (with mean values on the other covariates) has a predicted 64% of enrolling full time. The same student, however, with strong academic preparation in high school has a predicted 81% probability of enrolling full time.4


The matching procedure uses the propensity score to match every treated student with only one non-treated student with a very nearly identical likelihood of enrolling full time, based on the selection variables. For example, a female Hispanic student who is academically prepared for college and enrolls full time in her first semester is compared only with a student who has the same propensity to enroll full time based on the observable characteristics, yet did not do so. This process is repeated for every treated student in the dataset. If a match cannot be found, the record is discarded. The goal of this process is to establish a treatment and control group that are statistically indistinguishable in terms of the relationship of student characteristics to credit taking behavior. Table 4 presents t-statistics from logistic regression results from binary analyses of the student covariates and enrolling full time, for both the unmatched and matched datasets.



In the unmatched samples, nearly all of the selection variables are statistically significant at the 95% confidence level. Such a strong correlation between these characteristics in x and the treatment (enrolling full time) can produce bias in any estimate of the treatment. In all of the years, however, this fades in the matched sample (with the exception of two covariates in 2000), suggesting no major statistically significant differences between the treatment and control groups. Thus, I argue that I have achieved equivalent treatment and control groups based on observables and, to the extent that these covariates capture the propensity to receive treatment, may proceed with a treatment analysis to determine the relationship of full-time enrollment in the first semester and transfer to a four-year institution for community college students.


TREATMENT RESULTS


Once an equivalent control group has been established, I use a weighted logistic regression analysis to ascertain the relationship of full time enrollment and transfer. Specifically, all treated students are given a weight of 1 and matched control students are weighted by a factor of n1/n0 where n1 is the number of students in the treated subclass ([39_18153.htm_g/00020.jpg]) and n0 is the number of students in the control subclass. This approach guarantees that control students (of which not all are used) receive no more weight than treated students (Ho, Imai, King, & Stewart, 2005). The results for the treatment analysis indicate a positive relationship, in both the unmatched and matched samples, for enrolling full time on eventual transfer. Detailed results are provided in Table 5 and demonstrate that although the coefficient on enrolling full time is smaller in the matches samples, the positive relationship of enrolling full time and transfer does not vanish after the matching procedure. In 2000, the estimated coefficient for full time enrollment in the unmatched sample is 0.92, with a 95% confidence interval from 0.84 to 1.00. This translates to a mean difference in the transfer rate between those above and below full-time status of 20%, with a 95% confidence interval from about 18% to 22%. In the same year, the matched sample has an estimated coefficient of 0.79 (95% confidence interval from 0.65 to 0.92). This translates to a mean difference in the transfer rate between those enrolling full time and those who did not of 17.4%, with a 95% confidence interval from about 14% to 20% (see note 6). This pattern, though possessing a slight dip in 2001, is consistent across all years. Figure 2 provides a comparison of the estimated coefficients for full time enrollment in both the matched and matched samples, across all years, with mean differences highlighted and confidence intervals shaded.


[39_18153.htm_g/00022.jpg]


[39_18153.htm_g/00024.jpg]


Figure 2: Mean differences in treated/untreated groups, matched and unmatched samples for full time status and transfer outcome


[39_18153.htm_g/00026.jpg]



Results from the Sensitivity Analysis


Tables 6a–6d show the results from the sensitivity analysis. The first row presents the ATT estimate without any confounders7 and the second row contains a neutral confounder where p = .5 [39_18153.htm_g/00028.jpg] i, j. The remaining rows in the tables represent estimates for the ATT calculated under conditions where the distribution of u is comparable to the distribution of observed variables (Hispanic, Black, Asian, male, and economic disadvantage).


[39_18153.htm_g/00030.jpg]


[39_18153.htm_g/00032.jpg]


[39_18153.htm_g/00034.jpg]


[39_18153.htm_g/00036.jpg]



Overwhelmingly, the ATT estimates remain consistent, even after the introduction of confounders. In only four instances (2000: Hispanic; 2001: Asian; and 2002: Hispanic, Econ) does the ATT estimate differ by more than one percentage point from the baseline estimate. In no instance does the ATT estimate differ by more than two percentage points from the baseline estimate. Taken collectively, these results convey a strong impression of robustness of the baseline estimate.


Killer Confounder Results


Tables 7a–7d present the results of the killer confounders, by year. In each table, d (the effect of the confounder on transfer) and s (the effect of the confounder on assignment into treatment) are varied from 0.1 to 0.7 along the rows and columns, respectively. Overall, the ATT estimates remain positive and statistically significant until both the effect on transfer and the effect on treatment sizeable values. Indeed, the estimate for 2002 varies only by 3 percentage points for all simulations of assignment into treatment (s = 0.1 to .07) for low levels of the effect on transfer (d = 0.1). Additionally, the estimate approaches zero only for values of s and d greater than 0.5. More intuitively, these results show that only under extreme values (where an unobserved factor has a sizeable effect on both selection into treatment and eventual transfer) will the estimates of the for full time enrollment on transfer approach zero. Put differently, it would take a substantial confounder to “kill” the estimates.


[39_18153.htm_g/00038.jpg]


[39_18153.htm_g/00040.jpg]


[39_18153.htm_g/00042.jpg]


[39_18153.htm_g/00044.jpg]

DISCUSSION


Enrolling full time in the first semester at a community college has a significant positive relationship with transfer to a four-year university. Furthermore, this finding is present after reducing inherent sample selection bias to which any study of this nature is prone. In addition, the sensitivity analysis reveals that these results appear to be robust against confounding factors and are threatened only by the presence of so-called killer confounders that would have an extreme impact on the behavior of community college students. The magnitude of the killer confounders I have identified is greater than that of many factors often explored in the literature including part-time faculty (Jacoby, 2006), institutional enrollment and the percentage of enrolled minority students (Calcagno, Bailey, Jenkins, Kienzel, & Leinbach, 2008), as well as student behavioral characteristics (Hawley & Harris, 2006). This benefit of increased academic intensity in the first semester is consistent with the findings of Adelman (1994, 2004, 2006) and Doyle (2009), giving further support that by increasing student credit load to full time status, we will likely see increases in college student success.  


Furthermore, these results suggest a positive relationship between engagement (as defined by full-time enrollment) and student success (as defined by transfer to a four-year institution). Student engagement theory (Kuh, 2001) suggests that students who are more engaged will experience higher levels of academic success. Additionally, previous work has shown that full-time status typically results in higher levels of engagement (Kuh et al., 2005). Thus, combined with the results of this study, we can theorize that full time status among community college students is another significant measure of student engagement.


From a policy perspective, this study provides evidence there is value to state policy designed to encourage, incentivize, or require full-time enrollment in the community college sector: enrolling full time has a discernable, positive relationship with with transfer for traditionally aged first-time community college students.

 

LIMITATIONS


This study is subject to at least two limitations. First, financial aid information, a factor known to influence persistence and degree attainment, is not included in the model due to data limitations at the time of the analysis. Future analyses would benefit from the inclusion of such data; however, I am able to a student’s free and reduced lunch status from high school. Second, the data for this study encompass four cohorts that predate the U.S. economic recession of 2007–2010. Data from the National Student Clearinghouse Research Center (2014) show enrollment in the two-year sector increased during the recession with students who may have otherwise attended a four-year institution now attending a less-costly two-year institution. Given the influx of this particular student population, it is likely that estimating the effect of full-time enrollment during the recession would result in a larger effect than I have identified. Put differently, the students enrolling during the recession were those students whose intent all along was to attend a four-year institution, but were hampered by cost and attended a two-year school, yet enrolled full time and were also likely to transfer. Given this scenario, it is likely that I offer more conservative estimates and, in the absence of a recession, more realistic estimates of the relationship between enrolling full time and transfer.


Third, this study is limited in scope to data from the state of Texas, yet the results gleaned from this analysis are externally valid in at least three ways. First, Texas has a sizeable college-going population and has been highly active in its expansion in the community college sector, a trend also found nationally. Second, Texas has a diverse demographic composition and, in many ways, mirrors projections of the national racial composition. Finally, Texas has a varied landscape of institutions of higher education. Texas provides an excellent laboratory in which to study the community college given its sizeable college-going population, its increasing capacity in the two-year sector, and its shifting demographics. The student unit record dataset in Texas provides the opportunity to expand upon the existing body of knowledge and tap a vast wealth of previously unexplored factors. Furthermore, the TSMP data in Texas provides the opportunity to conduct analyses over several cohorts through recent years. Given the ever-changing landscape of the community college sector, this level of information is essential to understanding the complex role of the community college. With a rising community college sector enrollment and recent policy changes geared at increasing community college student transfer and degree attainment, as well as significant social, political, and economic shifts within its borders, Texas stands to become a pioneer in terms of practices and policies related to the community college.


CONTRIBUTIONS AND CONCLUSIONS


This analysis of community college transfer is particularly timely given the recent community college summit at the White House in early October 2010 where President Obama reinforced public faith in the community college as an important pathway to an undergraduate degree. With such an increased focus on community colleges and far too few students attaining an undergraduate degree, it is paramount that we identify the mechanisms that increase transfer from community colleges to four-year institutions. This study has provided evidence in support of a key policy lever for increase transfer rates already in place in a handful of states: encouraging, incentivizing, or requiring full time enrollment. The key, however, will be to develop policy that results in more students enrolling full time while also maintaining the open access mission of community colleges. While requiring students to enroll full time may not be appropriate in all contexts, states should seriously consider other ways to incentivize or, at a minimum, support and encourage full-time enrollment, particularly for first-time traditionally aged students.


Notes


The conclusions of this research do not necessarily reflect the opinions or official position of the Texas Education Agency, the Texas Higher Education Coordinating Board, or the State of Texas.


1. It must be noted, however, that national figures are calculated only for those who indicate a clear intention to complete a bachelor’s degree whereas Texas statistics are reported for those all students enrolling full time, making direct comparisons misleading. It remains clear, however, that degree attainment remains low for students beginning at the community college level in Texas.  

2. Full time enrollment is defined as 12 of more credit hours and was chosen, in part, due to the federal government’s definition of full-time undergraduate status as 12 credits, thus enabling students to receive federal financial aid.

3. Results and discussion are presented only for the nearest neighbor algorithm due to its simplicity and intuitive nature. Additional strategies (e.g., kernel matching) were implemented and produced similar results. Additional information on types of matching can be founded in Smith and Todd (2005) as well as Guo and Fraser (2009).

4. Considerable attention has been paid to the idea of the denominator—who to include in the sample—when calculating transfer/graduation rates. See Doyle (2009) for a summary.

5. Predicted probabilities for probit regression results are available from the author. A student who lacks academic preparation in high school is defined as having not taking an AP/IB course or trigonometry course and scoring two standard deviations below the mean on the state math exam. A student with advanced academic preparation has taken both an AP/IB course and a trigonometry course and scored at the mean on the math exam.

6. Detailed calculations of these percentages are available from the author.

7. This ATT estimate is similar, though not precisely the same as the previously estimated ATT estimate. This sensitivity analysis uses a different random sorting of the data. Results from the different sortings are very similar.


References


Adelman, C. (1999). Answers in the tool box: Academic intensity, attendance patterns and bachelor’s

degree attainment [Technical report]. Washington, DC: United States Department of Education.


Adelman, C. (2004). Principal indicators of student academic histories in postsecondary education

[Technical report]. Washington, DC: United States Department of Education.


Adelman, C. (2006). The toolbox revisited: Paths to degree completion from high school through college

[Technical report]. Washington, DC: United States Department of Education.


Agodini, R., & Dynarski, M. (2004). Are experience the only option? A look at dropout prevention

programs. The Review of Economics and Statistics, 86(1), 180–194.


Astin, A. W. (1985). Achieving educational excellence. San Francisco, CA: Jossey-Bass.


Bailey, T. R. & Karp, M. (2003). Promoting college access and success: A review of credit-based

transition programs. Washington, DC: U.S. Department of Education, Office of Adult and Vocational

Education.


Bound, J., Lovenheim, M. F., & Turner, S. (2009). Why have college completion rates declined? An

analysis of changing student preparation and collegiate resources. NBER Working paper. Retrieved from

http://www.nber.org/papers/w15566.pdf


Bradburn, E. M., & Hurst, D. G. (2001). Community college transfer rates to 4-year institutions using

alternative definitions of transfer [Technical report]. Washington, DC: United States Department of

Education, National Center for Education Statistics.


Braxton, J. M. (2006, June). Faculty professional choices in teaching that foster student success.

Washington, DC: National Postsecondary Education Cooperative.


Braxton, J. M., Milem, J. F., & Sullivan, A. S. (2000). The influence of active learning on the college

student departure process: Toward a revision of Tinto’s theory. The Journal of Higher Education, 71(5),

569–590.


Calcagno, J. C., Bailey, T., Jenkins, D., Kienzel, G., & Leinbach, T. (2008). Community college student

success: What institutional characteristics make a difference? Economics of Education Review, 27(6),

632–645.


Center for Community College Student Engagement. (2009). Making connections: Dimensions of student engagement (2009 CCSE Findings). Austin, TX: The University of Texas at Austin, Community College Leadership Program.


Chaney, B., Muraskin, L. D., Cahalan, M. W., & Goodwin, D. (1998). Helping the progress of

disadvantaged students in higher education: The federal student support services program. Educational

Evaluation and Policy Analysis, 20(3), 197–215.


Checkley, K. (2001). Algebra and activism: Removing the shackles of low expectation: A conversation

with Robert P. Moses. Educational Leadership, 59(2), 6–11.


City University of New York. (2008). A new community college concept paper. New York: The City

University of New York. Retrieved from http://www1.cuny.edu/portal_ur/cmo/i/8/24/nccconceptpaper.pdf


City University of New York. (2011, September 29). State approves CUNY’s new community college. The

University. Retrieved from http://www1.cuny.edu/mu/forum/2011/09/29/state-approves-cuny’s-new-

community-college/


College Board Advocacy & Policy Center. (2010). The financial aid challenge. Reston, VA: The College

Board.


Connecticut Community Colleges System. (2011). Financial aid services. Retrieved from

http://www.commnet.edu/finaid


Doyle, W. R. (2009). Impact of increased academic intensity on transfer rates: An application of matching

estimators to student-unit record data. Research in Higher Ed, 50(1), 52–72.


Fike, D. S., & Fike, R. (2008). Predictors of first-year student retention in the community college.

Community College Review, 36(2), 68–88.


Fletcher, J., & Tienda, M. (2009). High school classmates and college success. Sociology of Education,

82(4), 287–314.


Fletcher, J., & Tienda, M. (2010). Race and ethnic differences in college achievement: Does high school

attended matters? Annals of the American Academy of Political and Social Science, 627(1), 144–166.


Hawley, T. H. & Harris, T. A. (2006). Student characteristic related to persistence for first-year community

college students. Journal of College Student Retention: Research, Theory, & Practice, 7(1-2), 117–142.


Heckman, J. J. (1979). Sample selection bias as a specification error. Econometrica, 47(1), 153–161.


Heckman, J., Ichimura, H., Smith, J., & Todd, P. (1998). Characterizing selection bias using experimental data (No. w6699). National Bureau of Economic Research.


Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2005). Matching as a nonparametric preprocessing for

reducing model dependence in parametric causal inference. Cambridge, MA: Harvard University Press.


Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association,

81(396), 945–960.


Ichino, A., Mealli, F., & Nannicini, T. (2008) From temporary help jobs to permanent employment: What

can we learn from matching estimators and their sensitivity? Journal of Applied Econometrics, 23(3),

305–327.


Jacoby, D. (2006). Effects of part-time faculty employment on community college graduation rates. The

Journal of Higher Education, 77(6), 1081–1103.


Klopfenstein, K., & Thomas, M. K. (2005). The link between advanced placement experience and college

success. Retrieved from https://www.utdallas.edu/research/tsp-

erc/pdf/wp_klopfenstein_2006_link_advanced_placement.pdf.pdf


Kuh. G. D. (2001). Assessing what really matters to student learning: Inside the National Survey of

Student Engagement. Change, 33(3), 10–17.


Kuh, G. D., Cruce, T. M., Shoup, R., Kinzie, J., & Gonyea, R. M. (2008). Unmasking the effects of student

engagement on first-year college grades and persistence. The Journal of Higher Education, 79(5), 540–

563.


Kuh, G. D., & Hu, S. (2001). The effects of student-faculty interaction in the 1990s. The Review of Higher

Education, 24(3), 309–332.


Kuh, G. D., Kinzie, J., Buckley, J., Bridges, B., & Hayek, J. C. (2007). Piecing together the student

success puzzle: Research, propositions, and recommendations. ASHE Higher Education Report, 32(5).

San Francisco, CA: Jossey-Bass.


Kuh, G. D., Kinzie, J., Schuh, J. H., White, E. J., & Associates (2005). Student success in college:

Creating conditions that matter. San Francisco, CA: Jossey-Bass.


Lee, V. E., & Frank, K. A. (1990). Students’ characteristics that facilitate the transfer from two-year to four-year colleges. Sociology of Education, 63(3), 178–193.


Lee, V. E., & Smith, J. B. (1997). High school size: Which works best and for whom. Education Evaluation

and Policy Analysis, 19(3), 205–227.


Lemieux, T., & Milligan, K. (2006). Incentive effects of social assistance: A regression discontinuity approach. Statistics Canada, Analytical Studies Branch.


Long, B. T., & Kurlaender, M. (2009). Do community colleges provide a viable pathway to a

baccalaureate degree? Educational Evaluation and Policy Analysis, 31(1), 30–53.


Long, M. C., Iatarola, P., & Conger, D. (2009). Explaining gaps in readiness for college-level math: The

role of high school courses. Education Finance and Policy, 4(1), 1–33.


Marti, C. N. (2009). Dimensions of student engagement in American community colleges: Using the community college student report in research and practice. Community College Journal of Research and Practice, 33(1), 1–24.


McCauley, D. (2007). The impact of advanced placement and dual enrollment programs on college

graduation. Applied Research Projects, Texas State University-San Marcos. Paper 206. Retrieved from

http://ecommons.txstate.edu/arp/206


Melguizo, T., & Dowd, A.C. (2009). Baccalaureate success of transfers and rising 4-year college juniors.

Teacher College Record, 11(1), 55–89.


Mohammadi, J. (1994). Exploring retention and attrition in a two-year public college. Martinsville, VA:

Patrick Henry College.


Moltz, D. (2009, February 6). Shaking up the community college concept. Inside Higher Ed. Retrieved

from http://www.insidehighered.com/news/2009/02/06/cuny


Moltz, D. (2011, March 3). When more costs less. Inside Higher Ed. Retrieved from

http://www.insidehighered.com/news/2011/03/03/california_community_colleges_encourage_full_time_en

rollment_with_financial_aid


National Student Clearing House Research Center. (2014). Signature report 6 state supplement.

Herndon, VA: Author. Retrieved from http://nscresearchcenter.org/wp-

content/uploads/NSC_Signature_Report_6_StateSupp.pdf


Niu, S. X., & Tienda, M. (2011). High school economic composition and persistence. Princeton, NJ:

Princeton University. Retrieved from http://theop.princeton.edu/reports/wp/College%20Persistence.pdf


Pace, C. R. (1984). Measuring the quality of college student experiences. Los Angeles, CA: University of

California, Higher Education Research Institute.


Pascarella, E. T. (1997, January-February). It’s time we started paying attention to community college

students. About Campus, 14–17.


Pascarella, E. T., & Terenzini, P. T. (2005). How colleges affect students: A third decade of research (Vol.

2). San Francisco, CA: Jossey-Bass.


Powers, E. (2007, November 12). Rules of (community college) engagement. Inside Higher Ed. Retrieved

from http://www.insidehighered.com/news/2007/11/12/ccsse#sthash.O1eCE1Zz.dpbs


Reynolds, C. L., & Desjardins, S. L. (2009). The use of matching methods in higher education research:

Answering whether attendance at a 2-year institution results in differences in educational attainment. In J.

C. Smart (Ed.), Higher education: Handbook of theory and practice (Vol. 24, pp. 47–97). Amsterdam,

NL: Springer.


Rosenbaum, P., & Rubin, D. (1983). The central role of propensity score in observational studies for

causal effects. Biometrika, 70(1), 41–55.


Rothstein, D. S. (2007). High school employment and youths’ academic achievement. The Journal of

Human Resources, 42(1), 194–213.


Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies.

Journal of Educational Psychology, 66(5), 688–701.


Rubin, D. B. (1976). Assignment to treatment groups on the basis of a covariate. ETS Research Bulleting Series, 2, 1–26.


Schuetz, P. (2008). A theory-driven model of community college student engagement. Community

College Journal of Research and Practice, 32(4–6).


Smith, J. A., & Todd, P. E. (2001). Reconciling conflicting evidence on the performance of propensity-

score matching methods. The American Economic Review, 91(2), 112–118.


Surette, B. J. (2001). Transfer from 2-year to 4-year college: An analysis of gender differences.

Economics of Education Review, 20(2), 151–163.


Texas Higher Education Coordinating Board (2010). 6-year Graduation Rates of First-Time Entering

Undergraduates, Fall 2003. Retrieved from

http://www.thecb.state.tx.us/reports/PDF/2057.PDF?CFID=11243290&CFTOKEN=41746812)


Terenzini, P. T., Springer, L., Pascarella, E. T., & Nora, A. (1995). Academic and out-of-class influences

on students’ intellectual orientations. The Review of Higher Education, 19(1), 23–44.


Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition (2nd ed.). Chicago,

IL: University of Chicago Press.


Tinto, V. (1999). Taking retention seriously: Rethinking the first year of college. NACADA Journal, 19(2),

9.


Tierney, W. G., Colyar, J. E. & Corwin, Z. B. (2003). Preparing for college: Building expectations,

changing realities. Los Angeles, CA: University of Southern California, Center for Higher Education Policy

Analysis.


U.S. Department of Education. (2005). Beginning postsecondary students: Data analysis system

[Technical report]. Washington, DC: National Center for Education Statistics.


U.S. Department of Education. (2010). Digest of Education Statistics 2009 (NCES 2010-013).

Washington, DC: National Center for Education Statistics, Institute for Education Sciences, U.S.

Department of Education.





APPENDIX


Sensitivity Analysis

Although I have tested for equivalent treatment and control groups by regressing the selection variables, x, on the treatment variable z, based on the work of Rosenbaum and Rubin (1983) and Ichino, Mealli, and Nannichini (2008), I now proceed as though assignment to treatment is not unconfounded with the observable variables. Additionally, I establish a binary covariate, u, which is associated with both the treatment and the outcome.  In other words:


Pr(z=1 | y0, y1, x) ≠ Pr(z=1 | x)


Furthermore, given an unobserved binary covariate u, I establish:


Pr(z=1 | y0, y1, x, u) = Pr(z=1 | x, u)


The sensitivity analysis assesses the point estimates for the ATT under the propensity score matching technique by imposing values of the factors that determine u and then using a predicted value of u for each of the treated and control students to re-estimate the ATT using the predicted u. By changing the specification of u, I am able to assess the robustness of the ATT estimate under different hypotheses about the unobserved confounder. Furthermore, I am able to determine if there is a hypothesis about u that could drive the ATT to zero. However, if the findings for ATT under multiple hypotheses are consistent, unbiased inference becomes more concrete (Ichino et al., 2008).


Specifically, I consider the case of y0, y1 [39_18153.htm_g/00046.jpg] {0,1} where y = z×y1 + (1 – zy0 represents the observed outcome for a given student and treatment/control classification. In this situation, I characterize the distribution of u such that:


Pr(u=1 | z=i, y=j, x) = Pr(u=1 | z=i, y=j) [39_18153.htm_g/00048.jpg] pij


Just as y0, y1 [39_18153.htm_g/00050.jpg] {0,1}, I establish i, j [39_18153.htm_g/00052.jpg] {0,1}. I then learn the probability pij in each of the four categorizations established by combinations of the treatment and outcome values. By using different specifications of pij I can attribute different values of u to each of the students in both the treatment and control groups. Then, I treat u as any other selection variable used in the creation of the propensity score and the resulting ATT estimate. By repeating this estimation many (1,000) times, I can obtain an ATT estimate that is robust to different specifications of the unobserved variable u. Such an estimate is consequently free from bias of any underlying observed variable.


As Ichino et al. (2008) demonstrate, any threat to the ATT estimate would arise from the situation where u has both a positive effect on the control group outcome (where p01p00 > 0, an “outcome effect”) and the selection into the treatment (where p1*p0* > 0, a “selection effect”). Thus, I am able to interpret the results from the sensitivity analysis by focusing on confounders that produce this outcome. To gauge the magnitude of bias caused by u, I calculate the outcome effect as an average of the odds ratios from logistic regression model of Pr(y=1 | z=0, u, x) as:


Pr(y=1 | z=0, u=1, x)

Pr(y=0 | z=0, u=1, x)

Pr(y=1 | z=0, u=0, x)  [39_18153.htm_g/00054.jpg] Γ

Pr(y=0 | z=0, u=0, x)


Then, to gauge the magnitude of bias caused by u, I calculate the selection effect as an average of the odds ratios from logistic regression model of Pr(z=1 | u, x) as:


Pr(z=1 | u=1, x)

Pr(z=0 | u=1, x)

Pr(z=1 | u=0, x)  [39_18153.htm_g/00056.jpg] Λ

Pr(z=0 | u=0, x)


Estimates for Γ and Λ, above as well as the ATT estimates produced under these scenarios will allow me to determine whether the original ATT estimates generated from the propensity score matching technique (that is, the estimates where u is not included) are robust to different specifications of an underlying variable. In the most simple case, I set pij equal to 0.50 for all i and j combinations to simulate a “neutral” confounder. I then simulate pij to mimic the set of selection variables. For example, if 45% of the students who enrolled full-time and successfully transferred are male, I set p11 equal to 0.45 and similarly for the other covariates and pij values. Taken collectively, the ATT estimates produced under these various parameterizations of pij will shed light on the robustness of the original ATT estimates.


Next, I describe the so-called killer counfounders. As established:


ATT = E(y1 | z=1) – E(y0 | z=1)


Also, as aforementioned, y0 is not observed when z = 1 and, thus, the term E(y0 | z=1) cannot be estimated from the available data. It is possible, however, to establish non-parametric bounds for the ATT estimate by substituting E(y0 | z=1) with its largest and smallest obtainable values:


E(y1 | z=1 ) – 1 [39_18153.htm_g/00058.jpg] ATT [39_18153.htm_g/00060.jpg] E(y1 | z=1 )


Furthermore, I establish the difference d as d = p01p00 (an unconditioned estimate of the effect of u on the control students successfully transferring) and the difference s as s = p1*p0* (an unconditioned estimate of the effect of u on assignment into treatment). Thus far, I have fixed both s and d to be zero; however, by varying these terms along with the associated Γ and Λ terms, I am able to ascertain how severe the impact of u would need to be on the outcome effect and/or the treatment effect in order to produce a substantially biased ATT estimate. In other words, I am able to determine how strong the effects of u on either treatment and/or outcome would need to be in order to “kill” the ATT estimate.




Cite This Article as: Teachers College Record Volume 117 Number 12, 2015, p. 1-34
https://www.tcrecord.org ID Number: 18153, Date Accessed: 10/22/2021 4:26:50 PM

Purchase Reprint Rights for this article or review
 
Article Tools
Related Articles

Related Discussion
 
Post a Comment | Read All

About the Author
  • Toby Park
    Florida State University
    E-mail Author
    TOBY J. PARK, PhD, is an assistant professor of economics of education and education policy, and an associate director of the Center for Postsecondary Success, at Florida State University. His research interests include policy related to improving enrollment and completion, particularly for community college, traditionally underrepresented, and nontraditional students. Park’s resent work has appeared in Educational Researcher, the Journal of Higher Education, and Research in Higher Education.
 
Member Center
In Print
This Month's Issue

Submit
EMAIL

Twitter

RSS