

High School Advanced Placement and Student Performance in College: STEM Majors, NonSTEM Majors, and Gender Differencesby Phillip L. Ackerman, Ruth Kanfer & Charles Calderwood  2013 Background/Context: The past few decades have seen an explosive growth in highschool student participation in the Advanced Placement program® (AP), with nearly two million exams completed in 2011. Traditionally, universities have considered AP enrollment as an indicator for predicting academic success during the admission process. However, AP exam performance may be predictive of future academic success; a related factor in gender differences in major selection and success; and instrumental in predicting STEM persistence. Purpose: This study focused on determining the influence of patterns of AP exam completion and performance on indicators of postsecondary academic achievement. These patterns were examined in the context of gender differences and for the prediction of grades, STEM persistence and graduation rates. Subjects: The sample consisted of 26,693 students who entered the Georgia Institute of Technology (Georgia Tech) as firstyear undergraduate students during the period of 19992009. Research Design: Archival admissions records and college transcripts were obtained for entering firstyear (nontransfer) students, to examine patterns of AP exams completed and performance on the exams, as they related to indicators of college academic performance, inflow and outflow STEM majors and nonSTEM majors, and attrition/timetodegree criteria. For predicting college performance, patterns of AP exams were examined in isolation, exams grouped by domain, and instances of multiple examinations completed (e.g., three or more AP exams in the STEM area). These patterns of AP exams were evaluated for predictive validity in conjunction with traditional predictors of postsecondary performance (e.g., highschool GPA and SAT scores). College course enrollment patterns were also examined, in conjunction with AP exam patterns, to determine the associations between AP exam performance and coursetaking patterns in postsecondary study. Data Collection and Analysis: Admissions records were obtained from Georgia Tech, including highschool grade point average information, along with college transcripts, including initial and final major declaration, attrition, and graduation data. Course enrollments were classified by level and by domain. Advanced Placement exam and SAT records were obtained from the College Board, and matched to the Georgia Tech records. Conclusions/Recommendations: Although student completion of AP exams was positively related to postsecondary grades and graduation rates, this overall pattern masks the relation between AP exam performance and postsecondary success. Students who did not receive credit tended to perform at a level similar to those students who did not complete any AP exams. Increasing numbers of APbased course credits were associated with higher GPAs at Georgia Tech for the first year and beyond. Students with greater numbers of APbased course credits tended to complete fewer lowerlevel courses and a greater number of higherlevel courses. Such students graduated at a substantially higher rate and in fewer semesters of study. Average AP exam score was the single best predictor of academic success after high school GPA (HSGPA). The most important predictors of STEM major persistence were receiving credit for AP Calculus and if the student had successfully completed three or more AP exams in the STEM areas. Men had substantially higher rates of these AP exam patterns, compared to women. Given that slightly over half of the AP exams are now completed by high school students prior to their senior year, it is recommended that admissions committees consider use of actual AP exam performance data, in addition to, or instead of AP enrollment data as indicators for predicting postsecondary academic performance. BACKGROUND The Advanced Placement program has been in existence since the 1950s (DiYanni, 2009), but the program has markedly changed over time, especially in the past decade. Although the original goals of the program (to allow students to obtain collegelevel credit for advanced study during high school) have not changed, the program has expanded in scope, from an initial set of 10 exams in core areas of study (e.g., English composition, literature, Latin, French, German, Spanish, mathematics, biology, chemistry, and physics [DiYanni, 2009]) to 33 exams that span the original areas, but also other diverse domains such as Art History, Environmental Science, Human Geography, and Macroeconomics. In addition, there has been an explosive growth in the number of AP exams administered, from about 10,000 in 1960 to a halfmillion exams in 1990, 1.5 million exams in 2002 (DiYanni, 2009), and 3.36 million exams in 2011 (completed by 1.97 million students; College Board, 2012a). For the purposes of the current investigation, it is important to distinguish between two different aspects of the AP program: participation by students in AP courses during high school, and performance on the AP exams. Historically, most AP exams were completed at the end of the students senior year, after the college admission process has been completed. Thus, participation in the AP courses became one of a set of variables that are considered by many selective institutions as part of the students admissions portfolio (e.g., see Kopfenstein & Thomas, 2010). In many cases, bonus points are awarded to a students HSGPA for enrollment in AP courses (along with other advanced courses). The use of information about AP participation has perhaps been an unintended consequence of the popularity of the AP program. Advocates of the use of this information have argued that a students enrollment in AP courses is indicative of a more rigorous high school curriculum. Critics have noted several concerns about the reliability and validity of such indicators, in addition to public policy concerns about the availability of AP courses in schools with fewer curricular resources, compared to schools with greater curricular resources. (See for example, Sadler, Sonnert, Tai, and Klopfenstein [2010] for a discussion of these issues.) The other aspect of the AP program (performance on AP exams themselves and the provision of collegelevel credit) has been perhaps less controversial. However, a cursory review of AP credit policies at colleges and universities around the United States indicates that there is substantial variance in the score thresholds set for awarding credit, and even whether credit is awarded, regardless of AP test scores. Nonetheless, a large number of postsecondary institutions provide collegelevel course credits for successful performance on AP exams. This policy has the potential for students to do one of several things, such as: (a) graduate more quickly than students without APbased credit; (b) complete more advanced courses than students without APbased credit in the same number of college/university terms; or (c) take fewer credits per term in order to have a lighter workload or to pursue other activities (e.g., take oncampus or offcampus jobs, internships or coop opportunities, etc.). Several studies have been reported indicating that students with APbased credits tend to have higher grades and higher graduation rates, compared to students who do not receive such credits (e.g., see Ewing, 2006; Eykamp, 2006; Morgan & Klaric, 2007; Sadler & Tai, 2007; Shaw & Barbuti, 2010), although there remain questions about the potential for other influences being partly responsible for such associations (e.g., ability differences, demographics) that covary with AP participation. This research project does not address the causal determinants of individual differences in enrollment in AP courses, school differences in AP course availability, nor do we investigate patterns of students who enroll in AP high school courses, yet do not complete the AP exams (e.g., see discussions by Camara & Michaelides, 2005; Geiser & Santelices, 2004). Instead, our focus is on the number of AP exams completed, and the scores obtained on the exams, in their relations to key college success criteria of grades and graduation rates at one selective state postsecondary institution. We also examine broad aspects of course enrollment patterns of students, conditionalized on their APbased credits. INTRODUCTION For students who complete AP exams in their major area, it is possible to bypass survey level courses and begin advanced study in their major much earlier in their college experience. Successful completion of AP exams is also an influential predictor of college success overall (e.g., see Morgan & Klaric, 2007). However, little is known about the optimal patterns of AP exams or portfolios of AP courses for success in science, technology, engineering, and math (STEM) areas, and even less is known about how students (and other stakeholders, such as parents, teachers, counselors) select specific AP courses. The fact that there remain large discrepancies between young women and young men in patterns of AP exam completion suggests that optional/elective curriculum decisions made at the highschool level may have important consequences for later enrollment and success, particularly in STEM areas. For example, although women took 314,947 more AP exams (54.7%) than young men (45.3%) in the 2011 administration of the AP exams, the distribution across STEM areas was strikingly different (College Board, 2012a). In the STEM areas, men took 21,843 more AP Calculus exams. The numbers in other areas were Chemistry, 7,160; Physics, 44,870; and Computer Science 13,139. Only in the STEM domains of Biology (28,766) and Statistics (1,637) were more AP Exams taken by young women than young men. The bulk of the additional AP exams taken by young women than young men were in the arts, humanities, foreign languages, and social sciences. The goal of this project was to determine whether there are configurations of Advanced Placement (AP) exam completion that are optimal for success in the STEM majors, that is, that are most highly associated with success (in terms of GPA, persistence in STEM majors, and degree attainment). If it can be determined that there are optimal APtype portfolios for success in the STEM areas, we expect that such information could be disseminated to stakeholders at the high school level, to provide the foundation that many students (especially young women) who wish to pursue STEM majors might not otherwise have. To accomplish this goal, we sampled the entire set of students with an initial matriculation (i.e., no transfer students) from 1999 to 2009 at the Georgia Institute of Technology (Georgia Tech)a selective, STEMintensive public institution. Transcript and admissions records were obtained, along with selected College Board data for these students, to examine patterns of AP exam completion, exam scores, and other predictors (HSGPA and SAT scores), against criteria of major choice, grades, and graduation rates at Georgia Tech. In this paper, we describe the sample and address the relations between APexam related variables and criteria of grades and graduation rates, with specific attention to gender differences and to outflow of students from initial STEM majors to nonSTEM majors. SAMPLE The initial sample included all students entering as firstyear students (i.e., no transfers) at Georgia Institute of Technology from fall, 1999 to fall, 2009 (total N = 26,985; 18,869 Men [69.9%], 8,116 Women [30.1%]). For analysis of APprogram related issues, students who completed an International Baccalaureate (IB) program and did not complete any AP exams were excluded from the sample. Specifically, of the 551 students who completed an IB program, 292 completed no AP exams. Thus, the total sample with usable data totaled N = 26,693. AP EXAM COMPLETION After exclusion of the IB students who did not participate in the AP Exam program, the mean number of AP exams completed by students who matriculated at Georgia Tech was 3.23 exams, with a standard deviation of 2.96 exams. A total of 7,703 students completed no AP exams, and the maximum number of exams completed was 20. Because there were relatively small numbers of students who completed more than 10 AP Exams (N = 376), for most analyses, we truncated the distribution to include counts up to 10 AP Exams, and then a final category of 10 or more exams. For those students completing AP exams, the average score obtained was 3.46 (sd = .871). For comparison purposes, the national average score on AP exams in 2011 was 2.84 (College Board, 2012a). RESULTS The analyses of these data proceeded along two major themes, as follows: (a) The first theme concerns the associations of AP exam completion and AP exam performance as predictors with college performance indicators of GPA, rates of graduation, and timeto graduation as criteria. (b) The second major theme pertains to the determination of patterns of AP exam portfolios that may be associated with overall college performance and with STEM major inflow and outflow. Interspersed among these analyses are treatments of gender differences in the AP predictor measures and college performance criteria. Hypotheses that were generated prior to the analyses of the data are presented here, but they are derived from a variety of different informal and technical report sources (citations provided where appropriate), and are not based on any overarching theory per se. Before addressing the general findings of the study, we take account of changes in the nature of the different cohorts of students that makeup the overall sample. Table 1 provides a snapshot of the differences of the two extreme cohort groups (1999 vs. 2009). During this period, Georgia Tech had relatively stable enrollment totals, but saw an increase in selectivity, in terms of rejection rates (an increase of 9.7%), a mean increase of 51 points on the SAT (Verbal + Quantitative). There was also an increase of 5.2% in the percentage of women students enrolled. Most notably, in concert with the expansion of AP programs in general (1,149,515 AP exams were completed in 1999; 2,929,929 AP exams were completed in 2009; College Board, 2012b), there was an increase in average number of AP exams completed by matriculating students (from 2.28 in 1999 to 4.05 in 2009). Average exam scores were also higher in the more recent cohorts (3.30 in 1999; 3.59 in 2009). As a result of these differences, we computed results for individual cohort groups in addition to the results across the cohort groups. Although small differences were noted in magnitudes of correlations, for example, none of these differences affected the overall pattern of results. Therefore, the results reported in the rest of this paper are based on the total sample (or, for graduation rates, on the 19992005 cohorts). Table 1. Snapshot and comparisons of 1999 and 2009 cohort groups.
Note. **p < .01, sd in parantheses The first hypothesis pertained to AP participation (indexed by completion of one or more AP exams). The specific hypothesis was as follows: HYPOTHESIS 1: Completion of AP exams will be associated with higher postsecondary grades and a higher likelihood of completing a baccalaureate degree (i.e., graduating). To address this hypothesis, we created contrasts for data from those students who completed no AP exams (and did not participate in the alternative IB program) from those who completed at least one AP exam. In addition, we examined how performance on the AP exams was related to the key criterion variables of GPA and graduation rates, using either scores of 3 or higher, or scores of 4 or 5 as indicators of qualified or well qualified/extremely well qualified performance levels. POSTSECONDARY GRADES Average grades for students completing at least one AP exam were markedly higher at each year at Georgia Tech. (Year 1, M = 2.77, 2.99, t (25095) = 23.86, Cohens d = .32; Year 2, M = 2.82, 3.02, t (19122) = 19.97, d = .32; Year 3, M = 2.88, 3.06, t (15891) = 18.23, d = .31; Year 4, M = 2.80, 2.94, t (8413) = 11.25, d = .27, respectively for no AP exams and one or more AP exam groups). For students in the 19992005 cohorts (later cohorts not having reached a fiveyear graduation threshold), completion of at least one AP exam was also associated with higher cumulative GPAs, M = 2.75 vs. 2.98, t (16054) = 18.60, d = .31 (for no AP and at least one AP exam groups, respectively). Generally, there was a monotonically increasing relationship between the number of AP exams completed and the firstyear GPA obtained at Georgia Tech (r = .23). Figure 1 shows the mean firstyear GPA by number of exams. In addition, the figure plots the number of AP exams where scores of 3 or higher were obtained (r = .29), and the number of AP exams where scores of 4 or higher were obtained (r = .34). In each curve, it is clear that more exams completed leads to higher average firstyear GPA levels, on average. However, keeping in mind that the mean firstyear GPA for students who did not complete any AP exams was 2.77 (sd = .700), it is also clear that completing AP exams without scoring 3 or higher on any of the exams is associated with GPAs that are poorer on average (N = 944; M = 2.48, sd = .633) than those students who completed no AP exams (t (7959) = 11.87, d = .43). In addition, those students who completed AP exams, but did not obtain a score of 4 or 5 on any of the exams also obtained significantly lower firstyear GPAs (N = 3,649, M = 2.65, sd = .647), in comparison to those students who completed no AP exams (t (10664) = 8.38, d = .18). Figure 1. Firstyear GPA by number of AP exams completed overall, number of exams completed with scores of 3 or higher, and number of exams completed with scores of 4 or 5.
Based on these results, participation in the AP program is associated with higher grades in postsecondary study, with the qualification that those students who completed AP exams, but did not obtain a score of 3 or higher on any of the exams tended to obtain grades that were, on average, lower than those students who did not complete any AP exams. Completion of larger numbers of AP exams with scores of 3 or higher was associated with higher firstyear GPAs, and completion of larger numbers of AP exams with scores of 4 or 5 yielded even higher firstyear GPAs. For example, students who completed at least 3 exams with scores of 3 or higher obtained mean firstyear GPAs of 3.12 (sd = .64), while students who completed at least 3 exams with scores of 4 or 5 obtained mean GPAs of 3.25 (sd = .61), d = .21. GRADUATION RATES Graduation (completion of a BS degree) for the 19992005 cohorts was at a rate of 78.7%. Overall participation in AP (completion of one or more AP exams) in this sample was at a rate of 67.9%. The first analysis involved evaluating whether AP participation was associated with higher likelihood of graduating within a five or more year period. For students across these cohorts with no AP exams, 3,778 graduated and 1,419 did not, for a total graduation rate of 72.7%. In contrast, for students who completed one or more AP exams, regardless of scores on the AP exams, 8,956 graduated, and 2,025 did not, for a total graduation rate of 81.6%. That is, students who completed at least one AP exam graduated at a rate that is 8.9 percentage points higher than students who did not complete any AP exams. However, when graduation rates are conditioned on the number of AP exams with either 3 or higher or only 4/5 AP exam scores, the data indicate that just completing AP exams, per se, does not yield an increase in graduation rates. For the 714 students in the 19992005 cohorts who completed at least one AP exam, but did not exceed a score of 3 on any of the exams, the graduation rate was slightly lower than that of students who did not complete any AP exams at all (68.4% vs. 72.7%, respectively). With increasing numbers of AP exams with scores above 3, graduation rates exceeded those of the students who did not take the AP exams, and increased in a quadraticshaped function (i.e., large gains in graduation rates as number of AP exams increased, but diminishing returns for additional AP exam with scores >= 3). When considering only the number of AP exams with scores of 4 or 5, graduation rates were higher by a few percentage points across the total number of AP exams, compared to using a criterion of AP scores of 3 or higher. Based on these results, it is clear that both participation overall, and increasing numbers of AP exams with qualifying grades (3 or higher) were associated with higher graduation rates compared to students who did not complete any AP exams, with the qualification that completing AP exams but receiving no scores of 3 or higher resulted in slightly lower graduation rates than for students who did not complete any AP exams. A further analysis of graduation rates was especially illuminating, with respect to successful completion of AP exams. For the 19992005 cohorts, overall fouryear graduation rates were 31.7% and overall fiveyear graduation rates were 70.2%. A comparison of four and fiveyear graduation rates plotted against the number of AP exams with scores of 4 or 5, shown in Figure 2, demonstrates that increased numbers of AP exams with scores of 4 or 5 are associated with higher fiveyear graduation rates, compared to students who did not complete any AP exams. For example, students who had 4 or more AP exams with scores of 4 or 5 had a graduation rate of 82.8%, compared to students who did not complete any AP exams (63.1%). Figure 2. Upper panel. Four and fiveyear graduation rates, conditional on total number of AP exams with scores of 4 or 5. Lower panel. Frequencies for each category.
The graduation rate gradient associated with increased numbers of AP exams with scores of 4 or 5 and fouryear graduation was much stronger than that of the fiveyear graduation curve. The fouryear graduation rate for students who did not complete any AP exams was quite low (20.1%), but there was a near linear increase in graduation rate with each increasing number of AP exams with scores of 4 or 5. For example, the students (N = 1,215) with four or more AP exams with scores of 4 or 5 graduated within four years at a rate of 50.8%; a rate that was over twice that of the students who completed no AP exams. APBASED COURSE CREDITS For the second set of analyses, we focused on the number of semester hour course credits associated with AP exam performance. With few exceptions, Georgia Tech provides course credit when AP exam scores of 4 or 5 are obtained. The exceptions include Calculus BC and Music Theory (where scores of 3 are also awarded credit), Biology and Environmental Science (where scores of 5 only are awarded credit), a few exams where course credits are given for common courses (e.g., students may only receive credit for a single foreign language, regardless of how many different language exams are completed) and several other exams (e.g., Italian, Chinese, Human Geography, etc.) where no Institute credit is given. Most AP credits range from three and four semester hours, though a few areas are awarded six semester hour credits (e.g., foreign languages and Computer Science AB). For the 26,693 students included in this analysis, 11,745 (44%) received no APrelated course credit, and a small number (71) received 36 or more credits (the maximum obtained was 62 credit hours). Mean number of credit hours across the entire sample (including those who did not complete AP exams) was 5.58 (sd = 7.19), and the distribution was decidedly not normal (kurtosis = 2.76, skewness = 1.60). Mean number of credit hours for only those students who completed at least one AP exam was 7.85. For ease of comparisons, students awarded 36 or greater credit hours were classified into a single category. With this recategorization, the mean and standard deviation for the entire sample did not substantially change (M = 5.57; sd = 7.14 respectively), but the kurtosis was reduced (kurtosis = 2.11; skewness = 1.52). The second set of hypotheses pertained to the relationship between the number of semesterhour course credits awarded to students on the basis of their AP exam scores and their overall performance. Specifically: HYPOTHESIS 2: Receipt of more APbased course credits at Georgia Tech will be associated with overall improved performance and graduation success, compared to fewer APbased course credits. The correlation between number of AP exambased credit hours and firstyear GPA was r = .34. For the 19992005 cohorts (N = 16,056), the correlation between number of AP exambased credit hours and cumulative GPA was r = .29. Overall, it is clear that the number of APbased semester course credit hours is markedly related to first year and cumulative GPA at Georgia Tech, especially in comparison to traditional predictors of GPA (to be discussed in a later section). Because of outliers, the correlation somewhat obscures the pattern of results and this relationship can be better demonstrated by a regression plot of Year 1 GPA against number of course credits, shown in Figure 3. Figure 3. Mean firstyear GPA by number of APbased course credits (semester hours). Frequencies by number of credits inset. Line is represents quadratic regression.
We performed an additional analysis to determine whether there are gender differences in the relationship between the number of APbased course credits and firstyear GPA. As is generally found, women tended to obtain higher firstyear GPA than men (M = 3.02, sd = .62 for women; M = 2.89, sd = .71 for men; t (25,095) = 13.29, p < .01, d = .20.) There was a significant main effect of number of credits (F (6, 25083) = 449.65, p < .001), a main effect of gender (F (1, 25093) = 45.92; p < .001), but no significant interaction between number of credits and gender (F (6,25083), = .80, ns). That is, women had higher firstyear GPAs, compared to men, throughout the distribution of APbased course credits. Or, to put it another way, the positive effects associated with increasing APbased course credits accrued equally to men and women. AP EXAMS WITH NO COURSE CREDIT VERSUS NO AP EXAMS A further analysis was conducted of grades and graduation rates for students who did not complete any AP exams, two categories of those who completed one or more exams, but did not score well enough to obtain any course credits (exams with scores of 1 and 2 only, exams with scores of 3), and those who scored well enough to obtain course credit (scores of 4 or 5). These results are presented in Table 2. The group that performed least well was the composed of students who completed one or more AP exams, but received only scores of 1 or 2. Mean firstyear GPA for this group was 2.48, and these students had the lowest comparative graduation rates. Students who completed no AP tests represent a mixture of those who did not elect to take the exams and those for whom AP courses were not available (e.g., some foreign students). The no AP group performed at a level similar to those students who completed AP exams, but who did not receive any scores above 3. The final group, who scored 4 or 5 on at least one AP exam performed the best throughout their time at Georgia Tech and graduated with at the highest rate of all of the groups. Table 2. Selected academic criteria by AP exam performance categories.
Note. ^{a}Cumulative GPA, final STEM major and Graduation rates for 19992005 cohorts only. For the group of students who completed one or more AP exams, but received no course credit, the relationship between the number of AP exams they completed and firstyear GPA was essentially zero (r = .003, ns). These results may be especially salient for admissions staff, in that nearly half of all AP exams are completed by high school seniors, whose AP exam scores are not available until well after the end of the application season. Just knowing that a highschool senior is enrolled in one or more AP courses may not provide sufficient diagnostic information for predicting initial postsecondary performance. These results are consistent with those obtained by other investigators (e.g., see Adelman, 2006; Geiser & Santelices, 2004; Klopfenstein & Thomas, 2010). To further explore this effect, we created an additional graph (see Figure 4), where firstyear GPA was statistically adjusted for student differences on HSGPA and SAT scores (verbal and math). The regressionadjusted firstyear GPAs were then plotted against the number of AP exams completed and the scores obtained on the exams. These results appear to further support the inference that in the absence of AP exam score information, just knowing that the student was enrolled in one or more AP courses will not provide sufficient information for predicting initial postsecondary performance.^{1} Figure 4. Adjusted Firstyear GPA by number of AP exams completed, but with no credits awarded for any exam. (Firstyear GPA for students who completed no AP exams also provided. Adjustments are made on the basis of high school GPA and SAT (verbal and math) scores. Solid lines are linear regression lines. APBASED COURSE CREDITS AND TOTAL CREDITS AT GRADUATION One issue that arises about students who receive APbased course credits is whether with increasing AP credits, they tend to reduce the overall number of oncampus credits completed. From an overall course credits perspective, there is a clear pattern of decreasing total course credits (not counting those for which the students received APbased credits), but the decline is clearly not directly compensatory to the number of credits awarded. For students with no APbased course credits at graduation, the mean number of course hour credits was 129.05 (sd = 15.53). For students who received 16 course credits, the mean total number of course hours was 128.88 (sd = 13.91), for 712 course credits 127.10 (sd = 13.58), for 1318 course credits, 124.48 (sd = 15.16), for 1924 course credits, 121.57 (sd = 15.58), 2530 course credits, 120.56 (sd = 16.67), and for more than 30 credits, 119.34 (sd = 20.00). That is, even with 30 or more course credits, students on average still completed only about 10 fewer course credits at Georgia Tech. Students receiving 30 APbased course credits, graduated with an average of more than 149 credit hours, nearly 20 more credit hours than students who did not participate in the AP program.
APBASED COURSE CREDITS AND CURRICULAR PATTERNS Another claimed advantage of obtaining collegebased credits through the AP exam program is that students are able to obtain credit for surveylevel courses and thus enroll in more advanced courses during their undergraduate programs. The actual pattern of enrollments for the Georgia Tech students showed a general pattern of fewer 1level (freshman level) courses that is proportional, but not at a 1:1 ratio, to the number of APbased course credits. The total course hours by course levels for students receiving various amounts of APbased course credits are shown in Figure 5. Keeping in mind that students with many APbased course credits tended to complete fewer courses overall, students with no APbased credits completed an average of 36.22 hours of 1level courses. Students who obtained 16 credits completed 35.05 hours, 712 credits completed 32.56 hours, 1318 credits completed 29.90 hours, 1924 credits completed 26.96 hours, 2530 credits completed 25.40 hours, and students who obtained 30 or more credits, completed an average of 21.80 1level course hours. Figure 5. Course enrollment hours by number of APbased semester course credit hours, and by course level (1level = freshman, 2 = sophomore, 3 = junior, 4 = senior). Only students who obtained 19 or more APbased course credits completed fewer 2level courses than those with no or smaller numbers of APbased course credits. Even those with the highest level of APbased course credits completed only an average of 4 fewer 2level course hours. In contrast, those with the highest levels of APbased course credits tended to complete a greater number of advanced 3 and 4level courses. For example, the average difference between those with no APbased course credits and those with 30 or more credits was roughly 3.5 hours for 3level courses, and 5.4 hours for 4level courses. Thus, the principal differences between students who received APBased course credits and those who did not, pertained not only to the overall number of course hour enrollments, but also to the pattern of enrollment by course levels. Students with greater numbers of APbased credits enrolled in fewer 1level courses, but greater numbers of higherlevel courses. For students with no APbased credits, nearly 60% of courses completed were 1level and 2level courses. For students with larger numbers of APbased credits, enrollment in the 1level and 2level courses represented about 50% of their enrollment hours (e.g., 57.4% for students with 712 APbased credit hours, 53.4% for students with 1924 APbased credit hours). Gender Differences Consistent with national trends, we expected that gender differences would be found for patterns of AP exams completed (especially in the contrast between STEM and nonSTEM domains). We also expected that overall STEM/nonSTEM patterns of AP exams would, in turn, also be reflected in initial and final STEM majors. Specifically, HYPOTHESIS 3: Choices of particular AP courses (STEM vs. nonSTEM) in high school will reflect substantial gender differences. These gender differences will also be reflected in whether the students ultimately major in a STEM domain. First, it must be noted that the counts of AP exams completed by men and women who matriculated at Georgia Tech were essentially identical (M = 3.197 for men and 3.199 for women; sd = 2.92 and 2.79, respectively; t (26691) = .06, ns, d = .00). A small but significant difference was found for the respective numbers of AP exams with scores of 3 or greater (M = 2.66 for men and 2.54 for women; sd = 2.72 and 2.56 respectively; t (26691) = 3.38, p <.001, d = .04). The difference between genders was larger for the respective numbers of AP exams with scores of 4 or 5, but still a relatively small effect (M = 1.80 for men and 1.58 for women, sd = 2.28 and 2.08, respectively; t (26691) = 7.41, p < .001, d = .10). Just limiting the gender analysis to AP exams completed, the gender breakdown of topic domains is clear. To simplify examination of the general results, AP exams were grouped into 7 different thematic domains, as follows: Physical Sciences, Biology, Computer Science, Math, Social Sciences, Foreign Languages, and Humanities. Mean number of AP exams in each category and percent of the sample that completed exams in each area, conditioned by gender, are shown in Table 3. For exams in the Physical Sciences, Math, and Computer Science domains, men took more exams than women. For exams in the Humanities and Foreign Languages, and to a lesser degree, Social Sciences, women took more exams than men. Biology was the only STEM domain where the number of exams completed by women exceeded those taken by men. Thus, even though the average total number of AP exams completed by women was the same as those completed by men, the mean profiles indicate greater numbers of exams completed by men in STEM areas and greater numbers of exams completed by women in nonSTEM areas. These differences are largely concordant with current and historical patterns of gender differences in AP exam completion (e.g., see College Boards AP exam statistics; College Board, 2012a). Table 3. Mean count and Participation Rates of AP exams completed, by topic area and overall, by gender.
Note: N_{men} = 18,669; N_{women} = 8024. MAJORS AT MATRICULATION AND GENDER Given the strong STEM reputation and programs at Georgia Tech, it comes as no surprise that a majority of entering students declared an intention to major in STEM fields. In the 19992009 cohorts, 23,438 (87.8%) students declared STEM major intentions, compared to 8,024 (12.2%) who declared nonSTEM major intentions. The breakdown was roughly the same (though a little higher for STEM) for the 19992005 cohorts (88.3% vs. 11.7%). For the whole sample, there were significant differences in STEM/nonSTEM major intentions, when conditioned by gender. STEM major intentions were made by 92.2% of men, in comparison to 77.7% of the women (φ= .203, p < .0001). Breakdowns of mean number of AP exams taken by topic domain, conditioned by gender and STEM vs. nonSTEM major are shown in Table 4. These results are striking in several ways. First, for students who intended to major in nonSTEM areas, the overall average number of AP exams completed by women substantially exceeded those taken by men (2.19 for men vs. 2.78 for women, t (3253) = 6.53, d = .23). The overall average number of AP exams completed by those who intended to major in STEM areas was essentially identical for men and women (3.28 for men and 3.32 for women, t (23436) = .85, d = .01), but substantially higher than the average number of AP exams taken by students who intended nonSTEM majors. Table 4. Average Number of AP exams Completed, by STEM/nonSTEM Initial Major and Gender.
F exams df = 1, 26689; *p < .05; **p < .01 Women with nonSTEM major intentions completed more AP exams than men in all areas except for Physical Sciences and Computer Science (where the base rates were very low for both groups). For those students with STEM major intentions, women completed fewer AP exams in all of the STEM areas except for Biology, and more AP exams in Social Sciences, Foreign Languages, and Humanities domains. Across these major categories of AP exams, significant effects were found for main effects of STEM/nonSTEM major intentions and gender; significant effects were also found for the interaction between STEM and nonSTEM major intentions and gender. The vast majority of variance accounted for was by STEM/nonSTEM major intentions (for exams in the STEM areas), and by gender (for exams in the nonSTEM domains). The expectation regarding AP coursework in high school and university study was that students with greater numbers of AP exams overall, and STEM AP credits in particular would be more likely to enter and persist in STEM majors. Specifically: HYPOTHESIS 4: Students with fewer AP credits or those with only nonSTEM AP credits would be more likely to leave a STEM major for a nonSTEM major (and would be less likely change from a nonSTEM major to a STEM major). Table 5 shows the mean number of AP exams taken by students, conditioned by their major intention at matriculation and their final majors at graduation. For this analysis, there was a total of 13,728 students who had both initial and final major information available. As expected the outflow from STEM majors to nonSTEM majors was at a substantially higher rate (15.0%), compared to the outflow from nonSTEM majors to STEM majors (8.1%). However, the pattern of AP exam completion among the groups that shifted between STEM and nonSTEM majors is stark. Table 5. Average number of AP exams completed, by STEM/nonSTEM initial major and final major (inflow and outflow from STEM/nonSTEM majors). Number of students indicated in parentheses. Total AP Exams Completed
Note. Only students who have graduated are included in this analysis. From the earlier analysis, it is clear that the overall number of AP exams completed by those students who declared STEM major intentions (M = 3.14) was higher than the number of AP exams completed by students who declared nonSTEM major intentions (M = 2.39). In analyzing the number of AP exams by students who changed from STEM to nonSTEM majors and vice versa, the patterns are striking. The number of AP exams completed by those students who left STEM majors to nonSTEM majors (2.42) closely resembled those students who had nonSTEM major intentions at matriculation. Similarly, the number of AP exams completed by those students who left nonSTEM majors for STEM majors (3.30) very closely resembled those students who had STEM major intentions at matriculation. Although these are historical data, one might be tempted to argue that the differences in preparation of these groups allowed for (in the case of nonSTEM to STEM major changes) or was instrumental in (in the case of STEM to nonSTEM major changes) the shift to or from an orientation towards STEM majors. INFLOW AND OUTFLOW STEM/NONSTEM BY NUMBER OF APBASED CREDITS Table 6 shows the subtotals of students in the sample who completed an undergraduate degree, conditioned by initial and final major category (STEM vs. NonSTEM), and by number of AP exambased course credits awarded by Georgia Tech. For students expressing an initial STEM major (N = 12,052), 85% completed a degree in one of the STEM areas. The rate was lower (79.1%) for those initial STEM majors who received no APbased course credit, and the rate increased with increasing numbers of STEM course creditsto 95.9% STEM degrees for those initial STEM major students being awarded more than 18 credit hours in the STEM areas. Table 6. Frequencies and Percent of students who completed degrees in the same major as original intention (STEM vs. NonSTEM) by APbased course credits in STEM domains and nonSTEM domains, by initial STEM vs. NonSTEM major
For students who had initial STEM major intentions, those who received no nonSTEM course credits completed a STEM degree at a rate of 83.1%. Interestingly, increasing numbers of nonSTEM AP course credits actually related to a higher rate of STEM degree completion. Students with 7 or more credits in the nonSTEM area completed STEM degrees at a rate greater than 90%. In contrast, students who had a nonSTEM initial major (N = 1,676) completed nonSTEM degrees at a rate of 91.9% (that is, 8.9% outflow from nonSTEM to STEM degrees). The outflow from nonSTEM initial major to STEM degrees was greater for the small number of students with higher rates of STEM APbased course credits (e.g., 15% outflow for students receiving 16 course credits, and 19.4% for students receiving 712 course credits). Interestingly, for the nonSTEM initial major students, attaining a moderate number of nonSTEM course credits resulted in more outflow from nonSTEM to STEM areas (e.g., for 0 nonSTEM course credits, the outflow from nonSTEM to STEM was 6.1%, but for 16 nonSTEM course credits, the outflow from nonSTEM to STEM was 15.9%). Together these results suggest that, ceteris paribus (i.e., all else being equal), for students intending to major in STEM areas, more AP credits (both in STEM and nonSTEM areas) are associated with lower outflow from STEM. For students intending to major in nonSTEM areas, more AP STEM credits were associated with a higher likelihood of shifting from nonSTEM to a STEM major, as were increasing numbers of nonSTEM AP credits, though the numbers of students shifting from nonSTEM to STEM majors was relatively modest. Outflow from STEM to nonSTEM majors was associated with several indicators, including the number of AP exams with scores of 4 or 5 (outflow students M = .99, sd = 1.59; students who remained in STEM majors M = 1.89, sd = 2.25; t (12,050) = 16.27, d = .46), and the average scores on AP exams (outflow students M = 3.11, sd = .88; students who remained in STEM majors M = 3.59, sd = .84; t (8,669) = 18.09, d = .56). The outflow students had lower SAT Verbal scores (M = 630 vs. 641; t (10,968) = 5.31, p < .01; d = .15), SAT Math scores (M = 666 vs. 699; t (10,968) = 20.74; d = .55), and lower HSGPAs (M = 3.78 vs. 3.83, t (10,547) = 8.13, p < .01; d = .22),^{2} but the archival data indicate that the most salient correlate of outflow from initial STEM major to final nonSTEM major is firstyear GPA. For those students who had both initial and final STEM majors, the mean firstyear GPA was 3.11 (sd = .56), but for the students with initial STEM major and final nonSTEM major, the mean firstyear GPA was 2.55 (sd = .72), t (12,045) = 38.18, p < .01, d = .87, a difference of means of almost one standard deviation in magnitude. Although a higher percentage of women (20.1%) with initial STEM majors switched to nonSTEM majors than men (13.0%), the pattern of GPA differences for both gender groups was quite similar (M = 2.48 for men, M = 2.69 for women). In addition, considering only completion/performance on the AP Calculus exams (AB and BC), 60.4% of the men in general did not receive a 4 or higher score on either Calculus exam, but 76.7% of the men who switched from STEM to nonSTEM majors were in this group. For women, 70.1% of the women in general did not receive a 4 or higher score on either Calculus exam, but 81.7% of the women who switched from STEM to nonSTEM majors were in this group. STEMBASED COURSE CREDITS AND APBASED STEM CREDITS Table 7 shows the number of college STEM course credits (excluding the APbased course credits) completed at graduation across the seven major categories of degrees (Physical Sciences, Biology, Technology, Engineering, Math, Social Sciences, and Languages). For those students who completed degrees in the STEM areas, excluding Math (the first four categories), increasing numbers of APbased STEM course credits were associated with either a minor drop in college STEM course credits (about 36 hours), or no consistent differences. For the small numbers of Math degree recipients, increasing APbased STEM credits were associated with small increases (about 3 hours) in college STEM course credits. For Social Sciences degree recipients, there appeared to be an increase in the number of college STEM course credits with increasing APbased STEM credits, though the largest differences are associated with a very small number of students (i.e., those receiving more than 13 APbased STEM course credits). Table 7. Mean Total STEM course credits (not including AP course credits) at graduation by Major domain and by number of STEM APBased course credits awarded (Number of students in parentheses).
Notes. Phys. = Physical; Tech. = Technology; Eng. = Engineering Soc. Sc. = Social Science. Categories with 10 or fewer students have been omitted. Collapsing across the five STEM degree categories, and examining the results by gender, indicates that men completed about 4 more college course credits in the STEM area (M = 94.89, sd = 15.64) than women (M = 91.41, sd = 17.83) overall, but that women with more APbased STEM credits showed no appreciable difference compared to those with no APbased STEM credits. Men who received 18 or more STEM credits showed about a 3hour difference from those who received no APbased STEM credits. PORTFOLIOS The concept of a portfolio of AP exams represents the focus of the next set of analyses. We start with analysis of the impact of individual AP exams, then pairs of AP exams, and finally multiple exam patterns, to examine whether there are exams or combinations of exams that are associated with success in STEM and nonSTEM areas. Specifically: HYPOTHESIS 5: Individual AP exams, pairs of AP exams, and sets of three AP exams will show substantial differences, especially in comparing STEM and nonSTEM areas. Students with specific STEMdominated portfolios of AP courses will have a higher likelihood of performing well in STEM majors, compared to students with fewer AP credits, or the majority of AP credits in nonSTEM areas. INDIVIDUAL EXAMS The results of analyses of student performance and graduation rates, conditioned on APbased exam credit for individual AP exams, are presented in Table 8. The table provides frequencies for students with initial STEM major and nonSTEM major intentions, point biserial correlations with firstyear GPA, and 5year graduation rates, respectively, for students who did or did not obtain course credit for the individual AP exams. It is important to keep in mind that the students who are identified as not receiving credit include both those who did not take the AP exam and those who completed the exam, but did not obtain a score high enough to receive Georgia Tech course credit. Table 8. Single APbased credits, by STEM/NonSTEM initial majors. Frequencies, Point Biserial Correlations with firstyear GPA, Mean FirstYear GPA, and Graduation rates.
Notes: correlations not otherwise indicated are significant p < .01; *=p < .05; ns = not significant ^{}Graduation rate for cohorts 19992005 only. The first noteworthy aspect of the table is that, as would be expected, relative frequencies for credit on the STEM AP exams were much higher for students with initial STEM major intentions, compared to those with nonSTEM major intentions. The STEM exams with the largest number of STEM students receiving credit were Calculus, Chemistry, and Physics. Initial STEM majors received credit for Calculus at a rate of 43.6%, while students with nonSTEM initial majors received credit for AP Calculus at a rate of 17.7%. Initial STEM majors received credit for Chemistry and Physics at a rate of 12.0% and 9.7%, respectively, while students with nonSTEM initial majors received credit at rates of 2.0% and 0.9% for Chemistry and Physics, respectively. Comparatively large percentages of initial STEM major students also received credit for nonSTEM AP exams in the areas of U.S. History (17.3%), English (18.4%), and Economics (10.2%). Among nonSTEM initial majors, the most frequent credits were obtained in U.S. History (16.9%), English (20.2%), Calculus (as noted above), and Psychology (7.7%). Point biserial correlations and mean firstyear GPAs for students receiving or not receiving credit for the individual AP exams indicate that with few exceptions, receiving credit for any of the AP exams was associated with higher firstyear GPAs. For STEM majors, the overall mean firstyear GPA was 2.92, but for students receiving AP credit, the mean GPAs were in a range from 3.15 to 3.43. For nonSTEM majors, the overall mean firstyear GPA was 3.01, but for students receiving AP credit, the mean GPAs were in a range from 3.20 to 3.55 (excluding those exams for which there were fewer than 50 students receiving credit). Graduation rates for those students receiving AP credit were approximately 10% higher than those students who did not receive AP credit, across all of the AP exams and for both STEM and nonSTEM majors. It is interesting to note that the highest graduation rates (for AP exams where more than 50 students in a category received credit) for STEM majors were obtained by those students who received credit for the Latin AP exam (91.5%), and for nonSTEM majors, those who received credit for the Calculus BC exam (91.5%). One needs to keep in mind also that, ceteris paribus, the exams with the relative frequencies most discrepant from 50/50 will have mean GPAs and graduation rates with the highest variability. To evaluate the relative importance of successful performance on individual AP exams (where successful performance was operationalized as scores of 4 or 5) for predicting firstyear GPA, multiple regressions were conducted with stepwise entry of the individual exams, for initial STEM and nonSTEM majors, separately. For STEM majors, 17 of the AP exams provided incremental predictive validity for prediction of firstyear GPAs. For nonSTEM majors, 12 of the AP exams provided incremental predictive validity. Across the two groups, 3 of the 5 most highly predictive exams were the same (Calculus AB, Calculus BC, and English Literature). The other two highest predictors for STEM majors were Chemistry and U.S. History; the other two highest predictors for nonSTEM majors were U.S. Government and Biology. These subjects represented some, but not all of the most frequently completed AP exams for the two groups (see Table 8). In the aggregate, the AP exams account for 13.3% and 13.8% of the variance in firstyear GPAs for STEM and nonSTEM initial majors, respectively. A parallel set of analyses predicting first yearGPA for STEM and nonSTEM majors was conducted with AP exams categorized into topical domains. In these analyses, we also created regressions for men and women in each group separately. The results of these analyses are provided in Table 9. The key results from these analyses show that the most influential predictors for performance in the STEM and nonSTEM majors were the same for men and women in each group. The main difference between salient predictors for STEM and nonSTEM majors was that the number of AP exams with scores of 4 or 5 in the Physical Sciences and in Foreign Languages were significant predictors for the STEM majors, but not for the nonSTEM majors. In the aggregate, AP exams by categories accounted for 12.2% (men)/15.2% (women); 14.4% (men)/10.6% (women) of the variance in firstyear GPAs for men and women STEM and nonSTEM initial majors, respectively. Table 9. AP Exam Counts by Category (with scores of 4 or 5) leading raw and standardized predictors (via Stepwise entry) of FirstYear GPA for STEM and nonSTEM initial majors, by gender and by order of contribution estimates. Initial STEM Majors
Initial NonSTEM Majors
**p < .01 To provide an overarching perspective on examtakers who completed AP exams in different domains, we computed the raw and relative frequencies of students who completed exams across the broader domains of Physical Sciences, Biology, Computer Science, Math, Social Sciences, Foreign Languages, and Humanities. Most noteworthy among these results were the high levels of crossdomain examtaking between Physical Sciences and Math (88.8%), and the low relative frequency of crossdomain examtaking between Foreign Languages and every other domain except for Humanities (72.1%). In addition, there was a fair amount of crossdomain examtaking between Physical Sciences and Humanities (61.5%), and Physical Sciences and Social Sciences (71.1%). These results suggest that patterns of AP examtaking were relatively broad across topic domains. Such results provide a reasonable basis for proceeding with analyses of multipleexam patterns. EXAM PAIRS A set of parallel analyses of frequencies, relations with firstyear GPA, and graduation rates were conducted for exampairs that had reasonably high joint frequencies (over 1,000 for STEM majors, over 100 for nonSTEM majors). For these analyses, exams in the same domain were combined (e.g., English Language and English Literature; Calculus AB and Calculus BC; Physics C: Mechanics and Physics C: Electricity and Magnetism), so that students who completed either of the exams within a category were counted as yes and students who did not complete either of the exams were counted as no. Among initial STEM majors, the most frequent exam pairs that were awarded credit were those paired with Calculus: English/Calculus, U.S. History/Calculus, Chemistry/Calculus, and Physics/ Calculus. Among nonSTEM initial majors, the most frequent exam pairs were English/U.S. History, English/Calculus, and U.S. History/Calculus. For the initial STEM majors, the mean firstyear GPA overall was 2.92, and the range of mean firstyear GPAs for the exam pairs was from 3.21 to 3.45. For the initial nonSTEM majors, the mean firstyear GPA was 3.01, and the range of exam pairs was 3.33 to 3.52 (for exam pairs with more than 50 students receiving credit). That is, for students who received credit for the various exam pairs, the average firstyear GPA was roughly 0.40 higher than for those students who did not receive credit. The only overall pattern that emerged from these analyses was that students who were initial nonSTEM majors and who received credit for exam pairs that included Calculus tended to have higher grades (about 0.1 GPA points) than those who received credit for exam pairs that were only in nonSTEM areas. Graduation rates for the various exam pairs were uniformly higher for both STEM and nonSTEM initial majorswith a graduation rate advantage of about 10% higher than for students who did not receive credit for the exam pairs. THREE OR MORE EXAMS To determine whether particular multipleexam portfolios of APbased exam credits were more or less advantageous, we computed six different groupings that included the following combinations of exams: (a) 3 or more AP; (b) 3 or more STEM AP; (c) 3 or more nonSTEM AP; (d) 3 or more AP without Calculus; (e) 2 or more STEM, 1 or more nonSTEM; and (f) 2 or more nonSTEM, 1 or more STEM. The results of these analyses are presented in Table 10, along with a breakdown of percentages for men and women for each grouping. The results are consistent with the earlier analyses that point to the association between larger numbers of APbased exam credits leading to higher firstyear GPAs, and higher fiveyear graduation rates. Table 10. Multiple exam portfolios. Upper panel: Frequencies, Point Biserial with Year 1 GPA, Mean Year 1 GPA, and graduation rates (19992005 cohorts only for graduation rates) by STEM/NonSTEM initial majors. Lower panel: Percentages of portfolios by gender.
Notes: Correlations not otherwise indicated are significant p < .01; ns = not significant ^{}Graduation rate for cohorts 19992005 only. There were two salient results from these analyses. The first result pertains to the initial STEM majors and the lack of a benefit associated with obtaining credit for three or more AP exams, if the students did not receive credit for Calculus. For these students, the firstyear GPA (M = 2.95) was nearly identical to that obtained by the overall firstyear GPA (M = 2.92). These students also had a graduation rate that was less than 5 percentage points higher than the overall graduation rate. NonSTEM initial majors who similarly obtained credit for three or more AP exams but not Calculus, in contrast, had much higher GPAs and graduation rates that were 11.1 percentage points higher than the overall sample. The second result from this analysis concerns the frequency breakdown by gender. In particular, gender differences were modest across all the groupings, with the exception of the 3 or more STEM AP exam credits. For this grouping, 11% of the men who were initial STEM majors received credit, while only 5.2% of the women who were initial STEM majors received credit. That is, over twice as many men than women obtained credit associated with 3 or more STEM AP exams. SUMMARY For combined STEM and nonSTEM majors, successful completion of AP exams in Calculus and English Literature were the factors most highly associated with higher grades in the first year of study at Georgia Tech. For STEM majors, successful completion of AP exams in Chemistry and U.S. History were also associated with higher grades; for nonSTEM majors, successful completion of AP exams in U.S. Government and Biology were also associated with higher grades. From a higherlevel perspective, more successful AP exams in Math and Physical Sciences domains contributed the most to higher firstyear grades for the STEM majors, and more successful AP exams in Math, Social Sciences, and Humanities were the most salient positive correlates of firstyear grades for nonSTEM majors. The combination of successful AP performance in Calculus and an AP exam in the humanities or social sciences was especially related to high firstyear GPAs for nonSTEM majors, but various pairs of exams were nearly equally beneficial for the firstyear GPAs of STEM majors. When considering multiple sets of AP exams (portfolios) and STEM majors, three or more AP exams in the STEM domain or two or more AP exams in the STEM area, when combined with more than one AP exam in a nonSTEM domain, were similarly associated with higher firstyear GPAs and graduation rates. The combination of 3 or more AP exams without calculus was not an indicator for higher grades for STEM majors, and only indicative of a moderately higher graduation rate. That is, for STEM majors who failed to receive credit for one of the AP Calculus exams, their performance was equivalent to students who did not participate in the AP exam program at all, or who failed to receive any APbased credits. (See Sadler & Sonnert, 2010; Sadler & Tai, 2007 for similar conclusions with other samples.) For nonSTEM majors, completion of 3 or more AP exams in STEM areas yielded higher GPAs and graduation rates, compared to those students who did not participate in the AP exam program, but so did completion of 3 or more AP exams in the nonSTEM areas. That is, for nonSTEM majors, the number of successful scores on the AP exams was the main contributing factor to increased grades and graduation rates, and the AP exam domain in particular appeared to have little overall influence. However, it is perhaps worth noting that although 9.4% of initial STEM majors successfully completed 3 or more STEM AP exams, only 0.7% of the initial nonSTEM majors did so. Based on the general tendencies for women to have a lower proportional enrollment in STEM majors, and a higher rate of outflow from initial STEM majors to nonSTEM majors than men, we expected that gender differences in AP portfolios would reflect these differences. Specifically: HYPOTHESIS 6: Women will have a lower relative frequency of optimal portfolios for STEM areas than men. From the 2011 national AP exam statistics, the Calculus AB exam was completed by 14.9% of the young men, and 11.0% of the young women. For the Calculus BC exam, 5.5% of the young men and 3.0% of the young women completed the exam. By way of comparison with the 19992009 data on Georgia Tech students reported in this investigation, 35.9% of the men and 35.4% of the women completed Calculus AB, and 23.5% of the men and 17.8% of the women completed Calculus BC. These data most likely reflect both the selectivity of the institution and the reputation of the institution, especially for majors in the STEM domains. If women at Georgia Tech had completed the Calculus BC exam at the same rate as the men at Georgia Tech, an additional 457 women would have completed the Calculus BC exam over the 19992009 period. Of the 1,141 men who started with a STEM major intention, but ended up with a nonSTEM major at graduation (13.0% outflow), only 114 of them received credit for Calculus BC (9.99%). Of the 663 women students who started with a STEM major intention, but ended up with a nonSTEM major at graduation (20.1% outflow), only 38 of them received credit for Calculus BC (5.73%). Similarly, where 11.0% of the men with an initial STEM major received credit for 3 or more AP exams in the STEM domain, only 5.2% of the women did so. If the women with initial STEM majors had completed 3 or more STEM AP exams at the same rate that the men with initial STEM majors, about 361 additional women would have had this portfolio of exams. Of the 1,141 men who started with a STEM major intention, but ended up with a nonSTEM major at graduation, only 46 of them received credit for 3 or more STEM AP exams (4.03%). Of the 663 women who started with a STEM major intention, but ended up with a nonSTEM major at graduation, only 5 of them received credit for 3 or more STEM AP exams (0.08%). These data are strictly correlational, yet the coincidental associations between successful completion of Calculus BC and/or 3 or more AP exams in the STEM area and outflow for both men and women suggests that one potential indicator of the higher outflow of women from the STEM majors might be related to the relatively lower levels of completion of Calculus BC in particular, and 3 or more AP exams in the STEM area in general, compared to the men in the sample. CORRELATES OF AP EXAM COMPLETION In this section, we review the associations among traditional predictors of postsecondary performance, and indicators associated with AP exam performance, as independent and joint predictors of performance at Georgia Tech. Table 11 shows the correlations between HSGPA, SAT Verbal, SAT Math, and the number of AP exams completed by the students, both by broad category and overall. Most interesting in these results is that HSGPA is significantly, but only modestly correlated with the number of AP exams across all the categories, with a correlation of r = .171. In contrast, the respective correlations between SAT scores and number of AP exams completed were r = .318 for SAT Verbal, and r = .278 for SAT Math. The pattern of correlations between the respective SAT scores and AP exam completion follow thematic lines. That is, the correlations are higher for SAT Math and AP exam counts in the Physical Sciences, Math, and Computer Science, and they are higher for SAT Verbal and Humanities, Social Sciences, and Foreign Languages, and they are approximately equal for Biology. When it comes to average performance on the AP exams, the correlation with HSGPA was still relatively modest (r = .161), but the correlations with SAT scores were substantially higher (SAT Verbal, r = .406; SAT Math r = .417). Based on these results, it appears that entry into AP courses is at least partly associated with ability (as reflected in SAT scores), and entry into specific courses is partly differentiated by ability profile (e.g., verbal vs. math). Performance on the AP exams is moderately correlated with abilities (with the same general pattern of math/physical sciences AP exam scores being more highly correlated with SAT Math, and nonSTEM AP exam scores being more highly correlated with SAT Verbal ability). However, it is important to note that only about 28% of the variance in overall average AP exam scores are accounted for by HSGPA and SAT exam scores (R = .529, df = 3,17573), suggesting that variables other than ability and grades may account for substantial variance in AP exam performance. Table 11. Correlations between High School GPA, SAT Verbal, SAT Math, and completion of AP Exams by Domain
N = 24,965 for High School GPA correlations, N = 25,675 for SAT correlations. **p < .01 PREDICTORS AND CRITERIA RELATIONS In Table 12 we provide means, standard deviations, and intercorrelations among traditional predictor measures for university grade criteria, including SAT Critical Reading; SAT Mathematics; HSGPA; an index used by Georgia Tech that reflects an optimal weighting of these predictors (SAT Index); a set of variables associated with the AP program (total number of AP exams completed, total number of AP exams with scores of 3 or greater, total number of AP exams with scores of 4 or 5, Number of APbased course credits awarded at Georgia Tech, and average scores on the AP exams); and cumulative GPA at Georgia Tech for Years 1 through 4. Table 12. Means, standard deviations, sample size, and correlations among predictor and criterion variables.
Notes. GT = Georgia Tech, Cum. = Cumulative *Because the distribution of exam counts was substantially skewed by a relatively small number of students who completed more than 10 AP exams, the AP Exam Counts were limited to 10 (any number greater than 10 was set to 10). All correlations are significant beyond the p < .01 level. ^{a}The SAT Index is a weighted average of SAT Math, SAT Verbal, SAT Writing, and High School GPA used for admission purposes by Georgia Tech. We included the average AP score variable as a predictor for two reasons. First, previous research (Ackerman, Bowen, Beier, & Kanfer, 2001) indicated that average AP scores provide robust predictions of other indicators of the depth and breadth of academic knowledge and abilities. Second, average AP scores also show substantial correlations with personality/interest/motivation/ability trait complexes that are indicative of both the direction and intensity of effort in academic settings. Specifically, average AP scores were significantly positively correlated with science/math trait complex scores and with verbal/intellectual trait complex scores, which are themselves positively associated with knowledge in a variety of domains. The average AP scores were significantly negatively correlated with two broad social/extroversion trait complexes and with a traditionalism/worry/emotionality trait complex, which are in turn, negatively associated with knowledge in a variety of domains. As expected, SAT scores and HSGPA provide significant and substantial correlations with grades for each year at Georgia Tech, even though there is marked restrictionofrange for all of the predictor measures; correlations with the GPA criteria decline with each additional year after the first, consistent with historical findings in the literature (e.g., see Humphreys, 1968; Juola, 1966). Of the APrelated predictors, all of them provide significant correlations with both the other predictors and with the GPA criteria. The largest correlations with the criteria are found for the number of APexam based course credits and with the average AP exam scores, with the largest correlations for the average AP exam scores. In a multiple regression equation predicting firstyear GPA, the two SAT scores and HSGPA yield R = .440, accounting for 19.3% of the variance in firstyear GPA. Adding the average AP exam scores, yielded an incremental variance accounted for of 6.58%, for a final R = .509, accounting for 25.9% of the variance in firstyear GPA (F to add = 1488.06, df = 1,16765).^{3} Although the variance accounted for at Year 4 cumulative GPA is lower for the SAT and HSGPA (R = .369, R^{2} = 13.6%), the incremental variance accounted for (5.34%) by the average AP exam scores remains significant and substantial, with a final R = .436 (F to add = 355.85, df = 1,5395). These results compare quite favorably to Georgia Techs SAT Index which includes only the SAT scores and HSGPA. For Year 1 cumulative GPA, r = .464 for the SAT Index vs. R = .509 for the equation that also includes average AP exam scores. For Year 4 cumulative GPA, r = .400 for the SAT Index vs. R = .436 (where the Average AP exam score was entered). For comparison purposes, multiple regressions were performed with different combinations of predictors with the Average AP exam score. When HSGPA was removed from the prediction equation, there was about a 10percentagepoint loss of the firstyear GPA variance accounted for. One implication of these results is that HSGPAs that reflected bonuses for honors and AP courses do not eliminate the influence of the actual AP exam scores in predicting firstyear GPAs. However, when only the HSGPA and Average AP exam scores were entered (i.e., without SAT scores), there was a minor loss in variance accounted for (less than 1% decline in variance accounted for), suggesting that the variance in average AP scores reflects similar degrees of variance accounted for with SAT scores (the partial correlation between firstyear GPA and SAT [Verbal + Quantitative] scores, with AP average exam scores partialled out was r (17,550) = .07, p < .01). Of course, use of such a formulation would not be practical in an actual selection situation, because nearly half of the AP exams are only completed at the end of the students senior year in high school, and the full set of AP scores is not available by the time that selection decisions are made. Because it was not possible to identify when in the course of the high school experience the AP exams were completed in this sample, it remains to be determined how much variance the average AP exam scores for exams taken prior to 12th grade would account for in GPA at Georgia Tech. Nonetheless, it should be noted that, as of the 2011 national administration of the AP exams, 59.3% of school students completing AP exams were in the ninth11th grades at the time of the exams (1,103,758 students from ninth11th grade vs. 756,856 in 12th grade, College Board, 2012a), a percentage that has seen steady growth in the last decade (e.g., for the 1999 administration, only 42.1% of the exams were completed by students prior to the 12th grade). With this as background, it may be possible that inclusion of an average AP exam score variable in the prediction equation during university selection will increase the predictability of firstyear postsecondary grades, in conjunction with traditional SAT and HSGPA scores. QUALIFICATIONS Before drawing any overall conclusions from this investigation, it must be noted that there are two levels of selection associated with the sample under consideration in this paper. Student selfselection takes place on several levels, including in terms of financial issues (instate vs. outofstate tuition), the reputation of the institution (e.g., as specializing more in engineering than the arts), the fact that the institution is in an urban setting, and many other features that determine whether a potential student even applies for admission to Georgia Tech. The second aspect of selection is explicit, in that Georgia Tech received, for example, 13,553 applications for fall enrollment in 2010, and accepted 6,976. Of the accepted group, selfselection yielded a group of 2,650 new first year students in 2010. As with most selective institutions, Georgia Tech does not publicly discuss the exact selection criteria, but from public information and the data provided for this sample, it is clear that entrance exam scores (SAT/ACT) and high school grades are instrumental variables for the explicit selection. For the 25,675 with SAT scores in the 19992009 samples, the mean for SAT Critical Reading was 635.01 (sd = 70.57), and for SAT Math was 686.30 (sd = 64.55). For a gender breakdown, SAT Critical Reading scores were not significantly different (635.40, sd = 71.31 for men, 634.08, sd = 68.79 for women, t (25,673) = 1.37, ns). However, for SAT Math, men had scores that were about 30 points higher, on average, compared to women (695.67, sd = 63.67 for men, 664.33, sd = 62.69 for women, t (25,673) = 36.53, p < .0001). For comparison purposes, the 2010 SAT Critical Reading national norms were M = 503, sd = 114 for young men, and M = 498, sd = 111 for young women; and the SAT Math national norms were M = 534, sd = 118 for young men, and M = 500, sd = 112 for young women; a 5 point gender difference for Critical Reading and a 34 point difference for Math. In comparison to the national norms, the average matriculating student at Georgia Tech had scores equivalent to the 87th percentile for Critical Reading and the 91st percentile for Math. Coupled with the smaller variance in SAT scores for the Georgia Tech student population in comparison to the national norms, these data are consistent with the assumption that Georgia Tech is a highly selective institution when it comes to SAT scores, and thus the population under consideration is substantially restricted in rangeoftalent. OTHER RESEARCH In a recent study, Shaw and Barbuti (2010) examined persistence in STEM majors in a national sample of students. The samples and the variables under consideration were different from those examined in this report. However, their conclusions are largely concordant with those reported here. For areas of overlap between investigations, they found that highschool involvement in math and science courses in general, and completion of AP exams in the STEM areas in particular, were positively related to STEM persistence in college/university study. They also found that students who left the STEM majors had lower grades at college than those who persisted. Although 59% of their sample were identified as switchers (changing from an intended major in a STEM area to a nonSTEM area), compared to 15% in our sample, two differences are important to note. First, the indicator Shaw and Barbuti used for initial major intention was collected in the junior or senior year of high school, and the indicator we used was initial major intention at matriculation to Georgia Tech. Second, the diversity of schools sampled in their study included many schools that did not have the strong STEMthemed educational programs that are most identified with Georgia Tech. Nonetheless, we see our results as largely complementary to the results of their study. Shaw and Barbuti were able to examine variables such as selfefficacy, parental income, and other demographic variables, whereas we were able to examine performance on the AP exams, breakdowns of courses completed in college, and the relations among patterns of different AP course completions along with traditional predictors of college performance criteria. DEMOGRAPHIC CHANGES Over the course of the 11 student matriculation date cohorts (19992009), there have been some changes in the frequencies for completion rates for the APrelated variables identified as integral to STEM major completion. In 1999, 15.3% of the men completed Calculus BC, compared to 12.5% of the women. By 2009, there was a marked increase in completion rates for Calculus BC29.9% for men and 19.6% for women. Although the rate for women increased, it did so at a much lower rate than for men. For the variable of 3 or more STEM AP exams, in 1999, 6.5% of the men met this criterion, compared to 2.6% of the women. By 2009, 14.4% of the men completed 3 or more STEM AP exams, compared to 6.4% of the women. Gains were clearly made by both men and women, but the women students still lag men on this variable. CONCLUSIONS While we acknowledge that this is strictly a study of archival data, the results of our analyses indicate that successful completion of AP exams in general, and depending on the students major, specific AP exams in particular, are associated with higher GPA, higher rates of STEM persistence, higher graduation rates, and fewer semesterstograduation. In contrast, simply having completed AP courses in high school without obtaining successful exam performance was not associated with a more successful experience on these indicators of academic success. Specifically, the major findings of this investigation were as follows: 1. Participation in AP exams had an overall positive association with grades, for students who matriculated at Georgia Tech from 19992009. Increasing numbers of AP exams with scores of 4 or 5 had the strongest association with performance criteria, as did the closely related number of APbased semester course credits. (Students who completed AP exams but who did not receive scores of at least 3 on the exams tended to perform at a level similar to those students who did not complete any AP exams.) 2. With respect to graduation rates, greater numbers of AP exams completed with scores of 4 or 5 were associated with substantially higher fiveyear graduation rates. An even stronger gradient was found for fouryear graduation rates, where students with four or more AP exams with scores of 4 or 5 were found to have double the fouryear graduation rates, compared to students who did not complete any AP exams. 3. Although women obtained higher grades overall, in comparison to men, the relationship between AP exam performance and grades at Georgia Tech was uniformly positive for both men and women. 4. Receipt of credit for AP Calculus and AP English Literature was most highly associated with higher student grades at Georgia Tech, for both STEM and nonSTEM majors. Both men and women who received credit for Calculus BC and for three or more AP exams in the STEM domains were substantially less likely to switch from an initial STEM major to a nonSTEM major. Women had higher outflow from STEM to nonSTEM majors and substantially lower rates of completion for either Calculus BC or for three or more STEM AP exams, compared to men. 5. Average scores on the AP exams completed by students matriculating at Georgia Tech wellpredicted grade and graduation criteria, resulting in the greatest amount of variance accounted for in grades, after consideration of HSGPA (keeping in mind that there is a substantial restriction of range in HSGPA for this sample). Together, HSGPA and average AP exam scores accounted for 25 percent of the variance in firstyear GPA at Georgia Tech. For students interested in STEM majors, it is apparent that successful completion of a maximal number of AP exams in the STEM areas is an important determinant of collegelevel performance and persistence. One possibility that stakeholders might consider is to increase availability and lowering entry barriers to STEM AP courses for qualified and interested/motivated students, perhaps even if students only have a moderate level of interest in a future STEM major. We cannot determine from the current study whether lowering the barriers to entry to STEM AP courses will result in larger numbers of students who are successful in the subsequent AP exam tests. However, given the modest relationship between HSGPA (even with honors/AP bonuses factored in) and AP participation, it is not clear what criteria are currently used to determine access or availability to such courses. Nonetheless, it is critical to recognize that the traditional math sequence from middle school through high school is often determined through formal or informal tracking (starting with algebra). By the time students reach high school, whether or not they can complete an AP Calculus course is typically predetermined. Given the important contribution of completing AP Calculus in predicting success at Georgia Tech, some additional attention should be given to the determination of which students start the sequence that will allow them to complete Calculus courses by the end of high school. In general, it appears that the probability of both men and women successfully persisting in STEM majors at Georgia Tech is importantly associated with curricular decisions (namely AP participation and success) that take place prior to matriculation at Georgia Tech. On the basis of the current investigation, our conjecture is that reducing outflow from STEM majors to nonSTEM majors might be accomplished by dissemination of detailed information about the importance of STEM AP courses/exams to various stakeholders, and by efforts to expand opportunities for students with potential STEM major intentions to complete additional AP courses in the STEM areas. Finally, the results in this investigation and other extant data (e.g., Geiser & Santelices, 2004) suggest that using HSGPA bonus points for student enrollment in AP courses may be a suboptimal strategy for prediction of college performance, especially given the fact that nearly 60% of AP exams are now completed by students prior to their senior year of high school. Evaluating the predictive validity of those AP exam scores for exams taken prior to the date of the college application appears to be a highly promising avenue for future admissions decisions. A key advantage for some decisionmakers is that the AP exam content is based on explicit curricula, in contrast to other indicators (e.g., SAT scores) that are loosely based on a general curriculum that is common to high school, and are strictly normreferenced. The potential advantage of using AP exam scores is that the structured syllabi used in AP courses provide clear indicators to the applicant and other stakeholders about the knowledge necessary to attain high scores on the tests. Acknowledgments The authors wish to acknowledge the support and able assistance of the College Board in sponsoring this research project and providing archival record information. In particular, we are deeply grateful for the support provided by Wayne Camara and Maureen Ewing. In addition, we wish to acknowledge the extraordinary assistance of David Cauble of the Institutional Research and Planning office at Georgia Tech, without whose diligent work in tracking down and parsing records from many different sources, this project would have been impossible to complete. The ideas expressed in this article are those of the authors and do not reflect the opinions or position of the College Board. Correspondence concerning this paper should be addressed to Phillip L. Ackerman, School of Psychology, Georgia Institute of Technology, 654 Cherry Street, MC 0170, Atlanta, GA 303320170. Email: phillip.ackerman@psych.gatech.edu Notes 1. One reviewer suggested that, in addition to Figure 4, the bulk of the remaining analyses reported here should be adjusted by statistically partialling out the influence of individual differences in SAT scores, on the premise that (a) the SAT is a highly stable estimate of academic aptitude, and (b) that by taking account of aptitude differences, the influence of individual differences in aptitude on AP course enrollment and performance can be statistically removed. The premise is that whatever individual differences in AP performance remain, are (at least statistically), independent of student aptitudes. In the current analysis, we did statistically remove the influence of both SAT and HSGPA. In a later analysis, when we consider different models for predicting postsecondary grades in a selection context, we report incremental predictive validity of AP enrollment and performance, after SAT scores and HSGPA variables were entered into the regression equation. However, the remaining analyses focus mainly on AP enrollment and performance measures in isolation. There are two major reasons for this decision: First, partialling out the influences of SAT (and both SAT and HSGPA) accords any common variance among SAT, HSGPA, and AP to the variables that are being partialled out, even though it is arguable whether the common variance, for example, between the SAT and the AP performance measures is more appropriately assigned to one or the other set of measures, or part to each. As several researchers have noted (e.g., Anastasi, 1970, 1983; Humphreys, 1973), the dividing line between intelligence, aptitude (e.g., SAT), and achievement (e.g., AP) is not at all theoretically or practically clear. Second, because AP tests are taken as early as ninth grade, and in the most recent norms (College Board, 2012a), slightly over half of the AP tests are taken by students prior to 12th grade, it is unclear whether the AP experiences have a direct or indirect influence on SAT scores. If that is the case, then AP and SAT measures are confounded to an unknown degree, and partialling out the variance in one measure may render the variance remaining variable impossible to interpret. 2. A logistic regression was conducted to predict those students who changed from STEM to nonSTEM majors and those who remained STEM majors to graduation. In the first step, SAT Verbal, SAT Math, and HSGPA were entered into the equation. Only SAT Math and High School GPA had significant contributions to predicting STEM persistence (χ^{2}(3) = 467.58, p < .01). In the second step, number of AP exams with scores of 4 or 5 was entered. Even after allowing SAT scores and HSGPA to account for all common variance among predictors, the Calculus credit variable was a significant contributor in accounting for STEM persistence (χ^{2}(1) = 114.30, p < .01). Similar results were obtained in the second step using average AP exam scores (χ^{2}(1) = 170.61, p < .01). 3. In other words, if one assumes that there is no directional or bidirectional influence between SAT scores, HSGPA, and AP exam performance over the course of the students high school experience, individual differences in average AP exam performance account for an additional 6.2% of variance accounted for in firstyear postsecondary grades. If, in fact, there are benefits from AP course enrollment on SAT scores and/HSGPA, the influence of AP courses is underestimated from this statistic. (Although it is possible that AP course enrollment could have negative influences on SAT scores, it seems theoretically implausible. Less certain is whether AP course enrollment affects HSGPA. Most schools provide direct GPA bonus points for AP course enrollment, partly to recognize the generally greater rigor of such courses in comparison to the standard curriculum, but the actual net influence of AP course enrollment on HSGPA is unknown.) References Ackerman, P. L., Bowen, K. R., Beier, M. B., & Kanfer, R. (2001). Determinants of individual differences and gender differences in knowledge. Journal of Educational Psychology, 93, 797825. Adelman, C. (2006). The toolbox revisited: Paths to degree completion from high school through college. Washington, DC: U.S. Department of Education. Anastasi, A. (1970). On the formation of psychological traits. American Psychologist, 25, 899910. Anastasi, A. (1983). Evolving trait concepts. American Psychologist, 38, 175184. Camara, W. J., & Michaelides, M. (2005). AP use in admissions: A response to Geiser and Santelices. Retrieved August 23, 2011 from http://www.collegeboard.com/research/pdf/051425Geiser_050406.pdf College Board (2012a). National Summary. Downloaded on May 20, 2012, http://professionals.collegeboard.com/datareportsresearch/ap/data. College Board (2012b). Annual AP Program Participation 19562011. Downloaded on May 19, 2012. http://professionals.collegeboard.com/datareportsresearch/ap/data. DiYanni, R. (2009). The history of the AP Program. Retrieved September 7, 2009 from http://www.collegeboard.com/apc/public/courses/21502.html. Ewing, M. (2006). The AP® program and student outcomes: A summary of research. Research Notes RN29. New York, NY: College Board. Eykamp, P. W. (2006). Using data mining to explore which students use Advanced Placement to reduce time to degree. New Directions for Institutional Research, 131, 8399. Geiser, S., & Santelices, V. (2004). The role of Advanced Placement and honors courses in college admissions. Berkeley, CA: Center for Studies in Higher Education, UC Berkeley. Retrieved August 20, 2011 from http://escholarship.org/uc/item/3ft1g8rz. Humphreys, L. G. (1968). The fleeting nature of the prediction of college academic success. Journal of Educational Psychology, 59, 375380. Humphreys, L. G. (1973). The misleading distinction between aptitude and achievement tests. In D. R. Green (Ed.), The aptitudeachievement distinction. Proceedings of the Second CTB/McGrawHill Conference on Issues in Educational Measurement (pp. 262285). Carmel, CA: CTB/McGrawHill. Juola, A. E. (1966). Prediction of successive terms performance in college from exams and grades. American Educational Research Journal, 3(3), 191197. Kopfenstein, K., & Thomas, M. K. (2010). Advanced placement participation: Evaluating the policies of states and colleges. In P. M. Sadler, G. Sonnert, R. H. Tai, & K. Klopfenstein (Eds.), AP: A critical examination of the Advanced Placement program (pp. 189218). Cambridge, MA: Harvard Educational Press. Morgan, R., & Klaric, J. (2007). AP® students in college: An analysis of fiveyear academic careers. College Board Research Report No. 20074. New York, NY: College Board. Sadler, P., M., & Sonnert, G. (2010). High school Advanced Placement and success in college coursework in the sciences. In P. M. Sadler, G. Sonnert, R. H. Tai, & K. Klopfenstein (Eds.), AP: A critical examination of the Advanced Placement program (pp. 119163. Cambridge, MA: Harvard Educational Press. Sadler, P. M., & Tai, R. H. (2007). Advanced Placement exam scores as a predictor of performance in introductory college biology, chemistry, and physics classes. Science Educator, 16(2), 119. Shaw, E. J., & Barbuti, S. (2010). Patterns of persistence in intended college major with a focus on STEM majors. NACADA Journal, 30(2), 1934.





