Analysis of the National Science Foundation’s Discovery Research K–12 on Mathematics and Science Education for English Learners

The push for all students, particularly English Learners (ELs), to develop fundamental understanding of science, technology, engineering, and mathematics (STEM) and to fully participate in STEM has become more urgent and complex due to multiple factors. First, while ELs are the fastest-growing population in the nation’s schools, achievement gaps in mathematics and science have persisted between ELs and their non-EL peers (National Center for Education Statistics, 2013). Second, recent educational policy reforms, including the Common Core State Standards (CCSS) for mathematics and the Next Generation Science Standards (NGSS), are language intensive and thus present both challenges and opportunities for language learning and content learning for ELs (Lee, Quinn, & Valdés, 2013). Finally, today’s technological and global society requires that all students learn challenging mathematics and science for personal and social reasons as well as for college and career readiness.

Educational and societal phenomena can converge to draw attention to a new focus, such as ELs and STEM, and then trigger new research interests. A funding program can play a critical role in shaping these new research interests by prioritizing specific research topics and designs or by requiring particular specializations of researchers. The magnitude of funding alone can also determine the nature, scope, and impact of projects by individual researchers who without such financial support would be limited to projects of a smaller scale. A funding program, therefore, can shape research agendas by providing targeted and deliberate funding for priority areas (Ferris, Hentschke, & Harmssen, 2008; Hansen, 2008).

The purpose of this study was to explore whether funding provided through the National Science Foundation’s (NSF) Discovery Research in K–12 (DR K–12) program has made a unique contribution to the research in the fields of science and mathematics education for ELs. The study compared the portfolio of DR K–12 projects focusing on EL science and mathematics education to the literature of non DR K–12 projects in terms of research topics, design, methods, scale, samples, and outcomes. The study also examined the disciplinary expertise of the DR K–12 investigators. The access to the portfolio of DR K–12 projects offered a unique opportunity to examine the role of funding in shaping a field. However, funded projects that propose studies and studies that have already been conducted are not directly comparable. As such, this analysis was not a literature review of existing research in the fields of science and mathematics education for ELs, but rather a comparative study of the research characteristics of the portfolio of DR K–12 projects and the literature of non DR K–12 projects. As the DR K–12 projects mature over time, a comprehensive review of the literature may be conducted.

The study is intended for multiple audiences. One audience involves the principal investigators and research teams of the DR K–12 projects to gain insights from the findings of the study, gauge how their projects relate to other projects in the portfolio of DR K–12 projects, and articulate how their projects could contribute to the portfolio specifically and the field broadly. Another audience involves the NSF and other funding agencies to reflect on and assess how their funding policies and programs could make a contribution to a new or emerging field. Still another audience involves educational researchers and practitioners who are interested in learning about research on mathematics and science education for ELs.

SCIENCE AND MATHEMATICS EDUCATION WITH ELS

Research has traditionally given limited attention to content area instruction for ELs, specifically in science and mathematics (August & Hakuta, 1997; Janzen, 2008). However, knowledge about science and mathematics instruction for ELs has been expanding in the last decade (Buxton & Lee, 2014). One reason for this change is that science and mathematics education researchers are broadening their research to encompass EL issues, while second language acquisition researchers are increasingly examining EL issues in the context of science and mathematics. This movement has increased the number of studies that focus on the specific challenges that ELs experience in content areas, and instructional strategies to address these challenges (Davison, 2006). Most of these studies have been published since the mid-1990s.

Lee (2005) conducted a comprehensive review of the literature in science education with ELs. In terms of methods and scale, Lee found that few of the studies used either an experimental or quasi-experimental design, and it was rare that intervention-based studies were conducted on a large scale. Rather, most studies used qualitative approaches only and tended to produce small-scale, descriptive research that was conducted by individual researchers. Furthermore, hardly any studies evaluated the impact of interventions on English proficiency as well as science achievement outcomes, examined achievement gaps among linguistic groups, reported results across different levels of English proficiency, or considered linguistic/semiotic approaches. Lee characterized the attention to linguistic issues as “uneven,” stating that most studies did not consider the “complexities inherent in the construction of language” (p. 511).

Although no equivalent systematic review exists for mathematics, the literature search that was conducted for this study indicated that even fewer studies addressed ELs in mathematics education than in science education over the same time period (1982–2004). One review of the literature on teaching ELs explored research in four content areas, including mathematics (Janzen, 2008). Janzen noted the dearth of research on teaching mathematics to ELs, which she attributed to the misguided belief that mathematics is less difficult for ELs because it is based on a language of numbers (Veel, 1999). Additionally, for each of the content areas, Janzen highlighted four strands of research—linguistic, cognitive, sociocultural, and pedagogical—and noted that a large portion of the research on mathematics instruction focused on sociocultural issues, specifically teaching students of Latino/Latina descent (e.g., Daisey & José-Kampfner, 2002; Gutiérrez, 2002; Gutstein, Lipman, Hernandez, & de los Reyes, 1997). Across all four strands, research on mathematics instruction for ELs tended to be descriptive (e.g., Gutiérrez, 2002; van Garderen, 2004) and lacked investigations of large-scale interventions.

Lee (2005) proposed an agenda for moving forward with research involving ELs in science education, which is largely applicable to mathematics education as well. Her proposed research agenda for the field suggested (a) an increased focus on student outcomes; (b) a more explicit integration of science learning and English language development, particularly in terms of merging research perspectives from the science education and EL areas; (c) identification of linguistic and cultural resources that ELs bring to the classroom; (d) more extensive exploration of approaches to science inquiry and the use of technology specifically for ELs; (e) further examination of the effect of policy changes on high-stakes assessments for ELs; and (f) the design of programs to effectively prepare teachers to teach science to ELs.

ROLE OF FUNDING AGENCIES IN CONTRIBUTING TO THE FIELD

Both governmental and nongovernmental organizations are increasingly influential in the development and implementation of education policy by means of grant funding (Ferris et al., 2008; Hansen, 2008). In essence, such entities are able to sway the education research and reform agendas by providing targeted and deliberate funding opportunities. In 2011, the President’s Council of Advisors in Science and Technology convened a special committee to look at federal investments in STEM education across agencies and found that 13 federal agencies spent $3.4 billion in supporting STEM education (National Science and Technology Council, 2011), with the largest investment made by NSF (U.S. Government Accountability Office, 2012).

NSF is the federal agency created in 1950 “to promote the progress of science; to advance the national health, prosperity, and welfare; to secure the national defense …” (National Science Foundation Act of 1950 [Public Law 81-507]). NSF pursues its mission by supporting research and education in all fields of science and engineering. It accomplishes this through awarding grants and cooperative agreements to over 2,000 higher education institutions, K–12 school systems, businesses, informal science organizations, and other research organizations in the United States (NSF, 2013).

Within NSF, the process used to determine funding awards begins with the development and release of solicitations targeting specific funding priorities. In response to these solicitations, researchers and other entities submit proposals that are reviewed by panels of experts, with final award recommendations made by NSF staff members who have expertise in the field (NSF, 2013). Consequently, both the requirements of the solicitations and the peer review panels play a role in what is funded.

The DR K–12 program is one of NSF’s six education research and development programs, and is aimed at enhancing the teaching and learning of STEM. It focuses specifically on research projects that study the “development, testing, implementation effectiveness, and/or scale-up of innovative resources, models and tools (RMTs)” (NSF, 2011, p. 2). The DR K–12 program accepts proposals for exploratory projects, full research and development projects, and synthesis projects, as well as for conferences and workshops related to improving STEM education. DR K–12 projects begin with a research question or a hypothesis about how to improve pre-K–12 STEM learning and teaching through the development and testing of RMTs. The definition of RMTs is broad—it could include research-based materials for use by students and teachers to facilitate learning and/or teaching, innovative frameworks to support increased teacher collaboration, or new technologies that measure student learning or effective teaching.

The DR K–12 program strives to fund projects that respond to the needs of an increasingly diverse student population. Although the DR K–12 program solicitations do not make reference to ELs specifically, they do call for projects that broaden access to and success in STEM, and that take into account the cultural, linguistic, economic, and educational diversity of learners in K–12 settings. In this way, DR K–12 funding may be contributing to the knowledge base in the critical fields of EL science and mathematics education for diverse student groups including ELs. Of the 295 DR K–12 projects funded between 2007 and 2011 (cohorts 1–5), 34 projects (12%) focused on ELs. Of these projects, 15 focused on science, 16 on mathematics, and three on both science and mathematics.¹

RESEARCH QUESTIONS

Lee’s (2005) proposed research agenda served as a focal point for a session convened by the research team at the 2009 annual Principal Investigators’ (PIs) meeting for the NSF DR K–12 program. This session included a discussion of ways in which current research, and specifically the research being funded by DR K–12, might be extending the lines of inquiry in science and mathematics education for ELs. Several hypotheses emerged. First, the DR K–12 projects may be addressing different research topics within EL science and mathematics education than what research in the field has traditionally pursued; for example, they may be focusing more on curriculum and instruction than on student learning. Second, they may be using more experimental or quasi-experimental designs and be fairly large scale, involving more than one school district or collecting data from multiple states. As a consequence, the DR K–12 projects may be more likely to utilize quantitative methods instead of or in concert with qualitative methods. The PIs acknowledged that some of these differences may be occurring in response to the DR K–12 solicitation, which emphasizes the development and testing of STEM resources, models, and tools for teachers and students, rather than basic research in the STEM disciplines. These projects may also be more likely to examine science outcomes instead of language and literacy outcomes, and may focus on secondary education rather than on elementary education. Finally, in terms of disciplinary expertise, the DR K–12 EL projects may be more likely to be conducted by researchers who possess expertise in mathematics and/or science education than in EL/English language arts (ELA). Should these hypotheses hold true, the session’s attendees concluded, the ongoing projects within the DR K–12 portfolio have the potential to make a considerable contribution to the knowledge of science and mathematics education for ELs. The research team designed and conducted this study to examine the hypotheses put forth by the DR K–12 PIs.

Building on the hypotheses generated during the session convened by the investigators of EL science and mathematics DR K–12 projects at the 2009 PI meeting, this study investigated the following research questions:

(1)

What are the research characteristics of the portfolio of DR K–12 projects and the literature of non DR K–12 projects in the field of EL science education? How do they compare?

(2)

What are the research characteristics of the portfolio of DR K–12 projects and the literature of non DR K–12 projects in the field of EL mathematics education? How do they compare?

(3)

What is the disciplinary expertise of the investigators (PIs and co-PIs) in the portfolio of DR K–12 EL science and mathematics projects?

METHOD

The primary method used in this study was content analyses of the portfolio of DR K–12 projects and the literature of non DR K–12 projects in the fields of EL science and mathematics education. To develop comprehensive lists of the literature in these fields, two separate literature searches were also conducted. Finally, content analyses of the curricula vitae (CVs) of the DR K–12 projects’ PIs and co-PIs were undertaken, as described further below.

ANALYSIS OF THE DR K–12 PORTFOLIO OF EL PROJECTS

To examine the research characteristics of the portfolio of DR K–12 EL projects, a careful review of the work being funded by the DR K–12 program was required. An initial set of DR K–12 projects in cohorts one through five (2007–2011) was identified as EL projects through an annual review conducted by the research team. Projects were considered EL projects if they focused exclusively on ELs or explicitly stated that they planned to examine ELs as a subgroup. DR K–12 investigators reviewed this initial set of projects to ensure that only those DR K–12 projects that targeted ELs were included. For analyses, the three projects that focused on both science and mathematics were counted in the final analysis samples for both fields, resulting in a total of 18 science and 19 mathematics projects. The DR K–12 EL projects involved 80 investigators (PIs and co-PIs) who were seeking to develop new knowledge, instructional approaches, and educational materials for ELs and their teachers in science and mathematics. (See Appendix A for brief descriptions of the DR K–12 EL projects included in this study.)

A content analysis of project materials was then conducted. Project materials included project abstracts, proposal narratives, responses to NSF reviewer questions, and annual reports that the investigators had shared with the research team. Basic attributes for each project, such as type of research design, disciplines addressed, and outcomes, were pulled from the broader portfolio analysis of DR K–12 projects that was conducted by the research team on an annual basis. Additional attributes specific to this review were then coded to capture other areas of interest, such as study topic, intervention type, EL focus, participants and research setting, and research methodology. The analytic project database merged codes from these two sources of attributes for each of the projects.

Four coders collected and analyzed the data using an Access database, SAS, and Excel. Four randomly selected projects were double coded, and conflicts with the coding protocol were discussed and resolved after double coding. Periodic meetings were held to resolve issues that arose and ensure consistency among the coders.

LITERATURE SEARCHES OF NON DR K–12 RESEARCH

To identify projects in the fields of EL science and mathematics education that were not funded by the DR K–12 program, a literature search that identified peer-reviewed, published articles in each field was conducted. The ERIC and EBSCO databases² were searched using the following terms: “science” or “science educat*”³ or “mathematics” or “mathematics educat*” in combination with “English Language Learner,” “ELL,” “Dual Language Learner,” “DLL,” “bilingual,” “Limited English Proficient,” “LEP,” “English for Speakers of Other Languages,” and “ESOL.” Additional searches were conducted in the journals published by AERA (i.e., American Educational Research Journal, Educational Evaluation and Policy Analysis, Educational Researcher, and Review of Educational Research), in math or science education (Journal of Research in Mathematics, Journal of Research in Science Teaching, Mathematics Education, and Science Education), and in other top-rated journals (Teachers College Record, The Elementary School Journal, and Harvard Educational Review).

Abstracts for a potential pool of articles were reviewed to select those articles meeting the following seven criteria. The first five were similar to those utilized in Lee’s synthesis (2005), while the last two were specific for this study.

(1)

Were published in peer-reviewed journals (conference proceedings were not included);

(2)

Directly addressed ELs in mathematics and/or science education;

(3)

Focused on K–12 EL education;

(4)

Analyzed empirical data using any of the following designs or methods: experimental, quasi-experimental, correlational, descriptive, survey, ethnographic, qualitative, or case study (practitioner-oriented articles, literature reviews, and conceptual pieces were not included);

(5)

Were written in English;

(6)

Were conducted within the United States; and

(7)

Did not report on research funded by the DR K–12 program.

As the science search was intended to update the list of projects included in a preexisting literature synthesis covering 1982 through 2004 (Lee, 2005), only peer-reviewed articles published between 2005 and March 2013 were considered. The search for mathematics literature included all articles published since 1982. A total of 88 articles were identified—35 in science education, 40 in mathematics education, and 13 involving both science and mathematics education. Since the unit of analysis for this review was the project, articles that reported on the same research project were not double counted; the 88 articles represented 77 distinct projects. Of these 77 projects, 32 focused on science education, 38 focused on mathematics education, and seven focused on both science and mathematics education. The latter projects were included in the final analysis samples for both science (n = 39) and mathematics (n = 45) since analyses were conducted separately by field and these dual-focus projects were assumed to be contributing to both fields. (See Appendix B for a list of the projects included in this review.)

The full texts of the articles were reviewed and the research characteristics of the projects were coded with the same protocol that was used for the DR K–12 portfolio analysis described above. Three coders double-coded three articles, adapted the coding protocol for the articles,⁴ and met routinely to resolve issues with coding the literature. The data were collected and analyzed using an Access database, SAS, and Excel. The research characteristics of the literature of non DR K–12 projects in EL science and mathematics education were combined in a matrix with the research characteristics of the DR K–12 EL projects to facilitate comparison between these two types of projects. 

CV ANALYSIS OF DR K–12 INVESTIGATORS (PIS AND CO-PIS)

To examine the disciplinary expertise of the investigators working on the DR K–12 EL projects, an analysis of the CVs of all 80 investigators (PIs and co-PIs) listed in project materials was conducted. The approach was based on the assumption that investigators include their most prominent work in their CVs—publications, conferences, courses, and grants—which in turn speak to their most salient interests and areas of expertise. Internet searches were done for the CVs of PIs and co-PIs, and copies were directly requested in cases where CVs could not be found or were not dated; CVs were secured for 96% (n = 77) of all investigators and 100% (n = 33) of PIs. CVs varied in length and the amount of information they contained; 10 CVs were two or three pages, while nine were 30 or more pages. The mean length was 16 pages. When CVs were limited, investigators’ websites were searched for pertinent information (e.g., course listings) to supplement the CV data. The majority of CVs that were dated (n = 64) were relatively recent: Only five were dated before 2009, while the rest were equally distributed across 2009, 2010, 2012, and 2013.⁵

CVs were coded using a two-step process. First, all relevant items on investigators’ CVs were coded into one of five categories of professional activities considered indicators of disciplinary expertise: field of highest degree, articles published in peer-reviewed/refereed journals (since 2000), grants awarded (since 2000), papers presented at conferences (since 2000), and courses taught. Second, within each category of professional activity, items were coded into one or more of five areas of disciplinary expertise.⁶ In particular, the coding captured the distinction between expertise in the STEM content areas of science and mathematics versus the more language-oriented EL/ELA area by coding the five categories of professional activities as (a) mathematics, (b) mathematics education, (c) science (including computer science and psychology when not specified as educational psychology), (d) science education, and/or (e) EL/ELA.

To determine the disciplinary expertise of an investigator in each of the five categories of professional activity, the percentage of items within each category of professional activity was calculated. An investigator was credited with disciplinary expertise in a category of professional activity when at least a quarter of the items within that category were coded. Using the cut-off of a quarter was appropriate because it required an investigator to demonstrate a serious commitment within a disciplinary area while still allowing for the possibility of having expertise in more than one area.

RESULTS

Results of the content analyses of the portfolio and the literature are presented by field: EL science education (research question 1) and EL mathematics education (research question 2). Findings from the CV analysis, which encompassed both fields, are presented last (research question 3).

CHARACTERISTICS OF EL SCIENCE EDUCATION RESEARCH

Lee’s (2005) review of the emerging field of EL science education provided a baseline from which to evaluate the growth of the field over the last decade. In this section, the changes in the research characteristics of the literature of non DR K–12 projects since 2005 are discussed and compared to the portfolio of DR K–12 projects, in terms of both Lee’s characterizations of the research—including research topics, methods, designs, scale, samples, intended outcomes—and her proposed research agenda. The full DR K–12 portfolio included 167 science-related projects,⁷ 18 (11%) of which included a focus on ELs.

Research Topics

Lee’s (2005) review examined the topics addressed by the literature in EL science education in terms of five areas: (a) science learning, (b) science curriculum (including computer technology), (c) science instruction, (d) science assessment, and (e) science teacher education, including both pre-service preparation and in-service professional development. Because the field was young, most of these areas were categorized as having only limited or emerging bodies of research. Since 2005, the distribution across the five areas has remained fairly stable in the non DR K–12 projects (see Table 1), with a continued emphasis on science learning and science instruction, limited focus on science curriculum and assessment, and little research on teacher preparation. The distribution of topics addressed by the DR K–12 projects was slightly different from that of the larger field of non DR K–12 projects, with less of a focus on science learning and a greater focus on science instruction, curriculum, and teacher preparation (see Table 1).   

Table 1. Topics Addressed by Non DR K–12 EL Science Education Projects Pre- and Post-2005 and by DR K–12 EL Science Education Projects

Note. Percentages sum to over 100% because projects could address more than one topic area.

^a As described in Lee (2005).

Research Methods, Designs, and Scale

As illustrated in Table 2, the comparison of the larger field pre- and post-2005 indicates substantial changes in research methods. The larger field shifted from a reliance on mostly qualitative methods to an increased use of quantitative and mixed methods. In the last eight years, the percentage of projects using qualitative methods only decreased to just under a third, while the percentages using quantitative methods only and mixed methods increased (to 44% and 26%, respectively). Similarly, in terms of research design, while descriptive projects were still the majority (56%), 44% of projects used a quasi-experimental approach and 8% used an experimental design. The scale of research projects also changed. Prior to 2005, projects in the field tended to be fairly small scale, infrequently conducted by research teams or in multiple sites (defined as two or more districts). However, since 2005, the projects in the larger field increased in scale, with a majority of projects (79%) now conducted by research teams and over a third conducted in multiple sites.

Table 2. Research Method, Design, and Scale of Non DR K–12 EL Science Education Projects Pre- and Post-2005 and of DR K–12 EL Science Education Projects

Note. Totals do not necessarily sum to 100% due to rounding or to the fact that projects could include multiple research designs or outcomes.

^a As described in Lee (2005).

^b The three projects that focused on both science and mathematics were included in this total.

^c Projects that collected data in at least two school districts were considered multi-site projects.

The examination of the characteristics of the DR K–12 EL science education portfolio indicated similar trends compared to the larger field before and after 2005 in all areas, but with some important distinctions (see Table 2). First, the vast majority (89%) of DR K–12 projects used both quantitative and qualitative (mixed) methods to answer their research questions, while only a small number of projects (26%) in the larger field used this approach. The DR K–12 projects were also much less likely than the larger field to use qualitative methods only (6% versus 31%). Although the majority of projects in both the larger field since 2005 and the DR K–12 portfolio were still descriptive, there has clearly been a move toward efficacy and effectiveness designs. In the larger field since 2005, this movement was evident in the 44% of projects that used a quasi-experimental design. Similarly, the DR K–12 portfolio included 17% of projects that used a quasi-experimental design; however, unlike the larger field since 2005, 28% of the DR K–12 EL science education projects used an experimental design, while only 8% of projects in the larger field used an experimental design.

The DR K–12 and non DR K–12 projects were similar in terms of research scale, as evidenced by similar percentages of projects being conducted in multiple sites (44% and 36%) and by teams of researchers (83% and 79%).

Samples

One area that the Lee (2005) review does not address, but that comes to light when the larger field since 2005 is compared to the DR K–12 portfolio, is the target grade levels of students in the study samples. As can be seen in Table 3, one important difference was that the DR K–12 projects were more likely to be focused on middle school than the larger field (72% versus 46%) and even include some attention to preschool. Both of these age groups are key EL populations: middle school students because they have to master challenging subject matter while learning English, and preschoolers because this is where early language and literacy learning is critical.

Table 3. Grade Levels Addressed in the Non DR K–12 EL Science Education Projects Since 2005 and in DR K–12 EL Science Education Projects

Note. Percentages add to over 100% because some projects focused on students and/or teachers of students across several grade levels, such as both the fifth and sixth grades or teachers of the secondary level (grades 7 to 12).

Intended Outcomes

Lee’s purpose in examining intended outcomes was to determine if projects were focusing on students’ learning and, in particular, if they were taking ELs’ linguistic and cultural resources into account. If projects included language, literacy, student engagement, agency, or empowerment outcomes in addition to science achievement outcomes, or if they considered linguistic or semiotic perspectives, this would indicate an awareness of the importance of the linguistic and cultural assets that ELs bring to the classroom. Lee’s (2005) review indicated that few projects measured student outcomes, including language outcomes, and that a focus on linguistic or semiotic perspectives was rare (see Table 4).

Table 4. Intended Outcomes of Non DR K–12 EL Science Education Projects Pre- and Post-2005 and of DR K–12 EL Science Education Projects

Note. Totals do not necessarily sum to 100% due to rounding or to the fact that projects could include multiple research designs or outcomes.

^a As described in Lee (2005).

^b The three projects that focused on both science and mathematics were included in this total.

Since 2005, close to two thirds of the projects in the larger field (64%) measured students’ science achievement, 18% measured English proficiency or literacy, and 13% measured student engagement, agency, or empowerment, indicating a major shift toward examining the effects of EL science interventions on student outcomes. In addition, the post-2005 research clearly showed a remarkable shift and included a linguistic or semiotic perspective in the majority of projects (59%).

In comparison to the larger field since 2005, the DR K–12 projects were more likely to measure student outcomes in science achievement (78% versus 64%), English proficiency or literacy (39% versus 18%), and student engagement, agency, or empowerment (33% versus 13%). However, despite being more likely to include English proficiency or literacy outcomes, the DR K–12 projects were less likely to consider a linguistic and/or semiotic perspective than the larger field (44% versus 59%).

Research Agenda

In terms of Lee’s (2005) research agenda, as mentioned above, the majority of DR K–12 projects and the larger field included science achievement as an important outcome and over a third of the DR K–12 projects included English proficiency or literacy outcomes, indicating attention to language learning. As can be seen in Table 5, the DR K–12 projects were more likely than the larger field since 2005 to consider the linguistic and cultural resources that ELs bring to the classroom from their home environments (28% versus 15%), to explore the critical skills needed for inquiry-based teaching and learning (78% versus 38%), and to investigate the use of technology in science curriculum and instruction for ELs (44% versus 18%). Because evaluating policy is not the focus of the DR K–12 program, it is understandable that no DR K–12 projects examined the effect of policy changes on high-stakes assessments for ELs. Non DR K–12 projects did not examine this issue either. In terms of teacher preparation, Table 1 indicates that more than four times as many DR K–12 projects compared to the larger field addressed teacher preparation (22% versus 5%).

Table 5. Research Agenda Topics Addressed by Non DR K–12 EL Science Education Projects Pre- and Post-2005 and by DR K–12 EL Science Education Projects

Note. Percentages do not necessarily sum to 100% because projects could address more than one research agenda topic area.

^a As described in Lee (2005).

CHARACTERISTICS OF EL MATHEMATICS EDUCATION RESEARCH

At the initial meeting that served as the impetus for this study, interest was expressed in EL mathematics education as well as in EL science education. During the period of 2007–2012, the DR K–12 program funded 145 projects in the field of mathematics education, 13% of which (n = 19) focused on ELs. Thus, it was possible to broaden the analysis to compare the portfolio of DR K–12 EL mathematics education projects to the literature of non DR K–12 EL mathematics education projects. Since no literature review comparable to Lee (2005) existed for the area of EL mathematics education, the single point of comparison was to projects since 1982 that were identified in the literature search (n = 45).

Research Topics

Table 6 indicates that the most common topic addressed in the larger (non DR K–12) EL mathematics education field was mathematics learning (47%), followed by mathematics instruction and assessment (29% each).⁸ Mathematics curriculum was infrequently addressed (13%) and there were no projects about teacher preparation. The DR K–12 projects, on the other hand, were much less likely to focus on mathematics learning (5% versus 42%), and were more heavily focused on mathematics instruction (89% versus 29%) and teacher preparation (21% versus 0%). The tendency of the larger field to focus more on students, while the DR K–12 projects focused more on teachers, mirrors what was found in the EL science analysis described above.

Table 6. Topics Addressed by Non DR K–12 EL Mathematics Projects Since 1982 and by DR K–12 EL Mathematics Projects

Note. Percentages do not necessarily sum to 100% because projects could address multiple research topics.

Research Methods, Designs, and Scale

As shown in Table 7, the larger field was more likely to focus on either quantitative or qualitative methods only, as opposed to the predominantly mixed methods approach in the DR K–12 portfolio (84%). A majority of projects in both the larger field and the DR K–12 portfolio included descriptive research designs (69% and 61%, respectively). Compared to the larger field (2%), DR K–12 projects were more likely to include an experimental design (17%). In terms of research scale, the two groups of projects were similar, with over half of the projects being conducted by research teams (69% for non DR K–12 and 68% for DR K–12) and almost half being conducted in multiple sites (40% for non DR K–12 and 44% for DR K–12).

Table 7. Characteristics of Non DR K–12 EL Mathematics Projects Since 1982 and by DR K–12 EL Mathematics Projects

Note. Percentages do not necessarily sum to 100% because some projects utilized multiple research designs.

^a The conference project was not included in the research design or research scale counts.

^b Projects that collected data in at least two school districts were considered multi-site projects.

Samples

Table 8 shows the grade level emphasis in the larger field of EL mathematics education and the DR K–12 EL mathematics education. While the larger field was more likely to focus on the primary grades, the DR K–12 projects were more likely to focus on the middle grades. There was also some attention paid to the preschool years in the DR K–12 portfolio of projects that was not evident in the larger field.

Table 8. Grade Levels Addressed in Non DR K–12 EL Mathematics Projects Since 1982 and in DR K–12 EL Mathematics Projects

Note. Percentages do not necessarily sum to 100% because some projects focused on students and/or teachers of students across several grade levels, such as both the fifth and sixth grades, teachers of the secondary level (grades 7 to 12), or all grades (pre-K to 12).

Intended Outcomes

As shown in Table 9, mathematics outcomes were assessed in just under three quarters of the projects in the larger field and the DR K–12 portfolio (71% and 74%). Student engagement, agency, or empowerment outcomes were included in almost a quarter of the projects (24% and 22%). The larger field was twice as likely to include EL proficiency or literacy outcomes as the DR K–12 portfolio (22% versus 11%), but slightly less likely to consider a linguistics or semiotics perspective (29% versus 37%).

Table 9. Intended Outcomes in Non DR K–12 EL Mathematics Projects Since 1982 and in DR K–12 EL Mathematics Projects

^a The conference project was not included in the intended outcome count.

Research Agenda

Finally, in terms of a research agenda for the future, the recommendations made by Lee (2005) can be considered for EL mathematics education as well as for EL science education. As Table 9 indicates, the majority of projects from the literature of non DR K–12 projects and the portfolio of DR K–12 projects included mathematics achievement as an important outcome (71% and 74%). Twice as many projects in the larger field included English proficiency or literacy outcomes, indicating attention to language learning (22% versus 11%). As shown in Table 10, neither the larger field nor the DR K–12 projects demonstrated much attention to integrating knowledge that ELs bring from their home environment or investigating the use of technology for ELs (all percentages were less than 25%). The larger field was, however, slightly more likely to consider the role of technology than the DR K–12 projects (24% versus 16%). As described above (see discussion of Table 5), it is understandable that no projects examined the effect of policy changes on high-stakes assessments for ELs. Table 6 indicates that while 21% of DR K–12 projects addressed teacher preparation, no projects in the larger field did so.

Table 10. Research Agenda Topics Addressed by Non DR K–12 EL Mathematics Projects Since 1982 and by DR K–12 EL Mathematics Projects

Note. Percentages do not necessarily sum to 100% because projects could address more than one topic area.

EXPERTISE OF PROJECT INVESTIGATORS FOR DR K–12 EL SCIENCE AND MATHEMATICS

EDUCATION

The goal of this analysis was to understand whether and how the DR K–12 research teams were bringing expertise from a variety of disciplines (e.g., science/science education as well as EL/ELA) to bear on their research.⁹ Results are presented about the areas of disciplinary expertise of individual investigators, the distribution of disciplinary expertise across key investigators within teams, and the additional expertise contributed by advisory group members.

As described in the method section, CVs were reviewed for evidence of disciplinary expertise. Table 11 indicates that data for all five categories of professional activities were available for three quarters of the investigators. Missing data in a category resulted in no expertise being recorded in that category for the investigator.

Table 11. Availability of Data on Professional Activities for CV Analysis, All DR K–12 Investigators (n = 80)

^a The total for this category is 67 instead of 80 because 13 investigators (eight PIs, five co-PIs) were not professors at universities, as indicated by their current title and/or employer, and did not have any data in this category.

Areas of Disciplinary Expertise

Analyses of the CVs revealed several key findings. First, among the group of DR K–12 project investigators, a range of disciplinary expertise was reflected in each of the five categories of professional activities. Moreover, in all categories, at least one investigator demonstrated expertise in each of the disciplinary areas (see Table 12).

Table 12. Areas of Disciplinary Expertise, All DR K–12 Investigators (n = 80)

Note. Percentages across rows do not necessarily sum to 100% because investigators could have other disciplinary expertise besides the five areas of expertise that were coded, or investigators could have expertise in multiple areas.

Second, expertise in science/science education was most prevalent, followed by mathematics/mathematics education, and then EL/ELA-related areas. This general pattern held across all of the five categories of professional activities. Looked at another way, for science/science education projects, over half of the investigators (53%) demonstrated expertise in science/science education and more than a third of investigators (39%) demonstrated expertise in mathematics/mathematics education in any of the five professional activities, while just under a fifth of investigators (19%) demonstrated expertise in EL/ELA in any of the five professional activities.

Third, as shown in Table 13, while most investigators (43%) had expertise in only one area, it is notable that over a third (36%) had expertise in two areas. In addition, 13% had expertise in three or more areas.¹⁰ The most common area of expertise for those researchers with expertise in only one area was mathematics education (20%). Of those with expertise in two areas, the most common combinations were science and science education (13%) and science education and EL/ELA (9%). Mathematics/mathematics education was less likely to be paired with EL/ELA expertise than with science/science education. Investigators with three or more areas of expertise were most likely to have science, science education, and EL/ELA.

Table 13. Numbers of Areas of Disciplinary Expertise, All DR K–12 Investigators

Note. Percentages do not necessarily sum to the total percentage for each category due to rounding.

Distribution of Disciplinary Expertise

Because research is often conducted by research teams, this analysis investigated whether, and if so how, disciplinary expertise in multiple areas was distributed across projects. Of the 34 DR K–12 EL projects, 25 listed a PI and at least one co-PI, so that multiple key investigators could serve as sources of expertise. The expertise of the research teams was analyzed to gauge the different skillsets of the research teams. Table 14 shows that all but one project team demonstrated expertise in the content area that was the focus of their projects (mathematics or science or both), and that even the one project that did not demonstrate specific science expertise included investigators who were general education experts. Two mathematics project teams, eight science project teams, and one mathematics/science project team consisted of investigators with demonstrated expertise in both the specific content area and EL/ELA. This means that while a majority of science projects had key investigators with EL/ELA expertise, this was not the case for the mathematics or mathematics/science projects.

Table 14. Disciplinary Expertise Within DR K–12 Research Teams (PIs, Co-PIs, and Advisory Groups)

^a This team had general education expertise; however, it is not shown in this table because the expertise was not specifically in mathematics education or science education.

Given that not all projects had key investigators with EL/ELA expertise, the analysis considered the collective expertise for each project provided by the addition of advisory group members. The additional expertise contributed by advisory groups is shown with X’s in Table 14. The most common area of expertise by far to be gained through advisory group members was EL/ELA expertise (seven of nine additions), and this was more likely to be added to mathematics projects that lacked key investigators with EL/ELA expertise. Once advisory group members’ expertise was accounted for, 10 of the 15 “science” projects (67%; second panel in Table 14), two of the three “science and mathematics” projects (67%; third panel in Table 14) and seven of the 16 “mathematics” projects (44%; first panel in Table 14) had expertise in both the target content area(s) and EL/ELA.

DISCUSSION

At the outset of this article, we identified three audiences for which the study is intended: (1) PIs and research teams of the DR K–12 projects, (2) NSF and other funding agencies, and (3) educational researchers and practitioners. Although the results of the study are relevant to all three groups, these results and discussion below may be particularly informative to educational researchers and practitioners who are interested in learning about research on mathematics and science education for ELs.

The results of this study were shaped by the DR K–12 solicitation and the peer review process, both of which influenced which projects were funded. The results demonstrate that the DR K–12 EL projects in both science and mathematics education have made contributions to their respective fields in three areas in particular: (1) their use of mixed methods and experimental designs; (2) their emphasis on instruction and teacher preparation; and (3) their focus on middle school students.

The DR K–12 projects’ use of mixed methods, especially quantitative methods, and their use of experimental designs seems to reflect the increasing focus on intervention studies supported by federal funding agencies. In August 2013, NSF and the U.S. Department of Education published common guidelines for education research and development (U.S. Department of Education & NSF, 2013). The guidelines describe a scale-up approach to research funding in which projects begin as exploratory, move into the design and development of an intervention, and finally, scale up as evaluations of developed interventions. During the period reported on in this study (2007–2011), the DR K–12 program was utilizing this scale-up approach. The DR K–12 projects that were part of this analysis included both exploratory projects and full research and development projects that ranged in funding from $179,466 to $3,967,902. There was one grant that funded a conference in the amount of $105,493. The average amount of funding was $1,445,277 and the median was $1,090,995. This analysis indicates that DR K–12 funding provided financial resources to conduct intervention studies on a larger scale than is typical without such resources. In this way, the DR K–12 program has been able to influence research and evaluation in the fields of EL science and mathematics education.

In terms of areas of focus, the DR K–12 EL science and mathematics projects are more likely to focus on instruction and teacher preparation than the larger fields, which tend to concentrate on student learning. These foci are particularly critical in the areas of EL science and mathematics education. The K–12 EL population has grown from 3.5 million to 5.3 million in the last decade, an increase of 51%, while the overall school-aged population has increased only 7.2% (U.S. Department of Education, 2011). This means that more and more teachers are faced with the challenge of meeting the needs of a growing number of ELs in their classrooms. However, most teachers do not feel prepared to teach ELs and have had few opportunities to learn EL-specific strategies that would help ELs access and master content (Lucas & Grinberg, 2008). Therefore, research on pre-service and in-service training for content area teachers is necessary to identify effective approaches to improving their abilities to work with the growing numbers of ELs in their classrooms. The DR K–12 projects, with their focus on instruction and teacher preparation, have already begun to contribute to this critical area.

In terms of grade levels, the DR K–12 projects were more likely to focus on middle school students compared to the larger fields. The DR K–12 portfolio’s emphasis on adolescent ELs constitutes a critical contribution to an area that has traditionally been understudied. ELs in the middle grades are required to master complex course content as well as academic English. Because they are enrolling in U.S. schools at later grades when literacy skills are usually no longer taught, they often are placed with secondary teachers who have little training on how to teach basic literacy to adolescents (Khong & Saito, 2014; Short & Fitzsimmons, 2007). Therefore, the DR K–12 portfolio, which is exploring potential science and mathematics interventions specifically focused on ELs at the secondary school level, has promise for broadening the knowledge base about improving education for ELs at a particularly vulnerable age for educational failure.

The review of DR K–12 investigators’ CVs was conducted because it was hypothesized that the challenges in EL science and mathematics education would require investigators or research teams with expertise across areas, specifically science/science education, mathematics/mathematics education, and EL/ELA education. The results revealed that DR K–12 investigators are making connections across the content and EL/ELA areas, and are incorporating expertise from both areas, often through the addition of advisory group members. Although the DR K–12 researchers may not be representative of all investigators in conducting research in STEM education, the findings indicate that the DR K–12 program is building the capacity of a core group of EL science and mathematics education researchers. With the number of ELs in U.S. schools on the rise (U.S. Department of Education, 2011) and a growing demand for expansion and development of STEM education (National Science and Technology Council, 2013), encouraging the intersection of research from these two areas represents an important effort to understand and address pressing issues in U.S. schools.

Although collaboration among multiple areas of expertise across research teams in DR K–12 projects is promising, the results point to the need for science and mathematics investigators to expand their knowledge of ELs. For example, greater awareness of the specific issues that ELs face in learning science and mathematics could lead more investigators to conduct EL subgroup analyses as a matter of course, thus adding to the knowledge base about what works for ELs. The results also point to the need for more productive collaboration among investigators across areas. For example, unless more EL experts are involved in EL science and mathematics education, linguistic and/or semiotic perspectives will continue to be overlooked and ELs’ language needs will continue to be ignored. These traditionally separate research areas must come together in new ways to meet the needs of the growing population of ELs.

IMPLICATIONS

Efforts to understand and develop policies to better serve the EL population are imperative. Policymakers must identify the shortcomings in current practices and policies to make meaningful and effective reforms. Research plays an important role in identifying best practices and driving educational policies. However, there is a historical lack of research focused on ELs in science and mathematics education. This study found that through DR K–12 funding and emphases, NSF was able to expand and shape the research in these critical areas.

STRENGTHS AND LIMITATIONS OF THE STUDY

The study is unique in that it had access to the portfolio of a federal funding program. The analysis undertaken for this study was strengthened by the availability of detailed documents for the DR K–12 projects, which allowed for more accurate and reliable coding of project activities. However, the study also presents several limitations. One limitation was that the comparisons drawn here are based on coding of proposal materials and annual reports for the DR K–12 projects versus published papers for the non DR K–12 projects. What is proposed and what is ultimately implemented may differ; therefore, it would have been ideal to have had the same information for both DR K–12 and non DR K–12 projects. Another limitation was not being able to provide a comparison of the expertise of DR K–12 investigators with the larger fields. Such an analysis would have required a data collection effort that was beyond the scope of this study. Despite these limitations, the results have important implications for future research in the areas of EL STEM education.

IMPLICATIONS FOR FUTURE RESEARCH

Of the three intended groups of audience, implications for future research may be most relevant to PIs of the DR K–12 projects and other researchers who conduct their research in the fields of mathematics and science education with ELs. Although DR K–12 projects are at the forefront of the EL science and mathematics education fields, there is still much research to be done. STEM education research that concentrates on ELs constitutes a small percentage of research on STEM education. This study offers implications for future research on STEM with ELs.

First, this study uncovered interesting differences between the fields of EL science and mathematics education. For example, there has been more synthesis and analysis of the research in EL science education (Buxton & Lee, 2014; Lee, 2005) than in EL mathematics education. As a result, a more in-depth comparison could be made between the DR K–12 portfolio and the larger field in EL science education than in EL mathematics education. It would be beneficial to have a synthesis for EL mathematics education that mirrors the EL science education synthesis.

Second, from the perspective of the common guidelines for education research and development (U.S. Department of Education & NSF, 2013), more research is needed to test potentially effective STEM interventions for ELs and for teachers of ELs. Because these guidelines prioritize research findings that can speak to the effectiveness of interventions above other types of evidence, and because the larger fields consist of primarily descriptive research, the derivative conclusion is that more research is needed that can provide practitioners with evidence-based information about effective STEM interventions. However, it may be the case that by prioritizing a certain type of research, other types of research that would contribute to different knowledge bases are less likely to be pursued. The DR K–12 solicitation is clear in that its intent is to support “theory into practice.” In contrast, other programs within NSF are geared toward supporting foundational or theoretical research in education, for example, the Directorate for Education and Human Resources’ Core Research and Research on Education and Learning programs.

Third, this study presents the results of the portfolio of DR K–12 projects for the first five cohorts (2007–2011). The program has funded an additional three cohorts (through 2014) for a total of eight cohorts. As the program matures, future research may conduct integrative reviews or meta-analyses of research findings, analyze theoretical or conceptual frameworks of studies, or employ a particular theoretical perspective to interpret research findings. In addition, future research may compare research findings of DR K–12 projects to the literature of non DR K–12 projects. These future studies will indicate how research findings of the DR K–12 funding program contribute to the emerging literature in the fields.

IMPLICATIONS FOR FUNDING POLICY

Of the three intended audiences, implications for policy may be most informative for the National Science Foundation and other funding agencies to reflect on and assess how their funding policies and programs could make a contribution to a new or emerging field. This study indicated that one funding program—the NSF DR K–12 program—has been able to influence the research topics, methods, design, and samples in the areas of EL science and mathematics education. Researchers applying for DR K–12 funding responded to grant criteria, including research designs and methods that test the effectiveness of interventions and a focus on language diversity, and thus moved the fields forward in important ways. During the period of DR K–12 funding examined in the study (2007–2011), this program has made an impact on the research landscape in these fields, and the impact is likely to continue or even accelerate during the lifespan of the DR K–12 program.

The CCSS and the NGSS are likely to push research on ELs to another level because these new standards are both academically rigorous and language-intensive (Lee et al., 2013). This has implications for all students, but particularly for ELs, as the CCSS and NGSS present both language learning opportunities and demands (Lee et al., 2013). For example, engagement in the NGSS science and engineering practices involves intensive language use (e.g., engage in argument from evidence), as does engagement in the CCSS practices for English language arts (e.g., comprehend as well as critique, value evidence) and mathematics (e.g., construct viable arguments and critique the reasoning of others) (Stage, Asturias, Cheuk, Daro, & Hampton, 2013). Much research is needed to build greater understanding and knowledge about how to meet the language demands while capitalizing on language learning opportunities for ELs in the emerging policy context of the CCSS and NGSS. Federal agencies should continue to provide funding to support this research as a necessary step to improving the quality of science and mathematics education for ELs, closing the achievement gap, and enabling them to be ready for college and careers by the end of high school.

References

August, D., & Hakuta, K. (1997). Improving schooling for language-minority children: A research agenda.

Washington, DC: National Academy.

Buxton, C. A., & Lee, O. (2014). English language learners in science education. In S. K. Abell & N. G.

Lederman (Eds.), Handbook of research in science education (2nd ed., pp. 202–224). Mahwah, NJ:

Lawrence Erlbaum.

Daisey, P., & José-Kampfner, C. (2002). The power of story to expand possible selves for Latina middle

school students. Journal of Adolescent & Adult Literacy, 45, 578–587.

Davison, C. (2006). Collaboration between ESL and content teachers: How do we know when we are

doing it right? International Journal of Bilingual Education and Bilingualism, 9, 454–475.

Ferris, J. M., Hentschke, G. C., & Harmssen, H. J. (2008). Philanthropic strategies for school reform: An

analysis of foundation choices. Educational Policy, 22, 705–730.

Gutiérrez, R. (2002). Beyond essentialism: The complexity of language in teaching mathematics to

Latina/o students. American Educational Research Journal, 39, 1047–1088.

Gutstein, E., Lipman, P., Hernandez, P., & de los Reyes, R. (1997). Culturally relevant mathematics

teaching in a Mexican American context. Journal for Research in Mathematics Education, 28, 709–737.

Hansen, J. S. (2008). The role of nongovernmental organizations in financing public schools. In H. B.

Ladd & E. B. Fiske (Eds.), Handbook of research in education finance and policy (pp. 314–331). New

York, NY: Routledge.

Janzen, J. (2008). Teaching English language learners in the content areas. Review of Educational

Research, 78, 1010–1038.

Khong, T. D. H., & Saito, E. (2013). Challenges confronting teachers of English language learners.

Educational Review, 66, 210–225.

Lee, O. (2005). Science education and English language learners: Synthesis and research agenda.

Review of Educational Research, 75, 491–530.

Lee, O., Quinn, H., & Valdés, G. (2013). Science and language for English language learners in relation

to Next Generation Science Standards and with implications for Common Core State Standards for

English language arts and mathematics. Educational Researcher, 42(4), 223–233.

Lucas, T., & Grinberg, J. (2008). Responding to the linguistic reality of mainstream classrooms: Preparing

all teachers to teach English language learners. In M. Cochran-Smith, S. Feiman-Nemser, & J. McIntyre

(Eds.), Handbook of research on teacher education: Enduring issues in changing contexts (3rd ed., pp.

606–636). Mahwah, NJ: Erlbaum.

National Center for Education Statistics. (2013). The Condition of Education 2013 (NCES 2013-037). Retrieved from http://files.eric.ed.gov/fulltext/ ED542714.pdf

National Science and Technology Council. (2011, December). The federal science, technology,

engineering, and mathematics (STEM) education portfolio. Retrieved from

http://www.whitehouse.gov/sites/default/files/microsites/ostp/costem__federal_stem_education_portfolio_

report.pdf

National Science Foundation. (2011). Discovery research K–12: Program solicitation 11-588. Retrieved

from http://www.nsf.gov/pubs/2011/nsf11588/nsf11588.htm

National Science Foundation. (2013). Proposal and award policies and procedures guide. Retrieved from

http://www.nsf.gov/pubs/policydocs/pappguide/nsf14001/nsf14_1.pdf

National Science Foundation Act of 1950, Public Law 81-507. U.S. Code. Vol. 42, secs. 1861 et seq.

National Science and Technology Council. (2013). The federal science, technology, engineering, and

mathematics (STEM) education: Five year strategic plan. Retrieved from

http://www.whitehouse.gov/sites/default/files/microsites/ostp/ stem_stratplan_2013.pdf

Short, D., & Fitzsimmons, S. (2007). Double the work: Challenges and solutions to acquiring language

and academic literacy for adolescent English language learners—A report to the Carnegie Corporation of

New York. Washington, DC: Alliance for Excellent Education.

Stage, E. K., Asturias, H., Cheuk, T., Daro, P. A., & Hampton, S. B. (2013). Opportunities and challenges

in next generation standards. Science, 340, 276–277.

U.S. Department of Education. (2011). The growing numbers of English learner students, 1998/99–

2008/09. Retrieved from http://www.ncela.gwu.edu/files/uploads/9/ growingLEP_0809.pdf

U.S. Department of Education, Institute of Education Sciences & National Science Foundation. (2013).

Common guidelines for education research and development (NSF 13-127). Retrieved from

http://www.nsf.gov/publications/pub_summ.jsp?ods_key=nsf13126.

U.S. Government Accountability Office. (2012, January). Science, technology, engineering, and

mathematics education: Strategic planning needed to better manage overlapping programs across

multiple agencies (Report No. GAO-12-108). Washington, DC: Author.

van Garderen, D. (2004). Focus on inclusion reciprocal teaching as a comprehension strategy for

understanding mathematical word problems. Reading & Writing Quarterly, 20, 225–229.

Veel, R. (1999). Language, knowledge and authority in school mathematics. In F. Christie (Ed.),

Pedagogy and the shaping of consciousness: Linguistic and social processes (pp. 185–216). London,

England: Routledge.

Appendix A
Descriptions of the DR K–12 EL Science and Mathematics Education Projects

Appendix B
Non DR K–12 Funded Science and Mathematics Education Studies Included in the Review

Acknowledgments

This material is based on work supported by the National Science Foundation under Grant No. DRL-0822241. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

This work was conducted as part of the Community for Advancing Discovery Research in Education (CADRE). Contributors to this work included Sally Wu, Jackie Mendez, Rebecca Gotleib, Elisabeth Ericson, Elisabeth Copson, Megan Tiano, Nicole DellaRocco, Nathaniel Donoghue, Lisa Marco-Bujosa, and Hayley Didriksen. The authors thank NSF Program Directors Elizabeth Vanderputten and Julio E. Lopez-Ferrao, as well as the DR K–12 investigators, for their support of this work.

Notes

1. The DR K–12 program has funded three additional cohorts in 2012, 2013, and 2014, respectively. These were not included in this review because they had not been funded at the time of the analysis.

2. All journal databases available through EBSCO were searched, including: Academic Search Complete; Biomedical Reference Collection: Corporate; Business Source Corporate; EconLit; Environment Complete; MEDLINE; SocINDEX; and the Psychology and Behavioral Sciences Collection.

3. The asterisk is used to identify results with all possible endings such as education, educating, educator, etc.

4. The coding protocol for the literature was adapted only to accommodate for the nature of the materials coded. For example, coding involved identifying the APA citation and year of publication, but did not include DR K–12 program characteristics, such as the cycle of innovation and funding category, which were included in all proposals as dictated by the DR K–12 research solicitation.

5. Because researchers sent their most recent CVs at the time the study team requested them and because CVs were collected once in 2010 and once in 2013, no researcher submitted a CV dated from 2011.

6. Note that for each category of professional activity, items were not exhaustively coded on investigators’ CVs. Rather, only those items that fell into one of the five areas of disciplinary expertise were coded.

7. This total includes both projects that focus on only science and those that focus on science as well as other disciplines, such as mathematics, computer science, or engineering.

8. Percentages do not necessarily sum up to 100% because projects can address multiple research topics.

9. Note that this analysis was limited to the DR K–12 project investigators because their CVs were available to the research team through DR K–12’s resource network. No similar analysis could be done for the larger field due to the lack of easily available data on all study investigators.

10. The remaining 9% of investigators did not meet our criteria for expertise in a disciplinary area (25% of their reported works within any category of professional activity). This is in part due to missing data for these investigators.