Home Articles Reader Opinion Editorial Book Reviews Discussion Writers Guide About TCRecord
transparent 13
Topics
Discussion
Announcements
 

College Completion, the Texas Way: An Examination of the Development of College Completion Policy in a Distinctive Political Culture


by Denisa Gándara & James C. Hearn - 2019

Background: College-completion policies dominate state higher education policy agendas. Yet we know little about how policy actors make decisions—and what sources of evidence they use—within this policy domain.

Focus of Study: This study explores the use of evidence in college-completion policymaking in depth, focusing on Texas. In addition to exploring policymakers’ use of different types of information, this study examines the role played by intermediaries.

Research Design: We employed a qualitative case study design drawing on interviews with 32 policy actors engaged in college-completion policy in Texas. Our analysis consisted of both deductive coding (based on our a priori coding scheme) and inductive coding (based on emerging themes) to arrive at our four major findings.

Findings/Results: The analysis revealed four primary findings. The first theme suggests an insular culture of college-completion policymaking: Policymakers at various levels preferred Texas-based data and rejected the notion that external groups contributed to setting the college completion agenda in Texas. Second, business groups and a business ethos permeated college-completion policymaking in Texas. Third, research evidence was seldom employed in this policy process, partly because policymakers prefer concise and timely information. Finally, the study uncovered a new tactic for supplying research employed by certain intermediaries: punchy messaging, which was effective at garnering attention but also yielded unintended consequences.

Conclusions/Recommendations: Overwhelmingly, higher education policy actors tended to prefer Texas-based data. Respondents cited three major reasons for this preference: the high quality of the state higher education coordinating board’s data, Texas’s unique demographics, and the accessibility of statewide data. These findings reflect the mediating role that is played not only by state structural characteristics, but also by culture. Perceptions of Texas’s distinctive inward-looking nature permeated our interviews and set the stage for the role that intermediaries played in the state and the preferences for information. Intermediaries wishing to inform college-completion policy activity at the state level should consider the uniqueness of the state context in supplying information. For states that are more insular, like Texas, working through internal (in-state) intermediaries may be an effective strategy. In light of our findings of preferred types of information, those intending to influence policymaking should consider making information—especially research evidence—concise and easily accessible and establish relationships with policymakers and their staff members.



INTRODUCTION


In 2010, the College Board published a report in collaboration with the National Conference of State Legislatures (NCSL) titled “The College Completion Agenda: State Policy Guide.” In the first lines of the report, the authors described the urgency of addressing college completion in the United States. They called attention to the “crisis across the educational landscape” and in particular, the fact that “the United States is facing an alarming education deficit that threatens our global competitiveness and economic future” (College Board, 2010, p. 3).


Consistent with the College Board and NCSL’s call to action, numerous national organizations, including the National Governors Association (2010) and philanthropic groups (e.g., Lumina Foundation, 2013), have advocated for increased attention to college completion. Despite strong advocacy by individuals and organizations for college-completion policy activity in the states (Perna, Finney, & Callan, 2014), we know little about how state policymakers have approached the college-completion agenda. The present study explores one critical component of state-level college-completion pursuits: the extent to which higher education decision-makers use various types of information, including research evidence, to inform policy activity around the college-completion agenda. To date, policymakers’ utilization of information in higher education policymaking has gone virtually unexplored (Ness, 2010). This dearth in the higher education literature is especially problematic given the rapid diffusion of certain policies like performance-based funding for higher education (Snyder, 2015), which has occurred despite limited evidence of the policy’s effectiveness (e.g., Dougherty & Reddy, 2013; Rutherford & Rabovsky, 2014; Tandberg & Hillman, 2014). As such, it is critically important to examine the process leading to decisions regarding these policy instruments and the extent to which information, including research evidence, is used.


In parallel to exploring the degree to which policymakers use different types of information, this study examined the role played by intermediaries—groups that, among other roles, have the potential to serve as research brokers. Education policy scholars have begun to call attention to the distinct role intermediaries play in fostering the use of research and other forms of information in policymaking (DeBray, Scott, Lubienski, & Jabbar, 2014; Jabbar, LaLonde, DeBray, Scott, & Lubienski, 2014; Ness, 2010; Scott, Jabbar, LaLonde, DeBray, & Lubienski, 2015). Ness (2010) defines intermediary organizations as entities that “serve a translating function between two principals with different values and perspectives” (p. 37). While scholars have begun to document intermediaries’ influence over primary and secondary education policy in recent years (DeBray et al., 2014; Jabbar et al., 2014; Scott et al., 2015; Scott, Lubienski, DeBray, & Jabbar, 2014), their role in higher education, including in state-level college-completion activity, remains largely unexplored (Ness et al., 2015).


This study of information use and the role of intermediaries in higher education policymaking focuses on college-completion policy activity in Texas. As we discuss in greater detail in a subsequent section, we chose to examine this state because the Texas legislature (1) has robust resources, including professional staff, suggesting fertile ground for evidence use, and (2) was engaged in substantial college-completion policy activity prior to data collection, as revealed by preliminary document reviews. Together, these factors facilitated an in-depth analysis of the use of evidence in college-completion policymaking within a bounded context.


For this analysis, we drew upon a dataset that included interviews with 32 actors involved in higher education policymaking in Texas to examine the development of college-completion policy in the state. We focused on two primary research questions:


(1) How was information—especially research evidence—used in discussions and decisions related to college completion in Texas?

(2) What unique role did intermediaries play in enhancing or restricting the use of information in college-completion policy discussions and decisions in Texas?


Recently, Ness and colleagues (2015) urged scholars to pay greater attention to the roles played by “less obvious actors” (p. 158)—including researchers, business groups, and other intermediaries—in higher education policymaking. Employing a case study methodology, this study seeks to respond to that call by exploring in-depth the extent to which intermediaries help shape college-completion policy in Texas. Further, the study provides valuable insights into the policymaking process by examining whether and how policy actors use various types of information in decision-making processes in higher education.


CONCEPTUAL PERSPECTIVES


We anchored this analysis in the established literatures on policymakers’ research use and on intermediary organizations. Our analysis attended to the use of information more broadly, and not to research use exclusively. However, theoretical perspectives on policymakers’ use of research evidence were instructive since they provided insights on the types of information policymakers prefer and how they use this information. In addition to garnering insights from perspectives on research use and intermediaries separately, we drew on a developing body of research examining the distinct role of intermediaries in promoting research use in policymaking.


USE OF RESEARCH EVIDENCE


The early literature on research use in the policymaking process examined the extent to which policy actors (including elected officials and agency officials) relied upon research evidence in designing policy. Many early studies of research utilization suggested that researchers and policymakers belonged to “two communities,” and the two had separate priorities and cultures (Birnbaum, 2000; Caplan, 1979; Dunn, 1980; Snow, 1961). There is evidence of this phenomenon within higher education policymaking (Hearn, 1997). Education professionals working in state higher education agencies have also documented the disconnect between researchers’ agendas and state needs, noting that many research studies are either not relevant or timely enough to be used in policymaking (Conaway, Keesler, & Schwartz, 2015).


Studies of the “two communities” phenomenon (e.g., Caplan, 1979) have tended to emphasize the problem with the supply side of research, suggesting scholars can do more to enhance research uptake by policymakers (Ness, 2010). In contrast, studies that have explored the demand side of information use (e.g., Weiss, 1979) have focused on policymakers’ preferred information sources and the ways in which policymakers use evidence. For instance, studies have revealed legislators’ preference for information from constituents (Mooney, 1991; Webber, 1987), nonpartisan legislative research offices (Hird, 2006), and state government agencies (Hamann & Lane, 2004; Shakespeare, 2008). The present analysis focuses on both the demand for information by policymakers and the supply of information, including by intermediaries.


In addition to the supply and demand dynamics of information flow, this study garners insights from literature on how policymakers use information. Weiss’s (1979) landmark study was especially instructive. In that paper, she identified seven “meanings” of research use, which she extracted from the extant policy research on research utilization. The first is the knowledge-driven meaning of research use, through which basic research reveals an opportunity for applied research. Second, the interactive meaning involves using research as only one component of a complex decision-making process involving a variety of actors and considerations, including political calculations and pressure. Weiss’s third meaning, tactical use of research, captures policymakers’ use of the research process itself, rather than the content of the research, for practical purposes. For instance, policymakers can use the research process to delay decision-making, such as by requesting a research study to avoid taking immediate action on a policy proposal. In the fourth meaning, research as a process interacting with policy, not only does research inform policy, but policy also influences and motivates research.


The final three meanings have been the subject of most studies of research use. In the instrumental, or problem-solving, meaning, policymakers use research to inform the solution to a specific policy problem. This is the meaning we typically associate with research use and hope to observe in policymaking. Conversely, the political meaning refers to the use of research as “ammunition” to support predetermined positions (Weiss, 1979, p. 429). Discussing their findings of evidence use in district central offices, Coburn, Toure, and Yamashita (2009) captured the duality of our ideal for evidence use (i.e., the instrumental meaning) on one hand, and the reality we sometimes observe (i.e., the political meaning) on the other. They wrote, “Research and evidence often become further tools in the very political processes that they are meant to circumvent” (p. 1145). Finally, according to the conceptual or enlightenment meaning, concepts that have emanated from research infuse the policy process over time. Previous research has found that this is the most common use of evidence reported by policymakers (Amara, Ouimet, & Landry, 2004; Coburn, 2005; Weiss, 1977).  


INTERMEDIARY ORGANIZATIONS


In addition to theoretical perspectives on the use of research and other forms of knowledge in policymaking, this study built on literature related to intermediary organizations. Ness (2010) urged higher education scholars to examine the unique role of these boundary-spanning groups in studying the extent and ways in which policymakers use information. In the present study, we used Ness’s (2010) inclusive definition of intermediaries (i.e., entities that serve a translating function between two principals with different values and perspectives), although others define these groups more narrowly (e.g., as research brokers). Our definition also adhered to the taxonomy used in Scott and colleagues’ (2014) chapter on incentivist K–12 policies. In their classification, think tanks, foundations, advocacy groups, civil rights organizations, and the press are examples of intermediaries.


Philanthropic foundations, another type of intermediary, warrant further scrutiny given their ascending role in education policy in the United States and globally (Ball, 2008; Ball & Junemann, 2012). Researchers have documented these organizations’ significant role in setting K–12 education policy agendas (Reckhow, 2012; Reckhow & Snyder, 2014). Furthermore, through funding, they have played crucial roles in supporting data gathering, analysis, and distribution (Scott et al., 2014). In higher education, they have engaged in college-completion policy activity, primarily by funding other intermediaries (Gándara, Rippner, & Ness, 2017).


Beyond philanthropic foundations, numerous intermediary organizations—including the American Council on Education’s Commission on Education Attainment, the National Governors Association’s Complete to Compete, the Education Commission of the States, the Southern Regional Education Board, and the College Board—have been active in the college-completion policy arena. These types of groups can be classified as intermediary organizations based primarily on structural features—since they serve both policymakers and campus representatives—and functional characteristics, since they translate research and make policy recommendations (e.g., via policy reports and briefs and through legislative testimony).


Finally, one other type of organization was central to this analysis: state higher education agencies. These groups are government bodies, but can also serve as intermediaries (i.e., brokers between state higher education systems and institutions on one hand and the legislative and executive branches on the other). State agencies serve at least two important functions: mediator between federal and local policy (Hamann & Lane, 2004) and data source (Shakespeare, 2008). Studies have found mixed results with respect to state agencies’ influence on policymaking as compared to other suppliers of information, such as constituents and research offices (Gray & Lowery, 2000; Guston, Jones, & Branscomb, 1997; Webber, 1987). Ness (2010) has advocated examining these unique organizations in the context of higher education policymaking. Thus, within this analysis, we paid particular attention to the unique boundary-spanning function of the state higher education agency, the Texas Higher Education Coordinating Board (THECB).


RESEARCH USE AND INTERMEDIARIES


At the intersection of research use and intermediaries lies a small literature examining intermediaries’ roles in promoting evidence use. In primary and secondary education, Honig (2004) illustrated the role that intermediary organizations at the district level play in promoting the use of evidence-based curricula and pedagogical practices. Scott and colleagues (2014), for their part, proposed that intermediary functions, as they relate to information use, include producing, funding, interpreting, and disseminating information. According to this taxonomy, intermediaries serve as both suppliers and demanders (or consumers) of information, including research (Scott et al., 2014). The supplier role of intermediaries, particularly in producing and disseminating research, is most commonly acknowledged, especially since policymakers increasingly rely on these organizations to interpret research evidence (Finnigan & Daly, 2014).


RESEARCH DESIGN


This analysis of college-completion policy activity in Texas employed a case study design, which allowed for an “intensive, holistic description and analysis” (Merriam, 2009, p. 46) of the use of various types of information in state-level college-completion policy activity. In particular, this analysis focused on a unique or extreme case (Yin, 2009), given Texas’s distinctive level of legislative professionalism, as measured by legislative staff capacity, and high college-completion policy activity, as suggested by document reviews. By bounding the analysis geographically, we held constant various state-level characteristics, including political, socioeconomic, organizational, and higher education features of Texas policymaking that might influence college-completion policy activity.


CASE SELECTION


This study stemmed from a multiyear, multi-case project, in which we examined college-completion policy activity in five states: Georgia, North Carolina, South Carolina, Tennessee, and Texas. We selected these states for two major reasons. First, they shared relevant state characteristics for which we sought to “control” (e.g., location in the South). These states also had low educational attainment levels, which would suggest amplified attention to college completion relative to states with higher levels of educational attainment. Further, these states varied along dimensions that were of interest for the broader study, including type of higher education governing structure and levels of engagement in a national college-completion initiative. Specifically, three of the five states were members of the Complete College America (CCA) Alliance of States. Because we were interested in the role of CCA in college-completion policy activity for the larger project, we sought variability in membership in this organization.


From this group of states, we chose to focus on Texas for this analysis of the use of evidence and the role of intermediary organizations for the following reasons. First, given our interest in the use of research, we chose a state with considerable legislative staff capacity, since a legislature with sizeable legislative staff resources may lead to “better-informed legislators” (Squire, 2007, p. 214). According to the National Conference of State Legislatures, in 2015, Texas ranked second (after New York) in total staff employed during legislative sessions. Accordingly, we focused this analysis on a state with higher potential for evidence use to better glean preferences for various types of evidence and the mechanisms of evidence use.


Second, we focused on Texas for this analysis because of its robust college-completion policy activity, which enhanced opportunities for examining the use of evidence in college-completion policymaking. Document reviews preceding data collection revealed that Texas legislators were considering the following policies: a shift toward outcomes-based funding, expansion of dual credit, changes to developmental education, and modifications of the statewide need-based aid program. In addition, a task force was developing a replacement to the state’s access-centered master plan for higher education. The new plan, “60x30TX,” emphasized completion by calling for an increase in postsecondary degree or certificate attainment rates for adults between the ages of 25 and 34 from 38% to 60% by the year 2030 (THECB, 2015).


Beyond its uniqueness in both legislative staff capacity and college-completion policy activity, Texas constituted an apt case for this analysis given the state’s broader national significance. First, Texas is the second-largest state in the country in land size and population. It also ranks second (after California) in number of students enrolled in a degree-granting postsecondary institution (National Center for Education Statistics, 2014). Thus, college-completion policy activity in this state affects a considerable portion of the nation’s current and prospective college students. Beyond its size, the demographic composition of Texas epitomizes demographic trends across the United States, particularly given the fast-growing Hispanic/Latinx population. This demographic shift is especially important in the context of postsecondary attainment since for the past four years, Hispanics/Latinxs have been the largest minority group on college campuses. In addition, in 2010, Texas was one of four “majority–minority states,” meaning there were more non-White residents than there were White residents living in Texas (Murdock, Cline, Zey, Jeanty, & Perez, 2014). As such, understanding education policymaking within this state context is relevant beyond Texas.


DATA COLLECTION


Data for this study are from the aforementioned multiyear project, in which we examined college-completion policy discussions, decisions, and recommendations in five states. The data collection process entailed conducting individual interviews with actors involved in college-completion policy initiatives in Texas. For the broader project, we also collected documents and videos and engaged in participant observation of legislative hearings in which policymakers discussed college completion. These data enriched our learning throughout the study, helping us identify interview participants and construct the interview protocols, and supplementing our findings from the interviews.


Specifically, we conducted interviews with 25 policy actors in Texas and with seven intermediary (national or regional) officials. We used purposive sampling to identify participants. In particular, we drew on media accounts and on actors’ formal roles in government and higher education settings to identify actors who were involved with the supply or demand of information in college-completion policy processes in Texas. We targeted the following potential interview participants: (1) legislators (and their staff members) who had sponsored college-completion legislation; (2) leaders of Texas-based intermediaries who had been active in college-completion activity, per document reviews; (3) governor’s education staff members; (4) high-ranking higher education agency (i.e., coordinating board) officials; and (5) higher education leaders, including chancellors, vice-chancellors, and presidents of Texas colleges and universities who were active in college-completion policymaking, per our document reviews.


At the end of each interview, we asked participants to recommend other actors familiar with college-completion policymaking in Texas. In addition to snowball sampling (Patton & Patton, 2002), after the first round of interviews, we relied on document analysis and participant observation to identify additional interview participants. We also contacted journalists who covered higher education in Texas. Table 1 lists the number of interview participants included in this study by sector. Data collection ceased after we reached saturation, the phase at which minimal new insights were gleaned from the interviews (Lincoln & Guba, 1985).


Table 1. Number of Interview Participants by Sector


Sector

Interview Participants

Texas Higher Education Coordinating Board

4

Legislature

6

Multi-Campus System Office

5

Campus

3

Governor’s Office

1

Press

1

State-Level Intermediary

3

Education Consultant

1

Industry

1

Regional/National Intermediary

7

Total 

32


We used semi-structured protocols of open-ended questions to guide the interviews (Rubin & Rubin, 1995). Two sample interview protocols (one for intermediary officials and another for elected officials and their staff members) appear in Appendix A. The protocols included questions about the supply and demand dynamics of information acquisition and use in college-completion policymaking. The interviews also specifically addressed the role played by national, regional, and state-level intermediaries in college-completion policy activity in the state. With participants’ permission, the interviews were all recorded and transcribed.


In addition to the interview data, we drew on an archival inventory of college-completion policy activity in Texas. The corpus of documents (n = 58) comprise newspaper articles, meeting materials, legislative statutes and codes, intermediary documents and website content referencing Texas college completion, and higher education agency website content. We also collected over 12 hours of archived video footage documenting THECB meetings.


The documents helped inform the selection of the first round of interview participants and the creation of the interview protocols. For instance, following reviews of college-completion legislation, we asked about specific policies and the information that was used to inform their development. We also relied on our review of publications, including those by intermediaries, to inquire about specific decisions made by information suppliers. For example, we asked CCA officials about their decision to include relatively few footnotes in their reports, compared to those produced by similar organizations. On the demand side of information use, we asked policymakers about their engagement with specific reports or other publications. Document reviews also helped us identify interview participants (e.g., legislators to interview based on their sponsorship of college-completion legislation).


In addition to interviews and documents, we drew on participant observations of four legislative hearings—where legislators discussed and heard legislative testimony on higher education policies and on 60x30TX—and one THECB Strategic Planning Committee Meeting. Of the legislative hearings, two were in the Senate Higher Education Committee, one in the House Higher Education Committee, and one in the House Public Education Committee, where legislators discussed a college-readiness policy. In the THECB Strategic Planning Committee Meeting, committee members deliberated the development of the state’s new strategic plan for higher education (60x30TX). This study’s authors observed the proceedings in the audience and took notes, including notes on what types of evidence actors referenced in discussions and in legislative testimony. We also made note of who the key actors, beyond policymakers, were in these conversations. Following each event, we debriefed on our notes and identified additional interview participants based on the events.


DATA ANALYSIS AND TRUSTWORTHINESS


We coded the interview data using both descriptive and conceptual codes (listed in Appendix B). Descriptive codes included categories such as key policy actors and college-completion policies. The conceptual coding scheme comprised both a priori codes based on the conceptual framework on evidence use and intermediaries, as well as post hoc codes based on themes that emerged during the analysis. The a priori coding scheme included codes such as: the role played by an intermediary (e.g., information provider and policy advocate); characteristics of the supply of information; features of the demand for information; and state organizational and political characteristics (e.g., research capacity). Emergent codes included preferred types and sources of information, the legitimizing role of intermediaries (under “Role Played by Intermediary”), and state uniqueness. To perform the analysis, we used Dedoose, a mixed-methods data-analysis application. Examples of the coding process appear in Appendix C.


We arrived at the four primary themes that emerged from the analysis by using an axial coding technique to evaluate how our various categories “crosscut and link” (Strauss & Corbin, 1998). For instance, we found significant overlap in our themes on preferred sources of information, the role of intermediaries, and state characteristics. This analysis led to our first theme of insularity in college-completion policymaking in Texas. That theme captures the findings that policy actors prefer Texas-based data (partly because they view Texas as incomparable to other states) and that out-of-state, including national, intermediaries did not set the completion agenda in Texas.


Our second theme on the percolation of a strong business ethos into college-completion policymaking in Texas emerged from our analysis of the intersection between policymakers’ demand for information, the role played by intermediaries (including business groups), and state characteristics (including workforce/economic development). For our third theme, our analysis of actors’ preferences for different types of information, uses of the information, and differences by actors revealed a dearth of research (analytic) evidence in the policy process and dissimilarities in uses of evidence between policymakers and their staff. Finally, for our fourth theme, our analysis of the relationship between intermediaries’ roles and their preferences for supplying information uncovered a unique motivation for supplying information: to “shame” colleges and universities.


A team of five researchers coded the data. Each transcript was coded twice, by two separate researchers. During the second round of coding, the “secondary” coders took notes of any discrepancies between their coding decisions and those of the first coder. We discussed these differences and arrived at coding decisions with input from the full coding team during regular coding meetings.


To optimize the degree to which our findings represented the phenomena we examined, we followed a number of recommended methods (Merriam, 2009). First, we triangulated our findings by drawing on interviews with both Texas policy actors and intermediary representatives. Concurrence in interpretations of phenomena across actors in various roles enhanced our confidence in internal validity. In addition, a peer review process helped us compare individual interpretations of findings across researchers on the team and facilitated testing of rival hypotheses. The final technique used to enhance internal validity was adequate engagement with data collection (i.e., saturation after 32 interviews over a 19-month span).


TEXAS CONTEXT


The structural and policy contexts in Texas are unique and important for understanding the state’s use of information and the role of intermediaries in college-completion policy decisions. Before presenting the themes that materialized in this analysis, we first outline contextual elements that emerged from this study that are especially critical for understanding college-completion policy activity in Texas.


STRUCTURAL CONTEXT


Organizationally, the state is home to a complex network of higher education institutions. The public higher education institutional landscape, on which we focus in this analysis, comprises 50 community college districts and the Texas State Technical College System (with four technical colleges), 10 university systems, and 46 universities (including medical and health science centers). This landscape represents more public higher education institutions than in any other single state.


The higher education agency, the THECB, oversees all public colleges and universities in the state. This agency’s formal authority is primarily limited to approving degree programs and off-campus activities, providing recommendations on state appropriations to public colleges and universities, and approving most major construction projects. In addition to having limited formal powers, during the 83rd legislative session, the THECB lost authority during Sunset Review, the process by which legislators evaluate state agencies in Texas. In particular, legislators “required the coordinating board to involve colleges and universities in its decision-making processes on the front end rather than later on . . . [and] stripped the agency of the authority to close low-performing degree programs” (Hamilton & Ramshaw, 2013, para. 11). These dynamics constitute important context for this analysis, since relations with the coordinating board might affect legislative officials’ use of information from this agency.


Notwithstanding political dynamics surrounding the THECB, it is clear that the agency collects, produces, and distributes a plethora of information, much of which relates to college completion. As the first state to use a formula to allocate funds to public higher education institutions, Texas has had a robust data collection system for decades (McKeown-Moak, 1999). Three state agencies, including the THECB, worked collaboratively to create a longitudinal data system that tracks students from pre-kindergarten through the workforce.


POLICY CONTEXT


Texas policymakers have long been interested in issues related to college student access and success in the state, partly due to concerns with the state’s economy and partly as an acknowledgment of substantial demographic shifts (Malandra, 2012). Although the national college-completion movement began to gain momentum circa 2009, policymakers in Texas began focusing on completion well before 2000. Texas’s Closing the Gaps statewide master plan, implemented in 2000, included student success as one of four major goals. Another goal, which received the most attention, was increasing participation (or access), and “closing the gaps” in access and attainment between different demographic groups.


For over a decade, policymakers in Texas have been attuned to gaps, or differences in participation and attainment, particularly between racial/ethnic groups. This focus is reflected in the data the coordinating board collects and reports, which are often disaggregated by race and ethnicity, and sometimes by gender and by students’ socioeconomic profiles. Unfortunately, state policymakers have also taken steps that scholars have considered detrimental to educational attainment, especially for racial/ethnic minorities. For instance, tuition deregulation for public institutions in Texas resulted in decreased participation by Hispanic students (Flores & Shepherd, 2014). This state, like many others, has also divested in higher education; declines in financial support have been particularly harmful for community colleges (Perna et al., 2014).


Finally, one important feature of the Texas policy context is the substantial business influence in the state. Commenting on the business community’s prominence in Texas, political scientist Cal Jillson (2014) observed, “Groups that represent business come in many shapes and sizes, but together they are the dominant force in Texas politics” (p. 85). Some business leaders have played a crucial role in both the extent to which policymakers have focused on higher education policy and their preferred areas of emphasis. Malandra (2012) has reported that industry representatives, motivated by concerns with an adequately skilled workforce, were largely responsible for leading higher education reform. The present study examines how this context shapes debates on college completion specifically; this issue lies prominently at the intersection of multiple stakeholders and actors in the state (e.g., business, government, educators, students and families, and the media).


THEMATIC FINDINGS


Our analysis led us to identify four themes. The first—and most prominent—theme reveals the inward-looking nature of college-completion policymaking in Texas. This theme of insularity encompasses decision-makers’ preferences for Texas-based data and the perceptions that out-of-state influences on the college-completion agenda in Texas have been limited. The second theme confirms that the strong business ethos in Texas permeates college-completion policymaking in the state, and is especially apparent in business groups’ direct engagement in this process. Our third theme captures the dearth of analytical research in this policy process and discerns differences in evidence use by policymakers on one hand and their staffers and higher education agency officials on the other. Our final theme centers on the supply side: Some stakeholders deployed information tactically to shame colleges and universities.


THEME 1: LONE-STAR THINKING IN THE LONE-STAR STATE


The first theme captures two major phenomena: (1) decision-makers at various levels prefer Texas-based data to other sources of information, and (2) state-level policy actors rejected the notion that external actors helped set the college-completion agenda in Texas.


Preference for Texas-specific Data


Overwhelmingly, Texas policymakers—at the state legislative and higher-education system levels—expressed a preference for Texas-based (internal) data, including reports produced or repackaged by the coordinating board. Numerous reasons were given for this preference, including the quality of Texas’s longitudinal data system, the absence of states that are comparable to Texas, and the accessibility of Texas’s data. Identifying the coordinating board as her “first stop,” one senator’s aide responded, when asked about her preferred sources for information relating to college-completion policy:


I can talk to you generally about how I do research when issues come up during the interim, and . . . about how we go about researching other bills that come in. . . . I would say . . . my first stop, generally, is to contact the higher education coordinating board. They just have a wealth of data and they’re able to pull it pretty easily, just do pretty quick queries, and so as . . . an analyst at the state, they’re just an extraordinary resource. My next stop, usually, is . . . if it’s an issue related to a university, to call the government relations staff at different universities to get their specific data or experience or perspective, etc. If I’m . . . looking at questions related to community colleges, I’ll call specific community colleges or I’ll call . . . the Texas Association of Community Colleges. Oftentimes they’ll provide me with either . . . primary source in-house data, or . . . whatever reports have been developed. So by the time I’m actually getting to the literature, it’s usually kind of guided by who I perceive to be the experts. You know, we do—we use the CCA’s Game Changers publication, especially as it relates to like, outcomes-based issues and structured pathways research, and so that’s—that’s been helpful.


This respondent noted two clear examples of internal (Texas-based) data as preferred sources: data supplied by the THECB and “in-house data” from the Texas Association of Community Colleges. Beyond internal data, this aide, who worked for a legislator who served on the Senate Higher Education Committee, identified stakeholder input as a critical information source. After consulting those resources, she would refer to “the literature,” including intermediaries’ reports.


Another legislative aide, who also worked in the Senate Higher Education Committee, likewise identified Texas-specific data as his preference, pointing to Texas’s unique demographic landscape:


When you’ve got a state of 25 million people, 1.6 million college students, 37 public universities, 50 community college districts . . . it is always preferential to have Texas-based data. . . . Because if I have research that . . . is limited to private schools in the northeast, that is not especially relevant to a policy decision made that will impact . . . our own institutions. . . .We have a different population than Wyoming or Iowa or . . . even Florida, or Louisiana and . . . the more specific whatever it is can be to Texas, the better.


At the system level, a vice chancellor at one of the flagship university systems described his own data acquisition process around college-completion policy initiatives, identifying first system-level data, followed by national data for comparisons:


First of all, we look at four-year graduation rates . . . and we also look at semesters completed to assess how long the person take[s] to graduate from high school. And that’s how we devised our metrics, that’s how we devised our goal. And we typically have a group of peers that—that we bring together from national data and that’s how we set targets and goals. . . . So that’s how we do it.


One THECB official confirmed policymakers’ preference for internal data, claiming that the coordinating board’s data “carries the day, and rightly so.” This participant credited the quality of the data for this preference, since THECB data are subject to audits (as opposed to data submitted by institutions to the U.S. Department of Education to be included in the Integrated Postsecondary Education Data System). Beyond the quality of the coordinating board’s data, one legislative aide echoed the aforementioned legislative aide’s sentiment in pointing to the accessibility of the THECB’s data as his reason for preferring it:


I will just Google “THECB accountability” and follow the link, and just go from there. . . . You know, somebody called me the other day [asking] how many students are enrolled in our public four-year institutions and that’s a 45-second look at on the . . . Internet. I mean, I don’t have to keep things memorized because it’s—it is literally less than a minute away.


As illustrated previously, respondents also credited the unique demographic and structural state characteristics in the state for their preference for Texas-specific data. In parallel to expressing their partiality for Texas-specific data, some respondents cited the uniqueness of Texas’s demographics to explain their beliefs that cross-state comparisons were of limited value. One intermediary official, for example, observed that Texas is


. . . what the nation’s going to look like, and so we’re kind of like, on the cusp of what will be the reality as a nation, years to come, and so who can we look to? We look to ourselves and look to the people within those regions that are smart, that have—have a sensitivity to the population and know it best. That being said, yes, we can learn a lot from similar states like Florida and California, you know, perhaps New Mexico and maybe Colorado in terms of its Hispanic population, but we are . . . such a large state that, I mean, we can go from El Paso, and that’s very different from what’s going on in Houston, and then the [Rio Grande] Valley.


Likewise, an official in the governor’s office noted: “We look outside but I mean, a lot—but there’s—there’s just not a lot of states that are like us. So it’s hard to compare . . . New York’s not like us even though they’re the same size.”


Texas-driven Completion Agenda


Following this theme of insularity in college-completion policymaking, the analysis also revealed policymakers’ perceptions that out-of-state intermediaries did not influence college-completion agenda setting. That said, we found ample evidence of national intermediaries’ engagement in local (e.g., campus- and system-level) college-completion initiatives, especially by providing technical assistance. While some state-level policymakers recognized CCA—a single-issue group focused on elevating college completion on state policy agendas—and some were familiar with their reports, they rejected the notion that any external groups, including CCA, helped set the college-completion agenda. One legislator highlighted the state’s uniqueness in denying external groups’ role in agenda setting:


Our agendas—and maybe it’s the size of Texas, maybe it’s the number of institutions and flagship institutions and things like that—we’re driven . . . by mostly what goes on in the state of Texas. That doesn’t mean that our graduates aren’t going all over the country and things like that, but our system is so large—that’s most of our interaction. . . . If I talked to all the people around here that want to talk about higher education, quite frankly we don’t have time for [them]. . . . These organizations specifically, like I say, I have no bias against them, but—have they been providing us with a lot? No. Are they welcome? Yeah, to the degree that I [and staff] have time, sure, but . . . they shouldn’t expect us to . . . call them.


When asked about CCA’s role, this legislator replied, “I don’t know who the hell they are.” Another legislative staff member also was not familiar with this group. Only one of the three legislative staffers recognized the group; as mentioned previously, she was familiar with some of the “Game Changers,” CCA’s proposed policy solutions for improving college-completion rates.


Consistent with our review of documents, respondents noted that the completion agenda in Texas preceded the national momentum around this policy issue. One coordinating board representative described CCA’s non-role in initiating college-completion policy discussions in Texas by noting: “I don’t think . . . Complete College America has done much to sort of raise . . . the profile of the issue within the state, in large part because I think it’s already there.”


The interviews with intermediary officials serve to triangulate these findings from state policymakers about the challenges CCA has faced exerting influence at the state policy level. One senior CCA official, after discussing the organization’s influence in Georgia, credited state structural and political characteristics for CCA’s limited influence in agenda setting in Texas:


Texas has been more problematic. I think we probably went to Texas early, but it’s such a big state, it’s really hard to get a handle on how to start or to get a critical mass of people. The governor, this was not a priority of his, so we really couldn’t get much traction with the governor. We had a lot of support from the business groups, who have been our supporters there . . . and we’ve been to Texas quite a few times, but it’s been difficult.


Despite CCA’s challenges at the state policy level, this organization has been more successful working directly with the coordinating board—which, as previously mentioned, has limited formal and informal authority—and with business groups, as described in the previous quote. The THECB commissioner structured the 2013 State of Higher Education address around CCA’s “Game Changers,” the five college-completion policies for which CCA advocates. Regarding CCA’s relationship with business groups, one university system official remarked:


That’s how they have aligned. Less with legislators, more with Texas Association of Business to get to the legislators. . . . I think probably the—as you’ve noted, the lack of success [in influencing state-level policy agendas] probably has to do more with the—the vastness of the state, its regions, its economy, the whole host of issues. . . . It gets very complicating from a political perspective to get them all on the—some of the same page.


Indeed, the Texas Association of Business (TAB) has hosted CCA representatives to present at their higher education convenings. One CCA official confirmed that they will pursue a more “regional approach,” including working with systems and campuses, considering the challenges they have faced at the state policy level.


In addition to their work with the TAB and the coordinating board, CCA and other national intermediaries have engaged directly with some campuses and multi-campus systems, for example, by providing technical support to implement college-completion initiatives. One respondent described her perception that college-completion policy activity in Texas is less centralized than in Tennessee, another state with which she is familiar. Indeed, we found considerable evidence of local college-completion efforts and substantial intermediary influence at the localized level. CCA, for instance, has engaged with individual institutions, including the University of Texas Rio Grande Valley, and with multi-campus systems. For example, CCA is collaborating with the University of Houston System on an initiative called Guided Pathways to Success. Beyond CCA, respondents identified other national intermediaries active at the campus and system levels. The Texas State University System has worked closely with both the Education Advisory Board and Degree Compass, national groups that provide technical assistance on college-completion initiatives. In addition, community college districts in the state have collaborated with Achieving the Dream, also a national intermediary group, and the New Mathways Project, a Texas-based group with a national presence.


Not all multi-campus systems engaged with intermediaries, however. A senior official at a flagship university system described the key players in college completion, in his view:


In Texas, I mean it’s—to be honest with you, it’s going to be the coordinating board . . . and every system by itself. . . . We read about college completion and all that but you know, I haven’t worked with any [intermediaries] and we’ve been at it for a while . . .  working on that . . .  agenda . . .  [W]e [at the system] created an accountability program a long time ago . . .  where we started talking about four-year graduation rates and how do we . . . get them up to a much better rate than what we have and . . . how do we make use of community colleges and all that so that we increase the production of graduates? But we were not driven by, you know, sort of the intermediary organizations at all.


By noting that they “were not driven by” intermediaries, this official seems to be referring to intermediaries’ lack of influence over agenda setting specifically. A representative from the other flagship university system in Texas only mentioned one national intermediary, the Gardner Institute, and the Lumina and Gates Foundations, but could not speak to their impact. In light of these findings, it is conceivable that intermediaries are less influential with flagship institutions, perhaps because flagships’ internal capacity is more robust than other institutions’ given their higher levels of fiscal resources.


THEME 2: “PRO-BUSINESS, PRO-TEXAS”


Our second theme captures the strong business ethos in Texas—reflected in the slogan on the TAB’s business cards (Figure 1), “Pro-Business, Pro-Texas”—and, specifically, its influence over college-completion policy. First, the composition of the Higher Education Strategic Planning Committee demonstrates the central role of the business community in higher education policymaking in Texas. This committee, charged with replacing the state’s higher education strategic plan, was chaired by Woody Hunt, CEO of a real estate conglomerate.


Figure 1.

[39_22490.htm_g/00002.jpg]


The response of one legislator, when asked to identify the major players in this policy area, also illustrates the prevalent role played by the business community in college-completion policymaking in Texas: “[The] Texas Higher [Education] Coordinating Board, obviously. Us, because we’re the appropriating group. The business community in the State of Texas. Essentially, we’re producing a product for them, it could be said. They certainly think so.”


As further evidence of the business community’s role in higher education policy activity in Texas, the TAB has hosted annual higher education conferences to discuss college completion and productivity in higher education. In addition, on their website, “College Completion” appeared as one of 11 “Advocacy” issues. Others include Telemedicine, Title Insurance Reform, and the Trans-Pacific Partnership. Explaining the TAB’s rationale for engaging heavily in higher education policy activity, the organization’s president said, “Workforce is why we’re in the game. I mean, because it—it also happens to be the right thing to do.” Likewise, in 2014, leaders from the TAB went on a “jobs agenda” tour to more than 13 cities across the state, where they met with leaders of chambers of commerce. On the tour, they discussed the TAB’s agenda, which includes higher education proposals, such as performance funding for higher education.


In addition to business groups’ engagement in higher education policymaking, a strong business ethos influenced college-completion policymaking, including the supply and demand for information around this policy issue. On the supply side, at a legislative hearing we attended, Mr. Hunt commented on data comparing Texas’s educational attainment rates to other countries’, “I just translate those to dollars.” Specifically, he observed:


We’re much closer to Mexico than we are to Canada as far as educational attainment of our younger population, the 25- to 34-year-olds, and once again what I see there is income. In other words, when you measure our 25- to 34-year-olds against Massachusetts or Canada and you look at a gap that’s over 20%, I just translate those to dollars. . . . So that’s our challenge as a state, at least in my view, and why I was willing to participate and provide leadership to this committee. . . .


On the demand side, an official at the coordinating board noted that most of the information that policymakers or campus officials have requested from the coordinating board are either about dual credit or workforce outcomes, adding that, “education is an economic factor.” In response to decision-makers’ preference for these types of data, in the 2015 version of the Almanac—a document produced by the coordinating board that contains a plethora of information related to higher education—the coordinating board added a new section on graduates’ workforce outcomes. As another example, at the last few Strategic Planning Committee meetings, presenters referenced the connection between education and the economy. The presenters included representatives from the Georgetown Center for Education and the Workforce and the Organisation for Economic Co-operation and Development (OECD).


THEME 3: ANALYTIC EVIDENCE ON THE SIDELINES


The third theme captures the relative absence of research evidence in college-completion policymaking and policymakers’ attitudes toward research. Related to this finding, we present evidence regarding legislators’ general preferences for information that is concise and relevant to their agendas. This third theme also gleans differences between legislators on one hand and their staffers and coordinating board officials on the other in uses of information, with staff more likely to use information conceptually.


As noted in the first theme, policymakers preferred descriptive information, especially Texas-based data, to other types of information, including research evidence, as a guide to college-completion policymaking. Research evidence, for its part, was largely absent from college-completion policy deliberations in state-level policymaking. The only exceptions were references to reports by the Community College Research Center in Columbia University and by the Georgetown Center on Education and the Workforce.


When we asked one journalist who covered higher education policy about whether research played a role in college-completion discussions, he replied:


. . . not particularly, as far as I can tell. I mean, I’m sure the presidents of the universities and the—well, the chancellors are not necessarily academics. They’re mostly politicians. But the presidents, I’m sure, can have ways of making their feelings about [completion] known. But my impression is that we don’t see a lot of like—you’ll see professors give presentations at Coordinating Board meetings, but not at legislative hearings and you won’t see them cited in a lot of things. . . . But they also might be—you know, I might just be zoning out and not hearing it because they’re so dull that I’m missing them. [laughs]


In addition to the “dull” way in which it is presented, one possible reason for the dearth of research evidence in college-completion policymaking is policymakers’ preference for concise and relevant information. Exemplifying this preference, one legislator observed:


You can give me all the research you’ve done since you’ve gotten your doctorate and it probably is this thick [holds fingers two inches apart]. . . . I’m actually going to read the executive summary, which is this thick [holds fingers two millimeters apart]. . . . And I’m going to sit down with [the policy analyst] and say, what does this mean related to our legislative initiative? . . . It’s not meant to imply that people like me are either not diligent or intellectually capable, but we only have so much time and that research is your job, not mine. It’s my job to use it for some purpose in the milieu of public policy.


In addition to illuminating a preference for user-friendly information, this quote indicates the policymaker’s interest in examining evidence that relates to an existing “legislative initiative,” rather than information on an exploratory topic. Indeed, due to the time and resource constraints to which this legislator alludes, policymakers were less likely to use information conceptually than their staff members were. The three legislative aides interviewed all cited examples of conceptual use of evidence. One legislative aide observed that he looks at academic work but does not share it with elected officials for similar reasons to those cited by the legislator:


White papers probably are more utilized than the traditional academic articles. You know, I look at academic articles. I look at things written by faculty members, but it’s hard to translate that. It’s not hard—you can’t just give that to a member. You know, if I gave a member an R squared, that’s something that . . . would need to be translated. I mean, we can talk about the predictive nature of X . . . but I think, you know, anything beyond “this is predictive of college success” can get a little in the weeds.


In contrast to the legislator who preferred information to inform a contemporary issue, staff members provided numerous examples of conceptual uses of evidence (i.e., for enlightenment, rather than for direct application). Illustrating conceptual use of information, one legislative aide described that in his office, staff “don’t like to limit—at least initially—the person who gives us the information. I mean our—our job—I often say to people, too much information is my problem, not yours.” Describing what they do with information once they receive it, he noted, “I will review it and/or staff will review it, and then kind of, you know, put it in our black box in our files, so that we can take a look at it later.” This respondent contrasted his role as a policy analyst on one of the higher education committees to that of staff who have not been assigned to these committees, suggesting that legislative staff members who are not on higher education committees may not be as diligent about learning about higher education issues. He noted his sense of “. . . duty to be the most informed people in the . . . building with regard to higher education . . . that is our job. . . . People call us when they have questions and we should have the answers.”


Like legislative staff members, coordinating board officials also indicated familiarity with “best practices” reports by national intermediaries, including the Lumina Foundation and College Board. Further depicting conceptual uses of information, one coordinating board official claimed, “Most of the Texas stakeholders, at least in the [higher education] agency, are very well plugged in to what the national literature is saying.”


THEME 4: “SHAMING” AS A TACTIC FOR SUPPLYING INFORMATION


Our final theme from this analysis identifies a distinctive tactic for supplying information related to college completion: shaming public higher education institutions or states into improving completion rates. One particularly striking example, to which participants referred on multiple occasions, involved billboards. Specifically, the TAB, the most prominent business lobbying group in the state, erected billboards off major highways in Texas. One such billboard appeared near Austin Community College’s (ACC) main campus in 2011. The sign (presented in Figure 2) comprised pure text (no graphics), and the message was simple: “4% OF ACC STUDENTS GRADUATE IN 3 YEARS. IS THAT A GOOD USE OF TAX $? TX ASSOCIATION OF BUSINESS.” The president of the TAB, Bill Hammond, proudly noted in one of the interviews that those were the best $300 his organization had ever spent.


Figure 2.

[39_22490.htm_g/00004.jpg]


In response to the question of why the billboards were so impactful relative to other types of information (e.g., research studies or reports released by Lumina or other national intermediaries), Hammond credited the “shocking” nature of the data and the resulting press coverage for the billboards’ effectiveness in garnering attention:


Because [legislators] don’t read [studies or reports], and none of that stuff made the newspapers . . . you’ve got to create a fight or something . . . you’ve got to say something shocking, unfortunately, like we did with ACC. I mean, 4%. You’re just trying to . . . get the public going and get the establishment upset. . . . I mean, the billboard was a kind of radical idea—you know, became a discussion point.


The billboard example also illustrates an intermediary’s preferred method for supplying information: blunt messaging. As another example of employing blunt messaging with the intention of drawing attention to inadequate completion rates, CCA staff members title many of the organization’s reports using bold statements such as “Remediation: Higher Education’s Bridge to Nowhere”; “Four-Year Myth,” which reports on postsecondary students’ time to degree completion; and “Time Is the Enemy of Graduation.” CCA also displays, using infographics, completion statistics, particularly “on-time” graduation rates that as presented, would appear strikingly low. As an example, a Texas-specific report illustrates in a pie chart that 2% of all students complete their one- to two-year certificates on time. One CCA official reflected on this blunt-messaging approach, noting, “I think that in our advocacy mission . . . I think that we are overzealous or over simplifying sometimes.” The official continued by speculating that if they were “less inflammatory and more nuanced,” they would not “get the same attraction from people.”


These uses of data on graduation rates constitute examples of political uses of information. When asked what he liked about CCA’s data, the TAB’s president responded, “Because it validates what we all know. . . . [Institutions] are doing a lousy job.” As such, he was confirming his preference for data that bolsters his preconceived notions for policy foci.


In addition to the famous TAB billboards and to CCA, a prominent and potentially impactful source of information relating to college completion is the THECB, and, in particular, its yearly “Almanacs” (glossy 8 ½ in.-by-14 in. publications containing data for each postsecondary institution in the state). These reports are distributed to members of the legislature, trustees, businesspersons, executive branch officials, and others. The tone of these documents is more neutral. The message is not as “punchy” as that provided by the billboards or by CCA, and we found less evidence of “shaming” in the documents themselves. However, in an interview, a coordinating board official noted that the Almanac “puts pressure on institutions to do better.” This comment is consistent with other impressions of the role of data as a mechanism for shedding light on what some observers perceive as poor outcomes. Thus, the analysis revealed that intermediaries and stakeholders in Texas have employed a distinctively blunt method of disseminating information coupled with a shaming tactic to highlight poor college performance. In doing so, they have aimed to raise public awareness, motivate institutions to pursue improvements, and encourage policymakers to pressure institutions to do better.


The analysis also revealed a potential drawback to the punchy messaging technique coupled with a shaming strategy—higher education institution officials’ hesitation to provide information or data on their institutions. Some campus officials expressed reservations with sharing information on their institutions’ performance, given the potential uses of this information to attract criticism. One higher education official at a four-year regional university noted that, despite her support for data analysis and for efforts to improve her institution (and others), she has worried about what the press or state leaders might do with the data. Triangulating this finding, the leader of a state-level intermediary acknowledged some higher education leaders’ hesitation to provide data. Reflecting on a recent experience, he recalled:


I just was at a thing in the [Rio Grande] Valley and . . . I’ve never seen this before—[a university official] voluntarily said . . . we’ve worked really hard in the last six years and our graduation rate has gone, you know, from 35 to 46. I mean, it still wasn’t kicking it . . . but they were honest, and I actually called [the college’s president] out, because . . . I’ve never been at a meeting where . . . voluntarily . . . a college or university president has shared [graduation rates].


DISCUSSION AND IMPLICATIONS


At a time when college-completion policies continue to be ubiquitous on state-level higher education policy agendas, this in-depth case study shed light on the policy activity surrounding this issue in one state. In particular, this study examined Texas, which has a highly professional legislature and a long history with college-completion policy, both features that enhanced opportunities for examining use of evidence in college-completion policymaking.


Many of our findings are unique to Texas, given its distinct demographic, college-completion policy, and structural characteristics. The plethora of data collected by the THECB is rare across the states, but results in significant reliance on the coordinating board for information related to college completion. Indeed, overwhelmingly higher education policy actors tend to prefer Texas-based data, particularly that provided by the coordinating board. Respondents cited three major reasons for this preference: the high quality of the THECB’s data, Texas’s unique demographics, and the accessibility of the THECB’s data.


Also noteworthy is the perception among policy actors of Texas’s uniqueness. Like Texas, other states are becoming increasingly aware of their unique state characteristics, leading to a type of “policy filtering” (Rubin & Hearn, 2018). That is, Texas may not be alone in its scrutiny of cross-state comparisons. The widespread belief of the incomparability of demographics in Texas to those in other states, coupled with the critical role of the THECB in supplying college-completion information, appear to have limited external (out-of-state) influence over college-completion policymaking at the state level, especially in agenda setting. Indeed, external intermediaries played a largely supportive role, and a rather localized one, providing technical assistance as these initiatives move forward.


Turning to the information supplied by these external groups, policy actors in Texas used it but emphasized the limited value of cross-state comparisons given Texas’s unique demographics. These findings point to the importance of taking into account the context of the decision-making level (in this case, states) in examining the role of intermediary organizations and “new governance” arrangements, which recent studies have found to be significant and increasing (Ball, 2008). This finding also highlights the mediating role that is played not only by state structural characteristics, but also by culture. Perceptions of Texas’s distinctive inward-looking nature permeated our interviews and set the stage for the role that intermediaries play in the state and the preferences for information.


In addition to the policy-filtering phenomenon (Rubin & Hearn, 2018), several findings related to the use of information in the college-completion policymaking process have significance beyond this state. First, the role of the business community in college-completion policy in Texas is consistent with previous findings of the role of business in higher education policymaking, particularly around performance funding (Dougherty, Natow, Bork, Jones, & Vega, 2013). Given their prominence in supplying information for college-completion policy in Texas, future research should explore the unique role played by these organizations in higher education policy activity across states.


Second, consistent with previous literature (Lubienski, Scott, & Debray, 2011; Scott et al., 2014), systematic research evidence is far from the forefront of policymaking in Texas. It is important to note that relative to other education policy areas, such as charter schools or voucher programs, the literature base on college-completion policies is nascent. However, increasingly, higher education scholars, including ones working for research-based intermediaries (e.g., the Community College Research Center) and scholars who are not affiliated with research institutes, have begun to evaluate the effectiveness of certain college-completion policies, including remedial education approaches (e.g., Logue & Watanabe-Rose, 2014).


The work produced by the Community College Research Center and Georgetown’s Center on Education and the Workforce were the only research reports referenced in Texas in our data. Indeed, legislators noted their preference for concise and relevant information, which these organizations provide. In addition, these groups make themselves accessible to policymakers, which, as previous research has shown, matters greatly in enhancing the use of evidence in policymaking (Nutley, Walter, & Davies, 2007). Those intending to influence policymaking should consider making information, especially research evidence, concise and easily accessible and establish relationships with policymakers and their staff members.


Although research evidence was present and utilized, local information and localized initiatives seem to be most welcomed and effective. Intermediaries wishing to inform college-completion policy activity at the state level should consider the uniqueness of the state context in supplying information. For states that are more insular, like Texas, working through in-state intermediaries, such as the TAB, may be an effective strategy.


Our analysis also revealed that legislative staff members and coordinating board officials most commonly used information conceptually, which aligns with what previous research has found (Amara, Ouimet, & Landry, 2004; Coburn, 2005; Weiss, 1977). While researchers view instrumental use of information as somewhat of a gold standard, as Nutley and colleagues (2007) have noted, political use of evidence, particularly high-quality evidence, can promote sound policymaking. For instance, policymakers may reference data on workforce needs or low completion rates to support their argument for focusing on college completion; by visiting these sources, they become exposed to other important information (e.g., the need to focus on remedial education and the gap in workforce outcomes by racial/ethnic and socioeconomic groups) that can point to specific solutions.


Finally, we found that some intermediaries use a “shaming” tone with the intent to “pressure institutions.” The TAB and CCA’s blunt messaging approach appeared to be effective at garnering attention from policymakers and the public. Indeed, the TAB’s billboards featured prominently in our interviews, and many respondents were familiar with CCA’s reports. Despite this success at bringing attention to this issue, one possible unintended consequence of punchy messaging is campus representatives’ hesitation to share data, incited by a fear of becoming targets of “shaming.” In addition, punchy messaging runs the risk of omitting important nuances in evidence. For example, the graduation rates presented on the billboards, while striking to those who are unfamiliar with graduation rates across states, masks the challenges faced by institutions, particularly those that serve the students with highest needs, including community colleges. In addition, placing the onus entirely on institutions by “shaming” them can lead to policy initiatives that punish institutions that serve students with lower likelihoods of completing their degrees.


In conclusion, an overt recognition by policy actors of the preference for brief and simple information suggests that higher education stakeholders wishing to bring attention to this issue might consider simple messaging as a first step. Caution should be taken, however, to avoid shaming institutions and to forego nuances excessively, in light of potential unintended consequences.


References


Amara, N., Ouimet, M., & Landry, R. (2004). New evidence on instrumental, conceptual, and symbolic utilization of university research in government agencies. Science Communication, 26(1), 75–106.


Ball, S. (2008). New philanthropy, new networks and new governance in education. Political Studies, 56, 747–765.


Ball, S., & Junemann, C. (2012). Networks, new governance, and education. Chicago, IL: The Policy Press.


Birnbaum, R. (2000). Policy scholars are from Venus; policy makers are from Mars. Review of Higher Education, 23(2), 119–132.


Caplan, N. (1979). The two-communities theory and knowledge utilization. American Behavioral Scientist, 22(3), 459–470.


Coburn, C. E. (2005). Shaping teacher sensemaking: School leaders and the enactment of reading policy. Educational Policy, 19, 476–509.


Coburn, C. E., Toure, J., & Yamashita, M. (2009). Evidence, interpretation, and persuasion: Instructional decision making at the district central office. Teachers College Record, 111(4), 1115–1161.


College Board. (2010). The college completion agenda state policy guide. New York City, NY: The College Board. Retrieved from http://www.ncsl.org/documents/educ/policyguide_062810sm.pdf


Conaway, C., Keesler, V., & Schwartz, N. (2015). What research do state education agencies really need? The promise and limitations of state longitudinal data systems. Educational Evaluation & Policy Analysis, 37(1), 16S–28S.


DeBray, E., Scott, J., Lubienski, C., & Jabbar, H. (2014). Intermediary organizations in charter school policy coalitions: Evidence from New Orleans. Educational Policy, 28(2), 175–206.


Dougherty, K. J., Natow, R. S., Bork, R. H., Jones, S. M., & Vega, B. E. (2013). Accounting for higher education accountability: Political origins of state performance funding for higher education. Teachers College Record, 115(1), 1–50.


Dougherty, K. J., & Reddy, V. (2013). Performance funding for higher education: What are the mechanisms? What are the impacts? ASHE Higher Education Report, 39(2).


Dunn, W. N. (1980). The two-communities metaphor and models of knowledge use. Knowledge: Creation, Diffusion, Utilization, 1, 515–536.


Finnigan, K. S., & Daly, A. J. (2014). Using research evidence in education: From the Schoolhouse Door to Capitol Hill. New York, NY: Springer International Publishing.


Flores, S. M., & Shepherd, J. C. (2014). Pricing out the disadvantaged? The effect of tuition deregulation in Texas public four-year institutions. Annals of the American Academy of Political and Social Science, 655, 99–122.


Gándara, D., Rippner, J. A., & Ness, E. C. (2017). Exploring the ‘how’ in policy diffusion: National intermediary organizations’ roles in facilitating the spread of performance-based funding policies in the states. The Journal of Higher Education, 88(5), 701–725.


Gray, V., & Lowery, D. (2000). Where do policy ideas come from? A study of Minnesota legislators and staffers. Journal of Public Administration Research and Theory, 10, 573–597.


Guston, D. H., Jones, M., & Branscomb, L. M. (1997). The demand for and supply of technical information and analysis in state legislatures. Policy Studies Journal, 25(3), 451–469.


Hamann, E. T., & Lane, B. (2004). The roles of state departments of education as policy intermediaries: Two cases. Educational Policy, 18(3), 426–455.


Hamilton, R., & Ramshaw, E. (2013, May 16). Coordinating board nears end of tough Sunset process. Texas Tribune. Retrieved from https://www.texastribune.org/2013/05/16/coordinating-board-nears-end-tough-sunset-process/


Hearn, J. C. (1997). Research on higher education in a mass and diversified system: The case of the United States. In J. Sadlak & P. Altbach (Eds.), Higher education research at the turn of the century: Structures, issues, and trends (pp. 271–319). New York, NY: Garland Publishing, Inc.


Hird, J. A. (2006). Power, knowledge, and politics: Policy analysis in the states. Washington, DC: Georgetown University Press.


Honig, M. I. (2004). The new middle management: Intermediary organizations in education policy implementation. Educational Evaluation and Policy Analysis, 26(1), 65–87.


Jabbar, H., Lalonde, P., DeBray, E., Scott, J., & Lubienski, C. (2014). How policymakers define “evidence”: The politics of research use in New Orleans. Policy Futures in Education 12(8), 1013–1027.


Jillson, C. (2014). Texas politics: Governing the Lone Star State. New York, NY: Routledge.


Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills, CA: SAGE.


Logue, A. W., & Watanabe-Rose, M. (2014). Mainstreaming remedial mathematics students in introductory statistics: Results using a randomized controlled trial, presented at the Society for Research on Educational Effectiveness Fall 2014 Conference, Washington, DC, 2014.


Lubienski, C., Scott, J., & DeBray, E. (2011). The rise of IOs in knowledge production, advocacy, and educational policy [Commentary]. Teachers College Record.


Lumina Foundation. (2013). Strategic plan 2013 to 2016. Indianapolis, IN: Author. Retrieved from https://www.luminafoundation.org/files/resources/2013-lumina-strategic-plan.pdf


Malandra, G. H. (2012). Efforts to improve productivity: The impact of higher education reform in Texas. In A. P. Kelly & M. Schneider (Eds.), Getting to graduation: The completion agenda in higher education (pp. 246–268). Baltimore, MD: Johns Hopkins University Press.


McKeownMoak, M. P. (1999). Higher education funding formulas. New Directions for Higher Education, 1999(107), 99–107.


Merriam, S. B. (2009). Qualitative research: A guide to design and interpretation. San Francisco, CA: Jossey-Bass.


Mooney, C. Z. (1991). Information sources in state legislative decision making. Legislative Studies Quarterly, 16, 445–455.


Murdock, S. H., Cline, M. E., Zey, M., Jeanty, P. W., & Perez, D. (2014). Changing Texas: Implications of addressing or ignoring the Texas challenge. College Station, TX: Texas A&M University Press.


National Center for Education Statistics. (2014). Digest of Education Statistics. Washington, DC: U.S. Department of Education, Institute of Education Sciences.


National Governors Association. (2010). Complete to compete. Washington, DC: National Governors Association Center for Best Practices. Retrieved from https://www.nga.org/files/live/sites/NGA/files/pdf/1011COMMONTECHGUIDE.PDF  


Ness, E. C. (2010). The role of information in the policy process: Implications for the examination of research utilization in higher education policy. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. XXV; pp. 1–49). New York, NY: Springer.


Ness, E. C., Tandberg, D. A., & McLendon, M. K. (2015). Interest groups and state policy for higher education: New conceptual understandings and future research directions. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (pp. 151186). New York, NY: Springer.


Nutley, S. M., Walter, I., & Davies, H. T. (2007). Using evidence: How research can inform public services. Bristol, UK: Policy Press.


Patton, M. Q., & Patton, M. Q. (2002). Qualitative research and evaluation methods. Thousand Oaks, CA: SAGE.


Perna, L. W., Finney, J. E., & Callan, P. M. (2014). Hard choices ahead: The performance and state policies of higher education in Texas. In L. W. Perna & J. E. Finney (Eds.), The attainment agenda (pp. 137–167). Baltimore, MD: The Johns Hopkins University Press.


Reckhow, S. (2012). Follow the money: How foundation dollars change public school politics. Oxford University Press.


Reckhow, S., & Snyder, J. W. (2014). The expanding role of philanthropy in education politics. Educational Researcher, 43(4), 186–195.


Rubin, P. G., & Hearn, J. C. (2016, November). The policy filtering process: Understanding distinctive state responses to the national college completion agenda. Paper presented at the annual meeting of the Association for the Study of Higher Education, Columbus, OH.


Rubin, H. J., & Rubin, I. S. (1995). Qualitative interviewing: The art of hearing data. Thousand Oaks, CA: SAGE.


Rutherford, A., & Rabovsky, T. (2014). Evaluating impacts of performance funding policies on student outcomes in higher education. Annals of the American Academy of Political and Social Science, 655, 185–208.


Scott, J., Jabbar, H., LaLonde, P., DeBray, E., & Lubienski, C. (2015). Evidence use and advocacy coalitions: Intermediary organizations and philanthropies in Denver, Colorado. Education Policy Analysis Archives, 23(124), n124.


Scott, J., Lubienski, C., DeBray, E., & Jabbar, H. (2014). The intermediary function in evidence production, promotion, and utilization: The case of educational incentives. In K. Finnigan & A. Daly (Eds.), Using research evidence in education: From the schoolhouse door to Capitol Hill (pp. 69–89). New York, NY: Springer.


Shakespeare, C. (2008). Uncovering information’s role in the state higher education policymaking process. Educational Policy, 22(6), 875–899.


Snow, C. P. (1961). Science and government. Cambridge, MA: Harvard University Press.


Snyder, M. (2015, February 12). Driving better outcomes: Typology and principles to inform outcomes-based funding models. Retrieved on August 19, 2015, from http://hcmstrategists.com/drivingoutcomes/wp-content/themes/hcm/pdf/Driving Outcomes.pdf


Squire, P. (2007). Measuring state legislative professionalism: The Squire index revisited. State Politics & Policy Quarterly, 7(2), 211–227.


Strauss, A. L., & Corbin (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory. Los Angeles, CA: SAGE.


Tandberg, D., & Hillman, N. (2014). State higher education performance funding: Data, outcomes and policy implications. Journal of Education Finance, 39(1), 222–243.


Texas Higher Education Coordinating Board. (2015, April 20). 60x30TX: 60 percent of “Generation Texas” will have a postsecondary credential or degree by 2030. Retrieved on May 2, 2014, from http://content.govdelivery.com/attachments/TXHECB/2015/04/30/file_attachments/386255/TX_HESP_Draft_APR_2015v4.pdf

Webber, D. J. (1987). Legislators’ use of policy information. American Behavioral Scientist, 30, 612–631.


Weiss, C. (1977). Research for policy’s sake: The enlightenment function of social research. Policy Analysis, 3, 531–546.


Weiss, C. (1979). The many meanings of research utilization. Public Administration Review, 39, 426–431.


Yin, R. K. (2009). Case study research: Design and methods. Los Angeles, CA: SAGE.



APPENDIX A


Sample Interview Protocols


Interview Protocol – Intermediary Officials


1.

To start off, could you please provide an overview of [organization’s] major college completion initiatives?

a.

What are some specific policy proposals you pursued?

b.

What strategies did you employ to reach your college completion-policy related goals and do you feel like these were effective?


2.

From your perspective, what role, if any, does [organization] play in influencing college completion policy discussions and decisions in Texas?  

a.

Networking (connecting to other states or organizations)

b.

Legitimization (especially with governor and legislature)

c.

Expertise (providing information)


3.

Can you think of any ways in which [organization] influenced college completion policies in Texas?


4.

Who were the key people you targeted to inform or influence college completion policies (these could be elected officials, their aides, higher education agency officials, governor’s staff members)?

a.

When [specific state actors] reached out to you regarding college completion- related information, what was typically the nature of the discussions and/or requests?  


5.

In general, to what extent would you say research evidence that might be used to inform college completion policy is produced in your office?

a.

What type of information that might inform college completion do you produce?

i.

In what format (e.g., white papers, talking points, or research articles)?

ii.

What type (e.g., best practices or local descriptive information like district-level data or data specific to a college or university)?

b.

How do you distribute or promote the evidence that you produce?

i.

By whom is it used?

ii.

Do you ever distribute or promote information from other sources?

1.

What type of information do you distribute?

2.

How do you distribute or promote it?

iii.

Of the information you distribute, how much of it would you say is produced by your organization?

c.

Do you ever receive requests for evidence that might be used to inform college completion policy?

d.

How is the evidence you produce used?

i.

To inform a specific policy decision? (Instrumental)

ii.

To provide broad support for a policy approach? (Conceptual)

iii.

To support already determined policy preferences? (Political/Tactical)

e.

Would you say that [organization] is an important resource for information or evidence for Texas?

i.

How would you characterize your organization as a source of college-completion information relative to other organizations or potential sources of information?


6.

In general, to what extent would you say evidence is used in your office to inform college completion policy recommendations?

a.

What type of information do you use?

i.

In what format (e.g., white papers, talking points, or research articles)?

ii.

What type (e.g., best practices or local descriptive information like district-level data or data specific to a college or university)?

iii.

From what source (e.g., your office, state agency, regional organization, national organization, university, trade journal)?


7.

Can you think of any other individuals or organizations that might provide information relating to college completion?


8.

Do you have any additional thoughts on the role of intermediary organizations in the use of evidence to inform college completion policies in Texas?

a.

Individuals to interview

b.

Events to observe



Interview Protocol – Texas Elected Officials and Staff


1.

To start off, could you please provide a brief overview of college completion-related activity in Texas?


2.

In your view, what are the major influences on college-completion policy activity in Texas?


3.

In general, to what extent would you say research evidence, broadly defined, or other forms of information are used in your office to inform college-completion policy discussions, decisions, or recommendations?

a.

What type of information do you typically use (e.g., best practices evidence, legislative district-level data, system- or campus-level data, or others)?

b.

What are your most common sources for information?

c.

Do you prefer information in a certain format?  How and to what ends do you use research evidence or other types of information?

d.

Can you think of any examples of ways in which you’ve used research evidence?


4.

In your view, do certain state actors, including elected officials, influence the type of college completion information that you acquire or use?

a.

Do you think the professional staff’s capacity plays a role in the acquisition or use of college completion-related information?

b.

Can you think of any other state agencies or organizations (e.g., the university system offices) that play a role in the type of college-completion information that you acquire or use?


5.

From your perspective, what role, if any, do [intermediaries] play in influencing college-completion policy discussions and decisions in Texas?


6.

Would you say that [intermediaries] are important resources for information or research evidence?

a.

Do [intermediaries] ever provide information to you?

b.

How would you characterize the information that is most often provided to you by [intermediaries] (e.g., in terms of length, target audience, whether it is evidence-based)?

c.

When research evidence or information is provided to you, what do you do with it?

i.

What type of information do you prefer or are you more likely to use to inform college-completion policies?

ii.

At what stage in the policy process do you use information provided by [intermediaries]?

d.

Do you ever request information from [intermediaries]?

i.

What kind of information have you requested most frequently in the past?  

ii.

Typically, how would you go about requesting information (e.g., phone call, e-mail, in person, Internet search)?

a.

How do you use the research evidence or other forms of information?

b.

Can you think of any examples of ways in which you’ve used evidence?


1.

Can you think of any other individuals or organizations that might provide information relating to college completion?


2.

Do you have any additional thoughts on the role of intermediary organizations in the use of research evidence to inform college-completion policies in Texas?  

a.

Individuals to interview

b.

Events to observe


 

APPENDIX B

Coding Scheme


Descriptive Categorizations

State/governmental key players

In-state intermediaries

Executive

Business organization

Legislative

State-level foundation

Other state actors

Think tank

Campuses

 

Texas Higher Education Coordinating Board

College completion policies

UT Austin System

Outcomes/performance-based funding

Texas A&M University System

Remedial/developmental education

Texas State Technical College System

Guided/structured pathways

 

Articulation/transfer

Other states

Financial aid/affordability

 

Predictive analytics/advising

Researchers

Committed to excellence

 

College readiness

Press

Enhancing data systems

 

Competency-based education

National/regional intermediaries

Technical credentials

CCA

Adult students

SREB

Full time is 15/increase course load

Other intermediaries

 

NCHEMS

 

SHEEO

 

WICHE

 

Georgetown CEW

 

Lumina

 

Gates

 

Charles A. Dana Center/New Mathways Project

 

Achieving the Dream/ATD

 

Jobs for the Future/JFF

 

Community College Research Center/CCRC

 

National Conference of State Legislatures/NCSL

 

RAND Corporation

 

HCM/Nate Johnson

 



Conceptual Categorizations

Role played by intermediary

Supply

Information provider

Production (if original source)

Raise public awareness

Targets of distribution

Networking

Preferred kinds of information to supply

None

Redistribution (if not original source)

Legitimization

 

Funding

State characteristics

Advocate/policy promoter

Higher education governance

Technical assistance

Legislative professionalism

 

Party control/partisanship

Stage in policy process

Alignment between K–12 and higher education

 

Fiscal situation of state/funding for higher education

Demand

Educational attainment levels of state

Request/acquisition of information

Governor’s power

Use (from Weiss typology)

State demographics

Non-use

Campus demographics

Conceptual/enlightenment

Workforce/economic development

Instrumental

Research capacity

Political/tactical

State uniqueness

Significance of information

 

Preferred types/format of information

 

State data

 

Campus data

 

State comparisons

 

Academic research

 

Policy reports/best practices

 

Other

 

Preferred sources

 

Quantity

 

Absence of information

 

Quality

 




APPENDIX C


Coding Illustration


Note. Codes are as follows: Demand = D [preferred sources = Dps; preferred types = Dpt (academic research = Dptar; policy report = Dptpr); use = Du (non-use = Dunu)]; Other States = OS; State Characteristics = SC (state demographics = SCsd; state uniqueness = SCsu); State/Governmental Key Players = G (legislative = Gl)



Example 1 (Intermediary Official)

Interviewer: I wanted to ask about to what extent you view Texas as being different or not from some of the other states in terms of their use of information to inform college-completion policies, or their reliance on research, or even their data capacity within the state relative to other states.


Participant: We don’t really look at other states and try to copy them OS/Dunu, and I don’t think that’s really much of a pride issue, I think it’s because Texas is so diverse SCsd/SCsu. I mean, you have—you have urban, you have, you know, growing populations in certain cities, the valley where you have a huge region that is 80, 90% Latino . . . and so that’s really like—that’s what the nation’s going to look like, and so we’re kind of like, on the cusp of what will be the reality as a nation, years to come SCsd, and so who can we look to? We look to ourselves and look to the people within those regions that are smart, that have—have a sensitivity to the population and know it best Dps. That being said, yes, we can learn a lot from similar states like Florida and California, you know, perhaps New Mexico and maybe Colorado Dps/OS in terms of its Hispanic population SCsd, but we are—we are such a large state that, I mean, we can go from El Paso, and that’s very different from what’s going on in Houston, and then the valley SCsd/SCsu.


Example 2 (State Legislator)

Interviewer: Could you say a word about our—our theme of research evidence or data and what role those have played in driving the completion agenda and policymaking here?


Participant: I think they play a role Dptar, but you have to careful about burying people like me in data and research. That’s not my job Gl/Dpt. I need to know what the question is and—and what the problem is it’s meant to address Dpt. You know, it’s—my—mine are—mine are policy initiatives, not research Gl. It’s nice to know what it is as a technicality, or something. I just need to know empirically and very clearly and succinctly what the research is, what it’s meant to show Dptar, and then if anybody has any insights on paths of success or—or achievement, great Dptpr.




Cite This Article as: Teachers College Record Volume 121 Number 1, 2019, p. 1-40
https://www.tcrecord.org ID Number: 22490, Date Accessed: 11/27/2021 9:50:06 PM

Purchase Reprint Rights for this article or review
 
Article Tools
Related Articles

Related Discussion
 
Post a Comment | Read All

About the Author
  • Denisa Gándara
    Southern Methodist University
    E-mail Author
    DENISA GÁNDARA is an Assistant Professor of Higher Education at Southern Methodist University. Her research focuses on higher education policy and politics. Her work examines: (1) the policymaking process, with a focus on evidence use and representation, and (2) the relationship between higher education policies and equity outcomes for students and institutions. Recent publications include Outcomes Based Funding and Race in Higher Education (Palgrave Macmillan) and “Exploring the ‘How’ in Policy Diffusion: National Intermediary Organizations’ Roles in Facilitating the Spread of Performance-Based Funding Policies in the States” in The Journal of Higher Education.
  • James Hearn
    University of Georgia
    JAMES C. HEARN is a Professor in the Institute of Higher Education at the University of Georgia. His research and teaching focus on postsecondary education policy, finance, and organization. In addition to his work on policy development and implementation in state higher education governance, he has recently examined changes in postsecondary education’s faculty workforce and innovation and adaptation in the independent-college sector. Recent publications include “Conceptualizing State Policy Adoption and Diffusion” in Higher Education: Handbook of Theory and Research and “Mission-Driven Innovation: An Empirical Study of Adaptation and Change among Independent Colleges,” a report for the Council of Independent Colleges.
 
Member Center
In Print
This Month's Issue

Submit
EMAIL

Twitter

RSS