Operationalizing Culturally Responsive Instruction: Preliminary Findings of CRIOP Research

by Rebecca Powell, Susan Chambers Cantrell, Victor Malo-Juvera & Pamela Correll - 2016

Background: Many scholars have espoused the use of culturally responsive instruction (CRI) for closing achievement gaps, yet there is a paucity of research supporting its effectiveness. In this article, we share results of a mixed methods study that examined the use of the Culturally Responsive Instruction Observation Protocol (CRIOP) as a framework for teacher professional development. The CRIOP is a comprehensive model and evaluation tool that operationalizes culturally responsive instruction around seven elements: Classroom Relationships, Family Collaboration; Assessment; Curriculum/Planned Experiences; Instruction/Pedagogy; Discourse/Instructional Conversation; and Sociopolitical Consciousness/Diverse Perspectives.

Focus of Study: This study was designed to answer the following questions: (1) Do teachers increase their use of culturally responsive practices as they participate in CRIOP professional development? (2) What is the relationship between implementation of culturally responsive instruction and student achievement in reading and mathematics?, and (3) What are teachers’ perceptions of their successes and challenges in implementing culturally responsive instruction?

Participants: Twenty-seven elementary teachers participated in this study. Of the 27 participants, all were female, 26 were White, and all were native speakers of English. Student achievement data were collected from students enrolled in classrooms of participating teachers at the two schools in the study that administered the Measures of Academic Progress (MAP) test. Of the 456 students who were participants, 397 (87.3%) received free or reduced lunch, and 128 (28% of total sample) were classified as English Language Learners (ELLs).

Intervention: Three training sessions were held before school began and during the fall semester. Additionally, throughout the school year teachers received individual classroom coaching, on-site professional development, and instructional planning support. Participating teachers received an average of 50.4 hours of classroom-based coaching and mentoring during the intervention, which included observations, meetings with individual teachers and teacher teams, curriculum planning sessions, and collaborative creation of individualized action plans. The CRIOP was used as a professional development framework. The intended outcome of on-site support was to increase the incorporation of culturally responsive instruction in teachers’ daily practices, resulting in more culturally responsive classroom relationships, assessment and instructional practices, and use of discourse.

Research Design: This study utilized a concurrent triangulation mixed methods design. Data sources included classroom observations, student achievement results, and postobservation teacher interviews. The CRIOP instrument was used for classroom observations to determine the extent of implementation of culturally responsive practices. Following each classroom observation, field researchers conducted an audio-recorded semistructured interview using the CRIOP Post-Observation Teacher Interview Protocol and The CRIOP Family Collaboration Teacher Interview Protocol. These protocols were designed to elicit additional information that might not have been readily apparent from data gleaned during the observation. In addition, participants were interviewed to determine their perceptions of culturally responsive instruction. Three interview questions and responses were transcribed and coded for analysis: How do you define culturally responsive instruction? What are your biggest successes with using Culturally Responsive Instruction with your students? What are your biggest challenges with using Culturally Responsive Instruction with your students? Integration of quantitative and qualitative data occurred during data collection and interpretation.

Findings: Results of classroom observations showed that teachers had significantly higher levels of CRI implementation in the spring compared to fall. Data on student achievement indicated that students of high implementers of the CRIOP had significantly higher achievement scores in reading and mathematics than students of low implementers. The results of this study also suggest that teachers face several challenges in implementing CRI, including constraints imposed by administrators, high-stakes accountability, language barriers in communicating with families, and the sheer complexity of culturally responsive instruction.

Conclusions/Recommendations: Although numerous scholars have espoused the value of culturally responsive instruction (CRI), there is limited research on its effectiveness. The results of this investigation suggest that the CRIOP shows promise both as a framework for teacher professional development and as an observation instrument in investigations of culturally responsive instruction. Findings also indicate that one of the biggest challenges in implementing CRI is its multidimensionality in that it includes several components (e.g., student relationships, family collaboration, assessment practices, instructional practices, discourse practices, and sociopolitical consciousness), which together comprise the CRIOP model. Future research including an experimental design is needed to determine the effectiveness of the CRIOP as a measure of culturally responsive instruction and as a framework for intervention.

While numerous scholars call for teachers to consider students cultural knowledge in instructional decision making (Gay, 2000; Ladson-Billings, 1994; Lee, 2006; Nieto & Bode, 2008; Sleeter, 2011b) there is a need for classroom research that details instructional practices that are culturally responsive and that measures the effects of culturally responsive instruction (CRI) on student performance. This article documents a mixed methods study from the first year of a multi-year intervention designed to increase elementary teachers implementation of CRI. We present quantitative and qualitative data investigating the effects of intervention on teachers implementation of CRI, the relationship between CRI and student academic performance, and the experiences of teachers during CRI training and implementation.

BACKGROUND

For a number of years, multicultural scholars have advocated culturally responsive instruction (CRI) as a means for raising the achievement of students from historically underrepresented populations. Yet there is a paucity of research on the effects of CRI on achievement outcomes and on the challenges in implementing culturally responsive practices. Sleeter (2011a, p. 16) notes that [a]lthough there is quite a bit of research on culturally responsive pedagogy, far too little systematically documents its impact on student learning, and clarifies what sets of culturally responsive practices most strongly impact on students, and in what contexts.

CRI is also known as culturally relevant pedagogy (Ladson-Billings, 1994), culturally responsive teaching (Gay, 2000), culturally congruent instruction (Au & Kawakami, 1994), and culturally and linguistically responsive teaching and learning (Hollie, 2012). More recently, scholars have suggested that culturally sustaining pedagogy is a more useful term because it reflects the goal of upholding linguistic, literate, and cultural pluralism as part of a democratic agenda (Paris, 2012). In our work, we have chosen to use the term culturally responsive instruction because it is a designation that has wide acceptance and that embraces a broad range of elements related to effective instruction for students from underrepresented groups. CRI typically is conceptualized as instructional practices that connect learning to the cultural knowledge and experiences of students and that draws on students cultural and linguistic strengths and frames of reference in instruction, thereby resulting in higher levels of student achievement. At the same time, CRI validates and affirms students linguistic and cultural knowledge and seeks to empower students and their families by valuing their resources and by helping them to interrogate and act upon real-world issues.

While many educators have recognized culturally responsive instruction as a potentially useful paradigm for educating those students who tend to underachieve in our educational institutions, there has lacked a cohesive representation in the literature of what constitutes CRI, how it is defined, and how one might identify it in classrooms. In this article, we review the theoretical perspectives that have emerged from multicultural and equity education, culturally relevant/responsive pedagogy, and second language acquisition that provide the basis for our work. We argue that CRI is a complex phenomenon that incorporates several major elements, many of which have a relatively strong research base in contributing to student achievement. Absent, however, is a comprehensive model that combines these various elements into a single framework that can be used to assess and analyze CRI in classrooms and to investigate the overall effectiveness of culturally responsive teaching.

This study utilizes a model called the Culturally Responsive Instruction Observation Protocol (CRIOP), which operationalizes CRI using seven holistic elements: (1) Classroom Relationships; (2) Family Collaboration; (3) Assessment; (4) Curriculum/Planned Experiences; (5) Instruction/Pedagogy; (6) Discourse/Instructional Conversation; and (7) Sociopolitical Consciousness/Diverse Perspectives¹ (Powell & Rightmeyer, 2011). Originally developed as an observation tool for assessing culturally responsive teaching practices, the CRIOP has also been used to guide teacher professional development. In this article, we present results of an investigation on the first year of implementation of the CRIOP model in elementary classrooms. Findings indicate that the model shows promise as an effective framework for guiding teachers work in culturally responsive instruction and that high implementation of the CRIOP relates positively to student achievement.

CONCEPTUALIZING CULTURALLY RESPONSIVE INSTRUCTION

Early work in culturally responsive instruction was embedded within the multicultural education movement, which began in the 1970s out of concerns for persistent educational inequalities (e.g., Abrahams & Troike, 1972; Cuban, 1972). One of the pioneers of multicultural education, James Banks, has been instrumental in setting the stage for culturally responsive instruction. Banks (2008) suggests that multicultural education has five dimensions: (1) content integration; (2) knowledge construction; (3) prejudice reduction; (4) equity pedagogy; and (5) empowering school culture and social structure. Content integration refers to the extent to which teachers use information and knowledge from a variety of cultural groups to teach concepts in their particular subjects. Knowledge construction describes the procedures by which knowledge is created and the extent to which teachers help students understand that all knowledge is socially constructed. Prejudice reduction refers to pedagogical practices that intentionally help students develop positive racial and ethnic attitudes. Equity pedagogy uses teaching methods that have been found to facilitate the academic achievement of students from historically underachieving populations. Banks refers to this component as culturally responsive teaching: Research indicates that teachers can increase the classroom participation and academic achievement of students from different cultural and language groups by modifying their instruction so that it draws upon their cultural strengths (2008, p. 35). Finally, an empowering school culture and social structure refers to making changes within the school environment so that high expectations for every student are the norm and all students can experience success.

Other scholars have a broader concept of culturally responsive instructionone that encompasses several of Banks dimensions. Gloria Ladson-Billings (1994), in reporting her findings of the characteristics of highly effective teachers of African American children, conceptualizes culturally relevant teaching as a pedagogy that empowers students intellectually, socially, emotionally, and politically by using cultural referents to impart knowledge, skills, and attitudes (p. 18). She characterizes teachers who are culturally responsive as those who believe all students can succeed and who help students build conceptual bridges between home and school. Ladson-Billings also emphasizes the importance of developing collaborative relationships both within and outside the classroom. In culturally responsive classrooms, students are encouraged to assume responsibility for each others learning and to give back to the community. Thus, Ladson-Billings echoes many of Banks dimensions of multicultural education.

Elsewhere, Ladson-Billings defines culturally relevant teaching as a pedagogy of opposition not unlike critical pedagogy but specifically committed to collective, not merely individual, empowerment (2000, p. 208). Culturally relevant pedagogy, in her view, focuses on academic excellence, affirms students cultures, and develops critical consciousness in students. Similarly, in an earlier article, Banks and Banks (1995) provide an expanded conceptualization of equity pedagogy that focuses on human agency: Equity pedagogy creates an environment in which students can acquire, interrogate, and produce knowledge and envision new possibilities for the use of that knowledge for societal change (p. 153). This element of CRI reflects Freires (1970/1993; 1973/1998) notion of conscientization, whereby individuals are encouraged to interrogate the world in order to develop a critical understanding of their reality. Thus, CRI includes elements of equity pedagogy whereby students develop the skills and dispositions necessary for taking collective action on issues that affect the community.

Another prominent advocate of CRI, Geneva Gay, defines culturally responsive teaching as using the cultural knowledge, prior experiences, frames of reference, and performance styles of ethnically diverse students to make learning encounters more relevant to and effective for them (2000, p. 29). Compiling the ideas of several scholars, Gay suggests that culturally responsive teaching is:

"

validating, in that it affirms the cultural heritages of students and builds bridges of meaningfulness between home and school (p. 29);

"

comprehensive, in that in incorporates strategies that teach the whole child, i.e., that develop intellectual, social, emotional, and political learning (p. 30);

"

multidimensional, in that it incorporates multiple forms of expression and presents diverse experiences, contributions and perspectives;

"

empowering, which translates into academic competence, personal confidence, courage, and the will to act (p. 32);

"

transformative, in that it challenges the conventions of traditional educational practices; and

"

emancipatory, in that it redefines what constitutes truth and gives students a voice.

From these various perspectives of culturally responsive instruction/pedagogy, several points become clear. First, CRI is complex. Culturally responsive instruction cannot be reduced to a series of teaching practices; rather, it perhaps can best be conceptualized as a way of viewing teaching and learning that considers the social, emotional, cognitive, political, and cultural dimensions of every student. While the research on effective teaching is compatible with culturally responsive instruction, CRI suggests that teachers must not only employ best practices but also must consider the unique learning needs of students who have historically been underserved in our educational institutions. Second, CRI expands the traditional canon by incorporating perspectives of diverse others, affirming students cultural knowledge, and continuously reviewing and recreating knowledge in order to bring to the center the experiences of those who historically have been marginalized. Third, CRI is empowering in that students develop critical consciousness at the same time that they develop academic competence. A primary goal of CRI is to empower students to believe in themselves and their ability to bring about change.

Irvine and Armento (2001) argue that culturally responsive instruction is not a new concept: Traditional pedagogy has always been culturally responsive, that is, responsive to the culture of students who are primarily middle class and Euro-American (p. 13). Because educational institutions have been designed by and for dominant groups, schools have assumed that the content, values and learning styles of White, upper/middle class populations were appropriate for all students regardless of cultural background. The authors suggest that culturally responsive pedagogy widens our vision of effective instruction by incorporating instructional practices that are culturally appropriate for all students. We would add that through challenging how things have always been in schools and helping students to acquire a critical consciousness, CRI can be viewed as emancipatory education that has the potential not only to promote student success, but also to transform society.

CONTRIBUTIONS FROM LANGUAGE SCHOLARS

Work from the fields of sociolinguistics and second language acquisition has expanded our notion of CRI by examining how language is used as a social and cultural marker, and how speaking a primary language other than English affects learning. Sociolinguists examine the ways in which language operates within a hegemonic society, giving legitimacy to some voices while marginalizing others. For instance, Gee (1990) suggests that we all have a primary discourse that is the discourse of our home community, along with several secondary discourses. According to Gee, our discourse is essentially our identity kit. In a society marked by social stratification, dominant discoursesthose spoken by persons in powerare regarded as superior to the discourses of subjugated communities. As Delpit (1995) argues, it is critical that students learn power literacy so that they can have access to the dominant culture.

At the same time, sociolinguists are quick to point out that all languages are legitimate and that language use depends upon the situational context in which it is used (Stubbs, 2002). Thus, students home languages are validated as they discover that no language is inherently superior, but rather appropriate language use is defined by the social context (Halliday & Hasan, 1985; Powell, 1999). An important part of an empowering pedagogy, then, is teaching students when it is appropriate to use their home language, and when the situational context requires that they use the language of power.

An even larger body of research comes from the area of second language acquisition. Responding to the educational underachievement of English learners (ELs), scholarship in this area has focused on the importance of bilingualism and biliteracy and the need to understand the challenges faced by these students (August & Shanahan, 2008). Utilizing different terminology but endorsing many of the same principles as CRI, research on immigrant populations suggests that traditional education has been subtractive versus additive for these students (Valenzuela, 1999). That is, rather than building on the linguistic and cultural resources that second language learners bring to the classroom, our educational practices have diminished or ignored those resources in favor of promoting the language and culture of dominant (English-speaking) populations. Research in the area of multilingualism has found substantial evidence for the cognitive and social benefits of dual-language programs (Crawford, 1999; Krashen & Biber, 1988; Thomas & Collier, 2002). It also suggests that learning in a students first language transfers to learning in the second language and that conversing in ones home language can lead to greater conceptual understanding, thereby supporting the notion that students should be encouraged to converse in their native languages in classrooms (Cummins, 1996, 2000; Gutiérrez, Baquedano-López, & Alvarez, 2001; Moll & Díaz, 1987).

There have been several other notable contributions from the field of second language acquisition. The first has been the development of useful instructional models for students whose first language is not English. One popular model is the Sheltered Instruction Observation Protocol (SIOP), which includes eight components: Lesson Preparation, Building Background, Comprehensible Input, Strategies, Interaction, Practice/Application, Lesson Delivery, and Review/Assessment (Echevarria, Vogt, & Short, 2000). The SIOP model has been found to be moderately effective in raising the achievement of English Learners in classrooms where English is the language of instruction (Echevarria, Short, & Powers, 2006; McIntyre, Kyle, Muñoz, Chen, & Belden, 2010). Another useful framework has been developed by the Center for Research on Education, Diversity, and Excellence (CREDE). The CREDE model encompasses seven standards that reflect the principles of best practices for culturally and linguistically diverse students: Joint Productive Activity, Language and Literacy Development, Contextualization, Complex Thinking, Instructional Conversation, Modelling, and Child Directed Activity (CREDE, 2013). Research has found that the CREDE standards provide a useful instructional model, with the most rigorous findings supporting the element of Instructional Conversation for raising student achievement (Doherty & Hilberg, 2008; Doherty, Hilberg, Pinal, & Tharp, 2003; U.S. Department of Education, Institute of Education Sciences, 2006).

A second contribution from the second language acquisition field has been an emphasis on language development, and particularly academic language development, as a necessary component of instruction for English learners. Explicit language development is not just valuable for ELs, however. Fillmore and Fillmore (2012) include within the category of language minority students those who speak English only, but not the variety that is valued and promoted in the societys schools (p. 1). Explaining the difficulties faced by many students, the authors write that [t]he language used in complex texts differs enough from the English familiar to most students that it constitutes a barrier to understanding when they first encounter it in the texts they read in school (p. 1).

Research shows that providing myriad opportunities for students to be involved in interesting and engaging conversations (Goldenberg, 1991) can help all students develop language and conceptual understanding. It is important to note that research has demonstrated that engaging students in collaborative tasks with their peers promotes language acquisition, even when there are no students in the group who are proficient in English (Waxman & Tellez, 2002).

Language development occurs within all content areas, as students explore ideas using language and vocabulary from various disciplines. Because the language of academia differs significantly from the language of ordinary talk, however, it is important that teachers help students deconstruct the complex texts that they read. Thus, scholars advocate that teachers explicitly teach the grammatical constructions and vocabulary used in complex texts in various academic fields, and then give students the opportunity to use that language through authentic writing and discussion.

In a recent article, Goldenberg (2013) summarizes the research on English learners. Research suggests that, in addition to providing opportunities for students to develop English proficiency as discussed above, ELs benefit from instructional practices that have been found to be effective for all students. Beyond this, however, ELs benefit from additional instructional supports: building on students background knowledge, using graphic organizers, using pictures and other forms of realia, providing hands-on instruction, using gestures and visual cues, providing additional practice, incorporating language objectives, and using sentence frames and other language models. The consensus of research conducted over the past 40 years supports biliteracy and bilingual education and stresses the importance of using students home languages to promote learning. Further, providing instructional supports such as using cognates, teaching strategies and providing explanations in the students native language, and using the home language to preview and review important lesson concepts, have been found to benefit ELs.

Finally, the literature on culturally and linguistically appropriate teaching practices for English learners emphasizes instructional contextualization by including families funds of knowledge in the formal curriculum (González, Moll, & Amanti, 2005). The funds of knowledge research enhances the work in culturally responsive instruction by providing an intentional way to learn about students cultural knowledge that subsequently can be used to establish relationships with students and families and to build conceptual bridges in instruction.

To summarize, there is a rather large theoretical and research base that supports the use of instructional processes and strategies that are culturally and linguistically appropriate for students from underrepresented populations. This knowledge base emphasizes the need for best practices that have been found to be effective for all students, but suggests that these practices are not sufficient to close the achievement gap. Connecting the known to the unknown by using students home languages, experiences, and frames of reference in instruction, engaging students in peer collaboration and instructional conversations, using strategies to enhance academic language acquisition, and encouraging students to be change agents within a larger socio/linguistic community (both within and outside the classroom), are viewed as important instructional processes for these learners. It is also essential that teachers learn from students and their families, hold high expectations for all students, and value their cultural and linguistic knowledge. The CRIOP model incorporates all of these dimensions.

PURPOSE

The purpose of this study was to investigate an intervention designed to increase elementary school teachers use of CRI. Three research questions guided the study:

1.

Do teachers increase their use of culturally responsive practices as they participate in CRIOP professional development?

2.

What is the relationship between implementation of culturally responsive instruction and student achievement in reading and mathematics?

3.

What are teachers perceptions of their successes and challenges in implementing culturally responsive instruction?

METHODS

This study utilized a concurrent triangulation mixed methods design (Creswell, Plano Clark, Gutmann, & Hanson, 2003) to answer the research questions. Data were collected simultaneously from classroom observations, student achievement results, and teacher interviews. Integration of quantitative and qualitative data occurred during data collection and interpretation.

CONTEXT OF THE STUDY AND PARTICIPANTS

The seven elements of the CRIOP serve as the framework for the model. Each holistic element is accompanied by a number of specific indicators that are designed to assist teachers and observers in conceptualizing culturally responsive instruction, with examples and nonexamples included for comparison and evaluation. Thus, the elements, indicators and examples are used to guide teacher professional development in CRI as well as to assist observers in determining the extent to which CRI is being implemented in classrooms.

Participating Schools

Teachers from four elementary schools participated in the study. School A is located in a mid-sized Midwestern city while Schools B and C are in a nearby rural school district. School D is located in a small nearby town. Daily attendance rates at the schools were about 95%. School A had a free/reduced lunch rate of over 90% while Schools B, C, and D had free/reduced lunch rates at around 60%.

Teacher Participants

Twenty-seven teachers participated in this study. Teachers were all female, were predominantly White (n = 26, 86.7%; African American n = 1, 13.3%), and were all native speakers of English. Nineteen teachers taught all subjects in self-contained elementary classrooms (kindergarten n = 5, first grade n = 6, second grade n = 5, third grade n = 1), two teachers taught in self-contained classes containing both kindergarten and first grade, two teachers were special education teachers, one teacher taught physical education, and one teacher was a reading specialist/second grade teacher.

Student Participants

Student achievement data were collected from students enrolled in classrooms of participating teachers at School A and School D, the two schools in the study that administered the Measures of Academic Progress (MAP) test (Northwest Evaluation Association, 2013b). Student participants in this study consisted of 456 students325 students at School A and 131 students at School D (boys n = 229, 50.3%; girls n = 226, 49.7%, missing data n = 1). Of the 456 students, 397 (87.3%) received free or reduced lunch, and 128 (28% of total sample) were classified as English Language Learners (ELLs). Test results from the reading and mathematics sections of the MAP were collected from 430 students from the fall 2012 test administration and from 438 students from the spring 2013 test administration.

CRIOP Professional Development

During the summer preceding the 20122013 school year, teachers participated in a 6.5 hour professional development session led by the first author, who was also the project director. Activities during the workshop included inquiry-based learning, development of inquiry projects, family collaboration, and incorporating families funds of knowledge into classroom curriculum. Participating teachers also attended two full-day sessions during the fall semester that were conducted by the project director and an ESL consultant. These workshops focused on various elements of the CRIOP. The project director and a school-based coach also held a meeting with school administrators from the four participating schools to introduce the CRIOP model and to discuss the elements of the protocol.  

Throughout the 20122013 school year teachers received individual classroom coaching, on-site professional development, and instructional planning support. Participating teachers received an average of 50.4 hours of classroom-based coaching and mentoring during the intervention. Classroom support used the CRIOP as a professional development framework, and included observations, meetings with individual teachers and teacher teams, curriculum planning sessions, and collaborative creation of individualized action plans. The intended outcome of on-site support was to increase the incorporation of culturally responsive instruction in teachers daily practices, resulting in more culturally responsive classroom relationships, assessment and instructional practices, and use of discourse.

MEASURES, DATA COLLECTION, AND ANALYSIS

CRIOP Observation Instrument

The CRIOP measures teachers implementation of CRI and consists of an observation and an interview component. The observation assessment measures classroom CRI using a four-point Likert style scale: 1 = not at all, 2 = occasionally, 3 = often, and 4 = to a great extent. The interview component consists of a Post-Observation Teacher Interview and a Family Collaboration Interview. The CRIOP has been previously reported to produce Cronbachs alpha values of .88 and .94 (Malo-Juvera, Powell, & Cantrell, 2013). In the current study, the fall administration CRIOP yielded a Cronbachs alpha of .78, while the spring administration CRIOP yielded a Cronbachs alpha of .76.

Two field researchers observed 26 teachers in the fall and 27 teachers in the spring during the 20122013 school year. Teacher turnover led to the attrition of three teachers who were observed in the fall; therefore, there were 23 teachers who were observed during both fall and spring. Fall observations took place in October and November; spring observations were conducted in March, April, and May, at least five months after teachers initial observations. Observations occurred during literacy or content-area instructional times and included whole-class and/or small-group instruction and student independent activities. Learning events, teacher-student interactions, cooperative groups, and peer conversations were notated in the observations. Observations were conducted for at least 2.5 hours in each classroom, and researchers took field notes at five-minute intervals for the duration of the session. Observers looked for evidence of each CRIOP component in the detailed field notes and used that evidence in assigning a score for each component (14 as described in the previous paragraph). Appendix A shows an example of field notes taken during one observation. In this example, the teacher frequently assessed students responses to reteach or clarify when students needed support, and students demonstrated their learning in a variety of ways. Assessment practices (A/P) were observed to a great extent throughout the classroom observation, resulting in a CRIOP rating of 4 for the Assessment Practices element of the CRIOP. During this observation, the classroom teacher also frequently utilized strategies to encourage student conversations with their peers, and students engaged in academic conversations while using language for purposeful communication. Student discourse (D/C) was encouraged throughout the observation, resulting in a CRIOP rating of 4, or to a great extent, for Discourse/Instructional Conversation.

Observers, Training, and Interrater Reliability

Field Researcher One is a former elementary classroom teacher, holds a masters degree in literacy, is experienced in student teacher supervision, and was a full-time doctoral student at the time the study was conducted. Field Researcher Two is a former elementary classroom teacher, holds a masters degree in education, and was a full-time graduate student at the time the study was conducted. Field Researcher One conducted all fall observations, while Field Researcher One and Field Researcher Two conducted spring observations.

Field Researcher One and the project director participated in an observer training where they viewed videos of classroom instruction and scored the instructional practices using the CRIOP. Interrater agreement on final observations was 80%. Field Researcher Two participated in a small-group training session for the CRIOP in February with the project director, two professors, and Field Researcher One. The five participants viewed videos of classroom teachers, used the CRIOP as an evaluation tool for each video, and compared and discussed their ratings. Subsequently, Field Researchers One and Two conducted two classroom observations together in participating teachers classrooms. A total of three paired (one video and two in-class) observations were conducted. Inter-rater reliability analysis on these observations yielded a weighted Cohens Kappa = 0.84 (p < .001). A Kappa of .84 is considered to be an almost perfect strength of agreement (Landis & Koch, 1977, p. 165).

Teacher Interviews

Following each classroom observation, field researchers conducted an audio-recorded semistructured interview using the CRIOP Post-Observation Teacher Interview Protocol and The CRIOP Family Collaboration Teacher Interview Protocol. These protocols were designed to elicit additional information that might not have been readily apparent from data gleaned during the observation, e.g., whether the instruction observed was typical, the various ways that parents provide instructional support, and what else regularly occurs in the classroom that might not have been observed.

In addition, participants were interviewed to determine their perceptions of culturally responsive instruction. For this study, the following three interview questions and responses were transcribed and coded for analysis:

"

How do you define culturally responsive instruction?

"

What are your biggest successes with using Culturally Responsive Instruction with your students?

"

What are your biggest challenges with using Culturally Responsive Instruction with your students?

A two-phase method of analysis was utilized for analyzing data from fall and spring teacher interviews. Two sample transcripts are presented in Appendix B that illustrate how codes were assigned to each response, with italicized codes representing phase one codes and nonitalicized codes representing phase two codes.

In phase one of analysis, responses to each interview question were coded according to seven holistic elements of the CRIOP. These elements served as the macro-codes that were used to triangulate the observation and interview data. The second author and fourth author (also Field Researcher One) independently read and coded five teacher interview transcripts according to the CRIOP element most representative of each response. Subsequently, the two researchers compared the codes each had assigned. Initial agreement was 71%. The coders discussed their disagreements and arrived at consensus. Then, they coded five additional transcripts independently. The second round of coding comparisons resulted in an interrater agreement of 100%. The fourth author, alone, coded the remaining transcripts. Codes for each CRIOP component were counted for both fall and spring interviews to determine frequency, and illustrative quotes were selected for inclusion in the manuscript. Appendix C shows sample quotes for each CRIOP component.

Another phase of data analysis occurred to identify more specific themes in the interview data, beyond the categorical CRIOP elements. In this phase of analysis, the fourth author used analytic induction to create a code list. Utilizing Coffey and Atkinsons (1996) approach to qualitative data analysis, she began coding by reviewing each interview response and developing codes to represent particular ideas from the responses. After noting recurring patterns and themes from the compiled list of codes, conceptual links between codes emerged, and subgroups were collapsed into categories. For example, teachers repeated references to integrating standards and core content and administrative constraints were collapsed into instructional decision making. Using the list of inductive codes, the second and fourth authors independently read and coded five interviews, and interrater agreement was 88%. The fourth author independently coded the remaining interviews. The inductive analysis of transcription data provided depth and contextualized findings from the broader understandings captured through utilizing the CRIOP holistic elements for coding of interviews.

Linking of the interview data to the CRIOP elements enabled us to discern teachers perceptions about their implementation of the CRIOP elements in terms of those that were implemented most and least readily. Data are presented for both fall and spring interviews to indicate changes in teachers perceptions of implementation over time.

Measures of Academic Progress (MAP)

The Measures of Academic Progress (MAP) assessment, developed by the Northwest Evaluation Association (NWEA), is an adaptive computerized assessment aligned to state testing standards for reading and mathematics (2013a). This tool was used to measure student achievement because it is viewed by stakeholders within our state to be a valid measure of students academic progress. The MAP assessment provides data three times a year in math and reading performance and is utilized in many districts across our state to determine student progress in the elementary grades. It is accompanied by a manual that provides a comprehensive list of skills to be taught at each level. Student performance is reported through an RIT score, percentile, and a Lexile range.

MAP assessments were utilized as part of two participating schools regular assessments during the 20122013 academic school year. Achievement data for students enrolled in participating teachers classrooms were collected for the fall and spring administrations of MAP. An independent review of MAP provided evidence of its reliability and validity in terms of students relative standing and growth (Cizek, 2010).

RESULTS

Results are provided on teachers implementation of the seven CRIOP elements as determined through classroom observations, student achievement data, and teachers perceptions of their successes and challenges in implementing culturally responsive instruction.

TEACHERS OBSERVED PRACTICES

It was hypothesized that teachers culturally responsive instruction as measured by the CRIOP would significantly increase from the fall observation to the spring observation (p < .05). Of the original 26 teachers who were observed in the fall, 23 were observed in the spring. A repeated measures ANOVA was conducted using fall (x̄ = 16.35, s = 2.89) and spring observations (x̄ = 18.52, s = 3.25). Results show that culturally responsive instruction as measured by CRIOP was significantly higher posttest, Wilks’s Λ = .67, F(1, 22) = 13.64, p < .001, partial h² = .38. Post-hoc analysis was conducted on the seven individual CRIOP component scores using repeated measures ANOVAs to determine if effect of change was focused in any specific pillars. See Table 1 for change in specific CRIOP components from fall to spring observations.

Table 1. Change in CRIOP Components from Fall to Spring

	Fall		Spring		Sig		Effect Size
	Mean	SD	Mean	SD
Classroom Relationships	2.91	.87	3.36	.79	.019			.19
Family Collaboration	2.86	.71	2.73	.70	.209			.03
Assessment	2.32	.57	2.86	.77	< .001a			.25
Curriculum	2.23	.43	2.55	.67	.025			.17
Instruction	2.55	.60	3.05	.72	.001a			.37
Discourse	2.50	.67	2.59	.85	.576			.02
Sociopolitical	1.18	.40	1.59	.50	< .001a			.41

Bold = significant change from Fall to Spring (p < .05). a = significant utilizing Bonferroni adjustment. Values are calculated using repeated measures ANOVAs.

The largest effect sizes were found for Sociopolitical Consciousness (partial h² = .41), Instruction (partial h² = .37), Assessment (partial h² = .25), Classroom Relationships (partial h² = .19), and Curriculum (partial h² = .17). Mean scores increased for six of the seven elements, with Family Collaboration being the exception.

CULTURALLY RESPONSIVE INSTRUCTION AND STUDENT ACHIEVEMENT

High vs. Low Implementation of Culturally Responsive Instruction

In order to determine whether or not implementation of culturally responsive instruction was related to student learning, teachers were separated by scores on the spring CRIOP observations (x̄ = 18.4, s = 3.25) into High Implementation and Low Implementation groups. High Implementation teachers (n = 5; School A = 2, School D = 3) were defined as those who had CRIOP spring observation scores at or above one standard deviation from the mean (x̄ ≥ 22), while Low Implementation teachers (n = 5; School A = 4, School D = 1) were defined as those who had CRIOP spring observation scores at or below one standard deviation from the mean (x̄ ≤ 15). MAP reading and mathematics test results were used to determine relationships of CRI implementation to student achievement.

Reading Performance

Researchers hypothesized that students with teachers identified as High Implementers would have significantly higher reading performance on spring tests than students who had teachers identified as Low Implementers (p < .05). A one way ANCOVA was conducted using spring MAP reading tests as a dependent variable, implementation level (high or low) as fixed factor, and fall MAP Reading tests as a covariate. Results indicate that students with teachers identified as High Implementers scored significantly higher than students with teachers identified as Low Implementers on spring MAP reading tests F(1, 188) = 3.06, p < .05, h² = .02 (see Table 2 for adjusted means).

Table 2. MAP Reading High and Low Implementation Adjusted Means

        Dependent variable: Spring MAP Reading Test

Level	Mean	Std. Error
LOW	173.423a	1.063
HIGH	176.129a	1.115

^aCovariates appearing in the model are evaluated at the following values: Fall MAP Score Reading = 159.80.

Math Performance

Researchers hypothesized that students with teachers identified as High Implementers would have significantly higher mathematical performance on spring tests than students who had teachers identified as Low Implementers (p < .05). A one way ANCOVA was conducted using spring MAP mathematics tests as a dependent variable, implementation level (high or low) as fixed factor, and fall MAP mathematics tests as a covariate. Results indicate that students with teachers identified as High Implementers scored significantly higher than students with teachers identified as Low Implementers on spring MAP mathematics tests F(1, 186) = 25.39, p < .001, h² = .12 (see Table 3 for adjusted means).

Table 3. MAP Math High and Low Implementation Adjusted Means

        Dependent variable: Spring MAP Math Test

Level	Mean	Std. Error
LOW	176.817a	.692
HIGH	181.936a	.734

^aCovariates appearing in the model are evaluated at the following values: Fall MAP Score Math = 160.83.

TEACHERS PERCEPTIONS ABOUT THEIR IMPLEMENTATION

Table 4 shows the CRIOP elements that teachers addressed when asked about their definitions of CRI, successes, and challenges. The table reflects similarities in what was found in the observed practices detailed above and contextualizes the findings in terms of the elements that teachers implemented most and least frequently. Further, differences in interview responses from fall to spring as depicted in the table can be linked to changes that were observed in teachers CRIOP ratings.

Table 4. CRIOP Codes from Teacher Interviews

INTERVIEW QUESTION/CRIOP PILLARS	INTERVIEW SESSION
	Fall Participant Responses	Spring Participant Responses
Definition of Culturally Responsive Instruction
I.  Classroom Relationships	16	10
II.  Family Collaboration	0	0
III.  Assessment Practices	0	0
IV.  Curriculum/ Planned Learning Experiences	2	0
V.  Pedagogy/ Instructional Practices	8	13
VI.  Discourse/ Instructional Conversation	1	0
VII.  Sociopolitical Consciousness	0	0
Successes with Using Culturally Responsive Instruction
I.  Classroom Relationships	15	9
II.  Family Collaboration	6	1
III.  Assessment Practices	0	0
IV.  Curriculum/ Planned Learning Experiences	0	3
V.  Pedagogy/ Instructional Practices	7	8
VI.  Discourse/ Instructional Conversation	0	1
VII.  Sociopolitical Consciousness	0	0
Challenges with Using Culturally Responsive Instruction
I.  Classroom Relationships	7	5
II.  Family Collaboration	5	6
III.  Assessment Practices	1	0
IV.  Curriculum/ Planned Learning Experiences	7	5
V.  Pedagogy/ Instructional Practices	6	5
VI.  Discourse/ Instructional Conversation	0	2
VII.  Sociopolitical Consciousness	0	0

Classroom Relationships

As reflected in both sets of interviews, classroom relationships were central to teachers understandings of culturally responsive instruction. As part of their definitions of CRI, teachers emphasized the importance of valuing students cultures and experiences and building positive relationships with students. As was evident in the classroom observations, teachers reported more success with this element of the CRIOP than with any other element. In the fall, classroom relationships were mentioned as successes more than twice as often as any other element, and although teachers reported successes with a number of other elements in the spring, classroom relationships was still the most frequently mentioned success. Teachers did, however, articulate challenges with classroom relationships. As one teacher described:

Just not always understanding the different cultures, . . . because sometimes things are so ingrained. . . . I know its hard for me to think outside and say okay, what if I had not been born in the United States. What would I think about differently? So, just understanding exactly what is a part of the culture and what is just their [students] personality and how that kind of comes together.

A number of teachers pointed out challenges related to language barriers and described their efforts to mediate communication difficulties with both students and parents. Almost all teacher participants were White, monolingual females and all had nonnative English speaking students, primarily Spanish speaking, in their classrooms. These teachers made strong efforts to negotiate language barriers with their students. Teachers recounted their personal efforts to learn Spanish in order to communicate more effectively with their English learners and described their classroom instructional practices for facilitating their EL students comprehension, engagement, and learning. As one teacher with limited knowledge of Spanish stated:

I dont think I was ready. No one prepares you to be put in a classroom with 16 Spanish speaking children who come from a dominant Spanish household . . . and just kind of like them not understanding a certain word . . . because they may not understand that or never been introduced to that word, and so its really finding a way to connect with them. . . And to do that . . . I would look it up and learn how to say it in Spanish . . . Id say that and then Id say the English . . . and I know that has to help them because it kind of links it for them.

Instruction/Pedagogy

Interview data from fall to spring suggest teachers shifted in their definitions of CRI from an extensive focus on classroom relationships to a stronger focus on instruction and pedagogy. This finding is consistent with teacher observations indicating a significant increase in teachers implementation of the CRIOP element of Instruction/Pedagogy over the course of the year. During interviews, teachers gave examples of successful culturally responsive practices they had implemented, which included allowing students to respond in their native languages, teaching vocabulary explicitly, and providing opportunities for students to collaborate during learning activities. One teacher related her experiences with implementing CRI and emphasized the value of students serving as peer resources and supporting one another in learning:

I have one student who speaks entirely Spanish. And when I have kids that can fill in with a Spanish word or tie in, . . . and they feel comfortable with doing that and feel empowered about helping her in her language. To me, that is a huge success. Like when one of the other students helps her, then that to me, it kind of gives me goosebumps, because it is great. Like they are really wanting to help her understand.

Another teacher discussed the benefits of providing opportunities for students to engage in inquiry during instruction:

I see more improvement in their learning. Students are more eager in improving their learning . . . we implemented the inquiry [process] because it is just much more hands on, and it is really engaging for my students.

Several teachers mentioned challenges relating to their lack of knowledge of instructional strategies for nonnative English speaking students. One teacher related her struggles with accessing appropriate instructional strategies:

My biggest challenge is . . . with my Hispanic students especially, I say to myself, okay, I dont know how else to do this definitely, I dont know how else to help you understand this. Sometimes I say, okay, . . . I dont know any more ways to help you with this. I dont know. My resources are depleted. I dont know what else I can do for you. Sometimes it is a language barrier, but sometimes I just really dont know any more strategies to help make that light bulb come on.

Family Collaboration

Family collaboration was coded with a high level of frequency during both the fall and spring interviews; however, whereas one-fourth of teachers focused on successes with involving families during the fall interviews, teachers focused more heavily on challenges with this element in the spring interviews. Only one teacher mentioned family collaboration as a success during the spring interviews, but six of the teachers discussed challenges. Most specifically, teachers described their struggles with overcoming language barriers with family members. One teacher stated:

I think the biggest struggle is getting my parents involved and only because where I dont speak Spanish, . . . I think some of them are more hesitant to come in, knowing that I dont speak Spanish, that you cant always find a translator. So, I feel like I wish I could have more parent involvement and have them come in and read a story or help with a lesson or, like we did vegetables today, just to have them come in and work with a small group of kids. I think that is so important for them to see their parents come in, especially to see them in the school setting, and having my parents feel comfortable with coming into my classroom as well, . . . that is probably my biggest place Id like to grow, is getting my families more involved.

Another teacher with a large number of ELs in her class commented:

The biggest thing is Im used to just having constant parent communication, but unfortunately I cant in this location. I cant just zip a note. It has to go through the process of someone translating it. If I have parent teacher conferences, if I dont have a translator, that is a problem.

As the classroom observation data show, the teachers did not exhibit as much growth in the area of family collaboration as they did with other elements of the CRIOP. Although they reported successes with this element initially, they seemed to become discouraged in the face of persistent language barriers.

Curriculum

Curriculum was an element that proved difficult for a number of teachers who articulated challenges in both fall and spring interviews. Some interview responses reflected anxieties about meeting standards and adhering to prescribed curriculum mandates. One teacher illustrated:

I think the biggest problem is just trying to incorporate it [CRI] in every lesson. That is very difficult, mostly because of the tight bonds around curriculum. I think that that is one of the most difficult things.

Other teachers expressed that the pressures to incorporate standards into lessons and the expectations for state testing limited their implementation of culturally responsive instruction. One participating teacher described these constraints during her interview:

We would come back to administrative meetings and have it [culturally responsive lesson] questioned as to how were going to be testing that; how is that measurable. When youre excited about implementing it [culturally responsive instruction] and then get in your team meeting, and its questioned, thats been kind of defeating. Not kind of; thats been defeating.

The Elusive Elements

During the fall and spring interviews, teachers rarely addressed the elements of Discourse and Assessment, and teachers never discussed successes or challenges related to the element of Sociopolitical Consciousness. One teacher included Discourse in her definition of culturally responsive instruction during fall interviews and two teachers named Discourse as a challenge during the spring interviews. Classroom observations indicate no significant growth in teachers implementation of Discourse, and interview data suggest this element may not have been an area of focus for teachers in light of apparent focus on other elements such as Instruction/Pedagogy and Classroom Relationships. In contrast, classroom observations did reflect changes in teachers implementation of Assessment and Sociopolitical Consciousness, yet these elements were not addressed in the interviews.

The Complexity of Culturally Responsive Instruction

One aspect that we believe deserves attention is teachers perceptions of the complexity of the CRIOP model. As stated previously, we view culturally responsive instruction as multifaceted, encompassing not only generally effective pedagogical practices, but also practices that use and validate the language and cultural knowledge of students and their families and that encourage societal change. Comments made by teachers in structured interviews suggest that they grappled with understanding the many aspects of culturally responsive instruction. In discussing CRI, one teacher responded, Im not sure truly of what all it [CRI] should encompass, while another teacher described her biggest challenge with CRI as, just my lack of knowledge of it. . . trying to get used to wrapping my head around that framework. You know, I have taught one way for so long.

In order to make sense of the multifaceted nature of CRI, some teachers used terminology that implied that CRI was an instructional strategy versus a complex array of behaviors. For example, one teacher stated that the biggest challenge with the implementation of CRI was, just knowing that Im doing it right. . . doing it correctly. Another teacher described her biggest challenge with CRI:

Not knowing if I am using it correctly. Not really understanding if this is the right way of going with this. Like taking our standards and what we have to teach them and putting it into a lesson that is culturally responsive. That is kind of difficult sometimes.

Other teachers, however, recognized the multidimensional nature of CRI and acknowledged its complexity. For instance, when responding to an interview question about her definition of CRI, one participant answered: That is a tough one, even now, even after all of this. A combination of things. Another teacher responded:

How do I change what I have done for the last three years as a teacher to what I am doing now? Like am I doing it? . . . I feel like I try my best, but when it really comes across, am I doing it [CRI] the right way? If that makes sense. So I think the challenge is making sure I understand it correctly and how to implement it in the classroom to be the most effective, for the students.

These teachers comments reflect the challenges for teachers and teacher educators in conceptualizing and implementing a culturally responsive pedagogya point which will be examined in greater depth in the section that follows.

DISCUSSION

In this article, we have presented the results of a year-long professional development project that utilized the Culturally Responsive Instruction Observation Protocol (CRIOP) as a framework for intervention and as a tool for classroom observations. This study was designed to answer the following questions: (1) Do teachers increase their use of culturally responsive practices as they participate in CRIOP professional development? (2) What is the relationship between implementation of culturally responsive instruction and student achievement in reading and mathematics? and (3) What are teachers perceptions of their successes and challenges in implementing culturally responsive instruction? We investigated these questions using a concurrent triangulation mixed methods design that included classroom observations, teacher interviews, and student achievement data.

Results of classroom observations showed that teachers had significantly higher levels of CRI implementation in the spring compared to fall. This finding suggests that the intervention was effective, which supports previous research that indicates CRI can be taught using a coaching model (Teemant, Wink, & Tyra, 2011). It also suggests that when elements of CRI are clearly articulated and when teachers are supported in their attempts to implement the various facets of CRI, it can lead to changes in teacher behaviors. Thus, we believe these data have implications for preservice and inservice teacher preparation programs as they prepare teachers to teach in diverse classrooms.

These changes in behaviors were particularly evident in the areas of Classroom Relationships, Assessment, Instruction, and Sociopolitical Consciousness, although this latter element was rarely seen in classrooms. Teacher interview data provided insight into these findings, suggesting that participants felt most successful in building positive relationships within their classrooms and also felt they grew in their implementation of culturally responsive instructional practices through their participant in the CRIOP professional development project.

Researchers have demonstrated that classroom relationships are critical in influencing students engagement and school performance (Pianta, Hamre, & Allen, 2012; Reyes, Brackett, Rivers, White, & Salovey, 2012). Pianta et al. (2012) focus on four levers for changing classroom interactions: (1) teachers knowledge and cognitions related to their interactions with students, (2) availability of ongoing relational supports for teachers themselves, (3) teachers regular exposure to individualized feedback about their actual interactions with students, and (4) a standard and valid target around which to focus efforts to change interactions (p. 379). Pianta et al. (2012) have been successful in changing classroom relationships through their own professional development model. Similarly, professional development using the CRIOP framework provides the opportunity for all four of these conditions for teacher growth. Although teachers likely cared about their students and wanted to have positive relationships with them prior to their participation, they may not have known how to demonstrate that care, particularly for diverse populations of students. By operationalizing the element of classroom relationships, the CRIOP provided concrete guidance for implementing the identified practices.

Much of the professional development throughout the year that teachers received focused on their pedagogical practices as they related to the diverse learners in their classrooms. Because the professional development model involved on-site coaching, teachers were regularly provided with feedback on their instruction and on ways to formatively assess the effectiveness of that instruction. Thus, changes in culturally responsive practices in the areas of Instruction and Assessment were not surprising.

At the same time, using effective discourse practices remained a challenge, and the fact that teachers did not appear to improve in their use of these practices is problematic. Peer collaboration, instructional conversations, and using academic language in communicating ideas are essential components of CRI, as is the use of students native language in instruction. Research shows that the Initiate-Respond-Evaluate (IRE) discourse pattern dominates classroom interactions in U.S. classrooms (Cazden, 2001; Goodlad, 1984). This pattern may be so entrenched in the way we do school that it is difficult to overcome. Further, teachers in this study were not encouraged by their schools to use Spanish in instruction or to promote it in student conversation. Given that the majority of students in their classrooms were native English speakers, acquiring new habits of effective classroom discourse may be difficult.

Sociopolitical Consciousness is perhaps the most challenging element of the CRIOP to implement. Teachers must be committed to challenging social inequities and confronting stereotypes, which necessitates an understanding of white privilege and the courage to question the status quo. Kincheloe (2004) argues that teachers must first develop their own critical epistemology, which is something that generally is not emphasized in teacher preparation programs. Researchers have noted teachers inherent biases as a barrier to implementing culturally responsive instruction (Young, 2010).

In reporting challenges, teachers often mentioned language barriers, which hindered their ability to develop collaborative relationships with families and to communicate with their students. Interestingly, more teachers cited the Family Collaboration element as a challenge in the spring interviews as opposed to the fall. Many teachers attributed this challenge to communication barriers, which have been sighted in the literature as one of the obstacles to promoting parent involvement in schools for English learners (Peña, 2000; Turney & Kao, 2009). While communication difficulties were undoubtedly a factor, we hypothesize that this change in teachers perception may have been the result of a deeper understanding of what constitutes quality teacher-parent collaboration, which necessitates a genuine partnership between teachers and parents. The CRIOP model goes beyond traditional notions of appropriate parent participation in schools (Tett, 2001), encouraging the formation of more equitable relationships that allow teachers to learn from families (Endrizzi, 2008). Thus, teachers must acknowledge that families have important cultural knowledge to share and they must be willing to seek out that knowledge in non-traditional ways (González, Moll, & Amanti, 2005).

Finally, teachers noted curricular and administrative constraints in their ability to implement CRI, such as state testing mandates and rigid adherence to standards. This finding supports previous investigations that show that accountability policies can influence teachers implementation of culturally responsive pedagogy. For instance, in a five-year study, Achinstein and Ogawa (2012) found that new teachers of color face significant tensions relating to accountability and standardization that militate against implementing culturally responsive practices. Stillman (2011) found that school leadership was particularly important in mediating such external factors; when teachers were given the autonomy to resist objectionable reforms and to be innovative, there were positive results in instruction and teacher professionalism.

Curriculum theorists such as Michael Apple (2003) point out that the official knowledge of the school is entrenched within the politics of schoolinga system that values the knowledge of dominant groups while devaluing and even dismissing the contributions of nondominant groups. Thus, teachers are caught up in a political game where they are often told what to teach, thereby limiting their ability to enact a curriculum that would connect more directly to students lives. At the same time, Apple suggests that the politics of the curriculum also involves who should select it, how it should be organized, taught, and evaluated, and . . . who should be involved in asking and answering these questions (p. 7). Hence when the official knowledge of the school becomes tied to a high-stakes system of accountability, teachers may be limited in their capacity to implement culturally responsive practices.

STUDENT DATA RESULTS AND INTERPRETATION

To determine the effects of CRI implementation on student achievement, teachers were divided into High Implementers and Low Implementers of the CRIOP based upon their scores on spring observations. Findings indicated that students of High Implementers had significantly higher achievement scores in reading and mathematics than students of Low Implementers on the spring Measurement of Academic Progress (MAP) tests in reading and mathematics.

These findings empirically extend previous research on CRI in literacy and math. For instance, Lee (1995, 2001) implemented the Cultural Modeling Project with African American students in which students cultural knowledge and frames of reference were used to assist them with literary analysis. Lee found significant pre- and postachievement gains between students who used Cultural Modeling and those in control groups. Rickford (2001) found that low-achieving African American middle school students were able to respond to higher level comprehension questions when they were able to identify with the themes found in culturally relevant texts. Cammarota and Romero (2009) report on a social justice project geared toward Chicano students who were at risk of dropping out. As a result of their participation in the project, Chicano students outscored Anglo students in the same school on the states exams. McCarty (1993) reports on a program with Navajo youth called The Rough Rock English-Navajo Language Arts Program. Teachers created written materials in Navajo and developed thematic units that were culturally relevant to students. After the second year of their participation in the program, students reading scores on standardized reading tests progressively improved.

Similar findings have been reported in mathematics. In a number of articles, Lipka and colleagues chronicle research stemming from the Math in a Cultural Context (MCC) project, in which Yupik cultural knowledge is explicitly connected to math standards. Findings indicate that students in classrooms using the MCC curriculum made significantly greater gains than control groups in meeting state mathematics standards (Kisker et al., 2012; Lipka & Adams, 2004; Lipka, Webster, & Yanez, 2005). Research has also been done with African American middle and high school basketball players, who were able to solve math problems more easily when they were contextualized to their knowledge of sports (Nasir, Hand, & Taylor, 2008). Thus the findings of our investigation are consistent with the results of previous investigations that show that implementing culturally responsive practices can have a positive effect on student achievement.

CONCLUSION

While we consider the results of this investigation to be preliminary given the limited scope of this study, findings suggest that the CRIOP can provide an effective framework for teacher professional development, and that high implementation of culturally responsive practices can potentially lead to higher student achievement in both reading and math. At the same time, the results of this study also suggest that teachers face several challenges in implementing CRI, including constraints imposed by administrators and high-stakes accountability, language barriers in communicating with families, understanding CRI, and the ability to meet the needs of all of their students in a highly diverse classroom.

Beyond this, however, we suggest that one of the biggest challenges in implementing CRI is its multidimensionality. The CRIOP framework includes numerous characteristics associated with seven distinct elements: Classroom Relationships, Family Collaboration, Assessment, Curriculum, Instruction, Discourse, and Sociopolitical Consciousness. These characteristics encompass generally effective practicespractices that have been found through research to be effective for all studentsalong with instructional and behavior characteristics that incorporate and validate the language and experiences of the students and families we serve. Further, CRI is culturally congruent in that it uses the cultural preferences and resources of students, and it is also empowering, giving students a voice and potentially leading to social transformation. Incorporating all of these characteristics into a single framework can result in a perception that culturally responsive instruction is indeed very complex, as it did for many of the teachers in our study.

Despite its complexity, however, numerous books and articles have espoused the value of culturally responsive instruction; and while there have been several investigations on specific characteristics of CRI that have yielded positive results, our ability to assess CRI in classrooms has been limited by the lack of a comprehensive model and instrument that operationalizes the many effective practices discussed in the literature for culturally and linguistically diverse students. The Culturally Responsive Instruction Observation Protocol has been designed to address this need. The findings of this investigation suggest that the CRIOP shows promise as both a framework for teacher professional development and as an observation instrument in investigations of CRI.

At the same time, the authors of this article acknowledge several limitations of this study. The research project was constrained by the professional development model, which involved on-site coaching and therefore resulted in a small number of teacher participants. Larger scale studies involving more teacher participants might yield different results. Another clear limitation is the absence of a control group. Thus, the results of this study should be interpreted with caution. Future investigations are advised that would scale up the study to include more participants and a more rigorous experimental design.

Future research should also explore the usefulness of the CRIOP instrument as an effective measure of culturally responsive instruction. For instance, are there components of the model that are more salient and have greater impact on student performance than other components? In what ways do the various categories overlap? Is the instrument sensitive enough to capture pedagogical practices across a wide variety of instructional settings? Perhaps most important, in using instruments like the CRIOP, is there a danger in essentializing our conceptions of culturally responsive instruction? While we believe that the CRIOP provides a useful model for assessing teacher practices and growth, we are also mindful that our notions of CRI must change and evolve in order to meet the needs of each generation of students (Ladson-Billings, 2014, pp. 8081).

Finally, the many challenges in implementing CRI, and how teachers and administrators might overcome those challenges, are areas that deserve further study. As we continue to explore culturally responsive instruction as a paradigm for closing achievement gaps, we are hopeful other researchers will find the CRIOP to be a useful tool and model for the assessment and implementation of culturally responsive instruction, and we invite readers to join us as we continue to investigate its usefulness.

Acknowledgment

This study was supported by a National Professional Development grant from the U.S. Department of Education Office of English Language Acquisition. Any opinions, findings, and conclusions or recommendations expressed in this article are those of the authors and do not necessarily reflect the views of the U.S. Department of Education.

Note

1. The CRIOP has since been revised and now consists of six major elements: Classroom Relationships, Family Collaboration, Assessment Practices, Instructional Practices, Discourse, and Sociopolitical Consciousness.

References

Abrahams, R. D., & Troike, R. C. (Eds.). (1972). Language and cultural diversity in American education. Englewood Cliffs, NJ: Prentice-Hall.

Achinstein, B., & Ogawa, R. T. (2012). New teachers of color and culturally responsive teaching in an era of accountability: Caught in a double bind. Journal of Educational Change, 13(1), 1-39.

Apple, M. W. (2003). The state and the politics of knowledge. New York: RoutledgeFalmer.

Au, K. H., & Kawakami, A. J. (1994). Cultural congruence in instruction. In E. R. Hollins, J. E. King, & W. Hayman (Eds.), Teaching diverse populations: Formulating a knowledge base (pp. 5-23). Albany: SUNY Press.

August, D., & Shanahan, T. (Eds.). (2008). Developing reading and writing in second-language learners: Lessons from the Report of the National Literacy Panel on Language-Minority Children and Youth. New York: Routledge.

Banks, C. A. M., & Banks, J. A. (1995). Equity pedagogy: An essential component of multicultural education. Theory Into Practice, 34(3), 152-158.

Banks, J. A. (2008). An introduction to multicultural education. Boston: Pearson.

Cammarota, J. & Romero, A. (2009). The Social Justice Education Project: A critically compassionate intellectualism for Chicana/o students. In W. Ayers, T. Quinn, & D. Stovall (Eds.), Handbook for Social Justice Education. New York: Routledge.

Cazden, C. B. (2001). Classroom discourse: The language of teaching and learning (2nd ed.). Portsmouth, NH: Heinemann.

Center for Research on Education, Diversity and Excellence (CREDE). (2013). Retrieved from http://manoa.hawaii.edu/coe/crede/

Cizek, G. J. (2010). Review of measures of academic progress. In R. A. Spies, J. F. Carlson, & K. F. Geisinger (Eds.), The eighteenth mental measurements yearbook (pp. 327332). Lincoln, NE: University of Nebraska Press.

Coffey, A., & Atkinson, P. (1996). Making sense of qualitative data: Complementary research strategies. Thousand Oaks, CA: Sage Publications, Inc.

Crawford, J. (1999). Bilingual education: History, politics, theory, and practice (4th ed.). Trenton, NJ: Crane Publishing.

Creswell, J., Plano Clark, V., Gutmann, M., & Hanson, W. (2003). Advanced mixed methods research designs. In A. T. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social & behavioral research (Vol. 1, pp. 209240). Thousand Oaks: Sage.

Cuban, L. (1972). Ethnic content and white instruction. Phi Delta Kappan, 53(5), 270273.

Cummins, J. (1996). Negotiating identities: Education for empowerment in a diverse society. Ontario, CA: California Association for Bilingual Education.

Cummins, J. (2000). Language, power, and pedagogy: Bilingual children in the crossfire. Clevedon: Multilingual Matters.

Delpit, L. (1988). Other peoples children: Cultural conflict in the classroom. New York, NY: The New Press.

Doherty, R. W., & Hilberg, R. S. (2008). Efficacy of five standards in raising student achievement: Findings from three studies. The Journal of Educational Research, 10(4), 124.

Doherty, R. W., Hilberg, R. S., Pinal, A., & Tharp, R. G. (2003). Five standards and student achievement. NABE Journal of Research and Practice, 1, 195206.

Echevarria, J., Vogt, M. E., & Short, D. (2000). Making content comprehensible for English language learners: The SIOP model. Needham Heights, MA: Allyn & Bacon.

Echevarria, J., Short, D., & Powers, K. (2006). School reform and standards-based education: A model for English-language learners. Journal of Educational Research, 99(4), 195211.

Endrizzi, C. K. (2008). Becoming teammates: Teachers and families as literacy partners. Urbana, IL: NCTE.

Fillmore, L. W., & Fillmore, C. J. (2012). What does text complexity mean for English learners and language minority students? Retrieved from: http://ell.stanford.edu/sites/default/files/pdf/academic-papers/06-LWF%20CJF%20Text%20Complexity%20FINAL_0.pdf

Freire, P. (1970/1993). Pedagogy of the oppressed. New York: Continuum.

Friere, P. (1973/1998). Education for critical consciousness. New York: Continuum.

Gay, G. (2000). Culturally responsive teaching: Theory, research, and practice. New York: Teachers College.

Gee, J. P. (1990). Social linguistics and literacies: Ideology in discourses. New York: Falmer Press.

Goldenberg, C. (1991). Instructional conversations and their classroom application (Educational Practice Report 2). Berkeley, CA: The National Center for Research on Cultural Diversity and Second Language Learning.

Goldenberg, C. (2013). Unlocking the research on English learners. American Educator, 37(2), 411, 38.

González, N., Moll, L., & Amanti, C. (Eds.). (2005). Funds of knowledge: Theorizing practices in households, communities, and classrooms. Mahwah, NJ: Lawrence Erlbaum.

Goodlad, J. I. (1984). A place called school: Prospects for the future. New York: McGraw-Hill.

Gutiérrez, K. D., Baquedano-López, P., & Alvarez, H. H. (2001). Literacy as hybridity: Moving beyond bilingualism in urban classrooms. In M. de la Luz Reyes & J. J. Halcón (Eds.), The best for our children: Critical perspectives on literacy for Latino students (pp. 122141). New York: Teachers College Press.

Halliday, M. A. K., & Hasan, R. (1985). Language, context, and text: Aspects of language in a social-semiotic perspective. Victoria: Deaking University.

Hollie, S. (2012). Culturally and linguistically responsive teaching and learning: Classroom practices for student success. Huntington Beach, CA: Shell Education.

Irvine, J. J., & Armento, B. J. (2001). Culturally responsive teaching: Lesson planning for elementary and middle grades. Boston: McGraw-Hill.

Kincheloe, J. L. (Winter, 2004). The knowledges of teacher education: Developing a critical complex epistemology. Teacher Education Quarterly, 4966.

Kisker, E. E., Lipka, J., Adams, B. L., Rickard, A., Andrew-Ihrke, D., Yanez, E. E., & Millard, A. (2012). The potential of a culturally based supplemental mathematics curriculum to improve the mathematics performance of Alaska native and other students. Journal for Research in Mathematics Education, 43(1), 75113.

Krashen, S. D., & Biber, D. (1988). On course: Bilingual educations success in California. Los Angeles: California Association for Bilingual Education.

Ladson-Billings, G. (1994). The dreamkeepers: Successful teachers of African American children. San Francisco: Jossey-Bass.

Ladson-Billings, G. (2000). But thats just good teaching! In J. Noel (Ed.), Notable selections in multicultural education (pp. 206216). Guilford, CT: Dushkin/McGraw-Hill.

Ladson-Billings, G. (2014). Culturally relevant pedagogy 2.0: A.k.a the remix. Harvard Educational Review, 84(1), 7484.

Landis, J.R., & Koch, G.G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159174.

Lee, C. D. (1995). A culturally based cognitive apprenticeship: Teaching African American high school students skills in literary interpretation. Reading Research Quarterly, 30(4), 608630.

Lee, C. D. (2001). Is October Brown Chinese? A cultural modeling activity system for underachieving students. American Educational Research Journal, 38(1), 97142.

Lee, C. D. (2006). Culture, literacy, and learning. New York: Teachers College Press.

Lipka, J., & Adams, B. (2004). Culturally based mathematics education as a way to improve Alaska Native students math performance (Working Paper #20). Appalachian Collaborative Center for Learning, Assessment, and Instruction in Mathematics.

Lipka, J., Webster, J. P., & Yanez, E. (2005). Introduction: Factors that affect the performance of Alaska native students mathematical performance. Journal of American Indian Education, 44(3), 18.

Malo-Juvera, V., Powell, R., & Cantrell, S. (2013). Development, validation, and factor analysis of the culturally responsive instruction observation protocol. Paper presented at the 2013 annual meeting of the American Educational Research Association. Retrieved from the AERA Online Paper Repository.

McCarty, T. L. (1993). Language, literacy, and the image of the child in American Indian classrooms. Language Arts, 70(3), 182192.

McIntyre, E., Kyle, D. W., Muñoz, M. A., Chen, C., & Beldon, S. (2010). Teacher learning and ELL reading achievement in sheltered instruction classrooms: Linking professional development to student development. Literacy Research and Instruction, 49(4), 334351.

Moll, L. C., & Díaz, E. (1987). Change as the goal of education research. Anthropology & Education Quarterly, 184, 300311.

Nasir, N. S., Hand, V., & Taylor, E. V. (2008). Culture and mathematics in school: Boundaries between cultural and domain knowledge in the mathematics classroom and beyond. Review of Research in Education, 32(1), 187240.

Nieto, S., & Bode, P. (2008). Affirming diversity: The sociopolitical context of multicultural education (5th ed.). Boston, MA: Allyn & Bacon.

Northwest Evaluation Association. (2013a). Kentucky linking study: A study of the alignment of the NWEA RIT scale with Kentuckys Performance Rating of Educational Progress (K-PREP). Retrieved from http://www.nwea.org/sites/www.nwea.org/files/resources/KY_2012_LinkingStudy.pdf

Northwest Evaluation Association. (2013b). Measures of academic progress (MAP). Retrieved from http://www.nwea.org/products-services/assessments/help-all-kids-learn

Paris, D. (2012). Culturally sustaining pedagogy: A needed change in stance, terminology, and practice. Educational Researcher, 41(3), 9397.

Peña, D. C. (2000). Parent involvement: Influencing factors and implications. Journal of Educational Research, 94, 4254.

Pianta, R. C., Hamre, B. K., & Allen, J. P. (2012). Teacher-student relationships and engagement: Conceptualizing, measuring, and improving the capacity of classroom interactions. In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 365386). New York: Springer.

Powell, R. (1999). Literacy as a moral imperative: Facing the challenges of a pluralistic society. Lanham, MD: Rowman & Littlefield.

Powell, R., & Rightmyer, E. C. (Eds.) (2011). Literacy for all students: An instructional framework for closing the gap. New York: Routledge.

Reyes, M. R., Brackett, M. A., Rivers, S. E., White, M., & Salovey, P. (2012). Classroom emotional climate, student engagement, and academic achievement. Journal of Educational Psychology, 104, 700712.

Rickford, A. (2001). The effect of cultural congruence and higher order questioning on the reading enjoyment and comprehension of ethnic minority students. Journal of Education for Students Placed at Risk, 6(4), 357387.

Sleeter, C. E. (2011a). An agenda to strengthen culturally responsive pedagogy. English Teaching: Practice and Critique, 10(2), 723.

Sleeter, C. E. (2011b). The academic and social value of Ethnic Studies: A research review. Washington, DC: The National Education Association.

Stillman, J. (2011). Teacher learning in an era of high-stakes accountability: Productive tension and critical professional practice. Teachers College Record, 113, 133180.

Stubbs, M. (2002). Some basic sociolinguistic concepts. In L. Delpit & J. K. Dowdy (Eds.), The skin that we speak: Thoughts on language and culture in the classroom (pp. 6385). New York: The New Press.

Teemant, A., Wink, J., & Tyra, S. (2011). Effects of coaching on teacher use of sociocultural instructional practices. Teaching and Teacher Education, 27(4), 683693.

Tett, L. (2001). Parents as problems or parents as people? Parental involvement programmes, schools and adult educators. Journal of Lifelong Education, 20, 188198.

Thomas, W. P., & Collier, V. P. (2002). A national study of school effectiveness for language minority students long-term academic achievement. Santa Cruz, CA: Center for Research on Education, Diversity and Excellence, University of California-Santa Cruz.

Turney, K., & Kao, G. (2009). Barriers to school involvement: Are immigrant parents disadvantaged? The Journal of Educational Research, 102(4), 257271.

U. S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2006). Instructional Conversations and Literature Logs. Retrieved from http://ies.ed.gov/ncee/wwc/interventionreport.aspx?sid=236

Valenzuela, A. (1999). Subtractive schooling: U.S.-Mexican youth and the politics of caring. Albany, NY: SUNY Press.

Waxman, H. C., & Tellez, K. (2002). Research synthesis on effective teaching practices for English language learners. (Publication Series No. 3). Philadelphia: The Mid-Atlantic Regional Educational Laboratory at Temple University, Center for Research in Human Development and Education.

Young, E. (2010). Challenges to conceptualizing and actualizing culturally relevant pedagogy: How viable is the theory in classroom practice? Journal of Teacher Education, 61(3), 248260.

APPENDIX A

Sample Coding of Observation Field Notes

Note. CRIOP codes are as follows: Classroom Relationships = CR; Family Collaboration = FC; Assessment Practices = AP; Curriculum/Planned Learning Experiences = CPLE; Pedagogy/Instructional Practices =PIP; Discourse/Instructional Conversation = D/C; Sociopolitical Consciousness = SP.

Time	Objective Field Notes	Interpretations/ CRIOP Pillar
1:05	23 students. Teacher shows math talk text/chart on document projector. Ss seated on carpet, and T asks Ss to read the problem in their heads. T reads problem aloud and shows Skittles wrapper. T asks Ss to turn and talk to a partner about how to solve the problem. Text reads There are 20 Skittles in my bag. 5 are red. 5 are yellow. 3 are orange. How many are green? T asks Ss what is the problem? S responds that they have to guess how many are green. Student conversation as students talk with their partners.	Turn and talk; student engagement through conversation; CR; D/C
1:10	T uses call and response to get students attention. T asks Ss how they can help her solve this problem. S says use tally marks, or draw a picture, or solve an equation. T models drawing Skittles and Ss count Skittles by fives aloud as T draws them on the document camera. T asks Ss if shes done. Ss say no that she needs to color the Skittles. Ss use overlapping conversation to tell teacher how many of each color to color on the document camera. T asks Ss if they can think of an equation that she can use. S: struggles with answer, and T provides clarifying questions about the numbers in the problem.	Teacher modeling to solve math problem; formative assessment; P/IP; D/C; AP
1:15	Another S answers and T models writing 20 13 = 7 on document camera. T tells Ss that thats a subtraction problem, and if they can add to solve this problem. S answers 7 + 13 = 20. One student asleep at desk. (First grader in afternoon.) T demonstrates writing 5 + 5 + 3 = 13. T uses think aloud strategy as she models writing equations and writes 13 + ___ = 20. T affirms students by name who are paying attention.	Teacher demonstration; affirmation of student engagement; P/IP
1:20	T asks S who suggested using tally marks and tallies making 7 tally marks on document projector. T has high expectations and asks one S who is under table to come out and read the problem. T gives time for Ss to read the problem again. T re-reads problem aloud. T gives directions for math sheet and gives directions for completing math. T asks Ss to turn and talk with a partner about what theyre supposed to do. T dismisses Ss to tables and calls tables by color using Spanish for red, green, blue, and ?? Students move to six tables student conversation. T has distributed math sheets to each student seat and Ss get pencils to complete math problem. Students talk at each table and some solve equations or count aloud at their seats.	High expectations for students; turn and talk; teacher uses Spanish words to dismiss students to tables; CR
1:25	T circulates around the room and checks Ss work. T works with one student seated alone, reads problem and underlines parts of the worksheet in corresponding colors of Skittles. Ss help each other with task. Some students begin writing on math pages as conversation continues. Some Ss begin coloring Skittles on their papers. T asks one S: Whats the whole? Whats the total number? S answers and begins drawing and coloring Skittles on paper. T has positive relationship with students, and Ss cooperate with teacher.	Formative assessment; reteaches; teacher has positive rapport with students; AP; D/C; P/IP/ CR
1:30	Teacher asks Ss to put their heads down when theyre finished, but the whole table should have their heads down, because theyre supposed to be working together at their tables. T talks with 2 individual students on carpet. T tells Ss that shes going to give them five more minutes and to work together. T continues circulating around the room and checking student work. T works with one S and asks how many Skittles he has drawn, and how many he has to have. T tells S that he needs to ___, and S says add more. T checks back with S and asks if he has seven brown. T reads problem aloud and helps S color the correct number of each Skittle while pointing to the word problem.	Formative assessment; reteaches through questioning and scaffolding; AP; D/C; P/IP
1:35	T uses questioning with S to elicit what he needs to do. S says turn them green, and T affirms Ss answer. Students at tables have drawn and colored Skittles on their papers. S at one table: Guys, 10 are green. Students continue conversations while working on papers. Teacher continues circulating around the room and looking at students work. T asks one S what he does in the next box. T uses call and response, Class, class. Ss: Yes, yes. T calls Ss to carpet with their papers. Ss move from tables to front carpet with T seated in chair in front of class.	Formative assessment of individual student collaboration for task; AP; CR; P/IP
1:40	T asks Ss to put their papers in their laps right now. Ss have papers in their laps. T asks S to read the problem aloud. One S reads aloud. T has placed new math Skittles on document projector. T affirms S for putting paper in his lap and sitting quietly. One S gives answer aloud, and T puts her paper on the document camera. T describes what student did to solve the problem. T asks Ss to put a thumbs up in front of them if they got that. T asks for thumbs sideways if theyre not sure. T asks Ss what the whole is in this equation. T asks Ss if there are any numbers that she can put together easily in the problem. Eight hands raised. T circles 7 and 3 on overhead and explains how to solve the problem using Ss worksheet.	Engaging activity; formative assessment of whole class; reteaching; modeling; AP; P/IP
1:45	T affirms S by name for using tally marks to solve the problem as another way to solve the problem. T takes another Ss paper and displays paper, and explains how student missed the answer and how he could fix it. Students sitting quietly and listening to teacher during lesson. T asks Ss to take the papers home to show mom and dad what they learned in class. T refers to Ss as ladies and gentlemen and asks them to come back to the carpet. Student conversation as students move to carpet. T gives Ss two minutes to turn and talk about how to solve this problem. T moves two Ss who are lying on the floor and instructs them to turn and talk. Question on overhead is: There are 23 flowers in my yard. What is the problem? (Flower drawn next to problem.)	Teacher respect for students obvious through interactions; CR; AP; P/IP
1:50	T has high expectations most of the time. T affirms Ss for sitting and participating that they are good second graders. One S says that they could use tally marks. T says that we dont have a problem yet. We could use tally marks, but we dont know what to do with them. T tells another S that she needs some numbers. T says she doesnt know what to do with the flowers until they tell her. Two students give examples of possible math problems. T tells Ss that their answer is already up there. One S tells T that they have 19 flowers and they can plant 4 more and that will be 23. T tells S very good and gives him a high five. T asks if Ss can think of a subtraction problem. One S says she had 25 flowers, and two died, so she had 23.	High expectations; positive affirmation; CR
1:55	Ss give examples of subtraction problems using 23 flowers given in the initial question. T asks Ss to turn to a partner and tell them a problem based upon this story on the overhead. T asks two Ss if theyre ready to learn and pairs them with other Ss. T circulates around the room and listens to Ss answers. T stops and listens to Ss answers. T tells Ss that theyre now going to write a problem and make up a story to go with it to write in their math notebooks. T models writing a story on the overhead. T writes and reads aloud: There are 25 flowers. Two died. So how many are left? T writes 25 2 = 23 in middle column and draws pictures to demonstrate. T dismisses Ss by table to get out math notebooks and write.	Student engagement throughout activity; turn and talk with partners; formative assessment; teacher modeling/demonstrating; DC; P/IP; AP
2:00	Student conversation as they get out notebooks and begin writing. Ss work together. (Observation ends. 55 minutes.)	Cooperative learning activity

APPENDIX B

Sample Transcriptions with Inductive and A Priori Codes

Note. A Priori Codes of CRIOP Elements are as follows: Classroom Relationships = CR; Family Collaboration = FC; Assessment Practices = AP; Curriculum/Planned Learning Experiences = CPLE; Pedagogy/Instructional Practices =PIP; Discourse/Instructional Conversation = D/C; Sociopolitical Consciousness = SP.

Inductive Codes are as follows: Building Classroom Community = BCC, Respect for Students Cultures = RSC, Understanding Cultural Differences = UCD, Student Background Knowledge = SBK, Student Collaboration = SC, Improved Student Learning = ISL, Student Engagement = SE, Language Barriers Students = LBs; Language Barriers Parents = LBp, Parent Involvement = PI, Parent Communication = PC, Administrative Constraints = AC, Integrating Standards and Core Content = ISC, Lack of Knowledge of CRI = LKC.

Sample Interview #1

Interview question: How would you define culturally responsive instruction?

Respondent: I would define that as being conscious of all of the kids in your classroom of all the backgrounds of where they come from. Whether that is different kinds of families, whether that is different ethnicities, gender, you know? Whether they come from low income or high income and just making that, keeping that in the forefront of your lesson planning and making sure there are resources and your books, and your test material really is, just being mindful of those students, keeping in mind, making sure their culture is represented in everything we do in the classroom. CR/SBK

Interview question: What are your biggest successes with using culturally responsive instruction?

Respondent: I feel like, Im really more mindful of their backgrounds and just some decisions that I make. For instance when, its just been very eye opening, for instance when I had parent conferences, I had, just this Monday, I met with a few of my Hispanic families and we discussed homework. And I had one gentleman say, you know, I really cant help them with reading homework because I cant read in English. And so the resources that have been provided through the CRIOP, I have books now that are in Spanish and English. And so I pulled those out and said, you know, Its okay for you to not be able to read in English, because we want to celebrate that culture. And so they just lit up, and the mom, and I said to just pick one out and you can practice with him. And she was just happy that she could practice with him, and to know that it was okay to do that. And I said, As long as you are reading every day you are fine, that they know what it is supposed to say in Spanish, and then he can practice reading it in English and learn both of those languages. I think that is just going to make him an even stronger student, more confident in where he comes from. And so I feel like that has been eye opening. Just know, how my families, you know, feel about their culture. And I dont think that I took the time before, you know, to find out what my parents, you know, whether they spoke English or Spanish at home, you know, in the home? And so during this parent conference I asked another family, you know, Do you speak English at home, or Spanish at home, you know, because I was trying to understand, you know? Because the student that I am speaking of, he doesnt always understand what I am saying in class, and so I wanted to try to get a better grasp of, you know, where he is as a student. And so the dad said, Yeah some Spanish, some English. And so I could tell the dad like spoke a sort of broken English. And so I could see that that is what my student is picking up on. So hes hearing a little bit of Spanish, a little bit of English, and so he is coming to school with that, with that background. So, I just think that has been really eye opening to me. FC/ LBp/ LBs

Interview question: What are your biggest challenges with using culturally responsive instruction?

Respondent: Resources for one, but being a part of the CRIOP, I feel that has been a big help, getting books and materials and things like that. Also just having that barrier sometimes with the parents. One parent during the conference, I had to rely on an older sibling to communicate with mom. And so that is always, because I dont speak the language, and I took two years in high school of Spanish but, and thats the language we are talking about here at our school, I took two years of Spanish, you know, in high school. But if you dont use it, you know you lose that. And two years isnt really going to be anything that would help me really communicate with the parents, so that is kind of a big area. C/PLE / FC/ LBp

Sample Interview #2

Interview question: How would you define culturally responsive instruction?

Respondent: Culturally responsive instruction to me is just looking at what is important in other cultures and bringing in positive things from those cultures. So when we have studied some countries, I have made it a point to study some of those countries in Africa that my students were from. So, they could bring in things. One of them brought in a dvd and we watched it. So, you know, and they really enjoyed getting to share that. So I think getting to highlight the differences in cultures, just what makes us unique and why that is special. CR/ UCD

Interview question: What are your biggest successes with using culturally responsive instruction?

Respondent: I think when the kids have brought things from their culture. I have one student who speaks entirely Spanish, and when I have kids that can fill in with a Spanish word or tie in, I can see that she is confused, and they feel comfortable with doing that and feel empowered about helping her in her language. To me, that is a huge success. Like when one of the other students helps her, then that to me, it kind of gives me goosebumps because it is great, like they are really wanting to help her understand. CR/ SC

Interview question: What are your biggest challenges with using culturally responsive instruction?

Respondent: Definitely understanding different cultures. If you havent lived in another culture, I mean, I know what Ive read, and I know what Ive heard and things I have experienced, but I dont necessarily know, and I cant just assume that just because one student is from a Spanish speaking family that they are from Mexico or their culture is the exact same as another student, because culture varies, so I think understanding the different cultures. And then I think also when the kids are hesitant. CR/UCD

APPENDIX C

Sample Transcription Coding of Teacher Interviews

CRIOP Pillar	Examples of Teacher Responses
Classroom Relationships (CR)	I think the biggest successes I have had with being more culturally responsive is my classroom climate is wonderful. I feel like . . . I have built lots of confidence in my ELLs, because they feel like they are not only a part of the learning, but the teaching. And that makes them so proud. . . . A lot of the ELLs cant read in Spanish. They can speak it, but they cant read it. And so that exposure to that, it kind of has empowered them. . . . Also respecting where they come from and kind of telling them things about me that, you know, sharing experiences. I feel like those are definitely big successes, because I am very proud of this group of students. They have come a long way. (BCC) My kids have been able to start talking about, Oh, this is what my dad does, or, This is what goes on in my family. So, they are being able to tell each other stuff that is happening in their family, find similarities that they have. (BCC) When they start opening up their stories, they start saying Oh, well my family does that, too, but it just looks different. And its really interesting to see them engage in each others backgrounds and make those connections that I would have never been able to do, because I didnt know that. So just having them teach each other, as opposed to me just showing something, its really neat to see that. (SC) I think one of my biggest challenges would just be just trying to understand that everyone comes from something different, and Im trying, like I dont ever want to offend anybody. And that is just a challenge because you dont want to overstep your boundaries. So it is just a challenge, knowing that everyone is different and how you need to reach every one of those kids in the classroom. You know? That is a big responsibility on someones shoulders. (UCD)
Pedagogy/Instructional Practices (PIP)	I cannot assume that everybody is going to be on the same wavelength as I am. I am going to have to make sure that what I am saying and what I am doing, there is some kind of background that I am giving my students. So you are providing background, and I am explaining to, not dumbing anything down, but simplifying things enough for students to understand and then build from there. (SBK)   I think it (CRI) is just being mindful, not only of all cultures but of where kids come from. When kids come to this classroom, they come to me with varied backgrounds, varied experiences, and knowing that I need to design my instruction to meet the needs of those students and their cultures and their backgrounds and experiences. And, you know a lot of the kids at our school, unfortunately, dont have many opportunities and experiences. And so that really, I keep that in mind when I plan things . . . I think I have to be very mindful that they are not afforded those kinds of things, those kinds of opportunities, so like when we do a science experiment. And like vocabulary, that is a big thing. I just think a lot of their vocabulary is lacking because they dont have the experiences. And that is something we really try to work on this year, is to strengthen their vocabulary. (SBK) I think that he (ELL student) has found that its okay for him to talk in Spanish, and that he is able to understand, like reading stories or asking questions. Or Ill have my assistant, who is fluent in Spanish, ask him questions about the stories that weve read, and he can answer as long as he can answer in Spanish. . . . And we have talked to him about trying to build his vocabulary in English, so, and we are trying to help him with that by showing him a lot of pictures and, um, relating the pictures with the words. And Ive planned with our ESL teacher, we use the same pictures. So he is getting to see the exact same things, which will help him. But I think using those pictures has been helpful. (LBs) One of my students, he is still, absolutely no English. Actually about seven of my students couldnt speak any English. So I think just starting from scratch with them, think about how Id teach them, the pictures, allowing them to talk to each other in their native language, which is important. And you know, I speak a little Spanish, so talking with them was helpful. (LBs)
Curriculum/Planned Learning Experiences (CPLE)	I would define that as being conscious of all of the kids in your classroom, of all the backgrounds of where they come from, whether that is different kinds of families, whether that is different ethnicities, gender, you know? Whether they come from low income or high income, and just making that, keeping that in the forefront of your lesson planning. And making sure there are resources, and your books, and your test material really is, just being mindful of those students, keeping in mind, making sure their culture is represented in everything we do in the classroom. (SBK; RSC) I think we could still get more books and resources that were more, especially just knowing my population, Spanish to English books. I like the books that have it in English, but also have it in Spanish, because in first grade, theyre really learning to read. And so, the ones that are just in Spanish are not very helpful, so because we want them to be able to read in English. But I think also having it in Spanish is good for them too, because we want them to be bilingual. (LBs) I would like to have posters and pictures that are culturally diverse. Just having those resources and trying to find those. Even books, having good books that are culturally responsive. (RSC)
Family Collaboration (FC)	Weve also talked with parents about, we did a unit on community helpers, and we got some parents involved in that. A roofer came in and talked about what he does and brought his tools, which was really nice. And so the students got to see that their parents are important and what their job is, is important. We actually had a parent that worked at McDonalds, and we let them come in and talk about what they do and how they are a community helping. . . . (PI) Well, and I know all of my families work, and I would like to be able to bring them into the classroom more, because I used to always have parents in my room at my other school. And I would love for them to come in and participate more so its more of a community feeling in here, but I know that they are working all the time, they are busy, and so just finding ways for them to come in and participate would be good. (PI) We try to get parents to come in, but they work. We want to get people to come in to talk about weather, or something. Some cant pass the background check. It kind of eliminates half of our help. (PI)
Assessment Practices (AP)	I think the biggest challenge was something we talked about this weekend and I was very conscious about what I was saying today, because I have one student where I am not sure yet of whether he is really very, very low or its really such a language barrier to him. Still, and some days I think, well no, its just the language barrier, and then other days I think we maybe he is really that low. . . .And I really worked hard trying to draw him in, but he is very non-vocal, and it is a struggle to get him to answer, talk, and anyway, I am still at the point where I just dont know if this is a language barrier or if you are low, or if this is a combination of both. (LBs)
Discourse/Instructional Conversations (D/C)	. . . There is a lot I want to do next year, more language, sight words, more collaboration and making sure that they can both speak English and Spanish, but just making sure I can cultivate that environment for them to feel like they can do that. So, just discourse that they can do that. Doing call and response, so just next year just the discourse aspect, I need to grow on. Id like to have different phrases around the room that are in both languages, so they feel more at home. (BCC)

Note. Codes represent CRIOP holistic pillars: Classroom Relationships = CR; Family Collaboration = FC; Assessment Practices = AP; Curriculum/Planned Learning Experiences = C/PLE; Pedagogy/Instructional Practices = P/IP; Discourse/Instructional Conversations = D/C; Sociopolitical Consciousness = SC. Inductive Codes are as follows: Building Classroom Community = BCC, Respect for Students Cultures = RSC, Understanding Cultural Differences = UCD, Student Background Knowledge = SBK, Student Collaboration = SC, Improved Student Learning = ISL, Language Barriers Students = LBs; Language Barriers Parents = LBp, Parent Involvement = PI, Parent Communication = PC, Administrative Constraints = AC, Integrating Standards and Core Content = ISC, Lack of Knowledge of CRI = LKC.

Cite This Article as: Teachers College Record Volume 118 Number 1, 2016, p. 1-46 https://www.tcrecord.org ID Number: 18224, Date Accessed: 2/25/2022 2:42:36 AM Purchase Reprint Rights for this article or review