Subscribe Today
Home Articles Reader Opinion Editorial Book Reviews Discussion Writers Guide About TCRecord
transparent 13

Reconciling Data from Different Sources: Practical Realities of Using Mixed Methods to Identify Effective High School Practices

by Thomas M. Smith, Marisa Cannata & Katherine Taylor Haynes - 2016

Background/Context: Mixed methods research conveys multiple advantages to the study of complex phenomena and large organizations or systems. The benefits are derived from drawing on the strengths of qualitative methods to answer questions about how and why a phenomenon occurs and those of quantitative methods to examine how often a phenomenon occurs and establish generalizable, empirical associations between variables and outcomes. Though the literature offers many strategies, designing mixed methods research can be challenging in large scale projects when trying to balance reliability, validity, and generalizability. By supporting the findings with multiple forms of evidence mixed methods designs lend greater validity than mono-method ones. However to draw on the comparative advantages of these two paradigms, researchers must grapple with the challenges of working with more than one method.

Focus of Study: This paper discusses the benefits and challenges of collecting and interpreting mixed methods data in a large scale research and development project. Drawing on existing frameworks, we reflect on our strategies of mixed methods design, data collection, and analysis. We discuss the quandaries faced by researchers when discrepant findings emerge.

Research Design: The data come from a large, mixed methods case study focused on the practices that explain why some high schools in large urban districts are particularly effective at serving low income students, minority students, and English language learners. Undertaken in several phases, the work included sequential and concurrent designs. Incorporating a sequential explanatory design element, we first used quantitative data to identify schools in the district that were more and less effective at improving student achievement in English/language arts, mathematics, and science. We then used a combination of interviews, focus groups, surveys, classroom observations, and district administrative data—in a concurrent design—to try to understand what differentiated between the most and least effective schools in the district.

Conclusions: Based on our analyses, we provide examples of when mixed methods data converge, when they diverge but are complementary, and when they diverge and introduce a methodological quandary for researchers who must confront seemingly discrepant findings. In so doing, we discuss the tradeoffs encountered between the study design and the implications as we confronted them during analysis and suggest ways to balance the methodological demands of complex research studies. Seemingly discrepant findings, while challenging to reconcile, when considered for their potential complementarity, actually lead to a more complete understanding of the phenomena under study.

Keywords: mixed methods, discrepant findings, school reform, case study

To view the full-text for this article you must be signed-in with the appropriate membership. Please review your options below:

Store a cookie on my computer that will allow me to skip this sign-in in the future.
Send me my password -- I can't remember it
Purchase this Article
Purchase Reconciling Data from Different Sources: Practical Realities of Using Mixed Methods to Identify Effective High School Practices
Individual-Resource passes allow you to purchase access to resources one resource at a time. There are no recurring fees.
Become a Member
Online Access
With this membership you receive online access to all of TCRecord's content. The introductory rate of $25 is available for a limited time.
Print and Online Access
With this membership you receive the print journal and free online access to all of TCRecord's content.

Cite This Article as: Teachers College Record Volume 118 Number 7, 2016, p. 1-34
https://www.tcrecord.org ID Number: 20515, Date Accessed: 9/22/2021 10:17:38 PM

Purchase Reprint Rights for this article or review
Article Tools
Related Articles

Related Discussion
Post a Comment | Read All

About the Author
  • Thomas Smith
    University of California Riverside
    E-mail Author
    THOMAS M. SMITH is Dean and Professor in the Graduate School of Education at the University of California Riverside and Executive Director of the National Center on Scaling Up Effective Schools. His research interests include the impact of policy on teaching and learning, teacher development and retention, and continuous improvement research. Recent publications include “Understanding Differences in Instructional Quality between High and Low Value Added Schools in a Large Urban District” in Teachers College Record and “Research-Practice Partnerships to Support the Development of High Quality Mathematics Instructional Practices for All Students to Learn” in the Journal of Education for Students Placed at Risk.
  • Marisa Cannata
    Vanderbilt University
    E-mail Author
    MARISA CANNATA is Research Assistant Professor in the Department of Leadership, Policy, and Organizations at Peabody College of Education and Human Development, Vanderbilt University and Director of the National Center on Scaling Up Effective Schools. Her research interests include continuous improvement research, high school reform, charter schools, and teacher hiring and career decisions. Dr. Cannata is co-editor of School Choice and School Improvement (Harvard Education Press) and the forthcoming National Society for the Study of Education Yearbook, Mapping the High School Reform Landscape.
  • Katherine Haynes
    Vanderbilt University
    E-mail Author
    KATHERINE TAYLOR HAYNES is a senior research associate in the Department of Leadership, Policy, and Organizations at Peabody College, Vanderbilt University and the Assistant Director of the National Center on Scaling Up Effective Schools. Her current research focuses on the social context of education and public policy, school improvement, the implementation and scale-up of district designed school innovations and continuous improvement. She is co-author with Robert Crowson and Ellen Goldring of Successful School and the Community Relationship: Concepts and Skills to Meet Twenty-First-Century Challenges, published by McCutchan Publishing Corporation.
Member Center
In Print
This Month's Issue