Use of Visual Displays in Research and Testing: Coding, Interpreting, and Reporting Data
reviewed by Manuel S. Gonzȧlez Canchė - November 08, 2016
Title: Use of Visual Displays in Research and Testing: Coding, Interpreting, and Reporting Data
Author(s): Matthew T. McCrudden, Gregory Schraw, & Chad W. Buckendahl (Eds.)
Publisher: Information Age Publishing, Charlotte
ISBN: 1681231018, Pages: 346, Year: 2015
Search for book at Amazon.com
The development of innovative ways to visualize data has been prompted by the availability of large amounts of information along with increases in computing power. Education professionals and researchers alike who want to successfully summarize and explain quantitative or qualitative information benefit from investing in data visualization skills. In this respect, Use of Visual Displays in Research and Testing: Coding, Interpreting, and Reporting Data, edited by Matthew T. McCrudden, Gregory Schraw, and Chad W. Buckendahl, is an excellent resource for professionals and researchers interested in developing strong foundations in visual displays. It is worth noting that visual displays and data visualization are different constructs, but share the purpose of facilitating the comprehension and understanding of information. As is stated in the book, visual displays are graphic representations of information, whereas data visualization represents the underlying structure of data. From this view, visual displays may be representations of conceptual maps and/or frames of reference that guide qualitative, quantitative, or mixed-methods research to convey shared understandings without the need to use data. Accordingly, the most important difference between these constructs is that visual displays are not necessarily fed by qualitative or quantitative data, whereas data visualization techniques necessarily are. However, data visualization approaches should be guided by visual display principles as they aim for the effective and efficient transmission of information from researcher to reader for the purpose of facilitating knowledge, comprehension, and learning acquisition. From this perspective, the editors correctly note that the intended audience is quantitative, qualitative, and mixed-methods researchers in education interested in using visual displays to communicate their findings, information, and frames of reference in an effective and comprehensible way.
The organization of the book sections, and 11 chapters in total, flows naturally and provides readers with a structured and organized progression. It moves from a strong understanding of design principles and guidelines to real examples found in the literature with one possible exception detailed below. Readers who are new to visual display approaches will particularly benefit from reading the book in order. The second section of the book begins with Chapter Two where authors Gutierrez, Schraw, and Stefik delineate four theoretical frameworks and design principles typically used in visual display research. Lanes Chapter Three provides a more technical perspective that covers guidelines typically employed by graphical designers. The focus of the authors discussion is on displaying data points and emphasizing distributions across comparison groups. Lane excels at showing the relevance of using box plots as opposed to bar charts. However, more explanation is required for the pieces configuring each box plot as opposed to simply referring to other authors who have provided such explanations. Particularly valuable from Lanes work is the description of guidelines and integration of text in quantitative visual displays. Bringing Chapters Two and Three together, it is clear that even more seasoned readers of visual display techniques may benefit from the information provided by these authors.
The second section of the book closes with Chapter Four where Schraw and Gutierrez offer a data-driven classification of visual displays used in empirical research in the Journal of Education Psychology during the period 2010 to 2014. The authors careful attention to the methodological steps to build their typology is an important contribution and fits well with the purpose of the book. The main shortcoming of this chapter is the journal selected, which may not be representative of the education research spectrum due to its high level of specificity. Aside from this shortcoming, the methodological steps followed in this chapter are an important contribution and can be applied to other journal outlets to identify typologies of visual displays.
The third section of the book encompasses four chapters dealing with the use of visual displays of quantitative, qualitative, and mixed-methods research. Pastor and Finneys Chapter Five aims to discuss visual displays used in quantitative research, yet the authors completely focus on path diagrams typically employed in Structural Equation Modeling, thus narrowing its scope. The authors clearly convey that the use of path diagrams is not only theoretically-driven, but also has an empirical component where fit indices measure the extent that gathered data behave similarly to the theoretical model, making this technique an excellent teaching resource. In Chapter Six, Guetterman, Creswell, and Kuckartz show how the use of joint displays, which integrate quantitative and qualitative findings into a single visual display (usually in table format), can lead to comprehensive and holistic views of mixed-methods findings. The authors feature the MAXQDA software that ranges in price from around $950 to $2,500 for a perpetual single-user license. In this respect, the authors could have also reviewed freeware options that render the same or even more powerful analyses. This includes R for Qualitative Data Analysis, which can also integrate network visualization techniques using Network Analysis of Qualitative Data (NAQDA). The main advantage of NAQDA is that authors can use mathematical techniques to find structure in text data relying on network analysis principles (Gonzȧlez Canchė, 2016a), an approach yet to be broadly implemented in education research.
In terms of organization, perhaps readers would benefit more by reading Piano and Sanders’ Chapter Seven before reading Chapter Six. The latter is more practical in offering concrete templates and even resources such as MAXQDA. Although Chapter Seven is the product of an analysis of published articles as was done in Chapter Four, the authors successfully define visual displays used in mixed-methods research such as joint displays. In this view, the templates presented in Chapter Six are particularly well-informed by the information and recommendations discussed in Chapter Seven. Chapter Eight, by Feucht, Marchand, and Olafson, details the use of conceptual maps as strategies guiding qualitative research or facilitating shared understanding during instruction. Although the authors present very comprehensive information and examples, the main shortcoming of this chapter may be its dearth of discussion on using conceptual maps in quantitative studies, which represents a gap in this type of education research. An example of how conceptual maps can be used in quantitative research involves the criteria selection of analytic samples that lead to the identification of treatment and control groups (see Gonzȧlez Canchė, 2017, Figures 2 and 3 for an example). These decision diagrams ease clarification of analytic decisions and convey transparency in the types of conclusions and implications offered by quantitative researchers as a function of analytic decisions.
The final section of the book is the most quantitative in nature. It is interesting to note that the editors refer to this section as dealing with testing and assessment data, whereas from a broader education research perspective it is simply referred to as quantitative data on academic performance. In Chapter Nine, Zvoch and Stevens discuss regression discontinuity (RD) design and interrupted time series (ITS) as reasonable alternatives to experiments. While the authors present explanations and visual examples of RDs, readers may have equally benefited from a similar exposure to ITS examples. The authors also present the results of growth modeling, which is a conceptually-driven variation of multilevel modeling. In traditional multilevel modeling, individuals are nested within schools, whereas time is nested within individuals in growth models. This conceptual difference is omitted from the text. Nonetheless, the authors successfully showcase visual displays of their results. In Chapter Ten, Foley discusses best practices related to the use of visual displays to illustrate uncertainty by relying on confidence intervals or error bands that show what can be considered unexplained variation or noise in measurement efforts. The visuals presented by Foley are excellent examples of visual displays that incorporate textual explanations as depicted in the first section of the book. The author also shows a map where the coloring scheme serves to illustrate achievement scores across states. It is worth noting that Chapter Six also contains a geocoded visual display, a technique that remains underused in education research and represents an important area where researchers should invest more time and effort. The book closes with Chapter Eleven where Zenisky presents examples of visual displays of test data at the student-, group-, and state-levels and shows innovative strategies where comparisons at other levels can be conducted (e.g., region, state). Similar to the authors of Chapters Six and Ten, Zenisky also shows how visual displays can be geocoded to highlight state-level performance. It is worth noting that although the use of maps is important as a descriptive tool, spatial statistics techniques such as Morans I allow researchers to test for spatial dependence, a technique that remains underused in education research (see Gonzȧlez Canchė, 2014, 2016b, 2016c for examples of georeferenced visual displays and spatial statistics techniques).
In closing, Use of Visual Displays in Research and Testing comprises an excellent resource regarding best practices and frameworks involved in the design of visual displays. I would have valued more clarification regarding differences between data visualization and visual displays, the most important being that data visualization necessarily requires data, whereas the latter does not. At the center of most chapters discussion is the need to train researchers in the production of effective visual displays. In this respect, it should be mentioned that the book is not intended as a tutorial; researchers will not develop the skills required to generate their own visual displays by studying this text. This statement is especially true regarding the development of data visualization skills. Nonetheless, individuals interested in developing conceptual maps (from both quantitative and qualitative perspectives) and following important guidelines when selecting data visualization displays will benefit greatly from this books content. Overall, the message is clear: given the availability of large amounts of data, graduate programs in education should continue investing in the development of researchers' visual-display skills. This training will make them more marketable and also benefit the field in general.
Gonzȧlez Canchė, M. S. (2014). Localized competition in the non-resident student market. Economics of Education Review. 43, 21–35.
Gonzȧlez Canchė, M. S. (2016a). Network analysis of qualitative data (NAQD): Putting knowledge to action to achieve the promise of equal educational opportunity. Workshop Presented at the Higher Education Research Institute, UCLA, June 2016. 1–36. Retrieved from http://ihe.uga.edu/uploads/main/NAQD.pdf
Gonzȧlez Canchė, M. S. (June 2016b). The heterogeneous non-resident student body: Measuring the effect of out-of-state students’ home-state wealth on tuition and fee price variations. Research in Higher Education. 1–43. Retrieved from http://link.springer.com/article/10.1007/s11162-016-9422-2
Gonzȧlez Canchė, M. S. (2016c). Geographical network analysis and spatial econometrics as tools to enhance our understanding of student migration patterns and benefits in the U.S. higher education network. Review of Higher Education. 1–63. Retrieved from http://ihe.uga.edu/uploads/main/Geographical_Network_Analsys_and_Student_Mobility_msgc.pdf
Gonzȧlez Canchė, M. S. (2017). Financial benefits of rapid student loan repayment: An analytic framework employing two decades of data. The ANNALS of the American Academy of Political and Social Science. 1–39. Retrieved from http://ihe.uga.edu/uploads/main/Rapid_Repayment_Main_Document_msgc.pdf