 Assessing Contemporary Notions of Mathematical Competenceby Melissa Gilbert - June 14, 2013 With the implementation of the Common Core State Standards for Mathematics (CCSSM), there is a shared definition of what it means to know mathematics across most of the United States. This commentary highlights two issues that must be considered and accommodated if we hope to develop meaningful and rich assessments of the contemporary definition of mathematical competence represented in the CCSSM. The first issue is to identify and distinguish among multiple types of mathematical knowledge and the second is to include multi-faceted measures of productive disposition. Assessments that address these issues have the potential not only to refocus implementation on curriculum and instruction rather than on testing and evaluation but also to provide the more nuanced understanding of students’ mathematical competence that is needed to be college and career ready in accordance with the CCSSM. Content standards define what it means to know mathematics and inform the development of measures of mathematics achievement. In turn, these standards and achievement measures affect the curricula developed and implemented in schools. Until quite recently, differences in content standards across the United States have posed a challenge for curriculum developers, educators, and policy makers. At the same time, federal initiatives tied to school funding such as No Child Left Behind (2002) and the Race to the Top initiative have compared performance across states despite a significant variation in content standards. The implementation of the Common Core State Standards for Mathematics (CCSSM, National Governors Association Center for Best Practices, Council of State School Officers, 2010) and accompanying assessment development efforts (by the SMARTER Balanced Assessment Consortium, SBAC, and the Partnership for Assessment of Readiness for College and Careers, PARCC) significantly changed this situation (Porter, McMaken, Hwang, & Yang, 2011). Now, the vast majority (44/50) of states will use the same math content standards and comparable achievement measures. This is especially important given a sociopolitical context that focuses on the preparation of a skilled workforce across the United States and that ties school resources to student performance per federal mandates. This commentary considers the CCSSM within the current sociopolitical context and presumes that the Standards for Mathematical Content and the Standards for Mathematical Practice that comprise the CCSSM represent contemporary notions of mathematical competence and define what mathematically proficient (CCSSM, 2010, p. 6) K12 students should understand and be able to do (p. 5) in order to be college and career ready (p. 57). The purpose of this commentary is to discuss two issues that must be considered and accommodated if we hope to develop strong, accurate, assessments that provide meaningful and rich detail about K-12 students' mathematical competence as defined by the CCSSM. IDENTIFY AND DISTINGUISH AMONG TYPE(S) OF MATHEMATICAL KNOWLEDGE As the reference to mathematically proficient students in the CCSSM highlights, a core feature of these standards is that students demonstrate the multiple types of mathematical knowledge outlined in the strands of mathematical proficiency (NRC, 2001). These five strands represent a comprehensive, multi-faceted notion of mathematical knowledge. Four of the strands reflect more traditional mathematical competencies-- Procedural fluency (ability to apply procedures); conceptual understanding (ability to grasp mathematical ideas); adaptive reasoning (ability to explain and justify mathematical processes); and strategic competence (ability to solve contextualized problems). The fifth strand, productive disposition, is defined as a habitual inclination to see mathematics as sensible, useful, and worthwhile, coupled with a belief in diligence and ones own efficacy (NRC, 2001, p. 116). This strand highlights students motivation to learn mathematics as an important part of knowing the subject. The strands of mathematical proficiency are reflected in the CCSSM. The Standards for Mathematical Content are analogous to more traditional state content standards organized by grade level for K8 and by conceptual category (e.g., algebra, geometry, functions) for high school. The Standards for Mathematical Practice describe what mathematically proficient students (p. 6) understand and do across grades and conceptual categories. As illustrated by the specific examples in Table 1 (Content Standards) and Table 2 (Practice Standards), the CCSSM presume that students will not only be able to do (procedural fluency) and understand (conceptual understanding) mathematics but also actively engage with the content (productive disposition) to explain their thinking (adaptive reasoning) and solve contextualized problems (strategic competence). Table 1. Examples of Explicit Connections to the Strands of Mathematical Proficiency in the CCSSM Content Standards
Table 2. Explicit Connections to the Strands of Mathematical Proficiency in the CCSSM Practice Standards
Source: CCSSM (2010) In an analysis focused on grades 36, the CCSSM standards place a greater focus than prior state standards on demonstrating understanding (35.65% vs. 28.66%, respectively), an aspect of cognitive demand that emphasizes conceptual understanding and adaptive reasoning and includes mathematical behaviors such as communicating new mathematical ideas and explaining relationships between concepts (Porter et al., 2011). Further, the references to solving a word problem or real-world problem in a third (36/108) of the grades 36 content standards illustrate the focus on strategic competence. Clearly, the CCSSM expect students to demonstrate multiple types of mathematical knowledge. At the same time, research suggests that there are differences among these types of knowledge that have ramifications for diagnosing students mathematical difficulties. For example, work undertaken by Fuchs and colleagues (Fuchs et al., 2010; Fuchs et al., 2008) distinguishes between procedural calculations (related to procedural fluency) and problem solving (related to strategic competence). They find that different constellations of cognitive abilities are required to demonstrate these types of knowledge and argue that assessments need to distinguish between them in order to properly diagnose students mathematical difficulties. Research also indicates that some types of mathematical knowledge are more difficult for students than others. For instance, to demonstrate adaptive reasoning, researchers have found that students need to communicate mathematically, both orally and in writing, and participate in mathematical practices, such as explaining solution processes, describing conjectures, proving conclusions, and presenting arguments (Moschkovich, 2002, p. 190, emphasis mine). Yet, explaining what they have done mathematically (i.e., adaptive reasoning) is more difficult for students than finding a solution (e.g., procedural fluency) (e.g., Evens & Houssart, 2004). These findings regarding differences by type of mathematical knowledge underscore the importance of the first issue that must be considered and accommodated when assessing mathematical competence: What type(s) of knowledge (based on the strands of mathematical proficiency) is/are being assessed? If assessing more than one type, can the types be distinguished? The SBAC assessment claims offer a promising starting point. These claims are broad evidence-based statements about what students know and can do as demonstrated by their performance on the assessments (SBAC, 2013, p. i). Separate items provide evidence for the different claims. These claims include concepts and procedures (conceptual understanding combined with procedural fluency), problem solving (strategic competence), and communicating reasoning (adaptive reasoning) (SBAC, 2013). Thus, the SBAC assessment system will distinguish between two of the five types of mathematical knowledge. The PARCC (2012) performance levels also reference multiple strands (e.g., solving problems correctly, expressing mathematical reasoning). However, there is no evidence that the PARCC items or rubrics will be able to distinguish among the strands. Without such discernment, it will not be possible to evaluate student progress according to each type of mathematical knowledge students must demonstrate in order to meet the expectations of the CCSSM. INCLUDE MEASURES OF PRODUCTIVE DISPOSITION As noted in the previous section, the strand of productive disposition highlights students motivation to learn mathematics. Affective correlates of mathematics achievement, including attitudes, motivations, and goal orientation, have a long history in assessing mathematical competence (e.g., Ames & Archer, 1988; Eccles et al., 1983; Fennema & Sherman, 1976), and continue to be the focus of contemporary studies (e.g., Berger & Karabenick, 2011; Keys, Conley, Duncan, & Domina, 2012; Yildirum, 2012). Research by type of mathematical knowledge suggests that aspects of productive disposition differentially relate to other types of mathematical knowledge. For example, Stipek and colleagues (1998) found that efficacy positively related to procedural but not conceptual knowledge (similar to the strands of procedural fluency and conceptual understanding, respectively). Certain aspects of productive disposition may also be more strongly related to types of mathematical knowledge that are more difficult for students. For example, greater efficacy and perceived usefulness may be needed to persist through the challenge of explaining ones thinking in order to demonstrate adaptive reasoning. The developers of the strands of mathematical proficiency included productive disposition as a type of mathematical knowledge because of the evidence of its relevance for students mathematical competence. The CCSSM Practice Standards emphasize mathematical attitudes and behaviors associated with productive disposition. Standard 1 (see Table 2) references task engagement, metacognitive strategy use (monitor and evaluate, change course), and sustained effort (persevere) expected throughout the study of elementary and secondary mathematics. Other Practice Standards reiterate the sense-making (habitual inclination to see mathematics as sensible) facet of productive disposition. For example, Standard 2 explains that reasoning quantitatively entails habits of creating a coherent representation of the problem at hand&attending to the meaning of quantities, not just how to compute them& (CCSSM, p. 6) while Standard 3 expects that students at all grades can decide whether their peers arguments make sense (p. 7). To engage with mathematics in this way, students must perceive themselves as capable of learning mathematics and that doing so is worthwhile (Wigfield & Cambria, 2010). The expectations of productive disposition in the CCSSM and the theoretical and empirical support for differential relationships between productive disposition and other strands of mathematical proficiency underscore the importance of the second issue when assessing mathematical competence: How is students productive disposition being assessed? Only the PARCC web site (http://www.parcconline.org/) currently provides any insights for how this strand will be measured in assessments. The PARCC focus is on perseverance, one of the mathematical attitudes and behaviors associated with productive disposition. The PARCC assessment developers explain that selecting all three possible correct solutions among the five choices provided in a task prototype (4th grade, Buses, Vans, and Cars) demonstrates perseverance. However, the rubric does not distinguish between two groups of students who may demonstrate different levels of productive disposition. Half credit is awarded to those who select only two of the three correct solutions as well as to those who select all three correct solutions as well as an additional incorrect solution. The latter group of students may have demonstrated greater perseverance since they found all three correct solutions. Thus, the one assessment system (PARCC) that discusses how to measure productive disposition currently focuses only on perseverance and not other facets such as efficacy, task engagement, and metacognitive strategy use. Clearly, measures of multiple mathematical attitudes and behaviors associated with productive disposition are needed to provide a complete assessment of students mathematical competence. Such measures will be particularly important given the higher cognitive demand of the CCSSM as compared with prior state standards. For example, consider prealgebra students who demonstrate comparable levels of procedural fluency who are then asked to analyze a peers argument (part of Practice Standard 3). In my own work I have found that a measure of productive disposition that considered multiple facets including efficacy and perceived usefulness was significantly associated with differences in the depth of students analysis. Thus, among the benefits of including measures of productive disposition is explaining differences among students who demonstrate similar levels of competence for certain types of mathematical knowledge but not others. CONCLUSION With the implementation of the Common Core State Standards for Mathematics, there is a shared definition of what it means to know mathematics across most of the United States within the current sociopolitical context. This commentary highlighted two issues related to providing meaningful and rich assessments of this definition of mathematical competence, namely the need to identify and distinguish among multiple type(s) of mathematical knowledge and to include multi-faceted measures of productive disposition. Considering and accommodating these issues has at least two benefits for K12 curriculum, instruction, and assessment. First, by providing rich and detailed information about students mathematical competence, assessments have the potential to refocus implementation of the CCSSM on curriculum and instruction rather than on testing and evaluation as some early supporters-turned-critics have argued (e.g., Burris, 2013). Second, we will all gain a more nuanced understanding of students mathematical competence that highlights areas of strength as well as areas for growth across the multiple types of mathematical knowledge. Such an understanding is sorely needed if we are to discern whether students are indeed making the necessary progress in mathematics to be college and career ready. References Ames, C., & Archer, J. (1988). Achievement goals in the classroom: Students' learning strategies and motivation processes. Journal of Educational Psychology, 80, 260267. Berger, J.-L., & Karabenick, S. A. (2011). Motivation and students' use of learning strategies: Evidence of unidirectional effects in mathematics classrooms. Learning and Instruction, 21, 416428. Burris, C. (2013, March 4). Principal: I was naïve about common core [Web log post]. Retrieved from http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/03/04/principal-i-was-naive-about-common-core/ Eccles, J., Adler, T. F., Futterman, R., Goff, S. B., Kaczala, C. M., Meece, J. L., . . . Midgley, C. (1983). Expectancies, values and academic behaviors. In J. T. Spence (Ed.), Perspectives on achievement and achievement motivation (pp. 75146). San Francisco: Freeman. Evens, H., & Houssart, J. (2004). Categorizing pupils' written answers to a mathematics test question: 'I know but I can't explain'. Educational Research, 46, 269282. Fennema, E., & Sherman, J. A. (1976). Fennema-Sherman Mathematics Attitudes Scales: Instruments designed to measure attitudes toward the learning of mathematics by females and males. JSAS Catalog of Selected Documents in Psychology, 6(31), Ms. No. 1225. Fuchs, L. S., Fuchs, D., Stuebing, K., Fletcher, J. M., Hamlett, C. L., & Lambert, W. (2008). Problem solving and computational skill: Are they shared or distinct aspects of mathematical cognition? Journal of Educational Psychology, 100, 3047. Fuchs, L. S., Geary, D. C., Compton, D. L., Fuchs, D., Hamlett, C., L., Seethaler, P. M., . . . Schatschneider, C. (2010). Do different types of school mathematics development depend on different constellations of numerical versus general cognitive abilities? Developmental Psychology, 46, 17311746. Keys, T. D., Conley, A. M., Duncan, G. J., & Domina, T. (2012). The role of goal orientations for adolescent mathematics achievement. Contemporary Educational Psychology, 37, 4754. Moschkovich, J. (2002). A situated and sociocultural perspective on bilingual mathematics learners. Mathematical Thinking and Learning, 4, 189212. National Governors Association Center for Best Practices, Council of State School Officers. (2010). Common core state standards for mathematics. Washington, DC: Author. National Research Council. (2001). Adding it up: Helping children learn mathematics. J. Kilpatrick, J. Swafford, & B. Findell (Eds.). Mathematics Learning Study Committee, Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press. No Child Left Behind Act of 2001, Pub. L. No. 107110, Rec. 1425. 115 Stat. (2002). Partnership for Assessment of Readiness for College and Careers (2012). PARCC College- and career-ready determination policy in English language arts/literacy and mathematics & policy-level performance level descriptors. Retrieved from http://www.parcconline.org/sites/parcc/files/PARCCCCRDPolicyandPLDs_FINAL.pdf Porter, A., McMaken, J., Hwang, J., & Yang, R. (2011). Common core standards: The new U.S. intended curriculum. Educational Researcher, 40, 103116. Smarter Balanced Assessment Collaborative (2013). Initial achievement-level descriptors and college content-readiness policy. Retrieved from http://www.smarterbalanced.org/wordpress/wp-content/uploads/2012/11/Smarter-Balanced-Math-ALDs.pdf Stipek, D. J., Salmon, J. M., Givvin, K. B., Kazemi, E., Saxe, G., & MacGyvers, V. L. (1998). The value (and convergence) of practices suggested by motivation research and promoted by mathematics education reformers. Journal for Research in Mathematics Education, 29, 465488. Wigfield, A. & Cambria, J. (2010). Students' achievement values, goal orientations, and interest: Definitions, development, and relations to achievement outcomes. Developmental Review, 30, 135. Yildirum, S. (2012). Teacher support, motivation, learning strategy use, and achievement: A multilevel mediation model. The Journal of Experimental Education, 80, 150172.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||