Who’s Preparing our Candidates? edTPA, Localized Knowledge and the Outsourcing of Teacher Evaluation
by Alison G. Dover, Brian D. Schultz, Katy Smith & Timothy J. Duggan - March 30, 2015
This essay examines the impact of outsourcing teacher preparation and evaluation through high stakes teacher performance assessments like edTPA. In addition to undermining teacher preparation by marginalizing the local experts best situated to evaluate candidates’ performance, this has led to a growing industry of edTPA-related services. The authors use their own experiences with edTPA “coaching” and scoring to illustrate the inevitable consequences of shifting teacher preparation and evaluation to the private sector.
Despite widespread critique of the philosophical and empirical validity of standardized teacher performance assessment (Madeloni & Gorlewski, 2013a; Reid & Brennan, 2013), increasing numbers of states and accrediting bodies are requiring high-stakes assessments like edTPA for teacher licensure and teacher education program approval. Developed by Stanford University (via the Stanford Center for Assessment, Learning and Equity, or SCALE), the American Association of Colleges of Teacher Education (AACTE) and Pearson Education, edTPA is a requirement of teacher licensure in twelve states and the long-term expectation is that institutions of higher education, state education boards and professional standards boards throughout the United States will adopt edTPA as a mandatory requirement for the award of an education degree and/or for teacher licensure (AACTE, 2014b).
This trend reflects neoliberal rhetoric regarding the mediocre job performance of colleges of education (United States Department of Education, 2009), partnerships between regulatory bodies like AACTE and CAEP and corporate entities like Pearson Education (the sole administrator of edTPA), and proposed Federal Teacher Preparation Regulations that narrow measurement of teacher quality to items easily evaluated through standardized tests (Kumashiro, 2015). However, despite federal emphases on oversight in the name of rigor and accountability, the rise of teacher performance assessment undermines teacher preparation by marginalizing the local experts best situated to evaluate candidates performance, transforming student teaching from an educative experience to a prematurely evaluative one.
Moreover, the outsourcing of teacher evaluation has created cottage industriesincluding those related to the tutoring and scoring of edTPAthat are entirely unregulated and have the potential for disproportionate impact on underrepresented candidates. In this essay, we use a conversation between one of the authors and a private edTPA tutoring company to illustrate the inevitable consequences of shifting teacher preparation and evaluation to the private sector. While this move towards a market-driven educational commercialism is not entirely new, it represents a significant departure from the emphasis on localized knowledge and expertise that teacher education programs have long developed and practiced. Many scholars (e.g., Apple, 2006; Boyles, 2005; Lipman, 2011; Molnar, 2005; Saltman, 2012) have documented the perils of using students, schools, and more broadly education, particularly public education, as sites for making profit, commodification, and privatization. It is within these grave arguments in which we situate our critique.
OUTSOURCING OF CANDIDATE EVALUATION
Framed as an impartial process that requires candidates to actually demonstrate the knowledge and skills required to help all students learn in real classrooms (AACTE, 2014a), edTPA relies upon anonymous reviewers who assess candidates readiness based solely on their submission of three to five days of written instructional materials, two 10 minute video clips of their teaching, and approximately 25 pages of narrative analysis. Unlike local assessments of candidates, which involve multifaceted, contextual, and longitudinal evaluation by multiple school- and community-based educators, edTPA scorers determine proficiency based on a single, high-stakes snapshot. Candidates receive their scoresbut no feedbackthree to four weeks after submitting their portfolios, and $300, to Pearson Education. Moreover, while official edTPA guidelines encourage teacher education faculty to explicitly teach to the requirements of edTPA, faculty and cooperating teachers are prohibited from coaching or providing explicit feedback to candidates during the assessment process itself (SCALE, 2014). Given the time required to develop, teach, and reflect upon their edTPA portfolios, this means that many candidates are evaluated based upon materials created during their first weeks in the classroom.
As a high-stakes assessment intended to promote rigor and accountability among candidates and teacher educators alike, the developers of edTPA consider its relatively low pass rate (58-78%, depending upon the cut score used) evidence of a useful assessment. However, despite their claim that, prior to adoption, edTPA underwent two years of field-testing with 12,000 teacher candidates (AACTE, 2014a), published data from those field tests reveal that only 3,669 candidates successfully submitted materials for all 15 rubrics (SCALE, 2013). Of these candidates, 82% were White and only 3% spoke a primary language other than English. No data are available regarding the socioeconomic status of candidates, the universities they attended, or the demographics of their student teaching placement. However, SCALEs own research on earlier iterations of edTPA found statistically significant differences in candidates scores according to the racial and socioeconomic context of their student teaching placement, with candidates in suburban schools scoring higher than those in urban or inner-city schools (Pecheone & Chung, 2006, p. 29).
As faculty in a public, Hispanic Serving Institution that prepares culturally and linguistically diverse candidates for careers in urban education, we are troubled by the potential for a disparate impact on our candidates. Test developers and advocates dont share our concern. Pecheone & Chung (2006), for example, ultimately conclude thatdespite their findings regarding the impact of candidate placementteacher performance assessments are a valid measure of individual teacher competence for the purpose of teacher licensure and as a powerful tool for teacher learning and program improvement (p.22).
It is into this context that private edTPA tutoring and preparation companies emerged. In order to better understand the role these companies can play in candidate preparation, the authors of this articleposing as a candidate seeking assistancecontacted a private company that claims to offer step-by-step support and constructive feedback to candidates throughout their edTPA process. During the course of our 45 minute chat with this company, they quoted a price of $885 (which they reduced to $500 if paid by the end of the day) to rewrite part of a candidates edTPA portfolio, help the candidate select video clips, and continue revising the portfolio until the candidate passed. Moreover, they indicated that they had already assisted 50 candidates with edTPA and had a 100% pass rate, in part because some of their tutors were SCALE-trained edTPA scorers who knew how to effectively align portfolios to the official rubrics. The company with whom we spoke isnt unique: similar services are advertised on Craigslist, YouTube, and a growing number of personal websites. The market is ready to take advantage of the opportunity the outsourcing of teacher evaluation creates.
The availability of private coaching services raises multiple concerns regarding the validity of edTPA preparation and assessment. This companys assertion that it can ensure a passing score by revising candidates portfolios to include what scorers look for highlighted one of the pitfalls of external, standardized assessment of a contextual, reflective process like teaching. The edTPA scoring processes are necessarily more reflective of fidelity to the nuances of the task than to the quality of candidates instruction and assessment. Despite Pearsons claim that scorers are well qualified, training for edTPA scorers consists primarily of calibration activities; a practice that makes it easy for companies like this one to claim that passing edTPA is more about knowing what they want than the quality of a candidates teaching.
In order to better understand the training process for edTPA scorers, one of the authors of this paper applied for a position scoring edTPAs for Pearson. Ironically, current or retired P-12 teachers, university faculty, and student teaching supervisorsthe same people considered unqualified to evaluate their own candidatesare the target recruits for edTPA scoring. The author was offered the job after a five minute telephone interview that included no discussion of curriculum, pedagogy, student learning, or any other aspect of teacher preparation. Had she accepted the job, Pearson would have provided 19-24 hours of calibration training (compensated at $10 per hour), after which time she would have to spend four to six hours per week, over a period of at least three months, scoring edTPAs. She would be compensated $75 for each 50-80 page portfolio she scored; Pearson suggests scorers spend two to three hours on each portfolio. The low rate of compensation and high time commitment suggests that the available pool of scorers will include far more new, adjunct, retired, and underemployed educators than full-time teacher educators. It is no wonder that some edTPA scorers moonlight with companies like the one we contacted.
The financial incentive to score as many portfolios as possible in as short a time as possible also calls into question the expected thoroughness of edTPA review. Scoring protocols require edTPA scorers to highlight specific artifacts related to each rubric (look here, for that) and provide evidence (in the form of excerpts from the portfolio or video timestamps) to support their scores. However, in official scorer training materials provided to our university, evaluators considered the inclusion of specific artifacts, such as graphic organizers, evidence of a candidates proficiency without regard to the quality or usefulness of those artifacts.
Critics of edTPA cite concerns regarding the standardized assessment of a contextual process like teaching, and even TPA advocates admit the impact of classroom factors on candidates final scores (e.g., Pecheone & Chung, 2006). In describing edTPA implementation at his university, Au (2013) notes that edTPA effectively sanitized much of our students' work by limiting what they thought would be acceptable within the confines of the standardized test. In the process, many of my students felt they couldn't demonstrate what they were capable of and who they were as teachers (n.p.). Madeloni and Gorlewski (2013b) cite similar concerns among their candidates, describing one who was invited to complete his student teaching in a challenging urban classroom with a teacher for whom he had tremendous respect and admiration. . . but anticipating the judgment of an objective distant scorerone who might not understand why the classroom was not filled with compliant, well-behaved learnersmade him hesitate to accept (n.p.).
Rather than facilitating candidates reflection on complex questions of teaching and learning, the outsourcing of teacher assessment artificially decontextualizes teaching practice and encourages candidates to teach to the test. The lack of assessment of scorer credentials, standardized scoring protocols, and underestimation of the time required to comprehensively review candidates materials further de-professionalize this process. Under these conditions, scorers cannot comprehensively evaluate teacher performance, and thus the tutor we contacted was right to assure our candidate that if it looks good you can get the passing score. In its current incarnation, edTPA is neither educator preparation nor a comprehensive assessment of candidates readiness for the classroom.
At least the entire process can be subverted for the bargain price of $500.
Apple, M. W. (2006). Educating the right way: Markets, standards, God and inequality. New York: Routledge.
American Association of Colleges of Teacher Education. (2014a). edTPA Overview. Retrieved from http://edtpa.aacte.org/about-edtpa
American Association of Colleges of Teacher Education. (2014b). State policy FAQs. Retrieved from http://edtpa.aacte.org/state-policy
Au, W. (2013). Whats a nice test like you doing in a place like this? The edTPA and corporate education reform. Rethinking Schools, 27(4). Retrieved from: http://www.rethinkingschools.org/archive/27_04/27_04_au.shtml
Boyles, D. R. (Ed.). (2005). Schools or markets?: Commercialism, privatization, and school-business partnerships. Mahwah: Lawrence Erlbaum Associates.
Kumashiro, K. (2015). Review of proposed 2015 federal teacher preparation regulations. Boulder, CO: National Education Policy Center. Retrieved from http://nepc.colorado.edu/files/ttr10-tchrprepregs_0.pdf
Lipman, P. (2011). The new poetical economy of urban education: Neoliberalism, race, and the right to the city. New York: Routledge.
Madeloni, B. & Gorlewski, J. (2013a). Radical imagination, not standardization: Critical teacher education and the edTPA. Teachers College Record. Date Published: June 21, 2013. http://www.tcrecord.org ID Number: 17163.
Madeloni, B. & Gorlewski, J. (2013b). Wrong answer to the wrong question: Why we need critical teacher education, not standardization. Rethinking Schools, 27(4). Retrieved from http://www.rethinkingschools.org/archive/27_04/27_04_madeloni-gorlewski.shtml
Molnar, A. (2005). School commercialism: From democratic ideal to market commodity. New York: Routledge.
Pecheone, R. L., & Chung, R. R. (January/February 2006). Evidence in teacher education: The Performance Assessment for California Teachers (PACT). Journal of Teacher Education, 57(1), 2236.
Reid, J., & Brennan, M. (2013). The standards cage: A contradictory politics of control. In Lori Beckett (Ed.), Teacher education through active engagement: Raising the professional voice (111124). New York: Routledge.
Saltman, K. (2012). The failure of corporate school reform. Boulder, CO. Paradigm.
Stanford Center for Assessment, Learning and Equity [SCALE]. (2013). 2013 edTPA field test: Summary report. Stanford, CA: Author.
Stanford Center for Assessment, Learning and Equity [SCALE]. (2014) edTPA guidelines for acceptable candidate support. Stanford, CA: Author. Retrieved from https://www.edtpa.com/content/docs/guidelinesforsupportingcandidates.pdf.
United States Department of Education. (2009). Teacher preparation: Reforming the uncertain professionremarks of Secretary of Education Arne Duncan at Teachers College, Columbia University. Retrieved from http://www2.ed.gov/news/speeches/2009/10/10222009.html