A Grand Experiment in Reading Instruction: Interim Report 2
by Dick Schutz - January 17, 2014
The UK and the US are implementing very different models for teaching reading, constituting a Natural Experiment. Preliminary results of the experiment as of June 2013 are reported.
The experiment is a natural planned variations design involving two treatment models: the UK model and the US model.
The UK government is committed to teaching all children to read by the end of Year/Grade 2 using alphabetic code‐based instruction termed “synthetic phonics.” A 40‐item screening check marks the progress in accomplishing this goal. The test is administered to all children at the end of Year/Grade 1 and administered again at the end of Year/Grade 2 to children who did not pass the check the previous year. The test was rolled out in June 2012. The test is low stakes, about 10 minutes or less per child, and is administered by classroom teachers as a routine instructional activity.
The US governments Race to the Top initiative extends reading instruction from Kindergarten to Grade 12. The commitment is that all students will graduate from high school college and career‐ready” by 2020. Common Core State Standards for English Language Arts & Literacy in History/Social Studies, Science, and Technical Subjects have been formulated, and “new and better, next-generation” tests, to be administered annually in Grades 3–8 and 11, are being constructed. The tests will be rolled out at the end of the 20142015 school year. The tests are high stakes with security precautions, and the results are to be used, at least in part, to evaluate teachers for tenure and compensation purposes.
EVENTS IN THE UNITED KINGDOM
The specification Framework for the screening check and the Pilot Study results of the test administered in June 2011 were described in this journal in a First Interim Report (Schutz, 2012). The Framework specifications for the check make it possible to use a new parallel form of the test each year.
Table 1 shows the 40 items for 2012 and 2013. The check is divided into two sections, both to provide some degree of success for all students and to provide diagnostic information for teachers regarding additional instruction that may be needed. The test is scored by teachers as the total of the 40 items a student can read.
The check constitutes a Guttman-like scale rather than the comparative scale of conventional standardized achievement tests. That is, a capable reader can read all 40 items, and any lesser performance falls short. However, as with the Snellen eye test of visual acuity used in driver licensing, it is reasonable to cut some slack from 20-20 perfect vision to 20-40 pass. A pass score on the check has been set at 32 items.
Results are now available for Year/Grade 1 administrations of the check in June 2012 and 2013:
Overall, 69% Year/Grade 1 children passed the check in 2013 compared to 58% in 2012. In each year, the modal score was 40, a perfect score, and the overall distribution of scores was shifted upward in 2013 as shown in the graph in Figure 1 (BishopBlog, 2013).
What has received most notice in the UK is the dramatic jump in scores at 32, the cut-score for pass. However, this anomaly is an expected consequence any time a cut-score is set. The decision to set the cut- score at 32, although reasonable, was arbitrary. Disregarding the anomaly, the tails of the distribution show a dramatic jump in 2013 over 2012. At the upper end of the distribution, 45% scored in the 36-40 range in 2013 compared to 34% in 2012. At the low end, 4% scored in the 0-10 range, compared to 13% in 2012
When the results are disaggregated into the usual demographic subgroups of interest, we find the usual gaps as shown in Table 2. Girls outperform boys. Students not eligible for free meals outperform those who are eligible. Students without Special Needs outperform those with such needs. The news is that students for whom English is an additional language perform at exactly the same level as native English-speaking students. EAL students in the UK may well encounter differential instructional obstacles, but insofar as reading per se is concerned, the UK reading initiative has made these students as ready for advanced instruction as their native English-speaking peers.
Subgroup Results. 2012 and 2013
The UK Government has actively promoted the formation of Academies, which are similar to US Charter Schools. Sponsored Academies are schools which were low-performing, similar to US turnaround schools. Converter Academies voluntarily became an Academy to gain freedom in curriculum and financial matters. Free Schools are established at local parent initiative. The performance of Academies compared to non-Academies on the 2013 check is shown in Table 3.
Yr/Grade 1 Screening Check Performance in Academies/ Charters, 2013
Severing LEA ties, per se, does not appear to be a paramount factor in screening check performance.
The prime purpose of administering the screening check to students at the end of Year/Grade 1 is to identify students needing further intervention instruction in Year/Grade 2. The check is administered at the end of Year/Grade 2 to all students who did not pass the check the previous year. The combined Yr 1 2012 and Yr 2 2013 pass rate in 2013 is 85%. With Special Educational Needs students excluded, the pass rate is 92%, and 55% of SEN students passed the check. Some children are being left behind in reading at this point in the experiment, but there is reason to expect further improvement.
There is considerable variability in pass rates among Local Educational Authorities. LEA pass rates for Year/Grade 1&2 combined in 2013 range from 70% to 92%. The correlation between LEA Year/Grade 1 pass rates in 2012 and 2013 is .59. The Intervention gains correlation is .21 (correlation between the 2012 Yr 1 and the 2013 Yr 1 &2-combined pass rates). This variability reflects professional association opposition to the check:
The largest teachers union, NUT: schools believe the check provides no new information on pupils ability and that phonics should be used alongside other methods in the teaching of reading. This is indeed what happens now. . .The phonics check is clearly a waste of money (2013).
The school principals association, NAHT: In keeping with reducing bureaucracy, the phonics check is unnecessary and should be removed (2013).
The UK literacy association, UKLA: The Phonics Screening Check is not fit for purpose. . . Schools overwhelmingly felt that the check did not give any information that they did not already know, and it took valuable teaching time (2013).
There is no doubt also wide variability in performance at the school level. School level results are available on the Internet to educational authorities but are not publically available, and no analysis has yet been conducted of school level data. Data on school purchases of government-subsidized instructional materials and professional development training of teachers have also been collected but have not as yet been reported or analyzed. Additional analysis would respond to uninformed opposition to the check and would also provide useful instructional information for schools and teachers.
EVENTS IN THE UNITED STATES
The United States is implementing the four assurances of Race to the Top:
Increase teacher effectiveness and address inequities in the distribution of highly qualified teachers.
Establish and use pre-K through college and career data systems to track progress and foster continuous improvement.
Make progress toward rigorous college- and career-ready standards and quality assessments.
Support targeted, intensive support and effective interventions to turn around schools identified for corrective action and restructuring.
Insofar as reading instruction is concerned, the operative considerations are the Common Core State Standards for English Language Arts & Literacy in History/Social Studies, Science, and Technical Subjects and the new and better tests that are being developed under the auspices of two consortia of states.
The new tests will not be rolled out until the end of the 20142015 school year. However, results are now available for two states, Kentucky and New York, that have administered Common Core-aligned tests:
In fall 2010, Kentucky became the first of 45 states to adopt the Common Core, making the state a test case for the standards. So far, Kentuckys experience over the past three school years suggests it will be a slow and potentially frustrating road ahead for the other states that are using the Common Core. Test scores are still dismal, and state officials have expressed concern that the pace of improvement is not fast enough. Districts have also seen varying success in changing how teachers teach, something that was supposed to change under the new standards.
The first tests based on the Common Core standards were administered in Kentucky in spring of 2012, at the end of the first year of full implementation. Testing the harder standards produced worse results. Proficiency ratings were about 30 percentage points lower than they had been the year before. The same drop was seen in New York this spring when it became the second state to test under the new standards.
The news was only slightly better for Kentucky this year  . . . Statewide only about 40 percent of students scored at least proficient in math and about 50 percent in reading. And the gap has increased between the percentage of white students who are proficient and the percentage of African Americans (Butrymowicz, 2013).
Meanwhile, events are creating additional opportunities for variations in the natural experiment. Secretary of Education Arne Duncan has granted waivers to states and to one coalition of school districts seeking relief from No Child Left Behind requirements, which would have labeled most, if not all schools in the state failing in 2014. To receive the waiver, applicants had to agree to adopt college- and career-ready standards, focus on 15% of their most troubled schools, and create guidelines for teacher evaluations based in part on student performance. These stipulations are repeats of Race to the Top assurances the states had already made, but granting the waivers essentially nullified the NCLB commitment to teach all children to read by 2014-15.
In addition, whereas most states have adopted the national (Common Core State) standards, some have not; some are reneging on their adoption, and others are likely to do so as the cost of the implementation looms closer.
The Waiver-Non Waiver and the Core- Non Core states will constitute two treatment variations in the US Model in the natural experiment.
Some states and local education agencies received School Improvement Grants (investing in Innovation i3), which varied in amount, with some as much as $50 million. These innovations will constitute a third category of treatment variations.
Finally, what has not happened may well shape the entire US treatment. The US Congress, for all practical purposes, has deferred reauthorization of the Elementary and Secondary Education Act until after the congressional elections of 2014, or possibly until after the presidential election of 2016. The election results will be far more consequential in determining the shape of the reauthorization and the outcome of The Race to the Top than any professional or technical consideration.
The United Kingdom and the United States are on very different reading instruction trajectories. The natural experiment is in progress, but the results are still preliminary. Further reports will be forthcoming.
BishopBlog. (2013, October 5). Good news and bad news on the phonics screen. Retrieved from http://deevybee.blogspot.com/2013/10/good-and-bad-news-on-phonics-screen.html
Butrymowicz, S. (2013, October 15). What Kentucky can teach the rest of the U.S. about the Common Core. The Atlantic. Retrieved from http://www.theatlantic.com/education/archive/2013/10/what-kentucky-can-teach-the-rest-of-the-us-about-the-common-core/280453/
NAHT. (2013, October 29). NAHT response to Primary Accountability Assessment Consultation. Retrieved from http://www.naht.org.uk/welcome/news-and-media/key-topics/assessment/primary-assessment-and-accountability-naht-response/
National Union of Teachers. (2013, May 23). Evaluation of the phonics screening check: First interim report. Politics.co.uk. Retrieved from http://www.politics.co.uk/opinion-formers/national-union-of-teachers-nut/article/nut-evaluation-of-the-phonics-screening-check-first-interim1
Schutz, D. (2012). A grand educational experiment in reading instruction: Interim report 1. Teachers College Record, Date Published: June 01, 2012 http://www.tcrecord.org ID Number: 16786.
Schutz, R. (2012). A grand experiment in reading instruction: Interim report 1. Teachers College Record, Retrieved from http://www.tcrecord.org/content.asp?contentid=16786
UK Literacy Association. (2012, October). UKLA analysis of schools response to the Year 1 Phonics Screening Check. Retrieved from http://www.teachers.org.uk/files/y1psc-survey-october-2012.pdf