Home Articles Reader Opinion Editorial Book Reviews Discussion Writers Guide About TCRecord
transparent 13
Topics
Discussion
Announcements
 

A Grand Experiment in Reading Instruction: Interim Report 2


by Dick Schutz - January 17, 2014

The UK and the US are implementing very different models for teaching reading, constituting a Natural Experiment. Preliminary results of the experiment as of June 2013 are reported.

The experiment is a natural planned variations design involving two treatment models: the UK model and the US model.


UK MODEL


The UK government is committed to teaching all children to read by the end of Year/Grade 2 using alphabetic codebased instruction termed “synthetic phonics.”  A 40item screening check marks the progress in accomplishing this goal.  The test is administered to all children at the end of Year/Grade 1 and administered again at the end of Year/Grade 2 to children who did not pass the check the previous year.  The test was rolled out in June 2012.  The test is low stakes, about 10 minutes or less per child, and is administered by classroom teachers as a routine instructional activity.


US MODEL


The US government’s “Race to the Top” initiative extends reading instruction from Kindergarten to Grade 12.  The commitment is that all students will graduate from high school “college and careerready” by 2020.  Common Core State Standards for English Language Arts & Literacy in History/Social Studies, Science, and Technical Subjects have been formulated, and “new and better, next-generation” tests, to be administered annually in Grades 3–8 and 11, are being constructed.  The tests will be rolled out at the end of the 2014–2015 school year.  The tests are high stakes with security precautions, and the results are to be used, at least in part, to evaluate teachers for tenure and compensation purposes.


EVENTS IN THE UNITED KINGDOM


The specification Framework for the screening check and the Pilot Study results of the test administered in June 2011 were described in this journal in a First Interim Report (Schutz, 2012).  The Framework specifications for the check make it possible to use a new parallel form of the test each year.  


Table 1 shows the 40 items for 2012 and 2013.  The check is divided into two sections, both to provide some degree of success for all students and to provide diagnostic information for teachers regarding additional instruction that may be needed.  The test is scored by teachers as the total of the 40 items a student can read.


The check constitutes a Guttman-like scale rather than the comparative scale of conventional standardized achievement tests.  That is, a capable reader can read all 40 items, and any lesser performance falls short.  However, as with the Snellen eye test of visual acuity used in driver licensing, it is reasonable to cut some slack from 20-20 “perfect vision” to 20-40 “pass.”   A “pass score” on the check has been set at 32 items.  

Table 1

Screening Check Items, 2012 and 2013

Section 1

Section 2

 

2012

2013

 

2013

2013

1

pib

fot

21

kigh

quigh

2

vus

keb

22

girst

herks

3

yop

gan

23

baim

jorb

4

elt

ulp

24

yune

zale

5

desh

poth

25

flods

bluns

6

chab

shan

26

groiks

skarld

7

poil

veen

27

strom

splot

8

queep

quong

28

splaw

strabe

9

stin

drap

29

fair

toy

10

proom

flarm

30

flute

spike

11

sarps

lect

31

goat

fuel

12

thend

voisk

32

shine

name

13

chip

thin

33

crept

props

14

jazz

peck

34

shrubs

spoilt

15

farm

torn

35

scrap

scram

16

thorn

cheek

36

stroke

strike

17

stop

trap

37

index

panic

18

truck

snarl

38

turnip

second

19

jump

milk

39

waiting

trantrum

20

lords

moist

40

portrait

reach


Results are now available for Year/Grade 1 administrations of the check in June 2012 and 2013:

https://www.gov.uk/government/publications/phonics-screening-check-and-national-curriculum-assessments-at-key-stage-1-in-england-2012


https://www.gov.uk/government/publications/phonics-screening-check-and-national-curriculum-assessments-at-key-stage-1-in-england-2013


Overall, 69% Year/Grade 1 children “passed” the check in 2013 compared to 58% in 2012.   In each year, the modal score was 40, a “perfect” score, and the overall distribution of scores was shifted upward in 2013 as shown in the graph in Figure 1 (BishopBlog, 2013).  


Figure 1

[39_17386.htm_g/00002.jpg]

What has received most notice in the UK is the “dramatic jump” in scores at 32, the cut-score for “pass.”  However, this “anomaly” is an expected consequence any time a “cut-score” is set.  The decision to set the cut- score at 32, although reasonable, was arbitrary. Disregarding the “anomaly,” the tails of the distribution show a “dramatic jump” in 2013 over 2012.  At the upper end of the distribution, 45% scored in the 36-40 range in 2013 compared to 34% in 2012.  At the low end, 4% scored in the 0-10 range, compared to 13% in 2012


When the results are disaggregated into the “usual” demographic subgroups of interest, we find the usual “gaps” as shown in Table 2.  Girls outperform boys.   Students not eligible for free meals outperform those who are eligible.  Students without “Special Needs” outperform those with such needs.   The “news” is that students for whom English is an additional language perform at exactly the same level as native English-speaking students.  EAL students in the UK may well encounter differential instructional obstacles, but insofar as reading per se is concerned, the UK reading initiative has made these students as “ready for advanced instruction” as their native English-speaking peers.     


Table 2

Subgroup Results.  2012 and 2013




 

Met Standard

2012 (%)

Met Standard

2013 (%)

Gender

   

Male

54

65

Female

62

73

Free Meal

Eligible

   

No

63

72

Yes

44

56

Special Needs

   

No SEN

65

76

SEN

24

32

English an

Additional

Language

   

English

58

69

Other

58

69



The UK Government has actively promoted the formation of “Academies,” which are similar to US “Charter Schools.” “Sponsored Academies” are schools which were low-performing, similar to US “turnaround schools.” “Converter Academies” voluntarily became an Academy to gain freedom in curriculum and financial matters.  “Free Schools” are established at local parent initiative.    The performance of Academies compared to non-Academies on the 2013 check is shown in Table 3.


Table 3

 Yr/Grade 1 Screening Check Performance in Academies/ Charters, 2013


 

Schools

% Pass

Non-Academy

14,764

69

Sponsored Academy

    145

63

Converter Academy

     591

75

Free Schools

       32

71


Severing LEA ties, per se, does not appear to be a paramount factor in screening check performance.


The prime purpose of administering the screening check to students at the end of Year/Grade 1 is to identify students needing further “intervention” instruction in Year/Grade 2. The check is administered at the end of Year/Grade 2 to all students who did not pass the check the previous year. The combined Yr 1 2012 and Yr 2 2013 pass rate in 2013 is 85%.  With “Special Educational Needs” students excluded, the pass rate is 92%, and 55% of SEN students passed the check.   Some children are being “left behind” in reading at this point in the experiment, but there is reason to expect further improvement.


There is considerable variability in pass rates among Local Educational Authorities.  LEA pass rates for Year/Grade 1&2 combined in 2013 range from 70% to 92%.  The correlation between LEA Year/Grade 1 pass rates in 2012 and 2013 is .59.  The “Intervention gains” correlation is .21 (correlation between the 2012 Yr 1 and the 2013 Yr 1 &2-combined pass rates).  This variability reflects professional association opposition to the check:


The largest teachers’ union, NUT: “schools believe the check provides no new information on pupils’ ability and that phonics should be used alongside other methods in the teaching of reading. This is indeed what happens now. . .The phonics check is clearly a waste of money” (2013).


The school principals’ association, NAHT:  “In keeping with reducing bureaucracy, the phonics check is unnecessary and should be removed” (2013).


The UK literacy association, UKLA:  “The Phonics Screening Check is not fit for purpose. . . Schools overwhelmingly felt that the check did not give any information that they did not already know, and it took valuable teaching time” (2013).


There is no doubt also wide variability in performance at the school level.  School level results are available on the Internet to educational authorities but are not publically available, and no analysis has yet been conducted of school level data.  Data on school purchases of government-subsidized instructional materials and professional development training of teachers have also been collected but have not as yet been reported or analyzed.  Additional analysis would respond to uninformed opposition to the check and would also provide useful instructional information for schools and teachers.


EVENTS IN THE UNITED STATES


The United States is implementing the four assurances of Race to the Top:


Increase teacher effectiveness and address inequities in the distribution of highly qualified teachers.

Establish and use pre-K through college and career data systems to track progress and foster continuous improvement.

Make progress toward rigorous college- and career-ready standards and quality assessments.

Support targeted, intensive support and effective interventions to turn around schools identified for corrective action and restructuring.


Insofar as reading instruction is concerned, the operative considerations are the Common Core State Standards for English Language Arts & Literacy in History/Social Studies, Science, and Technical Subjects and the “new and better tests” that are being developed under the auspices of two consortia of states.


The new tests will not be rolled out until the end of the 2014–2015 school year.  However, results are now available for two states, Kentucky and New York, that have administered Common Core-aligned tests:


In fall 2010, Kentucky became the first of 45 states to adopt the Common Core, making the state a test case for the standards. So far, Kentucky’s experience over the past three school years suggests it will be a slow and potentially frustrating road ahead for the other states that are using the Common Core. Test scores are still dismal, and state officials have expressed concern that the pace of improvement is not fast enough. Districts have also seen varying success in changing how teachers teach, something that was supposed to change under the new standards.


The first tests based on the Common Core standards were administered in Kentucky in spring of 2012, at the end of the first year of full implementation. Testing the harder standards produced worse results. Proficiency ratings were about 30 percentage points lower than they had been the year before. The same drop was seen in New York this spring when it became the second state to test under the new standards.    


The news was only slightly better for Kentucky this year [2013] . . . Statewide only about 40 percent of students scored at least proficient in math and about 50 percent in reading. And the gap has increased between the percentage of white students who are proficient and the percentage of African Americans (Butrymowicz, 2013).


Meanwhile, events are creating additional opportunities for variations in the natural experiment. Secretary of Education Arne Duncan has granted waivers to states and to one coalition of school districts seeking relief from No Child Left Behind requirements, which would have labeled most, if not all schools in the state “failing” in 2014.   To receive the waiver, applicants had to agree to adopt college- and career-ready standards, focus on 15% of their most troubled schools, and create guidelines for teacher evaluations based in part on student performance.  These stipulations are repeats of Race to the Top assurances the states had already made, but granting the waivers essentially nullified the NCLB commitment to teach all children to read by 2014-15.


In addition, whereas most states have adopted the national (Common Core State) standards, some have not; some are reneging on their adoption, and others are likely to do so as the cost of the implementation looms closer.


The “Waiver-Non Waiver” and the “Core- Non Core” states will constitute two treatment variations in the US Model in the natural experiment.


Some states and local education agencies received School Improvement Grants (investing in Innovation i3), which varied in amount, with some as much as $50 million. These “innovations” will constitute a third category of treatment variations.


Finally, what has not happened may well shape the entire US treatment.  The US Congress, for all practical purposes, has deferred reauthorization of the Elementary and Secondary Education Act until after the congressional elections of 2014, or possibly until after the presidential election of 2016. The election results will be far more consequential in determining the shape of the reauthorization and the outcome of The Race to the Top than any professional or technical consideration.


CODA


The United Kingdom and the United States are on very different reading instruction trajectories.  The natural experiment is in progress, but the results are still preliminary.  Further reports will be forthcoming.


References


BishopBlog. (2013, October 5). Good news and bad news on the phonics screen. Retrieved from http://deevybee.blogspot.com/2013/10/good-and-bad-news-on-phonics-screen.html


Butrymowicz, S. (2013, October 15). What Kentucky can teach the rest of the U.S. about the Common Core. The Atlantic. Retrieved from http://www.theatlantic.com/education/archive/2013/10/what-kentucky-can-teach-the-rest-of-the-us-about-the-common-core/280453/


NAHT. (2013, October 29). NAHT response to Primary Accountability Assessment Consultation. Retrieved from http://www.naht.org.uk/welcome/news-and-media/key-topics/assessment/primary-assessment-and-accountability-naht-response/


National Union of Teachers. (2013, May 23). Evaluation of the phonics screening check: First interim report. Politics.co.uk. Retrieved from http://www.politics.co.uk/opinion-formers/national-union-of-teachers-nut/article/nut-evaluation-of-the-phonics-screening-check-first-interim1

Schutz, D. (2012). A grand educational experiment in reading instruction: Interim report 1.  Teachers College Record, Date Published: June 01, 2012  http://www.tcrecord.org ID Number: 16786.


Schutz, R. (2012). A grand experiment in reading instruction: Interim report 1. Teachers College Record, Retrieved from http://www.tcrecord.org/content.asp?contentid=16786


UK Literacy Association. (2012, October). UKLA analysis of schools’ response to the Year 1 Phonics Screening Check. Retrieved from http://www.teachers.org.uk/files/y1psc-survey-october-2012.pdf





Cite This Article as: Teachers College Record, Date Published: January 17, 2014
https://www.tcrecord.org ID Number: 17386, Date Accessed: 11/29/2021 9:09:08 AM

Purchase Reprint Rights for this article or review
 
Article Tools
Related Articles

Related Discussion
 
Post a Comment | Read All

About the Author
  • Dick Schutz
    3RsPlus
    E-mail Author
    DICK SCHUTZ is CEO of 3RsPlus, Inc. a firm conducting R&D and constructing educational products. He was formerly Professor of Educational Psychology at Arizona State University and Executive Director of the Southwest Regional Laboratory for Educational Research and Development. He has served as the founding editor of the Journal of Educational Measurement, the founding journal editor of the Educational Researcher, and editor of the American Educational Research Journal. His recent technical papers can be accessed from the Social Science Research Network http://ssrn.com/author=1199505
 
Member Center
In Print
This Month's Issue

Submit
EMAIL

Twitter

RSS