Evidence-Based Reform and No Child Left Behind: Next Time, Use What Works

by Robert E. Slavin - December 12, 2006

No Child Left Behind appeared to be a major victory for evidence-based reform in education, but it has instead been a major setback. Despite language throughout NCLB calling for the use of scientifically evaluated programs, such programs have in fact been largely shut out of Reading First and ignored in parts of the law such as supplemental educational services and turnaround programs for schools not meeting standards. This article recommends strategies to make evidence central to the reauthorization of NCLB. These include adding clarity about which programs have strong evidence of effectiveness and providing competitive preference points for proposals to implement proven programs.

In 2002, the U.S. Congress passed the No Child Left Behind Act. It contained a radical idea, long overdue: that schools receiving federal dollars should use programs and practices that have been proven to be effective in scientifically based research.

Evidence-Based Reform and NCLB: The Dream

NCLB appeared to be a great victory for proponents of evidence-based reform in education. Evidence-based reform means the adoption of policies that promote the use of programs that have been successfully evaluated in rigorous experiments. NCLB famously mentioned scientifically based research as a basis for what schools should use more than 100 times, in many parts of the law. It provided a detailed definition of scientifically based research.

In theory, at least, the idea of evidence-based reform is a no-brainer. Use what works. Who could disagree with that? Why shouldnt education finally join medicine, agriculture, and technology in embracing evidence as the basis for practice? At long last, here was the President of the United States and both parties of Congress openly endorsing the idea that evidence should matter in education, and supporting legislation designed to encourage schools and districts receiving federal funds to use programs with strong evidence of effectiveness.

Evidence-Based Reform and NCLB: The Reality

In practice, however, NCLB has been a major setback for evidence-based reform. In every area of the law in which evidence could have mattered, it did not, and if anything, programs with strong evidence of effectiveness were discouraged rather than encouraged. The winners in NCLB were the old-fashioned large publishers and other large companies, whose products lack evidence from rigorous experiments.

The flagship program for evidence-based reform in NCLB was Reading First, a $1 billion per year program designed to give high-poverty schools proven reading programs to use in grades K-3. Instead, Reading First money has gone primarily to traditional basal textbooks lacking any evidence of effectiveness, while programs that do have such evidence, such as our Success for All program (Slavin & Madden, 2001), and Direct Instruction (Adams & Engelmann, 1996) were largely shut out (Moss, Jacob, Boulay, Horst, & Poulos, 2006). In September 2006, the Department of Educations Inspector General (Office of the Inspector General, 2006) issued a scathing report on Reading First, documenting how department officials deliberately bent the law to favor certain programs and discourage others, without regard to evidence. Press reports have shown how the departments Reading First technical assistance contractors had serious conflicts of interest. The leaders of two of the three centers were on the design team for one of the most widely adopted remedial programs under Reading First, and were authors of one of the major basal textbooks (Manzo, 2006; Grunwald, 2006).

In other parts of NCLB, evidence requirements were equally ignored. Schools that failed to meet their adequate yearly progress (AYP) goals on state tests were encouraged to adopt proven turnaround programs, such as comprehensive school reform models with strong evidence. Not only has this provision been ignored, but the Bush administration also pushed for the abolishment of a Comprehensive School Reform funding program that had been helping schools adopt such models. Schools not meeting AYP were also supposed to provide supplemental educational services to children who needed them, usually in the form of group remedial instruction after school. Despite the laws discussion of using remedial programs with strong evidence, there has been no effort to focus SES on proven programs (Ascher, 2006).

Putting Evidence Back Into NCLB

At this writing, Congress is beginning to debate the reauthorization of NCLB. The fate of evidence-based reform is very much in the balance. Some argue that the widely acknowledged failure of evidence to affect NCLB shows that evidence-based reform is simply unworkable in federal legislation. Clearly, a new approach is needed. Earlier legislation establishing Comprehensive School Reform and the Reading Excellence Act also had language encouraging the use of proven programs, and this language also had little effect.

In order for legislation to genuinely encourage the use of programs with strong evidence of effectiveness, it is not enough to simply say that it should. The legislation needs to name names of proven programs, or describe mechanisms for doing so. The FDA does not just say to use proven medicines; it says use penicillin, dont use Laetrile. Lacking specificity about which programs have been proven in rigorous research is an invitation to cronyism, as lower-level federal, state, or local officials can decide for themselves what they consider to be proven, without open review or discussion.

Fortunately, the research and review of research have advanced considerably since NCLB was passed, and there will soon be several independent, trustworthy sources of information on effective programs. The long-delayed What Works Clearinghouse (2006), a federal initiative, will soon issue reports on effective programs in reading, elementary, and middle school mathematics, programs for English language learners, and so on. The federally funded Comprehensive School Reform Quality Center (CSRQ, 2006a and CSRQ, 2006b) has recently issued scientific reviews of research on comprehensive school reform models in elementary and secondary schools. The Best Evidence Encyclopedia (CDDRE, 2006), also part of a federal grant, summarizes reviews from many sources on effective programs for grades K-12, and contributes its own reviews using standards similar to those of the What Works Clearinghouse.

The reauthorization of NCLB should take advantage of these developments and begin to specify that schools should use programs that have been rigorously and successfully evaluated, as certified by the What Works Clearinghouse and other reviews that use similar standards. Schools should not be required to use proven programs, but rather, be given incentives to do so, such as a competitive preference of up to 10 points on a 100-point scale for schools applying to use programs with strong evidence (see Slavin, 2006). Schools should be free to use other programs, but the government has a legitimate interest in encouraging them to use federal funds on proven programs. This approach could work for the next round of Reading First, supplemental educational services, and turnaround programs for schools not meeting AYP. Funding for comprehensive school reform should be revived, but focused on programs with strong evidence of effectiveness. According to the CSRQ (CSRQ, 2006a and CSRQ, 2006b) reviews, there are more than a dozen CSR programs with substantially better evidence of effectiveness than that supporting any of the textbooks favored under Reading First. Beyond NCLB, the same approach could work in any area in which the federal government provides funding to schools to adopt programs that lend themselves to evaluation, such as Striving Readers (funding for secondary literacy programs) and E-Rate (funding to help schools purchase technology). In each case, in order to get their bonus points, schools would have to agree to implement their chosen program completely, with levels of resources and professional development comparable to those provided to schools in the studies establishing the programs effectiveness.

Effective policies promoting the use of proven programs would have an immediate benefit for children who receive better instruction, but they would also have a dramatic effect on the entire R&D enterprise. Publishers, developers, and others would begin to subject their programs to rigorous evaluations and to develop programs more likely to succeed in those evaluations. This could finally set education on the road toward genuine progress.

This paper was written under funding from the Institute of Education Sciences, U.S. Department of Education (Grant No. R305A040082). However, any opinions expressed are those of the author, and do not necessarily represent IES policies.

References

Adams, G.L., & Engelmann, S. (1996). Research on direct instruction: 25 years beyond DISTAR. Seattle, WA: Educational Achievement Systems.

Ascher, C. (2006). NCLBs supplemental educational services: Is this what our students need? Phi Delta Kappan, 88(2), 136-141.

Center for Data-Driven Reform in Education. (2006). The best-evidence encyclopedia. Retrieved November 30, 2006 from www.bestevidence.org.

Comprehensive School Reform Quality Center (2006a). CSRQ Center report on elementary school comprehensive school reform models. Washington, DC: American Institutes for Research. www.csrq.org/CSRQreportselementaryschoolreport.asp

Comprehensive School Reform Quality Center (2006b). CSRQ Center report on middle and high school comprehensive school reform models. Washington, DC: American Institutes for Research. www.csrq.org/MSHSreport.asp

Grunwald, M. (2006). Billions for an inside game on reading. Retrieved October 1, 2006 from http://www.washingtonpost.com/wp-dyn/content/article/2006/09/29/AR2006092901333.html.

Manzo, K. (2006). Federal review of Reading First identifies serious problems. Retrieved September 22, 2006 from http://www.edweek.org.

Moss, M., Jacob, R., Boulay, B., Horst, M., & Poulos, J. (2006). Reading First implementation evaluation: Interim report. Cambridge, MA: Abt.

Office of the Inspector General (2006). The Reading First programs grant approval process: Final inspection report. Washington, DC: U.S. Department of Education, Office of the Inspector General.

Slavin, R.E. (2006, October 18). Research and effectiveness: A '10 percent solution' that can make evidence-based form a reality. Education Week.

Slavin, R.E., & Madden, N.A. (Eds.) (2001). One million children: Success for All. Thousand Oaks, CA: Corwin.

Cite This Article as: Teachers College Record, Date Published: December 12, 2006 https://www.tcrecord.org ID Number: 12887, Date Accessed: 10/16/2021 7:04:53 AM Purchase Reprint Rights for this article or review