The need for more education R&D has been brought to the fore by our seven-year struggle to implement the No Child Left Behind Act (NCLB). Our current approach to education accountability assumes that districts and schools, newly armed with test-score data and motivated by NCLB’s incentives and sanctions, will be able to make the changes that are necessary to improve education processes. But there is a crucial gap between wanting to improve a school, and knowing how to improve it.

NCLB exhorts educators to adopt approaches grounded in “scientifically based research” (the phrase appears 110 times in the law). But the education sector has been notoriously lax when it comes to research and documentation of the effectiveness of different educational approaches (and is therefore vulnerable to reform fads). Educators have plenty of ideas about what might work—but good evidence to support their intuitions is generally hard to come by.

The “knowing what we don’t know” problem extends to accountability systems themselves. Policymakers and educators alike look to assessment results to inform their decision-making. Yet we know far too little about the validity of current accountability practices. And there’s been far too little research on how accountability systems are developed and implemented. How are national, state, and local government agencies working together to develop and implement effective accountability systems — here and abroad? What capacities do they need to do accountability right? We don’t know. But we need to.

Federal role

“While we spend roughly $400 billion annually in this country on public education,” President-elect Obama’s proposal reads, “we spend less than seven tenths of one percent of that—$260 million—figuring out what actually works. By comparison, the Department of Defense spends roughly ten percent of its annual budget on [R&D].” [2] Why should increases in education R&D funding come from the federal government? Although education is traditionally a state and local enterprise, R&D is a special case requiring federal support. Like national defense (President-elect Obama’s example), the knowledge produced by R&D is what economists call a “public good”: People can share it without using it up, and everybody involved (states, school districts, or education-related private companies) is free to use the results of research underwritten by somebody else. Rather than take on a proportionate share of research costs, many are likely to look for a “free ride” on others’ findings. [3]

Of course, some states and districts do invest in education R&D; New York’s Education Finance Research Consortium is a notable example. Other well-known efforts, like the Consortium on Chicago School Research, rely on philanthropic funding (though government representatives often participate in setting the research agenda). Most state and district budgets and personnel are stretched too thin to allow for much investment in R&D, however. In 2003, the National Research Council urged states to band together to form a Strategic Education Research Partnership, but the well-funded interstate compact they envisioned was never created due to lack of support from states. [4] State-led accountability collaboratives like Achieve’s American Diploma Project and the Council of Chief State School Officers’ State Collaborative on Assessment and Student Standards also have limited research agendas; they generally do not conduct longitudinal studies evaluating the effects of accountability programs.

The federal government has a critical role to play in funding more expensive studies, such as long-term or comparative studies, large-scale demonstration projects, and complicated randomized field trials. [5] As Eva Baker, Director of the UCLA-based National Center for Research on Evaluation, Standards, & Student Testing, pointed out at the Rockefeller Institute Symposium on Intergovernmental Approaches for Strengthening K-12 Accountability Systems, states tend to pursue a “requirements-driven” agenda and may need what T. K. Glennan terms “technology push” to help them anticipate what they might want in the future. To meet this need, federal funding is also necessary for exploratory studies and basic research. [6]

A new approach to education R&D?

Much of the current federal funding for education R&D flows through the Institute of Education Sciences (IES) and its What Works Clearinghouse. Established in 2002, IES funds research, evaluates federal programs, and reports statistics. Its What Works Clearinghouse (WWC) uses scientifically rigorous procedures to assess and summarize the strength of the evidence on the effectiveness of specific education interventions. WWC also issues practice guides (developed by expert panels) which recommend strategies for addressing education challenges.Observers have credited IES with promoting a new culture of rigor in education research—with “putting the ‘scientifically based’ in the scientifically based research mandated by NCLB.” [7] The extent to which any of this research or guidance has translated into improvements in educational practices is as yet unknown, however; it is probably too soon to tell.

President-elect Obama’s proposal is not simply to redouble the efforts of IES and WWC. The second element of the “Invest in What Works” proposal specifies that part of the funding increase will be used to commission

a blue-ribbon private sector panel of premier business leaders, educators, researchers, and others to make recommendations to the Secretary of Education on successful programs and innovations across the country that should be scaled. The panel will also be charged with making those successful practices and lessons learned universally available. [8]

Such a high-profile approach could complement IES and WWC’s by fast-tracking the dissemination and implementation of selected practices. But an important question remains: What criteria will panelists use to identify “successful” programs and innovations? Expert panels can, and do, play an invaluable role in the assessment of evidence and the development of recommended procedures. But this blue-ribbon panel might just as easily undermine the new culture of rigor by playing politics or promoting illusory silver bullets.

The Bill & Melinda Gates Foundation learned the hard way, with its $2 billion small schools initiative, that scaling up educational interventions prematurely can lead to disappointing results. As Bill Gates himself said, “Without evidence, innovation is just another word for ‘fad.’” [9] It’s time to get the evidence.