New Friedman Foundation report is less complete and careful than last November's report from the Urban Institute
Contact: Kevin Welner, (303) 492-8370; (email) kevin.welner@gmail.com
TEMPE, Ariz and BOULDER, Colo. (April 2, 2008) - Following closely on the heels of a November 2007 report released by the Urban Institute's CALDER center, a new report, Lost Opportunity: An Empirical Analysis of How Vouchers Affected Florida Public Schools from the Friedman Foundation, claims that "vouchers were a key element driving improvements in public schools from the A+ [accountability] program."
Both the Urban Institute and the Friedman reports contain overstated claims about the impact of the A+ program. The Urban Institute report did, however, include an appropriate description of the relationship between the accountability system and the higher test scores: described as a correlation or association, not a relationship where one is shown to cause the other. And the Urban Institute report made no claim that the effect of the Florida voucher program could be distinguished from the other parts of the state's accountability system.
Simply put, the actual effects of the voucher program are unknowable given the available data. Yet the new Friedman Foundation report disregards the data limitations and boldly concludes, for example, that the termination of the voucher policy "caused" test scores in public schools to drop (see page 18, where the report also wrongly uses the wording "is attributable to" and "produced").
The Friedman report is also sloppy in presenting its findings. While responsible reporting of results of regression analyses should include information such as standard errors, significance values for each coefficient included in each model, and omnibus statistics such as F-tests and R2 values, the Friedman report includes none of these. In effect, the report says, "trust us; don't verify."
In January of 2008, the Think Tank Review Project published a review of the more careful Urban Institute report. Those interested in this topic are encouraged to read the Urban Institute report and review. The new Friedman Foundation report contributes little if anything of value to the consideration of the impact of Florida's voucher program, since its analyses are presented in a biased and opaque manner.
The Friedman report is authored by Greg Forster, who is a Senior Fellow at the Friedman Foundation.
The earlier report from the Urban Institute's CALDER center was reviewed for the Think Tank Review Project by Damian Betebenner of the National Center for the Improvement of Educational Assessment.
About the Think Tank Review Project
The Think Tank Review Project (http://thinktankreview.org), a collaborative project of the ASU Education Policy Research Unit (EPRU) and CU-Boulder's Education and the Public Interest Center (EPIC), provides the public, policy makers, and the press with timely, academically sound reviews of selected think tank publications. The project is made possible by funding from the Great Lakes Center for Education Research and Practice.
Kevin Welner, the project co-director, explains that the project is needed because, "despite their garnering of media attention and their influence with many policy makers, reports released by private think tanks vary tremendously in their quality. Many think tank reports are little more than ideological argumentation dressed up as research. Many others include flaws that would likely have been identified and addressed through the peer review process. We believe that the media, policy makers, and the public will greatly benefit from having qualified social scientists provide reviews of these documents in a timely fashion." He adds, "we don't consider our reviews to be the final word, nor is our goal to stop think tanks' contributions to a public dialogue. That dialogue is, in fact, what we value the most. The best ideas come about through rigorous critique and debate."
CONTACT:
Kevin Welner, Professor and Director
Education and the Public Interest Center
University of Colorado at Boulder
(303) 492-8370
kevin.welner@gmail.com
**********
###
The Education Policy Research Unit (EPRU) conducts original research, provides independent analyses of research and policy documents, and facilitates educational innovation. EPRU facilitates the work of leading academic experts in a variety of disciplines to help inform the public debate about education policy issues.
Visit the EPRU website at http://educationanalysis.org
###
The Education and the Public Interest Center (EPIC) at the University of Colorado, Boulder seeks to contribute information, analysis, and insight to further democratic deliberation regarding educational policy formation and implementation.
Visit the EPIC website at http://epicpolicy.org
###
**********