Report’s Comparisons of Charter School versus Public School Productivity Suffer From Lack of Clarity and Validity

BOULDER, CO (May 10, 2018) – Bigger Bang, Fewer Bucks?: The Productivity of Public Charter Schools in Eight U.S. Cities, published by the University of Arkansas Department of Education Reform, contends that charter schools produce more achievement per dollar invested, as compared to public schools.

Professor Julian Vasquez Heilig of California State University Sacramento reviewed the report and identified a variety of methodological choices made by the authors that threaten the validity of the results. 

The report is focused on city-level analyses in eight U.S. cities (Atlanta, Boston, Denver, Houston, Indianapolis, New York City, San Antonio, and Washington D.C.) and uses cost effectiveness and Return on Investment ratios. It concludes that charter schools deliver a weighted average of an additional 4.34 NAEP reading points and 4.73 NAEP math points per $1000 invested. 

However, Professor Vasquez Heilig points out that the report fails to account for the non-comparability of the student populations in charter and comparison public schools. Four other problems also undercut the report’s claims. First, the report uses revenues rather than actual expenditures, despite well-established critiques of this approach. Second, the report’s lack of specificity (e.g., using state-level data in city-level analyses and completely excluding race and gender) plagues the accuracy and validity of its calculations. 

Third, the authors fail to reconcile their report with the extensive literature of contrary findings. Finally, even though the think tank’s earlier productivity report included a caveat saying that causal claims would not be appropriate, and even though the new report’s analyses also are insufficient to make causal claims, the new report omits that caution.

The evidence in this report is so flawed that it provides no valid guidance to educators or policymakers.

Find the review, by Julian Vasquez Heilig, at:
http://nepc.colorado.edu/thinktank/review-roi

Find Bigger Bang, Fewer Bucks?: The Productivity of Public Charter Schools in Eight U.S. Cities, written by Corey A. DeAngelis, Patrick J. Wolf, Larry D. Maloney, & Jay F. Mayand, and published by the University of Arkansas Department of Education Reform, at:
http://www.uaedreform.org/downloads/2018/02/bigger-bang-fewer-bucks-the-productivity-of-public-charter-schools-in-eight-u-s-cities.pdf

NEPC Reviews (http://thinktankreview.org) provide the public, policymakers, and the press with timely, academically sound reviews of selected publications. NEPC Reviews are made possible in part by support provided by the Great Lakes Center for Education Research and Practice: http://www.greatlakescenter.org

The National Education Policy Center (NEPC), housed at the University of Colorado Boulder School of Education, produces and disseminates high-quality, peer-reviewed research to inform education policy discussions. Visit us at: http://nepc.colorado.edu