'Everyone Wins' ignores factors besides competition to explain marginally improved public school achievement in NYC
Contact: Patrick McEwan, (781) 283-2987; (email) pmcewan@wellesley.edu
Gary Miron, (269) 599-7965; (email) gary.miron@wmich.edu
TEMPE, Ariz. and BOULDER, Colo. (November 17, 2009) -- A report released three weeks ago looked at the competition effects of New York City's charter schools, concluding that "students benefit academically when their public school is exposed to competition from a charter." A new review of that report finds that the report's own findings are pretty minimal, and even that slight improvement could be explained by factors other than charter-school competition.
The report is boldly titled, Everyone Wins: How Charter Schools Benefit All New York City Public School Students. It was written for the Manhattan Institute by Marcus Winters, a Senior Fellow at the institute. The report was reviewed for the Think Tank Review Project by Patrick McEwan, professor of economics at Wellesley College.
The report's focus reflects an important premise behind the market-competition approach to education reform. While critics of market theory contend competition from charter schools or other alternatives will end up penalizing the public schools that must educate most children, market advocates contend competition will help the entire school population. Competition, this argument runs, doesn't just expand choices for parents; it also prods existing public schools to improve in order to avoid losing students.
Everyone Wins draws on three years of test score data in mathematics and English Language Arts (ELA) from New York City public schools as well as data on the percentage of students leaving public schools for charter schools. As McEwan explains, the report uses the rate of departures for charter schools as a proxy to measure increased "pressure on public school administrators to ‘compete,' improve test scores, and staunch the flow of students to charter schools." Using appropriate statistical controls, the report finds that increasing competition does not appear to be associated with improved math test scores, while it has "small positive effects on ELA scores" that are "slightly larger among public school students with lower levels of achievement."
In his review, McEwan observes that the report itself is modest in its conclusions and that it "correctly notes that the statistical findings do not necessarily imply that increases in the measure of competition cause test scores to rise..." Unfortunately the report's title suggests a much more positive definitive outcome, and the nuances are lost as well in the executive summary. Causal claims also pervade Winter's own NY Post commentary about his study.
McEwan notes that a longer, dryer, and more accurate title for the study might be: "Some win and some lose, but, on average, they slightly win -- though mainly on the ELA test and with caveats about the causal interpretation of the effects."
The reviewer praises the report on several counts: its use of "a high-quality, longitudinal dataset of student achievement," its use of appropriate statistical methods, and its contribution to "an established literature that finds roughly consistent effects of ‘competition' that are often zero or slightly positive, depending on the state and method."
But he also points out important limitations. The statistical methods used, "while appropriate... cannot control for several potential biases," McEwan writes. "As a result, the measured effects of competition could also reflect the influence of shifting peer quality, declining class size, or other unobserved variables." Other sources of market pressure, such as private schools or schools of choice, are also not considered in this report.
The report notes in passing some of those limitations, but "does not make a serious attempt to assess their validity," the reviewer concludes. In the end, then, it does not deliver what it claims, and we do not know whether, in fact, "everyone wins."
Find Patrick McEwan's review on the web at:
http://epicpolicy.org/thinktank/review-Everyone-Wins
Find Everyone Wins: How Charter Schools Benefit All New York City Public School Students, by Marcus Winters and published by the Manhattan Institute, on the web at http://www.manhattan-institute.org/pdf/cr_60.pdf
CONTACT:
Patrick McEwan, Associate Professor
Department of Economics
Wellesley College
(781) 283-2987
pmcewan@wellesley.edu
Gary Miron, Professor of Education
Western Michigan University
(269) 599-7965
gary.miron@wmich.edu
About the Think Tank Review Project
The Think Tank Review Project (http://thinktankreview.org), a collaborative project of the ASU Education Policy Research Unit (EPRU) and CU-Boulder's Education and the Public Interest Center (EPIC), provides the public, policy makers, and the press with timely, academically sound reviews of selected think tank publications. The project is made possible by funding from the Great Lakes Center for Education Research and Practice.
Kevin Welner, the project co-director, explains that the project is needed because, "despite their garnering of media attention and their influence with many policy makers, reports released by private think tanks vary tremendously in their quality. Many think tank reports are little more than ideological argumentation dressed up as research. Many others include flaws that would likely have been identified and addressed through the peer review process. We believe that the media, policy makers, and the public will greatly benefit from having qualified social scientists provide reviews of these documents in a timely fashion." He adds, "we don't consider our reviews to be the final word, nor is our goal to stop think tanks' contributions to a public dialogue. That dialogue is, in fact, what we value the most. The best ideas come about through rigorous critique and debate."
**********
###
The Education and the Public Interest Center (EPIC) at the University of Colorado at Boulder and the Education Policy Research Unit (EPRU) at Arizona State University collaborate to produce policy briefs and think tank reviews. Our goal is to promote well-informed democratic deliberation about education policy by providing academic as well as non-academic audiences with useful information and high quality analyses.
Visit EPIC and EPRU at http://www.educationanalysis.org/
EPIC and EPRU are members of the Education Policy Alliance
(http://educationpolicyalliance.org).
###
**********