NEPC Review: Do Impacts on Test Scores Even Matter? (American Enterprise Institute, March 2018)
A new report from the American Enterprise Institute (AEI), Do Impacts on Test Scores Even Matter? Lessons from Long-Run Outcomes in School Choice Research, examines whether student achievement scores on math and English language arts tests align with “long-run” attainment outcomes such as high school graduation rates, college enrollment, and college graduation rates. Drawing on a systematic review of the literature, it concludes that the impacts of school choice programs on test scores are not well connected to such attainment outcomes, which are presented as more positive. This review considers two issues with the report: consistency and evidence. Regarding consistency, the report’s suggestion that achievement scores should play a smaller role in determining the efficacy of school choice models represents a stunning effort to move the goalposts in search of new justifications for supporting their preferred policies. After decades of pro-school-choice research and advocacy promoting test score comparisons with public schools as the primary measurement for evaluating school choice models (e.g., charters and school vouchers), the AEI report now suggests that less attention be given to these learning outcomes. Regarding evidence, the AEI report is riddled with numerous internal inconsistencies in its discussion and treatment of a set of studies that were selected by questionable methods. In view of the 180-degree turn based on questionable evidence, the report — despite the authors’ assertions — is of little use to policymakers.