A Fordham report highlights the historic academic progress of Black and Hispanic groups over the past two decades, at the elementary school level, on the NAEP exam. From this, the report offers the major claim, based on its author’s eyeball test, that the academic progress of students of color is attributable “mostly” to poverty reduction. The report, however, also acknowledges that correlation is not causation and calls for systematic statistical analysis to test the author’s proposition. This review responds to that call by examining the validity of the report’s arguments around progress and causes, looking to expanded data sources, including both family income and school expenditures. The review notes uneven patterns of achievement among grade levels and refutes the report’s claim that flat achievement trends among twelfth graders are a result of dropout reductions. 

This Urban Institute research report is aimed at assessing the impact of the Florida Tax Credit (FTC) scholarship program on college enrollment and graduation. Through matched comparisons, the study finds consistently positive effects of the program for both two-year and four-year colleges, with relatively stronger effects for four-year private college enrollment and for students who stayed in the program longer. This review acknowledges the study’s contributions, using expanded data tracking to inform school voucher policy debates, but the review also raises three critical questions about the validity of some methods, findings and conclusions. These concerns call into question the use and misuse of this study by voucher advocates; proper use of the study begins with an understanding of its limitations and of the need for confirmatory research and future exploration of potential mechanisms driving any increased college attainment.

The research report reviewed here concludes that many initially high-achieving students are falling further and further behind over the course of their years in school. The report intends to raise the alarm and to advocate for improved programs for these students. It is, however, a false alarm due to biased methodology and misleading arguments. The report’s norm-referenced framework guarantees “losers” as well as “winners,” regardless of any true improvement made by the students. Also, the “regression to the mean” effect produces a false illusion of a tradeoff of over-progress by low achievers at the cost of under-progress for high achievers. Finally, its prescription for stronger school accountability for high-achieving students under NCLB does not follow research-based guidance on how to improve student learning. Other research, including that conducted by this reviewer, finds that students who are high achievers and low achievers make approximately equal academic progress in reading and math, while the achievement gaps remain large. Moreover, a substantial part of the variation in student progress is attributable to teacher and school effects beyond students’ initial status and background characteristics.

Two authors of the report published a written reponse on October 30, 2011. That response is available here, and Dr. Lee's brief reply is published and downloadable below.

The Education Trust research report Stuck Schools suggests a framework for identifying chronically low-performing schools in need of turnaround. The study uses Maryland and Indiana to show that some low-performing schools make progress while others remain stagnant. The report has four serious problems of reliability and validity, however. First, the norm-referenced methodology guarantees "failed" schools independent of any true performance or improvement level by the school. Second, the report’s reliance on state assessment data is misleading, and some schools’ reported growth may be an artifact of regression to the mean and ceiling effects as well as instructional and testing practices. Third, the use of a linear growth model is questionable, since schools may not follow a strictly linear pattern of improvement. Fourth, the label of "stuck" becomes problematic given that there is no research-based guidance on how to improve schools other than vague prescriptions. In conclusion, the report’s methods are so simplistic, arbitrary and ill-fitting with its own assumptions that it is more harmful to sound policymaking than helpful. There remains an outstanding question of how to help struggling schools after identification, but we need to know first whether the identification is based on reliable and valid measures, and if so, what school factors account for these differences.

Suggested Citation: Lee, J. (2010). Review of “Stuck Schools: A Framework for Identifying Schools Where Students Need Change—Now.” Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. Retrieved [date] from http://epicpolicy.org/thinktank/review-stuck-schools

Public versus private school achievement gaps in general and the effects of school choice on academic outcomes in particular remain controversial issues. The author reviews two recent reports of empirical studies on this topic: one from the Milton & Rose D. Friedman Foundation (MFF) and the other from the Center on Education Policy (CEP). MFF presents its empirical analysis in the context of the larger policy question about the effect of school choice, whereas CEP simply attempts to answer a research question, with policy implications, about a possible public-private school achievement gap. Both studies contribute new evidence to the existing literature through secondary analyses of national high school student datasets — the Educational Longitudinal Study (ELS) and the National Education Longitudinal Study (NELS) databases. The two reports in tandem provide contrasting views and results regarding private school effects. MFF argues that private schooling is more successful at improving student test scores; CEP argues that public and private schools have relatively equal success. This review provides an independent cross-examination of the two data sources and shows that the public-private high school gaps in math achievement gain scores were almost null (in the NELS) or too small to be practically significant (in the ELS). Therefore, the seemingly divergent findings and conclusions at the first glance may have been largely due to their different interpretations rather than real differences in the results. Both studies could have given more useful guidelines for policy and practice if they had examined reasons for observed gaps (or the lack thereof) between public and private schools.

Suggested Citation:

Lee, J. (2007). Review of "Two Reports Addressing the Achievement of Students in Private and Public Schools."  Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. Retrieved [date] from http://epicpolicy.org/thinktank/review-two-reports-addressing-achievement-students-private-and-public-schools