Bunkum Awards 2008

The Fordham Institute and the Brookings Institution jointly win the Murray for their articles authored by Tom Loveless. His back-to-back winners came only three months apart but both showed his enduring commitment to convincing policy makers that too much effort is spent academically challenging the wrong children.

High Achieving Students in an Era of NCLB attempts to build the case that concentrating on low-achieving children diminishes the growth of the higher achievers, who accordingly become “languid.” Dr. Loveless comes to this conclusion by presenting NAEP score comparisons of trends among high- and low-scorers, showing faster growth at the bottom of the distribution. “[T]his trend,” the report concludes, “suggests a missed opportunity to promote achievement among high achievers.” The Think Tank Project reviewer, however, pointed out a troubling inconsistency. The report correctly notes that its correlational analyses can not be used to draw causal inferences, but it also makes patent causal inferences to bolster its policy recommendations. This over-reach is most apparent in the foreword by Fordham's Chester Finn and Michael Petrilli.

Loveless returns to the ‘wasted energies’ theme in The Misplaced Math Student: Lost in Eighth-Grade Algebra, published by the Brookings Institution. In this piece, Loveless contends that having low-achieving students in algebra classes with highly proficient students dampens opportunities for the best students and dooms many lower achievers to failure. Only one peer reviewed article is discussed, and it comes to a different conclusion – so Loveless criticizes it for selection bias. His research method is to use state NAEP scores and correlate them with algebra-taking rates in each state. Finding no relationship he concludes that his hypothesis is sustained, despite acknowledging that his correlational findings should not be used to argue that causal relationships have been found. Sound familiar?

This award is bestowed upon Public Charter Schools: A Great Value for Ohio’s Public Education System, authored by Matthew Carr and Beth Lear and published by Ohio’s Buckeye Institute. While many advocacy think tanks annually contend for this high honor, it was earned this year by the Buckeye Institute for its determined disregard of an extensive, non-partisan, and quite relevant state official report, its non-existent literature review, and its decisive failure to comprehend the state funding formula lying at the heart of its analysis. Our reviewer found that these bundled blunders lead the authors to the baseless conclusion that each charter student saves as much as $4,030 for the host public school district. He described the report as “unfounded and outlandish,” as well as “ridiculously false, deceitful, and patently misrepresent[ing] how the funding of public schools works.”

The Friedman Foundation garners the MIBTIWSIOMT award this year for building two distinct franchises on little more than fixated false claims.

One set of five cloned state studies offered the repetitive conclusion that high school dropouts would be reduced and economic prosperity advanced if voucher programs were introduced in each state. Although severely handicapped by the fact that there is no significant evidence that vouchers will reduce drop-outs, the author hangs his hat on a single 1998 study. Ironically, the Friedman reports themselves criticize the very approach used in this cherry-picked article: reliance on administrative counts (such as reports of school principals) to estimate high school graduation.

Even more impressive than this first effort, the Foundation stamped out ten separate state surveys, all of which came to the conclusion that potential voters in each state endorsed private school vouchers. Admittedly, the surveys suffered from biased questions and were administered to respondents whose own responses showed limited knowledge of the educational policy issues they provided opinions about. But the important thing is that these respondents’ conclusions were substantially more pro-voucher than the Kappan Gallup poll responses on the same topic (except that the Friedman survey reports never mentioned the Gallup research). Perhaps most damning, the authors reached so far that their pro-voucher conclusions were not even supported by their own problematic survey data.

In this third year of the Bunkum awards, the true barons of bunkum have bullied their way to the front. Perennial powerhouses Friedman and Fordham once again made the list for poor research and execution. Sadly, the Cato Institute in 2008 failed to hit its mark. Their entry into the competition was a global review called Markets vs. Monopolies in Education: A Global Review of the Literature. Normally, a report that excludes major studies, ignores selection bias, and oversimplifies the complex characteristics of educational markets would be a contender. Further, in an act of self-aggrandizing puffery, the report proclaims itself to be of “profound” importance to U. S. educational policy. Alas, they were knocked from this year’s rankings by their application of alleged lessons from the educational systems of Pakistan, India, Tanzania, Ghana and Nigeria to the fundamentally and structurally different system of the United States. This left our judges no choice but to dismiss the report as merely silly.