This marks our seventh year of handing out the treasured Bunkum. That’s seven years of honoring the lows and very lows, chosen from among those reports scrutinized by our expert reviewers. It’s an ugly business, akin to being hired to write weekly columns about Congress: there’s no glossing over the dysfunction, so we might as well have fun with it.
Watch the 2012 Bunkum Award Ceremony:
The nature of this irredeemably awful report is betrayed in the title, which seeks to alert readers to the evidently toxic combination of policy ingredients that, in the fevered imagination of the authors, amounts to a “race-based recipe for disaster.” Moreover, the imagined carnage would not be confined to the kitchen. In the apocalyptic metaphorical landscape of this report, aspects of our transportation system are also at risk: A “train wreck” resulting in massive “liabilities” of “billions of dollars” is the likely result of state policymakers colluding, in their promotion of race-based school reform policies, with advocates for busing and school funding. Our judges quickly checked the acknowledgements section to see if Chicken Little was listed as an advisor.
This exercise in hysteria was precipitated by a Minnesota Department of Education report on concentrated poverty and segregation, along with three other reports published by equity-focused organizations. These reports suggest policies such as a continuation of existing pro-diversity efforts, establishment of state standards for when equity could be considered achieved, a sharper focus on existing programs, and the encouragement of voluntary fair housing and magnet school programs.
The Center of the American Experiment’s 144-page counter-report, however, does not really address these seemingly sensible proposals. Instead it sets up straw men in the form of “busing” and mandated “de-segregation.” Neither of these policies was recommended by any of the reports being criticized.
Having created their straw men, the CAE authors then proceed with the dreary ritual of attacking them. Amusingly, they did a poor job even putting the lance into their own creations. Because school integration has been so well researched, the overall evidence base has been digested into authoritative literature reviews. The gist of these reviews is that school integration has several well-documented benefits, including improved intergroup relations for all students, and improved academic outcomes for students of color (along with no benefit or harm for white students). The authors do not seriously engage this research. Instead, while they do cite two literature reviews, they misrepresent the conclusions of those reviews.
All of this may strike readers as merely mundane, poor-quality arguments against equity and diversity. Fair enough. But what brought tears of appreciation to our judges’ eyes was the lengthy, heart-rending and compassionate soliloquy about the need to rectify the injustice of the achievement gap. The author’s stated passion for closing the achievement gap, accompanied by an equally passionate rejection of initiatives sensibly designed to close it, raises obtuseness to the level of performance art.
In the report’s grand finale, readers are told that universal salvation lies in Florida. Yes, at little or no cost, Minnesota can be the next proud owner of the Florida reform model. Of course, there is extensive literature (including several NEPC publications) explaining that the Florida reforms do not deliver on what they promise. But an excursion to Florida is invariably part of the annual itinerary of the Bunkum Award, and we thank CAE for making it possible in 2012.
As ridiculous as it is, this report is troubling on many levels: it sets up and attacks straw men; it provides a shoddy and unbalanced literature review designed to convince readers that desegregation and integration efforts have not been beneficial; and, worst of all, it reeks of hypocrisy. It self-righteously claims to be addressing the achievement gap while rejecting proposals with a positive record and embracing a set of proposals that, if they have any effect, would likely make the gap worse.
As one of our judges thoughtfully summarized, “This is just BUNK.”
Reading this report, one learns about a problem that few of us knew existed. Apparently, there is a great deal of confusion in disadvantaged communities where wealthy strangers have arrived laden with school-turnaround gifts. The patrons of these communities are inexplicably and unjustifiably seen as patronizing—or even as destructive intruders. Fortunately, the Public Agenda Foundation has stepped up with this report which outlines ways to help members of these communities to get their minds right.
The report examines why citizens have proprietary attitudes toward their community school and why they resist external “change agents” who are intent on improving those schools for the citizens’ own good.
In the view of this report, these uninformed and parochial parent attitudes are obstacles to the re-making and improvement of community schools. According to its authors, “Many parents do not realize how brutally inadequate local schools are.” As a result of their ignorance, parents have raised irrational and unwise objections to firing teachers due to low test scores or to their school being closed, privatized, broken-up.
While the report includes a section on “how the research was done,” it provides no research in any conventional sense of the term. It is instead an explicit effort to test and refine message framing and sound bites designed to engender negative attitudes toward public schools and promote positive attitudes toward charter schools and other “bold” turnaround approaches.
Our judges noticed that in the focus groups conducted by the researchers, the community members knowledgeably identified the problems of inadequate and inequitable resources, as well as the significant hurdles faced by their impoverished communities. The authors also note, “There was also a strong sense among the parents we interviewed that, in their view, the communities themselves should be seen as sources of new thinking.” The report’s authors honestly and clearly present such concerns and ideas. But the report never addresses these core problems and never suggests truly valuing local ideas. Instead, it recommends eight communications strategies—including such things as “tell lots of stories” and “communicate through trusted sources”—all designed to shift attention toward the favored “bold” turnaround approaches (and thus shift attention away from the real problems seen by the communities, as the report itself describes).
Amidst all this messaging, the report completely ignores the lack of evidence that the so-called turnaround strategies actually work to benefit communities. The strategies are just assumed to be desirable. As our reviewer concluded, “What’s Trust Got to Do With It? is ironically titled. Trust has everything to do with the problem. Yet, perhaps the greater problem is in the authors’ complete lack of trust in the views of the parents.”
These authors wander aimlessly around a data wilderness, searching for positive evidence about school vouchers. Their report attempts to make the case that New York City partial vouchers of $1,400 per year to attend private elementary schools for three years had later positive impacts on college attendance, full-time college enrollment and attendance at selective colleges for African American students. It received lavish media attention, including a foot-stomping commentary by the report’s authors in the Wall Street Journal that scolds President Obama for what they regard as his outrageous failure to line up behind voucher policies.
To help understand the problems with this report, let’s all mentally travel to Sunnyside, Nevada, which hit a high temperature of only 14°F on January 17, 2012. Even while the world was experiencing record heat, Sunnyside posted a record cold for that date. If we wanted to distract attention from overall warming trends, we might lead with this and other cherry-picked data. It’s an old trick that often works, if nobody pays attention to the overall trends and if nobody questions the cherry-picking.
Yet this is essentially the approach used by the Bunkum-winning Brookings report, which finds positive college-related impacts for African American students (but not for other students) who had received vouchers back in elementary school. The researchers, of course, had no a priori reason to think that African Americans would benefit in this way from vouchers, when other students do not. They simply explored the data, found lots of results showing no voucher benefits and then found this one (akin to Sunnyside, Nevada) that helped support their advocacy of vouchers.
Our reviewer did not criticize the decision to explore; instead, she points out that the results of foraging through data should be presented as such: “Had Chingos and Peterson framed the finding for African Americans as an encouraging, exploratory hypothesis deserving of further testing, I would not have been alarmed by the report. But the study’s results absolutely do not merit headlines such as ‘Vouchers promote college attendance for African Americans.’”
The reason is simple. As our reviewer indicates, “Contrary to how it was presented, the main finding of this new report should be that, using a rigorous experimental design in which vouchers were randomly assigned to students, the estimated college enrollment rates of students with and without vouchers were not different from one another.”
Buried on p. 12 of the report is the statement that for the total sample, there was “a tiny insignificant impact.” As for the claims of a positive effect on college attendance of African Americans, there were no statistical differences between ethnic groups. Yet the authors chose to trumpet a positive effect for African Americans.
If there were indeed masked positive effects for one subgroup yearning to express themselves, there also must be some negative effects cruelly neutralizing the hidden voucher gains. That is, there must have been some students doing worse. Who are they? The authors refused to say. And, despite the prodding by our reviewer, they did not make public their full findings; they simply expected readers to trust them.
This sort of substandard research practice is why the Bunkum Award was born. The evidence in this study—honestly read—simply does not say what the authors said it does.
After being shut out of the 2010 and 2011 Bunkum Award, four-time winner Friedman has returned in spectacular fashion. Seldom does a report hit the “trifecta:”
The problems begin with the report’s claims that test scores and dropouts have not shown any visible improvement between 1992 and 2009, during which time school staffing increased 2.3 times. Even setting aside problems with the staffing claim itself, our reviewer points out that the report’s fundamental premises asserting no improvements in test scores and an increase in the drop-out rate are flat wrong. In reality, there has been clear improvement in NAEP scores for all student subgroups, particularly students of color and younger students. And despite the change to a more stringent definition of drop-outs, graduation rates have increased, helping to raise college attendance to historic highs.
Soaring on the wings of flawed reasoning, with a strong updraft of chutzpah, the report’s author jumps from his platform of sham evidence to deliver three unsupported recommendations: a call for class size increases, a call for cuts in administrative and teaching staff and a call for increased school choice. As our reviewer points out, US public school class sizes are larger than those in our “competitor” OECD countries and are, in fact, larger than the idealized and attractive small classes in the private schools the Friedman Foundation touts. Small class sizes are apparently only bad and wasteful when they are in public schools. Similarly, there is the inconvenience that charter schools divert a higher proportion of their spending into administrative largesse.
Accordingly, not only does the report’s call for increased school choice have no visible relation to the data, it undermines two other recommendations from the same report. It uses bogus information to draw ungrounded causal conclusions that in turn lead to an unsupported series of recommendations that are in conflict with one another. Our judges were amazed.