"People collect statistics with a purpose. Sometimes the purpose is to advance the common good; sometimes it’s to simply advance the people who collected them or their agenda." Gerald Bracey

 

 

Two moments on social media speak in different ways about the essential problems with calling for scientific evidence to drive education policy and classroom practices.

First, a tremendous amount of misinformation and over-reaction have come in the wake of an article in Nature, forcing some to note that the article and research do not, as many claim, discredit education research:

A second example is that when I posted a blog about media coverage of reading reform, including research showing that there is no scientific evidence supporting a popular and often mandated teacher training program, LETRS, one of the most fervent advocates of the “science of reading,” responded in a very odd but telling way:

The public, politicians, and education reformers are prone, it seems, to seeing something as “science” if they believe it, but are skeptical of science when it doesn’t fit their beliefs or agenda; thus from Gerald Bracey: “People collect statistics with a purpose. Sometimes the purpose is to advance the common good; sometimes it’s to simply advance the people who collected them or their agenda.”

The quality of education research isn’t the dumpster fire some want to argue, but the field needs to do a better job with the quality and applicability of that research.

However, that really will not matter if the media continues to misrepresent and misread the research, if then politicians use that misinformation to drive policy, and if there remains little recourse for correcting both.

The current “science of” movement are ironically grounded in very strict demands for scientific research while being mostly driven by a podcast and a compelling but false story.

We lost a good scholar in education research, Bracey, way too early, and although his Reading educational research: How to avoid getting statistically snookered is now two decades old, I use it in my upper-level writing and research course.

It is a bit frustrating that Bracey’s warnings and analyses about education reform and think tanks remain just as valid as they did in 2006.

This book is accessible and smart. Here are the principals that ground his discussion:

• Principle: Make certain that any test aligned with a standard comprehensively tests the material called for by the standard.
• Principle: On a norm-referenced test, nationally, 50 percent of students are below average, by definition.
• Principle: A norm-referenced standardized achievement test must test only material that all children have had an opportunity to learn.
• Principle: Standardized norm-referenced tests will ignore and obscure anything that is unique about a school.
• Principle: Scores from standardized tests are meaningful only to the extent that we know that all students have had a chance to learn the material which the test tests.
• Principle: Any attempt to set a passing score or a cut score on a test will be arbitrary. Ensure that it is arbitrary in the sense of arbitration, not in the sense of being capricious.
• Principle: If a situation is as alleged, ask, “So what?”
• Principle: Achievement and ability tests differ mostly in what we know about how students learned the tested skills.
• Principle: Rising test scores do not necessarily mean rising achievement.
• Principle: The law of WYTIWYG applies. What you test is what you get.
• Principle: Any tests offered by a publisher should present adequate evidence of both reliability and validity.
• Principle: Do no use a test for something other than what it was designed for without taking care to ensure it is appropriate for the other purpose.
• Principle: Do not make important decisions about individuals or groups on the basis of a single test.
• Principle: In analyzing test results, make certain that no students were improperly excluded from the testing.
• Principle: In evaluating a testing program, look for negative or positive outcomes that are not part of the program. For example, are subjects not tested being neglected? Are scores on other tests showing gains or losses?

Click HERE for my PowerPoint overview I use for students also.

The publisher page also has a SAMPLE.

Recommended