Reviews and blog posts by others among new NEPC features
Contact:
Bruce Baker
(732) 932-7496, x8232
William Mathis, NEPC
(802) 383-0058
BOULDER, CO (May 26, 2011) – Passing Muster, a new Brookings Institution report authored by a prestigious group of researchers, attempts to offer guidance on how to evaluate teacher evaluation systems. Rutgers University professor Bruce Baker, however, took a close look at the report and raises serious questions about some of its key elements.
Baker’s review is accessible on the National Education Policy Center website [http://nepc.colorado.edu/thinktank/passing-muster-fails-muster]. It is made available as part of a new feature of NEPC’s “Think Twice” think tank review project. In addition to NEPC’s original reviews, other comprehensive reviews written by scholars for other venues are posted on the Think Twice webpage. In this case, Baker’s review was originally posted on his “SchoolFinance101” website.
At first blush, Baker writes, the Brookings web version of its report Passing Muster seems inviting. It even includes “a nifty calculation tool” for rating teacher evaluation systems. But when one looks at Passing Muster’s underlying assumptions, problems become apparent.
Perhaps most troubling, the report can’t seem to make up its mind about the importance of value-added approaches. Baker points out the following:
- An evaluation system must, according to the report, include student standardized test scores as a component of the teacher evaluation system.
- The report’s authors call themselves “agnostic” on the relative weight of this (and other) components.
- The authors fail to acknowledge the lack of expert consensus as to the validity or reliability of value-added approaches that tie teacher evaluation to student test scores.
- But – and here’s the key problem – the system advocated by the report requires that all other (non-valued added) components have their validity be evaluated based on the extent to which that component correlates with the subsequent year value-added measure.
As Baker explains, “This is rather like saying, ‘We remain agnostic on whether you focus on reading or math this year, but we are going to evaluate your effectiveness by testing you on math. Or more precisely, we remain agnostic on whether you emphasize conceptual understanding and creative thinking this year, but we are going to evaluate your effectiveness on a pencil and paper, bubble test of specific mathematics competencies and vocabulary and grammar.’”
The bottom line, according to Baker: “We technocrats have started to fall for our own contorted logic – that the available metric is the true measure – and the quality of all else can only be evaluated against that measure.” In pursuit of a clean solution and in deference to political fashions, “we’ve forgotten all of the technical caveats of our own work,” Baker writes – at the risk of being blind to “real ethical implications.”
Baker’s blog post is found in a new section on the NEPC’s website, under the “Think Tank Project” tab, “Reviews by Others.” You can find it here: http://nepc.colorado.edu/thinktank/passing-muster-fails-muster. It is also available on Baker’s own website at: http://schoolfinance101.wordpress.com/2011/05/18/passing-muster-fails-muster-an-evaluation-of-evaluating-evaluation-systems/.
A second new addition to the NEPC website is a collection of the best in recent education blogging, at http://nepc.colorado.edu/blog. Each day, one or more new blog entries are added, selected from contributions offered by a group of top-notch education bloggers.
The mission of the National Education Policy Center is to produce and disseminate high-quality, peer-reviewed research to inform education policy discussions. We are guided by the belief that the democratic governance of public education is strengthened when policies are based on sound evidence. For more information on NEPC, please visit http://nepc.colorado.edu/.