Contact:
Donald B. Gratz
(617) 333-2130
dgratz@curry.edu

William Mathis, NEPC
(802) 383-0058
William.Mathis@colorado.edu

BOULDER, CO (March 3, 2011) – A new report evaluates the incentive-based payment system for Texas teachers. A review released today finds that this report is professional and thorough, but limitations in the data and analyses caution against its usefulness in guiding policymaking, according to a new review of the document.

Think Twice think tank review project reviewer Donald Gratz, chair of the Education Department of Curry College, found that the report, published by Vanderbilt University’s National Center for Performance Incentives (NCPI), “suffers from two significant limitations: data not available and questions not asked.”

The review is published by the National Education Policy Center, housed at the University of Colorado at Boulder School of Education.

The NCPI report, District Awards for Teacher Excellence: Final Report, comes amid growing interest in ways to tie teacher pay to performance. As the largest program of incentive-based teacher pay in the nation, the Texas District Awards for Teacher Excellence (D.A.T.E.) plan is “bound to attract attention,” Gratz writes.

The experimental Texas plan is voluntary, with districts opting in if they want to share in a pool of $150 million to $190 million annually. This money funds merit awards that are made to teachers on the basis of their students’ performance on the state achievement test plus other factors determined by individual districts. Sixteen percent of the state’s districts participate, and these districts, which tend to be urban and poor, serve nearly half the students in the state.

Although Vanderbilt NCPI identifies the District Award report as “Final,” Gratz points out that the report is based on only one or two years of data. The analyses of teacher perceptions of the program and of teacher retention are based on a single cycle of awards for teachers. The analysis of student achievement data covers just two years. This is, Gratz notes, “not enough data to draw solid conclusions.” Moreover, he writes that “little is known about how districts actually implemented D.A.T.E., or about such factors as school culture, leadership and class size.” The report’s authors also acknowledge many of these limitations, stating for example, “It is important to emphasize again that this is a simple comparison, and does not account in any way for school or student characteristics that might also influence students’ gains” (p. 91).

Gratz found that the report focuses mostly on describing the Texas program, which districts took part, and why and how they designed their version of the program. As a program evaluation, the report “does not consider broader questions such as whether incentive pay produces positive results, not just higher test scores, and under what conditions.” Nor does the report consider the validity of using the Texas achievement test (TAKS) as the single measure of achievement.

The review also points out that while the report finds large differences in the how participating school districts implemented the program, “[t]he report can’t determine whether higher test scores are due to better teaching, more drill, professional development, or something else.”

Gratz credits the report for presenting “many intriguing findings about teacher attitudes, the potential impact of group versus individual awards, and other program factors.” He also praises the authors for noting many, although not all, of the key limitations of the report. However, he concludes that its evaluation design does not yield results from which policymakers can determine the success of the program. The report’s findings, according to Gratz, “are interesting but not sufficient for guiding policy.”

Find Don Gratz’s review on the NEPC website at:
http://nepc.colorado.edu/thinktank/review-district-awards.

Find District Awards for Teacher Excellence Program: Final Report by Matthew Springer and his colleagues on the web at:
http://www.performanceincentives.org/data/files/news/BooksNews/FINAL_DA…

The Think Twice think tank review project (http://thinktankreview.org), a project of the National Education Policy Center, provides the public, policy makers, and the press with timely, academically sound, reviews of selected publications. The project is made possible in part by the support of the Great Lakes Center for Education Research and Practice.

The mission of the National Education Policy Center is to produce and disseminate high-quality, peer-reviewed research to inform education policy discussions. We are guided by the belief that the democratic governance of public education is strengthened when policies are based on sound evidence. For more information on NEPC, please visit http://nepc.colorado.edu/.

This review is also found on the GLC website at http://www.greatlakescenter.org/