Should K-12 student surveys of their teachers be used to determine whether they get a boost in salary or be judged effective or ineffective?

The emerging answer, according to Amanda Ripley’s recent article, is yes. Based on surveys given to students about their teachers over the past decade, student judgments about their teachers are highly correlated with test scores. In particular, researchers have found these questions on a 36-item survey to bear the highest association with test scores. (see preliminary-findings-research-paper-1, pp. 11-16)

1. Students in this class treat the teacher with respect.

2. My classmates behave the way my teacher wants them to.

3. Our class stays busy and doesn’t waste time.

4. In this class, we learn a lot almost every day.

5. In this class, we learn to correct our mistakes.

These questions measure classroom control and the degree to which teachers challenge students to work harder with academic content and skills.

According to Ripley:

Memphis became the first school system in the country to tie survey results to teachers’ annual reviews; surveys counted for 5 percent of a teacher’s evaluation. And that proportion may go up in the future. (Another 35 percent of the evaluation was tied to how much students’ test scores rose or fell, and 40 percent to classroom observations.) At the end of the year, some Memphis teachers were dismissed for low evaluation scores—but less than 2 percent of the faculty.

The New Teacher Project, a national nonprofit based in Brooklyn that recruits and trains new teachers, last school year used student surveys to evaluate 460 of its 1,006 teachers. “The advent of student feedback in teacher evaluations is among the most significant developments for education reform in the last decade,” says Timothy Daly, the organization’s president and a former teacher.

In Pittsburgh, all students took the survey last school year. The teachers union objects to any attempt to use the results in performance reviews, but education officials may do so anyway in the not-too-distant future. In Georgia, principals will consider student survey responses when they evaluate teachers this school year. In Chicago, starting in the fall of 2013, student survey results will count for 10 percent of a teacher’s evaluation.

Of course, the cat is way out of the bag now. Once high stakes (e.g., salary, getting fired) are attached to a metric–student perceptions of teacher performance–then count on that measure being resented by teachers and then gamed. If the hope was that a useful tool such as student evaluations of teaching could be used to improve classroom practice, forget it. As one journalist put it:

I don’t doubt that student surveys could, in theory, be very useful in the large task facing administrators and teachers — how to make schools better and improve the quality of the education they provide. They would show where schools were weak and where they were strong; which teachers have managed to crack certain nuts where the rest of the faculty is having difficulty; that kind of thing. In short, they could be tools for diagnosing and improving the quality of a school’s education as a whole.

But the reformers rush straight past all that, and decide that the first best use of such data is to use it in performance reviews, and use it to give raises to good teachers and pink slips to bad ones.And, of course, the minute you start doing that, it becomes impossible to use the data for anything else, since the scores then become an end in themselves, rather than a means to an end.

In higher education, students have evaluated their professors’ teaching and course content for decades. Depending upon the institution (community colleges, small private colleges, large land-grant institutions, Ivy League schools), student ratings are used to varying degrees in salary and tenure decisions but they remain controversial. (see Student ratings of professors, American Psychologist).

Controversial or not in higher education, reform-driven policymakers and foundation officials, eager to find another metric beyond unstable end-of-year test scores that simply and inexpensively judges K-12 teacher performance, look to researchers to quantify student perceptions of how and what their teachers teach. Students do know a great deal about their teachers and professors; they sit in classrooms hundreds and thousands of hours each school year. Such information can be useful to help teachers and schools improve.

Not, however, if student perceptions of teaching are sliced and diced to fit into little boxes that can be checked off by principals and superintendents to determine teacher effectiveness and pay. Were that to occur, its usefulness will approach the likelihood of most people drinking rat poison.