Ballou's review offers insight into how the study can be improved and used to analyze the results. Unfortunately, The Times took the report placed it on page one and misused the data to sell newspapers.
Nicholas Kristoff, also of The Times, furthered the media blitz with: The Value of Teachers. His column took some wild leaps with the data.
Having a good fourth-grade teacher makes a student 1.25 percent more likely to go to college, the research suggests, and 1.25 percent less likely to get pregnant as a teenager. Each of the students will go on as an adult to earn, on average, $25,000 more over a lifetime — or about $700,000 in gains for an average size class — all attributable to that ace teacher back in the fourth grade. That’s right: A great teacher is worth hundreds of thousands of dollars to each year’s students, just in the extra income they will earn.
In his piece, Kristoff acknowledged that others warned him against the type of policy conclusions he was making, yet he made them anyway.
Suppose that the bottom 5 percent of teachers could be replaced by teachers of average quality. The three economists found that each student in the classroom would have extra cumulative lifetime earnings of more than $52,000. That’s more than $1.4 million in gains for the classroom.
Ballou specifically warns that using the data in the way Kristoff did is inaccurate, because there are too many variables which are not yet accounted for in the original study.
Inevitably, concern arises that the model has not controlled for enough other factors: that something omitted becomes confounded with the effect of the teacher. The mere fact that the model does not control for every influence on student achievement is not automatically a source of bias. Such influences need to vary systematically by teacher—that is, some teachers are normally assigned students who, for reasons not observed, score higher than expected, while other teachers are normally assigned students who do worse.
So while the report offers "one of the most dense, important and interesting analyses on this topic in a very long time" (Matthew DiCarlo, the Shanker Blog), Ballou's review should raise caution about overemphasizing the findings.
Thus, the report’s key findings linking teacher value-added scores to outcomes such as later earnings are not sufficiently validated; more evidence on that point needs to be presented before policy and practice are shaped in response to this report.
So much has been written about the Chetty study that blogger Larry Ferlazo has created The Best Posts On The NY Times-Featured Teacher Effectiveness Study.
Matthew Di Carlo at the Shanker Blog tackled the study with The Persistence Of Both Teacher Effects And Misinterpretations Of Research About Them.
Soon after the media blitz, Rutgers Bruce Baker wrote on his School Finance 101 blog: Fire first, ask questions later? Comments on Recent Teacher Effectiveness Studies.
Much has been written about the Chetty report on teacher value-added, but nothing as detailed as Ballou's review. I encourage you to read Ballou's review and dig deeper. Critical analysis of research, peer review, and policy implications need to be taken into consideration before we start jumping to conclusions.