As the idea of merit pay gains momentum, many teachers fear they will be unfairly evaluated, and according to the latest study by economics professor Jesse Rothstein, their fear is warranted.

Rothstein claims that value-added models aren't fair to teachers because students are not randomly assigned to their teachers. Principals routinely make assignment decisions, some for good reasons and some not so much such, as assigning problem students to the more unflappable teachers or giving the most favored teachers the best students. In particular, Rothstein contends that if one of those favored teachers had to teach the more difficult students, that teacher might not look so great after all, challenging an assertion made in a 2008 paper by his colleagues Doug Staiger and Tom Kane.

Rothstein also walks us through the tricky subject of the "longevity" of teacher effects. Students may make big gains one year under Teacher A, but Rothstein notes that too seldom do they achieve similarly big gains in the next year under Teacher B, no matter how effective Teacher B has otherwise been. What does this pattern suggest? He thinks it is a sign that some test score gains are real, but some are decidedly not, and that some teachers are getting credit they don't deserve. For instance, some teachers teach to the test, obtaining great test scores, but not very real or permanent results. On the other hand, some teachers may not be getting credit that is deserved, as their hard work may not show up in the test results for two or three years. E.D. Hirsch has also observed these problems in his critique of current language arts tests,which rarely link to the actual curriculum.

Rothstein doesn't write off value-added models. He believes they could be used to generate relatively unbiased measures of how teachers are performing, but designing the more accurate models would require insight into common assignment decisions made by principals across the country.