We know teachers are more than just the sum of their students' test scores. That's why teacher evaluations don't just look to VAM scores but also include observations—a measure which isn't actually all that reliable but it's the devil-you-know syndrome at work. 

Districts struggle with the desire to capture the many positive influences teachers have on their students apart from learning, influences which are much harder to quantify—such as students' enthusiasm for coming to school each day, their willingness to behave in some teachers' classrooms but not others, or their motivation to work hard in a course. 

Though it's early days yet, a newly-developed method may offer some solutions. This method, called N-VAM (the "N" stands for "non-tested"), is based on the same analysis framework as the more traditional test-based value-added modeling (VAM). While not the first to experiment with N-VAM, researchers Ben Backes and Michael Hansen of the American Institutes for Research tried it out in a recent study of TFA teachers' impacts on non-tested outcomes for students in the Miami-Dade County School District. The non-tested outcomes included the number of students' absences, GPA and the number of courses failed. 

Notably, the N-VAM data proved to correlate little with VAM scores, showing that a teacher's effectiveness on academic outcomes did not always correspond with her effectiveness for non-tested measures. This gives credence to the argument that measuring a teacher's effectiveness using test scores may overlook other key ways that teachers benefit their students, a finding that other studies support.

In any case, while this new methodology may not be ready for prime time, Backes and Hansen believe the results are suggestive and merit future research into this analysis framework. They express some concerns over the statistical validity of the results (identifying some areas in which the forecast bias was too great, meaning that the actual results and predicted results were too different for them to have confidence in their model). They also decline to make claims that the teachers caused their students' non-tested outcomes in this study.

Perhaps the N-VAM will turn out to be no more than a single shot across the bow, but the idea has so much merit, we're eager to see more.