As we discussed
earlier this month, holding teacher prep programs accountable for the
performance of their graduates is no easy task. The data is often scant and
researchers usually can’t distinguish any standouts in a sea of mediocre or weak
programs.
That’s why we are
pretty enthusiastic about a new study
from Matthew Ronfeldt and Shanyce Campbell of the University of Michigan. Previous
studies looked only to one data source—graduates’ value-added scores—to determine the
strength of program graduates. These
two researchers use multiple measures involving, first, teacher observation
scores and, second, value-added scores. They unearth clear evidence that others
have not: not all programs are created equal.
In the sample of 118
programs, 21 surface for graduating teachers who consistently have either
higher observation scores than most other programs, or, conversely, consistently
lower scores.
The waters do get
muddied a bit when folding back in the value-added measures. Not surprisingly,
programs that did really well or really badly on observation scores didn’t
always have similar results on value-added. In fact only about 40 percent of
the programs produced observation and value added scores that were similarly
positive or negative.
Nevertheless, if a
policymaker were to assess program quality by looking only at the overlapping
data, it seems safe to conclude that there are programs clearly succeeding or
failing–producing teachers who consistently get both great evaluations and great test score results or the
reverse.
When all was said and done,
there were #25 standout programs in the state, but as is the frustrating custom
of academic research, these programs were not identified.
These promising results
reinforce our interest in multiple measures for evaluating program quality. One
such additional measure could be provided by TPI-US, essentially a
comprehensive on-site inspection process imported from the United Kingdom. In its
assessment process, teams of four trained education professionals visit prep
programs to collect evidence on program quality as well as to provide actionable
feedback. They observe student teachers and course instructors, examine data on
candidate performance, and conduct interviews with key stakeholders, including
graduates and leaders at the schools that hire them—all of which could serve as
yet another source of data on a program’s quality.
