Research & Insights

This month, we analyze school calendars for the 2016-2017 school year to answer some key questions. Is school really starting earlier? How long is the school year for students and teachers?

It’s a popular movie theme–the story of that one teacher who really connects with
students, setting them on the right path to a productive future.

For most students, the start of middle school represents
some newfound independence. For the first time, they get to travel the halls
from class to class without being led by an adult.
Of course, this new freedom stems from how middle
schools are set up. Few teachers have the training to teach every subject in
the middle school curriculum so instead of having one primary teacher, there
are at least four. Students find themselves traveling to the teachers, each a
specialist in their subject area.
There’s been a lot of debate over the years about when
to introduce subject specialization. In fact many elementary schools introduce
the model at 4^th grade.
But a strong new study from Harvard economist
Roland Fryer calls into question that practice.
Fryer conducted a two-year experiment in 50 public
elementary schools in Houston, Texas. Half of the schools remained on a
traditional elementary school schedule; in the other half, principals assigned
teachers to teach specific subjects based on their observed strengths and
previous value-added scores.
For the most part, assigning teachers to specific
subjects didn’t make much of a difference—and when it did, that difference was
generally negative. Reading and math scores declined, while problem behaviors
and absences actually increased—albeit to a small degree. Basic economic theory
predicted an efficient assembly line in which every teacher installed her
piece; reality told another story.
It’s hard to know exactly why the experiment didn’t
have better results. Perhaps teacher specialization would have a positive
impact if teachers received extra training in their assigned subjects. Perhaps
lost time getting to know each individual student eliminates any benefits that
could come from specialization.
Regardless of the cause, the results serve as a
helpful reminder that elementary
teachers need a strong grounding in all four traditional content areas, not
just the ones they’re more drawn to, as principals can’t simply
rearrange staff based on interest or inclination and expect better student outcomes.
Further, Fryer’s study is robust, using a large sample size, multi-year
implementation, sound experimental design, and realistic policy implementation.
As this study failed to find a benefit to teacher specialization in Houston’s elementary
schools, it’s unlikely that other districts would fare better. 

The entire Teacher Preparation Strategies team was saddened to learn of the death of Sam Stringfield, a member of the Technical Panel for the Teacher Prep Review. Sam’s passion for better teacher preparation was evident in all of his advice, often expressed in no-holds-barred emails that overflowed with wit and wisdom.

One feature of an ideal school environment, we
believe, is that both students and their teachers reflect America in all of its
diversity. Everyone, including children coming from privilege, benefits from a
diverse school experience.
It’s no wonder that as the minority student
population has grown in numbers in the past few decades, surpassing 50 percent,
so many of us are troubled that the minority teacher population has failed to
keep pace, standing now at only 18 percent minority. The push to achieve racial
parity between teachers and students has never been stronger, with urgent calls
from school boards, states, and even the U.S. Department of Education for the
nation to make teacher diversity a top goal for school districts.
This sentiment is understandable, commendable, but
— at the risk of being a wet blanket — not even remotely achievable within the
foreseeable future.
No matter how you run the numbers, school districts
simply cannot recruit and retain enough black and Hispanic teachers to achieve
racial parity between the teacher workforce and the U.S. student body – no
matter how many reprimands HR officials have to face from the their school
boards for the paltry results.
Researchers from NCTQ and The Brookings Institution
recently analyzed what it would take to create a teaching workforce as diverse
as the students it serves. You can find a full report of our analysis and
findings here.
We estimate the effects of different interventions
that might increase the number of minority teachers, extrapolating population
projections for the next four decades to see how close they can come to
creating a racially representative teacher workforce.
The findings are startling: parity or even
significant inroads to parity will remain completely elusive unless we fire on
all cylinders. If we can somehow boost the rates of college completion,
interest in teaching, hiring, and retention so that these rates for black and
Hispanic college students and adults mirror the rates of their white
counterparts, then parity becomes an achievable goal – but is still several
decades away.
How can this be?
Let’s walk through what we found, drawing on a
model we built using U.S. Census projections and data about the current teacher
and student populations to estimate the impact of various interventions.
For example, let’s see where a big push on
retention of black teachers gets us. Currently, 16 percent of America’s public school
students are black, compared to 7 percent of teachers, creating a nine
percentage point gap in the diversity of students and teachers.
What could happen if schools took real and
substantive steps to retain their black teachers so that year in and year out
black teachers stay in the classroom at the same rate as white teachers
(improving their current attrition rate from 10 percent to mirror white
teachers’ 7 percent)? We still wouldn’t close the diversity gap even projecting
out to 2060 (the furthest year of Census projections available).
Next, imagine we committed ourselves heart, body,
and soul (as many districts are, in fact, trying to do) to hire more black
teachers. The rewards would be tiny even projecting all the way out to 2060.
Okay, so what about persuading more black college
students to consider teaching? Higher education could heavily promote teaching
with undergraduates, or graduate programs and alternative providers could
recruit more black candidates—something that many claim to be doing already.
But let’s imagine pouring some real resources into these strategies, such as increased
salaries, loan forgiveness, or more leadership opportunities to succeed in
persuading black adults to consider teaching at the same rate as white adults (currently
4.3 percent of black undergraduate students major in education compared with
6.9 percent of white undergraduates; we see similar disparities for graduate
college education degrees and alternative certification enrollment).
Again, the results would be fairly paltry – closing
the diversity gap by about two and a half percentage points by 2060.
I’m guessing that a lot of people reading this are
thinking “but look at the success of Teach For America, now recruiting new
cohorts which are about 50 percent teachers of color?” TFA was able to achieve
these huge gains in part because it developed unprecedented recruitment efforts
of minority students beginning in their freshmen year—a great strategy to
emulate. But keep in mind that the corps is tiny compared to America’s needs.
Teach For America supplies less than 3 percent of the nation’s teachers. Its
hard push on this problem still only produced about 800 new black corps members
in a year. That’s enough to translate into significant gains for TFA, which is
only recruiting some 4,100 teachers in a year, but remains a far cry from the
300,000 more black teachers needed to achieve parity across all American
schools.
Let’s go back to the point of greatest disparity
between black and white students: the college completion rate. If we invest
heavily to support black students and ameliorate their low college completion
rate so that they graduate college at the same rate as white students
(currently 28 percent of black 22-year olds have earned a bachelor’s degree,
compared with 47 percent of white 22-year olds), we still will not come close
to closing the gap by 2060.
Disheartening, no?
For Hispanic teachers, the dismal scenario is much
the same. In fact because the Hispanic population in the US is growing at such
a fast rate, much faster than the black population, the diversity gap is
expected to widen if we do not take action.
Only if we are able to graduate more Hispanic
teachers from college or draw more Hispanic adults into careers in teaching can
we reduce the growing diversity gap significantly, otherwise expected to be 22 percentage
points by 2060.
Clearly, the answer is to combine all these
interventions and be successful at
all of them (success defined here as achieving the same rates as their white
counterparts) to make any real dent. If over the next decade we were to improve
college completion rates, interest in teaching, hiring, and retention to mirror
that of white teachers at every point, we would actually achieve parity by the
year 2044 for black teachers and students. The picture is less cheery for
Hispanic teachers, only coming within the ballpark (three percentage points
away from racial parity) by consistent pushes through 2060.

So are we
suggesting we all throw up our hands and give up on achieving greater parity?
Absolutely not. But let’s not underestimate the ambition, commitment, and
persistence needed, or browbeat school superintendents or human resources
officials when they are only able to make incremental progress. Let’s also not
advocate for racial parity at the expense of quality. The research is clear
that students’ success still depends most on the quality of their teachers
In the meantime, other important solutions can
achieve greater equity in our schools. Let’s look for meaningful ways to ensure
that teachers – who want to do right by their students – don’t unintentionally
do harm to the kids who don’t look like them. Let’s go beyond the stuff of
current trainings in which teachers and teacher candidates engage in a lot of
reflecting on their implicit biases, analyzing their white privilege, or
developing cultural sensitivity, much of which hasn’t had much impact –
although these do play an important role. Let’s do more to have teachers
examine their daily interactions with students by asking themselves: which
students do they call on to answer questions? Are some students more likely to receive
harsher disciplinary actions than others? Do they select some groups of students
for more challenging work over others? Give teachers the training and tools to
make sure that they do not allow their biases along race, gender, class, or any
other lines to get in the way of helping every child thrive.

You
can’t teach what you don’t know—and
when it comes to math, some teachers just don’t know enough.  Why?  

Michigan
State University professor William H. Schmidt and his colleagues look at the
mathematics that American middle school teachers took during their teacher
preparation and produce some hard evidence of prevailing substandard
preparation.
First
though, recall some earlier
work in which Schmidt examined how teachers were prepared in 16
countries. There, he unearthed the essential knowledge for middle school
teachers based on the relationship between coursework and teachers’ strong
performance on an international assessment of math content knowledge and math
pedagogical content knowledge. The top performing teachers had all studied nine
topics, including linear algebra, calculus, probability, differential
equations, functions/equations and also had opportunities to analyze math
instruction.
This latest study looks
more closely at US preparation, examining how many US programs deliver this
essential content. Even on this basic question, Schmidt finds enormous,
inexplicable variations among institutions in what they consider to be
essential content. Schmidt estimates that only about a third of America’s
middle school teachers took coursework addressing this content. For the rest of
teachers, a sizeable portion of the content never gets covered. Compare that to
some of our international counterparts which include a number of countries
where 80 percent of all teachers learned essential content. 

Percentage
of Middle School Teachers Who Completed at Least Eight of Nine Essential Math
Courses

If half the battle is just showing up, some teachers have already lost the war.

That’s what we learned in our 2014 study Roll Call: The Importance of Teacher Attendance. In 40 large school districts, we found that while most teachers come to work regularly, a significant number of teachers (44 percent across the districts) missed a heck of a lot of school—11 or more days in an average school year.
Last month, the US Department of Education followed suit, releasing data that reinforces our findings. And, just like Roll Call, it revealed inexplicable differences among states, with Utah and South Dakota reporting relatively low rates of chronic absenteeism but others faring much worse, with 75 percent of teachers in Hawaii and 49 percent of teachers in Nevada missing more than 10 days per year.
Neither NCTQ nor the Education Department data sheds light on why some schools struggle so much with this basic personnel practice. So there’s much to be said for new research from Seth Gershenson which unearths at least one device that improves teacher attendance: threat of sanctions.
Looking at data from the early days of No Child Left Behind, Gershenson examines how K-5 teachers in North Carolina responded when their school failed to meet adequate yearly progress (AYP). Teachers in schools that failed to make AYP in 2003 significantly reduced their absences in the following year by about 10 percent. Chronic absenteeism (when teachers miss 15 or more days) fell by 20 percent.
Not surprisingly, these declines in teacher absences translated to real gains in student achievement.
NCLB was certainly not without its flaws, but this study serves as another reminder that strong systems of accountability lead to positive outcomes—and that every day a teacher spends in her classroom matters to her students. The challenge of teacher absences is unlikely to have a single solution, but we’re grateful for any additional insight.

We’re thrilled to welcome NCTQ’s newest addition to
our Board of Directors, Paul Kihn. Paul
brings with him the ideal resume to guide NCTQ into the future having taught
middle school both in the US and South Africa, earning his school district
stripes as the #2 in Philadelphia’s public schools and as an educational consultant for McKinsey &
Company. Welcome Paul! 

In order to be well prepared for daily instruction, teachers
need time without students to plan and collaborate. This month, Trendline delves into how long the teacher workday is and how much of that time
is devoted to planning and collaboration.

Every great teacher knows that her grading practices need to be fair. And any good policymaker should know that teachers expect the same of their own performance evaluations.

That’s one of the reasons why in 2014, the U.S. Department of Education offered states the option to delay using student growth data to evaluate teachers during the transition to new state standards and assessments. “These changes are incredibly important,” Arne Duncan
explained, “and educators should not have to make them in an atmosphere of worry.”

Still, many wondered: Would the new test scores
actually
have led to inaccurate value-added measures for teachers?
To
help settle the essentially theoretical question, Ben Backes and his colleagues
examined value-added
scores from before, during, and after states’ transition to new standards and
assessments. The five-state study, conducted with support from CALDER, included
changes that occurred during the Common Core era and as far back as 2001.

The
conclusion: assessment data from transition years would have produced the same
evaluation results for teachers as the older tests.

The
researchers analyzed the data from multiple angles, including the correlation
of value-added scores from year to year, the likelihood that a top- or
bottom-ranked teacher in one year would rank in the same range the next year,
and differences in teacher performance rankings by classroom type (advantaged
or disadvantaged). All in all, transition-year math assessments performed about
the same as in other years across all five states; in reading, two states saw
slightly more variability during transition periods.

It’s
good news, but it doesn’t mean the one-year moratorium was a mistake. For
teachers who were already skeptical of value-added in 2014, the message that
transition-year data would likely be much the same as data from any other year
probably wouldn’t have soothed any anxieties. A smart political decision, even
if there was no crisis to avert. 

Of late, I’ve been holding my own internal debate. Is getting states to change their teacher policies worth all the time and effort that it takes x 50?

For nearly a decade, NCTQ has taken an aggressive stance on the need to fix states’ teacher policies and we’ve had some great successes to boast about. Over that period, all but eight states have made significant improvements to their teacher policies. Yet, I’m still left with a nagging feeling that maybe these impressive changes aren’t having the game changing impact on teacher quality we anticipated. There’s all too much evidence of their derailment by school districts and teacher preparation programs.
But I’ve now encountered new evidence of the harm that even seemingly innocuous state policies have on teacher quality. While state policies will never be the full answer, I’ve concluded, they must be an essential part of the equation.
What’s that evidence? Last month we released a study looking at an array of essential content and skills that preschool teachers need but aren’t learning about in their teacher prep programs, skills like how to handle a disruptive four-year-old or how to build language skills and lay the foundation for learning to read.
As we dug into why so much essential—and noncontroversial—content was missing from these programs, we hit on an interesting pattern. It turns out that the more grade levels any one prep program tried to cover in teacher training, the less likely a program was to deliver the content preschool teachers need.
This chart explains more. It shows the big variations in grade spans among states for the purpose of certifying teachers. States, after all, don’t certify a teacher to only teach first grade but are more likely to certify a teacher to teach any elementary grade.
In states like Mississippi and West Virginia, the state requires that preschool teachers have to train with only aspiring preschool and kindergarten teachers. Accordingly their coursework is more geared to that specialized content. In other states, however, it is acceptable for a single program to prepare teachers for anything from preschool to grade 6 (Texas) or even birth to grade 6 (Wisconsin), setting the same course requirements for everyone in the program. In Oregon, it is even possible for preschool teachers to train with teachers intending to teach 8th grade!
It may be tempting to blame this problem all on the universities where the training occurs. After all state officials would be right in claiming that these teacher prep programs could choose to more narrowly train teachers no matter what state code dictates. They rarely do though.  It is surely considerably less expensive for a university to manage a one-size-fits-all program over several more narrowly focused programs.
This is not just a higher ed problem however. Teachers love the flexibility of broad certifications, as they can teach more grades without having to go back to get recertified. Principals and districts love it even more because the staffing flexibility makes scheduling a lot easier. 
I’ll never forget a conversation I had with some folks running what is largely regarded as one of the best STEM teacher prep programs in the country. They bristled at NCTQ’s criticism over the preparation of candidates under the state’s highly flawed general science certification. “We’re only giving the principals in this state what they want,” they asserted. “They only want science teachers who are certified to teach any science. They don’t really care that they’re not really all that well qualified.”
That brings me to my second point.  The only entity that can claim disinterest here is the state and that’s often the case. To the state falls the responsibility to do what is right by teachers, because neither higher ed nor schools are apt to work against their own self interests. All of which brings me back to my original concern: what happens even if the state does the right thing? In this case, if Texas, Wisconsin and Oregon (to name a few of the worst offenders) were to adopt more narrowly defined certifications for training preschool teachers, would the problem be fixed? No, but it’s the first hurdle. The second hurdle is to get prep programs to change what they teach. A move by the state not only shifts the onus of responsibility to teacher prep programs but it sends an important signal that how we prepare teachers for our young children matters. 

This month, the Trendline takes a look at substitute teachers’ required qualifications, salary, and benefits. We find some significant changes in the pay and benefits provided to substitute teachers since our last look only one year ago. 

30
million words. It’s a staggering number representing the tremendous gap in the
number of utterances (not distinct words) a child hears by age
four—depending on where his family falls on the socioeconomic spectrum.

Preschool
is regarded by many as the best opportunity for an early course correction,
allowing all children to be launched on a successful
academic—and life—trajectory no matter what is happening at home. Yet the jury
is still out on whether preschool can deliver on that promise. Research on the
long lasting contributions of preschool is at best mixed.
There’s
a lot of conjecture about why preschool investments may not
pay off as much as one might expect: the length of the preschool, parents’ involvement,
support services, low quality curricula, and poor oversight.
One
more plausible but largely ignored explanation remains: poor and/or
uneven preparation of preschool teachers.

While
preschool teachers’ dismally low salaries have garnered lots of attention, even this week, there’s
been little attention to the training given to preschool teachers before they
enter the classroom.
A new
NCTQ study to be released next week explores whether programs are
delivering essential content preschool teachers need. After all, beyond
being patient and caring, preschool teachers must be able to learn
and practice a wide swath of essential skills.
Put
more practically, when faced with the tremendous gaps in language skills in a
classroom of 20 four-year-olds, how many of us would have an inkling of where
to start?
Many
preschool advocates heavily champion the notion that all preschool teachers
must earn a bachelor’s degree in order to guarantee sufficient quality. Some 33
states now require that the preschool teachers in the programs they fund must
have bachelor’s degrees.
Though
there’s been little hard research to help guide that requirement, states made
what seemed like a safe bet, assuming that teachers who earn a bachelor’s
degree are more likely to learn how to create a higher quality preschool
environment than teachers who do not earn one.
Yet,
as it turned out, the assumption was just that, an assumption. 
Our
new study out on June 22nd, Some Assembly Required, takes a
look at a healthy sample of 100 teacher preparation programs located in 29
states. The vast majority of these 100 programs conferred
degrees: 54 led to a bachelor’s degree and 41 to a master’s, as
well as 5 at the associate’s degree level. We looked at course
requirements, course descriptions, syllabi, student teaching handbooks,
observation instruments, and textbooks to identify high-level evidence of key
content.
A review of these multiple factors reveals that almost all of these programs fail
to reference even basic
content needed for a teacher to learn how to build children’s language, a
precursor to becoming a successful reader.
In many programs, the instructional skills specific to the
job of preschool teaching are marginalized, buried by other content only relevant to a teacher of later grades. Even on paper,
programs do not claim to devote much—or any—time to training candidates in
developing young children’s language skills and vocabulary or building critical literacy skills.
It’s
not just language and literacy that get short shrift. We found even less
evidence of programs attending to the knowledge preschool teachers need to
build math skills, explore early science concepts, or help children develop
executive functioning skills. 
We’re
all for raising the bar for educators, especially those that play the crucially
important role of teaching young children. A bachelor’s degree makes
sense. But we need to make sure that the sheepskin earned by the preschool
teacher stands for something relevant and valuable.
Want
to learn more? Join the webinar next Wednesday, June
22.

In the face of all evidence on the (apparently) utter uselessness of most professional development, might there be something that works? A new NBER working paper claiming just that perked up our ears.

Researchers John Papay, Eric Taylor, John Tyler, and Mary Laski employed a relatively simple approach. High- and low-performing teachers in Tennessee were paired together based on their professional strengths and weaknesses and then asked to spend a year developing the low-performer’s skills that needed improving.
That’s it.
No new, mandatory meetings. No expensive coaches. No new policies. And cheap. The only real cost to schools was the time the teachers needed to work with each other—and even that was left to the discretion of each school.
The results were astounding: in classrooms taught by low-performing teachers, students scored, on average, 0.12 standard deviations higher than students in the control classrooms. To put that in perspective, the authors compare this improvement to what you would expect from a student assigned to a teacher ranked at the median rather than in the bottom quartile—or to an experienced teacher rather than a novice teacher. There was also evidence to suggest that the gains in teacher performance persisted or even grew in the year after the partnership was completed.
Might this be a one-hit wonder? After all, it only looked at 136 teachers who all taught in the middle grades. A study of the impact on a larger sample is in the works. For the cost, though, we think it might be worth a try elsewhere!

We’ve known for years that students of color are underrepresented in gifted education programs across the country.

Over the last few decades, the field of teacher education
has heavily promoted the use and even the mandatory adoption by states of
standardized assessments that can be used to judge how well a teacher candidate
can deliver a lesson. The push has come with a promise that these
“TPAs” fulfill three important and otherwise largely elusive functions:

1. 
They would help programs to better structure
training on how to plan and deliver instruction far better than the home-grown
efforts used by most programs.

2. 
They identify the quality of candidates and
therefore can serve a gatekeeper for entry into the profession.

3. 
They provide information which states could use
in the aggregate to hold programs accountable for their training.

We concluded four
years ago that the edTPA adequately fulfills the first function. TPAs are
an exponential improvement over the rubrics and observation forms most programs
use to assess a “live” lesson.

A new
study from three researchers at CALDER (Dan Goldhaber, James Cowan, and Roddy Theobald) provides evidence
on this second function—whether TPAs actually identify the more capable
candidates who deserve to be entrusted with a classroom of children.  That’s long overdue given the pressure that
AACTE and others have put on states to adopt an instrument without any evidence
that it was predictive of teacher performance.

As was widely
reported in the press last week, results from graduates of teacher prep
programs in Washington state (one of the first states to adopt the edTPA) are
mixed. Goldhaber et al. found that a passing score in the reading portion of
the edTPA is significantly predictive of teacher effectiveness in reading, but
not in the math portion.

Given that the edTPA is a lot of work for programs and is
costly to boot, is this enough bang for the buck? After all, instructions for
candidates entail 40 pages and candidates are alerted that they can be
evaluated on the edTPA on nearly 700 different items. The process consumes the
attention of teacher candidates and teacher educators in their programs for a
good share of candidates’ semester-long student teaching placement. 

There’s a strong argument that the complexity is merited if
it prevents unqualified persons from teaching—unless the same results could be
had with a lot less time and investment.  A recent
study on the measures employed by District of Columbia Public Schools to
screen its teacher applicants indicates that one of the components with
predictive validity is simply a 10-minute audition. A few more studies with
similar results may make it difficult to justify blanketing the nation with edTPA
requirements.
That leaves the third function: program accountability. According
to Goldhaber, the variation of scores of candidates within Washington programs is greater than the variation of scores across programs. This result, which
Goldhaber did not publish, means that all but the most egregiously low performing
programs are likely to have candidates whose scores vary considerably across
the range.  

The bottom line to date in this still-unfolding story about
TPAs: the edTPA 1) is a good organizing vehicle for training, 2) may produce
scores that at least partially discriminate among candidates in terms of
effectiveness—but through a process that appears to be unnecessarily
cumbersome, and 3) may produce scores that cannot be used to hold programs
accountable because they are insufficiently related to the quality of candidate training. 

This month’s Trendline takes a look at the length of
probationary periods across districts in the Teacher
Contract Database and how much flexibility districts have in the decision
to award tenure.

Isn’t it amazing how someone or some things gain traction when the facts clearly aren’t on their side? 
Just like the media handed off most of its airtime and column inches to elevate Donald Trump’s candidacy, so too is the media guilty of announcing a crisis in teacher supply when the facts just don’t support it. 

While there are no data to suggest we are in the midst of teacher shortage, there are certainly school districts that are experiencing real problems—but they’re largely the same school districts which have been struggling for years, even in times the media was reporting big teacher layoffs.

There are some data to suggest that enrollment has dropped considerably in some teacher preparation programs in some states. But as we’ll show, it’s a big leap to say that the drop in enrollment means there won’t be enough teachers.

These two graphs from economist
Dan Goldhaber illustrate the importance of taking the long view.



This first graph does indeed
show that the number of new teachers produced since 2008 has declined. But keep
in mind that that drop was preceded by a
three-decade
period of enrollment growth
, far outpacing the demand year-in and year-out
(as the second graph shows). America’s 1,450+ institutions which train teachers
have been OVER-enrolling for years.

The current decline is what we
normally see when unemployment dips and the pool of folks looking for work
isn’t as large as in other years.

And as programs have not traditionally
seen it as their responsibility to direct candidates to shortage teaching areas
(e.g. special ed), there continue to be massive misalignment between the types
of teachers trained and the types of teachers public schools need to hire.

Most notably, programs have been
routinely graduating twice as many new elementary teachers as public schools
hire each year.

Even when confronted with these
facts, many districts have begun to panic. In part they have grown accustomed
to dealing with a pretty distorted labor market and the low quality of many
teacher candidates. They’re used to having to sort through a pile of resumes to
find a single good hire. They are also often not nimble or flexible enough to
adapt their recruiting and hiring practices to a tighter job market.

 What then is a reasonable response to a
downturn in teacher production?  It’s not
to open the floodgates and let just anyone teach. We need to continue to
encourage teacher prep programs to become more selective and do a better job
preparing new teachers so that districts don’t have to count on 20 resumes to
find a single qualified teacher.

Districts would do well to tap
into the enormous pool of the many hundreds of thousands of people who were
certified to teach but never did. Some estimates put the percentage of new
teacher graduates who don’t actually teach at 50 percent. Neither states nor
districts make it easy for those folks to reconsider the profession a few years
down the road.

The headline of a new study from Matthew A. Kraft (Brown University) and David Blazar (Harvard Graduate School of Education) caught our eye: Intensive, personalized coaching programs can improve teacher performance substantially.

Professional development that works? Well… perhaps.

The two researchers examined New Orlean’s yearlong adaptation
of a coaching program from Boston-based MATCH School. Participants spend a week
in a summer workshop followed by four one-week sessions of observation and
feedback. Importantly, the participants teach a range of grades and subjects,
distinguishing the program from more common coaching interventions that focus
on early literacy.

Teachers who got the coaching scored significantly higher
than the uncoached teachers on an index of teacher quality that includes
classroom observation scores, principal evaluation, and student surveys. In
fact, the impact of having been coached exceeded even the impact of a teacher
gaining more experience. The difference between coached and uncoached groups
was nearly 50 percent higher than the difference between novice and experienced
teachers.

Coached teachers outperformed uncoached teachers by 0.59 standard deviations. For comparison, experienced teachers (3+ years experience) outperformed novice teachers (1 or 2 years experience) by a smaller margin of 0.44 standard deviations.

Despite relying on a generally strong methodology, the size of the sample involving only 30 matched pairs tones down our enthusiasm a tad. But more importantly, the sample is restricted to teachers from urban charter schools who wanted to participate in the program—meaning that this approach may not translate well to other settings or less willing groups of teachers.

Still, given how little we know about how to help teachers improve, even preliminary evidence of some ways professional development might become more effective is noteworthy.

On behalf of the entire Teacher Preparation Strategies team, I want to express our sadness at the death of Barry Kaufman and our condolences to his wife of 50 years, Gail. Barry provided NCTQ with invaluable assistance for close to six years as a member of the Technical Panel for the Teacher Prep Review. Barry’s expertise—gained from serving as an ed school dean and as a member of the California Commission on Teacher Credentialing—was combined with an unwavering commitment to improve teacher preparation. He always made himself available to NCTQ staff for consultation and helped us to ensure that the picture of teacher prep painted in NCTQ reports is accurate. We will miss Barry’s endless goodwill and spot-on advice! 

A
lot of us can rattle off the possible shortcomings of using value-added test
scores to evaluate teachers: The scores vary from year to year. They lack
transparency. They cannot control for other events going on in the classroom,
like broken air conditioning or teachers consistently being assigned
exceptionally motivated students.
Sad
to say, these problems aren’t relegated to the statistical wizardry behind
VAM—it turns out classroom observation scores may suffer from many of the same
ills.
New
research from Matthew
Steinberg of the University of Pennsylvania and Rachel Garrett of the American
Institutes for Research employed data from the Methods of Effective Teaching study to look at how classroom
composition relates to teachers’ observation scores.
First,
they found that teachers who were assigned to high-performing students were
more likely to earn higher observation scores. They also found that some
domains of the evaluation instrument used in this study (the Danielson framework)
appeared to give teachers undue credit for traits, achievement levels, and
other factors students arrived with at the start of the school year; domains
like “engaging students in learning” and “establishing a culture for learning,”
which rely heavily on student-teacher interaction, were the primary culprits.
The authors offer two competing hypotheses for these higher scores: they could
be a sign of observer bias—meaning, for example, that a teacher might get a
score boost for having an eager and well-behaved class, even if she basically
inherited her students that way—or they could indicate that teachers either
perform or become better when working with a class of higher-achieving
students.
Finally,
teachers who teach multiple subjects, like most elementary teachers, had observation
scores that were less related to their students’ incoming achievement—unlike
the scores for teachers who only teach a single subject (and therefore older
grades). This difference could be because teachers’ observations in older
grades are spread across multiple classrooms, or because teachers who spend
more time with one group of students are better able to adjust to their needs.
So,
would Steinberg and Garrett’s findings hold true elsewhere? After all, the MET
study observation data relied on an unusually robust approach to teacher
evaluation, using highly-trained off-site evaluators who rated videotapes of
lessons. In contrast, districts more often rely on in-person observations,
often by school principals and APs.
Unfortunately,
the findings from more typical on-site evaluations by school administrators may
look even worse, according to
Whitehurst, Chingos, and Lindquist. These Brookings researchers tackled
the observation issue a few years ago and found that outside observers produce more valid observations than school
administrators—so Steinberg and Garrett’s work could actually understate the
problem.

Read more about observations in our
previous month’s Teacher
Trendline. 

Earlier this year, the Every Student
Succeeds Act (ESSA) launched “nonacademic factors” into the national
discussion, as such factors can now be used as part of school accountability
metrics. As states explore which factors to use, they may want to consider a
recent working paper that finds that teachers who are rock stars at raising
student test scores may not be the same ones whose students are either
well-behaved or even particularly happy.

Using a large sample of upper
elementary classrooms across four districts, David Blazar of Harvard University
and Matthew A. Kraft of Brown University examine teacher influence on math test
scores and students’ self-reported behavior, self-efficacy, and happiness in
math class. Their work revealed not only that teachers do indeed have a
significant effect on non-tested outcomes, but also that the teachers who are
most effective at improving test scores are not necessarily the same teachers
who are most effective at improving non-tested outcomes—echoing the findings of
another recent study.

Although Blazar and Kraft don’t take
a stand on incorporating measures of non-tested outcomes into teacher
evaluations, their findings highlight the potential difficulty of doing so. In
the sample, more than a quarter of the most
effective teachers (based on test scores) were among the least effective when evaluated using student non-tested outcomes.
To further complicate matters, the
non-academic outcomes don’t always correlate. For example, teacher scores on
classroom organization had a positive correlation with student behavior but a
negative correlation with happiness in class.
These contradictory findings are
problematic since many of these factors influence long-term outcomes. Could it
be that teachers improve one important student outcome at the expense of
another? Or does this study highlight the complexity of the student experience
and the challenges of measuring it with a survey?
Clearly, this study raises more
questions than it answers. 

Of late, there’s been a lot of chatter among
teacher educators objecting to language in ESSA for “teacher preparation
academies,” relatively regulation-free routes into teaching. From the
perspective of much-maligned traditional teacher prep providers, it’s the feds
once again showing an unfair preference for alternative providers. The action
stings all the more because they see academies as an unproven model, an
objection with some merit in our eyes.

So what gives?
In return for escaping from most regulatory
requirements (such as the coursework that must be taught), teacher prep
academies would be held accountable for “the performance of their graduates,” which
has yet to be defined. In effect, this trade-off is similar to the deals
offered to K-12 charter schools—hence teacher educators’ christening of
academies as “charterized” teacher prep.
We have two questions: is this trade-off
realistic (i.e., practical), and is it fair?
Thinking of lessons to be learned from the early
days of the charter movement, there was a time, long before No Child Left
Behind, when schools offered up student performance results based on their own
definitions of accountability. The data weren’t always pretty and, in fact,
were sometimes nearly meaningless. It was only after data were standardized and
disaggregated that any degree of accountability was achieved and useful
insights began to emerge about promising models for school reform.
Data available on K-12 schools are light years
ahead of data on ed schools. In a recent report, Chaos to Coherence, Deans for Impact (a group of reform-minded ed
school deans) examined seven different data categories—things like completer
surveys, employer surveys, classroom observations, student achievement, and
teacher evaluation of graduates—and documented the disappointing degree to
which even the most progressive programs capture such data. 
So the rub is that ESSA essentially allows
states to hold academies accountable in the midst of a virtual data desert. Fewer
than half of states have yet to define a “teacher of record” or require
verification of class rosters—both fundamentals to hold academies accountable
for the performance of their graduates.
Further, long overdue regs on federal
requirements for state data systems aren’t imminent: nearly four years into a
rulemaking process that would help to close the black hole of accountability
for teacher prep programs, there is no clear timeline for finalizing them.
To answer our second question, proponents of
these new academies claim that with the right incentives and rewards, providers
will be highly motivated to fill the current data vacuum. But the current academies
don’t strike us as poster children for transparency. For example, all that’s
publicly available from Relay—perhaps the granddaddy of the academy movement—is
summary data on self-selected metrics. While Relay can be applauded for posting
this data, it is inadequate as a model for what should be collected and made
available as a condition for breaking free of state regulation.
Teacher preparation academies present a shiny
new reason for collecting objective teacher performance data. It’s ironic then
that the field of teacher education has consistently put up notorious
resistance to such data, given that it could be the ticket for releasing them
from the very regulations they find so burdensome.