Research & Insights

This month’s installment covers the new contracts in Sacramento City Unified School District, Baltimore County Public Schools, Billings Public Schools and Laramie School District One (WY).

A top-notch,
three-part series (here, here and here) running in
this week’s Education Week sheds new light on one of the most
non-transparent of states’ regulatory functions: their approval of teacher
preparation programs. Journalist Stephen
Sawchuk spent months doggedly investigating this largely dysfunctional
system, trying to figure out why states rarely yank program approval from any
of the nation’s 25,000 programs. The pursuit may not rise to the level of who
really killed Hae Min Lee (surely you’re listening to the hit podcast Serial)—but, in our view, it comes pretty close.
Sawchuk weaves a compelling narrative
about the reluctance of state officials to intervene (and how convoluted
intervention can become) by highlighting controversies involving programs in Michigan and New York. The third and last article grapples with the root
issue:  a fundamental lack of agreement
about what teacher prep should be about. As a quote from Jim Cibulka, president of the national accreditor CAEP (the Council for the Accreditation of Educator
Preparation) puts it,
the curriculum for teacher preparation “has been largely driven by ideology and
tradition, rather than empirical knowledge and investigation…We have allowed
a thousand flowers to bloom, including weeds, because there was no empirical
basis on which to separate the wheat from the chaff.” 
The next piece on Sawchuk‘s list should be a cold, hard look at the way that many
states’ oversight became so entangled with national accreditation, mostly
during the 1990s when NCATE was on a
roll. Only eight states now approve programs entirely independently of
accreditation. The arranged marriage certainly wasn’t appropriate under the NCATE regime and there’s a strong
argument to be made, no matter how weak or strong CAEP ends up being, that program approval should never be
intertwined with program accreditation. Sawchuk
references their entanglement, but it’s a huge topic in and of itself that
needs exploring.

Over the
years, there have been a gazillion studies examining the relative merit of
different pathways into the teaching profession. Almost all come up short,
often because the studies do not clearly define the type of pathway being
analyzed (i.e., just what does it mean to be alternatively certified?)
A new study from Gary T. Henry and his colleagues at Vanderbilt does not
disappoint, breaking down  the
definitions of assorted pathways into more meaningful categories as well as
examining the results of only new teachers (those with less than three years of
experience). After all, should we really judge a preparation program by the
teachers who graduated from Ol’ State U. in 1980 or by those who graduated in
2014?
Henry et al. disaggregate teaching
pathways into multiple distinct categories: out of state, in state, graduate,
undergraduate, public, private, Teach
For America and lateral entry (North
Carolina‘s own alternative entry).
As always, the researchers find more
variation within the groups than across; there are some notable differences in
this study from what previous research has found.
There is
some limited evidence of a “home team” advantage for specific teachers; teachers who were prepared in-state
were more effective than those from out-of-state programs in three of the eight subjects tested. Additionally, teachers prepared in private
institutions were no better than those from public institutions, a finding
we didn’t find surprising given that we haven’t found a discernible difference
between the two in the Teacher Prep Review.
One takeaway
consistent with other studies’ findings: TFA
comes out looking great. Students instructed by TFAcorps members annually gain approximately 18 days of additional
learning in elementary math, 11 days in elementary reading and an astounding 73
days in middle grade math.
While there
aren’t many TFA corps members in North Carolina, other alternatively
trained teachers (referred to as “lateral entry”) have a much harder time
posting gains. Findings conclude they are less effective (especially in STEM
subjects) or, at best, average.
Here’s a finding
we’ve never seen before: in both middle
grade math and reading, teachers trained in graduate school aren’t as effective
as teachers trained as undergraduates— though they do better in high school
science. This needs more research to figure out if those different outcomes are
due to the focus of masters’ degrees, if they were content specific, or in
education.

Districts
often ask NCTQ to identify the preparation programs that produce teachers who
will stay in the classroom through thick and thin. So we eagerly dove into two
recent articles, one by Richard
Ingersoll and
colleagues, the other
by Matthew
Ronfeldt and colleagues,thatexamine which aspects of teacher
preparation predict persistence – and we emerged a little perplexed.
Despite
sharing the same data sources (the US Department of Education’s School and
Staffing Survey and its companion Teacher Follow-Up Survey), the researchers
reach opposite conclusions. Ingersoll
et al. find “a large and cumulative relationship between pedagogy [i.e.,
methods courses, student teaching, etc.] and attrition.” First-year teachers
with the least training were three times more likely to leave teaching than
those with the most.
By contrast,
Ronfeldt et al. find that the whole
can be less than the sum of its parts. True enough, teacher candidates who take
more methods courses or who have more
weeks of student teaching are more likely to stay. But teachers who get full
doses of both student teaching and methods courses are actually
slightly more likely to leave teaching than those who just get one or the
other. Puzzling?  For sure.
The good news is that
both articles conclude that teachers who had a substantial stint of student
teaching are less likely to leave.
The bad news is that this finding doesn’t help us all that much in identifying
programs that prevent attrition, as most traditional programs now require such
a stint (i.e., a full semester).

How do academic research and world
exploration differ? In research, it’s good to discover the same thing
twice! 
Replication – research that repeats a
previous study – is a cornerstone of methodological rigor. But when was the
last time you read about a replicated education study? We won’t judge you if
you can’t remember. Turns out, few exist. Of those that do, their integrity may
be questionable.
In a recent study, Matthew Makel and Jonathan Plucker assess replication in education
science by looking into the publication history of the top 100 education
journals. When looking for replication reports specifically, their analysis
found that an infinitesimal number of the education articles published in these
journals were actual replications (221 out of 164,589). And while this may not
seem consequential at first, it matters. Even well-crafted experiments can find
anomalous results – replicating a study can confirm that the original findings
hold true or can turn up conflicting information that helps us better
understand the subject of analysis.
In nearly half of all replications, the same
research team that published the original article was also responsible for the
replication study. This is slightly worrisome given that the success rate (when
the replication has similar findings to the original study) is much higher (71
percent) when conducted by the original team than when there’s no author
overlap (54 percent). Although same author replications are beneficial to the
field, it’s clear that third-party authors are needed to increase reliability.

So why is replication not expected in
education research? The authors argue that social science publishers and
funders want new and exciting findings, so they’re more likely to fund, hire,
publish and promote groundbreaking studies rather than those that seem like
“old news.” Naturally, this kills any incentive to attempt replications.

Despite this unfavorable environment, some
researchers are bucking this trend. Check out Bacher-Hicks, Kane and Staiger (2014) and Rothstein (2014 working paper) – both of which
replicate an earlier study by Chetty, Friedman and Rockoff (2014)on the impact of teachers. Be on the lookout for a write-up on those
replication papers in next month’s TQB.

Earlier this month, the Missouri
Supreme Court declined to hear NCTQ’s appeal of a lower court
decision, which
blocked our request  that the University of Missouri turn over course
syllabi to us  for the purpose of
assessing them for the Teacher Prep
Review.  As is the custom of the
Supreme Court, no reason was given about its decision, but it came as a
surprise given the considerable support, including amicus briefs, from Missouri journalists. As UM Professor Mike Podgursky described, “The absurd
legal fiction that syllabi distributed to 35,000 UM students cannot be
disclosed” has been allowed to stand.
None of the legal decisions actually prevent us from traveling to Missouri and looking at syllabi, provided
no one has to copy them for us.  That was
an option we had not wanted to pursue, because of the principle at stake here
and the cost involved in sending numerous analysts to multiple Missouri campuses.
Missouri‘s press association
rightly calls the Supreme Court’s refusal to take up the case “devastating,” as
Missouri public agencies can now
refuse to disclose any copyrighted document. The implication of this decision
has not fallen on deaf ears — Missouri‘s
legislature is now considering a bill on the matter to ensure that it continues
to live up to its reputation as the “Show-Me State.”

Dr. Selma Botman
Dr. Botman has dedicated her career to higher education. She previously served as the Vice President for Academic Affairs at the University of Massachusetts, the president of University of Southern Maine, and the provost and Executive Vice Chancellor for Academic Affairs at The City University of New York. Dr. Botman has also taught at the College of the Holy Cross where she was involved in international programs and promoted Middle Eastern studies.
In her current role as Provost and Vice President for Academic Affairs at Yeshiva University, Dr. Botman is working with faculty and deans to expand curricular offerings for both undergraduate and graduate students. Along with her administrative responsibilities, Dr. Botman continues to teach undergraduates in the area of her academic specialty, modern Middle Eastern history and culture.

Dr. Botman holds a BA in Psychology from Brandeis University, a BPhil in Middle Eastern studies from Oxford University, and an AM in Middle Eastern Studies and PhD in History and Middle Eastern Studies from Harvard University.
John Connolly
John R. Connolly is the
co-founder and executive director of 1647, a non-profit start up that partners
with schools to implement effective family engagement strategies including
training teachers to conduct home visits. 
Prior to founding 1647, John spent six years as a Boston City Councilor
where he chaired the Committee on Education and co-chaired the annual review of
the Boston Public Schools’ budget. 
During his time as a City Councilor, John led the effort to reform the
Boston teachers contract, lengthen the school day, and fully staff school-based
social and emotional health services. 
John also led a Council investigation that exposed rampant mismanagement
of the Boston Public Schools’ food services department.  From 1995-1998, John taught seventh and
eighth grade at the Nativity Mission School on the Lower East Side of Manhattan
and taught sixth grade at the Boston Renaissance Charter School.  John has also worked as an attorney at Ropes &
Gray and Hanify & King, P.C. from 2001 – 2007.
John currently serves on the
board of FUEL Education.  He formerly
served on the board of Meridian Academy and Action for Boston Community
Development (ABCD).
Hugh Norwood
Hugh has been in and around
the education industry as a professor, administrator, marketer and serial
entrepreneur since 1995, with successful tenures at Emerson College, Lesley
University and Laureate Education, Inc., (home of Walden University, Kendall
College, National Technological and others). In 2007, he formed
Trinity Education Group, Inc. as a Maryland S-Corps, and has served as its
President and General Manager since. Hugh is a frequent presenter at
professional education conferences such as NAGAP, AACRAO SEM, SXSW Interactive,
SXSWedu and Connexions. He has held several board positions for corporations,
including his current role as Board Chairman for InSync Education. He is a
founding board member of SXSWedu and held a board position on Rice University’s
OpenStaxx OER Center until 2013. Hugh received his BA summa cum laude from Moravian College (1990) and his Master of Fine
Arts/Creative Writing from Emerson College (1995).

In the five
years since most states adopted Common Core State Standards or their own
version of college- and career-readiness standards, about one million new
teachers have graduated from teacher preparation programs in the United States.
With five years lead time, we thought the time was right to look at whether
states have aligned their requirements for teacher preparation and licensure
with the skills needed to prepare students for college and careers.
The 2014 State
Teacher Policy Yearbook finds that
most states have yet to make the critical changes this alignment necessitates.
With such profound changes occurring in K-12 standards, it would stand to
reason that requirements for new teachers would be changing, too. We looked
across states’ requirements for teachers’ subject-matter knowledge and skills
related to the instructional shifts demanded by the new standards— such as the
focus on the use of complex informational text across all subject areas, as
well as admission and accountability requirements for teacher preparation
programs. We found just five states – Indiana,
New York, North Carolina, Rhode Island and Texas – that are on the right track for ensuring new teachers are
ready.
In addition
to these five states, Arkansas is
also noteworthy, having done more than most other states to revise its
standards for teachers to reflect the instructional requirements of college-
and career-readiness standards. 
Most states
have considerable work to do. An area of particular neglect is preparation of
special education teachers, where the findings are especially grim:

– 34 states
offer or exclusively grant K-12 special education teacher licenses  that make no distinction between preparation
to teach elementary or secondary grades. This policy sends an important and
devastating message about special education: there is no need for any specific
subject or grade-level content or pedagogy knowledge for teachers of any
special education students ages 5-17.
– Just 15
states require any special education teachers to pass subject-matter tests. Although most special education students are
expected to meet the same high college- and career-readiness standards as other
students, too many states set a lower bar for licensing special education
teachers.

– Of the 18
states that require general education elementary teachers to pass a test of
effective reading instruction, seven do not require the test for elementary
special education teachers. Considering
that reading difficulties are the primary reason for referrals to special
education, the failure to transfer this important requirement to special
education teachers is nothing short of baffling

Search our dashboard or download a report here.

The debate over value-added models continues

Dallas Independent
School District
hires approximately 2,000 new teachers each year. It is transforming the way
its human capital team works, adopting data-driven strategies for recruitment,
selection and hiring. One of the many sources of evidence Dallas ISD uses is the Teacher Prep Review.
The district first evaluated past recruitment
efforts based on NCTQ rankings to determine quality and whether they would have
a future presence for recruiting season at the previously considered schools.
Some schools have moved down or off of the list, while schools that previously
were not targeted in the recruitment plans will become more of a focal point.
The talent acquisition team travels near and far to find top talent to educate
students. The team analyzed the NCTQ rankings carefully identifying the standards
that best fit the districts needs and projected a strategic distribution for
early contracts.
The new recruitment plan this year will strategically include places
such as University of Houston , Arizona State University and Northwestern State University of
Louisiana based on their rankings from NCTQ. Top students at these
universities can look for early contracts from Dallas. These new teachers have had the training and preparation
that will prepare them to meet the district’s needs and they’ll be welcomed
into a district whose goal is to develop, support and reward their teachers for
effectiveness in the classroom.

As the year draws to a close, we look back on the most talked about Teacher Trendlines from 2014. We covered everything from teacher salaries to leave policies and in this edition, we highlight some of the most popular Teacher Trendlines from the past year.

At least that is what I would like to believe.
This past September, America Succeeds hosted its inaugural conference in
Boise, Idaho. Actually it was billed as an “EDventure”— a thinly
veiled swipe at the typical conference fare we all know too well—involving
endless groups of panelists promising to limit their remarks to a few minutes
only to be shocked, just shocked, when the unfailingly polite (meek)
moderator suggests wrapping it up (“But I still have 10 slides to go!!”).
Anyway, the wonderful Tim Taylor who
is busily growing America Succeeds asked me to come, so I happily signed up.
At some point at EDventure there was probably some
education talky talk, but I was more intrigued by the “Recess”
session, featuring our choice of fly fishing, mountain biking, rock climbing or
bird watching. (I was incensed to learn that my assistant signed me up for bird
watching and in a huff switched myself to rock climbing. For heaven’s sake, I’m
not THAT old!)
Even more alluring was the Shark Tank Pitch. Head
honchos from Gates, Helmsley and Albertson foundations were on hand to hear and respond to anyone’s
5-minute pitch for a great education idea and fund it on the spot. No
10-page proposal, no string of unanswered emails, no sucking up. Instead, an
instantaneous $50,000 grant! Take a look at the video. Awesome!
Anyway, I wasn’t able to go because a very heavy
Adirondack chair dropped on top of my head, sending me out of commission with a
mild concussion. I lost my chance not only to rock climb (which, to be honest,
was a questionable choice), but also to pitch an idea which I had absolutely
no doubt NCTQ would have won, and we are now $50,000 poorer for it.
Who won my $50,000? Brenda Berg, who’s leading America Succeed’s newest affiliate, BEST NC. 
Brenda isn’t your typical TFA-Charter-Ed-Reformer, having recently sold
her own successful manufacturing company (baby furniture). If PIE had an award for “Most Likely to
Succeed” (take note, Suzanne
Kubach), Brenda would be
my bet.
Brenda won $50,000 even though not everyone on the
funders’ panel was gung ho. In fact, Brenda was, as she put it, the only one to
get “shark bit” when Jamie
MacMillan from Albertson
summarily dismissed Brenda’s idea. Brenda’s pitch was about funding a
large-scale effort involving some 180 stakeholders in North Carolina, convening them to collaborate and reach a consensus
on a major comprehensive reform plan in the state. That’s not an idea most of
us in the ed reform world love. As Michelle
Rhee famously said, “Collaboration is overrated.”  In declining to fund BEST NC, Jamie (likely
echoed by most everyone else in the room) said this: “… it’s too nice
for me.”
However, a few weeks ago, Brenda sold the ultimate
(albeit penniless) skeptic: me. I swear it was not the long-term effects of the
concussion. I was mostly just really impressed with her clearly capable persona
coupled with steely determination. She also reminded me of something we
reformers are too quick to forget —that consensus building has actually been
known to work and in some remarkable and sustainable ways, both in Massachusetts in the early ’90s and Tennessee in this decade. Brenda with
her extensive staff of one, former CarolinaCan
director Julie Kowal, has put in
place a wildly ambitious, furiously paced timeline involving 54 subcommittee
meetings, which convene three times over three months. Ouch. They intend to
leave even these role models in the dust (okay on that point, I’ll reserve
judgment).
So my bet is on Brenda.  And I
can probably speak for Jamie that she would be more than glad to be proven
wrong.

School leaders seeking to revamp their
teacher evaluation systems are biting into a meaty subject.
Behind the very public questions of whom to
evaluate and what to count is another layer most of us can overlook, but a
question school districts can’t: how should
test score data be measured? Making the right decision requires an
understanding of different value-added models and how to pick a model that best
suits the needs of the school district. 
There may not be one right answer. Darn.
A new study sets out to explain the
trade-offs of different value-added models. Mark Ehlert, Cory Koedel, Eric Parsons and Michael Podgursky explore different ways to build these models and find that some
of these variations make a big difference in teacher evaluations – and some
don’t – though they leave the big decisions to education officials rather than
throwing their weight behind one model or another.
The most significant decision facing those who
develop teacher evaluations in a state or district is whether they want a
“one-step” or “two-step” model— a decision with big policy
implications. (See our explanation on the difference between the
two and how the one-step model tends to favor teachers in more advantaged schools,
while the two-step is more likely to recognize stronger teachers within
disadvantaged schools.)
Deciding which student characteristics to
include in the model (e.g., free/reduced lunch status, language status, race),
also doesn’t seem to have a clear right – or wrong — answer. Different
combinations of these variables yield results that are highly correlated with
each other (in other words, removing or adding a variable does not drastically
change the outcome of the model). However, these changes do affect rankings.
So, for example, when two teachers have similar value added scores, one model
find that Teacher A is more effective, while a model with different
characteristics included may find that Teacher B is more effective – but, and
this is important, neither model is likely to see Teacher A go from highly
effective to ineffective.
Finally, these researchers also make the case
that using more years of previous test scores makes the data only a little more accurate; so little, in
fact, that it doesn’t merit the trade-off of evaluating fewer grades. It seems
that one previous year of data is generally good
enough…so out the window goes some advice we’ve been freely offering
states over the years.

It’s no secret that some elementary teachers
would rather walk across hot lava than teach a science lesson and that means
less science gets taught.  A new study
shows that improving teachers’ content knowledge may be the best way to
increase their students’ science learning, suggesting that elementary teachers
should take more rigorous science classes. 
That might sound like a no brainer, but it is amazing how often teacher
content knowledge has been missing from the nation’s STEM conversation.
When Brandon S. Diamond, Jaime Maerten-Rivera,
Rose Elizabeth Rohrer and Okhee Lee examined teacher characteristics that influenced student science
learning, they found that teachers’ science knowledge, measured by a test, was
the strongest predictor of student success in science.
Other measures were largely irrelevant,
including self-reported teacher science knowledge, highest degree obtained and
even the number of science courses that teachers had taken in college. The last
finding is particularly intriguing because one would think that more science
courses equal more science knowledge. However, there are two major shortcomings
in the courses elementary candidates do take
which perhaps go a long way to explain this finding.
First, candidates are often not required to
take basic courses that would give teacher candidates a breadth of knowledge
across important science areas. Second, elementary candidates frequently have
too many choices about which science courses they can take – meaning that they
may choose courses that are not relevant to teaching elementary students. So
instead of taking a relevant and basic course like “Introduction to Biology,”
they may take “Introduction to Astronomy” – a topic that comes up far less
often in elementary schools. Or, they may select a course like “Earthquakes and
Society” which, while fascinating, means that teacher candidates will be
unprepared to teach anything beyond this narrow topic scope. NCTQ’s Teacher Prep Review has found that
almost 70 percent of undergraduate
programs do not
require that teacher candidates take even one basic science survey course.
The answer is not more classes, but more
comprehensive coverage of the science topics elementary teachers must be able
to teach. Shouldn’t elementary students have the opportunity to learn science
from a teacher who knows about earthquakes AND the rock cycle, dinosaurs AND
the six kingdoms of life – and much more?

For nearly a decade, NCTQ has been trying to figure out how to
evaluate teacher prep coursework for its rigor. It’s an endeavor to which both
of us brought no small measure of personal interest due to our own experiences
in teacher prep programs.
We think the wait was worth it. Our latest report, Easy A’s and
What’s Behind Them, is a twofer
that both addresses the grading standards in teacher prep and puts forward a
plausible theory for the high grades.

Our president, Kate Walsh, first hit on the novel idea of using
brochures from spring graduation ceremonies as the data source for teacher
candidates’ grade point averages. (Grade-based Latin honors are often noted in
brochures.) We brainstormed any number of ideas for how to obtain those
brochures — until we realized that they were readily available on websites and
from registrar’s and commencement offices. The fact that it took over 5 hours
on average to wrestle into spreadsheets the data from each of over 500
brochures — well, that’s a mere technicality, not a deterrent.
The second prong of this two-pronged report is the categorization
of coursework assignments (be they in teacher prep or any other major) into one
of two types. One type (criterion-referenced) facilitates real learning by focusing on content and skills. The other (criterion-deficient) involves overly broad or subjective assignments that not only artificially raise grades but also seriously weaken the quality of training. The identification of these two
basic assignment types grew out of Julie’s years of perusing coursework for our
teacher prep studies. Now she feels pretty stupid for not identifying the
distinction earlier. After all, it’s so evident once it’s explained that we’ve
included a do-it-yourself categorization quiz in the report.
One more note on the personal dedication our team brought to the
analysis for this report: our categorization of over 6,000 teacher prep coursework
assignments was done largely by Christina Perucci, a very clear-eyed NCTQ
analyst. The vacuous nature of the courses Christina took for her reading
specialist degree from Teachers College at Columbia University (historically the
premier teacher education institution in the nation) is still a sore point for
her.
Importantly, our work on improving rigor in teacher prep won’t
stop with this report. The report has laid a foundation for a new standard — the
Rigor Standard — for the Teacher Prep
Review. We have first assessed teacher prep programs on the alignment of
their grades with the institution at large, but we will soon also be looking
for an adequate representation of criterion-referenced assignments—the
assignments which keep grades in balance and help to truly prepare teachers—in
a sample of program coursework.
We hope this is work that the field welcomes, having heard from
many deans over the years that they’d like to find some plausible ways to
reduce the high number of “A” grades their faculties award. We’ve
taken great pains in this report to provide resources for teacher educators on
how to easily transform a criterion-deficient assignment into a
criterion-referenced one. 
We’ll know we’re having a real impact when the
litany of teacher candidate tweets at #edmajor which boast about the low-level
demands of teacher prep assignments (coloring assignments and making
marshmallow snowmen are now coming up) begin to change to boasts about meeting
its real challenges.

We love this study for what it says about
great teachers. 
People generally agree that an effective
English teacher may not make such a great Calculus teacher, or vice versa.
While people may assume that good teaching probably won’t translate across subjects, few have considered whether
good teaching can translate across all students.

New research asks this very question, looking
specifically at whether teachers who have high scores on value-added measures
(VAM) with non-English learner (EL) students have similarly high VAM scores
with EL students. The study, by Susanna Loeb, James Soland and Lindsay Fox, finds that, yes, good teaching
for one group of students remains effective for another group with different
needs in the same class.
Examining seven years of data from teachers
who had both EL and non-EL students, the researchers found that consistency in
effective teaching holds true for both math and reading, although the
relationship is stronger with effective math teachers. More than half of the teachers
considered to be in the top-fifth of all teachers for non-EL students are also
in the top fifth for EL students. For reading, this overlap is slightly
lower: a little under half of the top teachers for non-ELs are also the top
teachers for ELs.
A few characteristics, however, are
associated with teachers being more effective with EL students. Specifically,
teachers who have a bilingual certification or are fluent in Spanish tend to be
more effective with their EL students relative to their non-EL students.
The implication for those hiring new teachers is
that their best option is to find a good teacher – period. However, if the
school system has lots of EL students, finding a good teacher who is also
fluent in Spanish (assuming that’s the language commonly spoken by EL students)
or who has a bilingual certification may mean a bigger boost for EL students.