Imagine you're a district with access to information that can improve student achievement, but you have to wait a full academic year to act upon it. Further, consider that in that next year, you will be required to subject additional students to the same or worse outcomes. On the one hand, you possess a valuable tool to boost performance, but on the other hand, your inability to act immediately means you're most likely committing a cohort of students to weeks or months of learning loss. As theoretical as it may seem, according to an Annenberg working paper authored by Lauren Sartain and Matthew P. Steinberg, this is the reality for Chicago Public Schools.
Standing as the third largest school district in the country, Chicago Public Schools (CPS) implemented REACH teacher evaluations starting with the 2012-2013 academic year. REACH ratings are based on a combination of student growth measures and classroom observations, for which the latter serves as 70% of a teacher's final score. Teachers receive formal observation feedback (including their score) in the year the observations are completed, but final REACH ratings are not generated until a few months into the following academic year. It's the effect of this delay and the fact that teachers and the district possess knowledge of the lion's share of the final rating a year in advance that the researchers examined.
As is common practice across districts, non-tenured teachers in CPS receive no protections and can be removed at any point in their pre-tenure years, meaning principals can dismiss low-performing non-tenured teachers based on observation ratings alone. The result is that non-tenured teachers are about twice as likely to be replaced in the observation year than tenured colleagues who earn equivalent REACH ratings (when they become known the following year).
Tenured teachers are only subject to involuntary removal after REACH ratings are released the year following their observations. Tenured teachers who receive an "unsatisfactory" rating are given 90 days of remediation before receiving an updated observation score, which is used to recalculate their REACH rating. This means that unsatisfactory teachers will be well into the second semester of the year after receiving poor observation scores before receiving an updated and deterministic REACH rating. Only if the unsatisfactory rating persists is the teacher subject to dismissal.
When compared to the observation year, tenured teachers who earn unsatisfactory ratings are 50% more likely to leave the district in the year they receive their deterministic REACH rating (i.e., in the year after they received poor observation scores). In other words, even when low-performing teachers learn their observation scores (that account for 70% of the eventual REACH rating) in the current year, they frequently continue teaching until they are involuntarily removed from the district the following year.
Ultimately, this leaves two important questions: How do tenured teachers respond to remediation (the primary reason for giving them an additional year in the classroom), and does replacing low-performing teachers improve teacher quality? When the researchers examined the first question, they found that not only did teachers earning unsatisfactory ratings fail to improve; on average, they actually performed worse the following year. As a result, the timeline of REACH evaluations means CPS is ultimately assigning students to a teacher who is known to be low-performing and unlikely to improve.
To the more important question of what happens when low-performing teachers are removed, the study found that whether replacement teachers are transfers or new hires, their classroom observation scores, as well as reading and math student growth scores, are all significantly higher than the growth scores of those teachers rated unsatisfactory. Within the context of REACH ratings, which range from 100 to 400 points (with 210 being the line for the unsatisfactory rating), replacement teachers earn an average score of 304.6 compared to the 188.4 points earned by unsatisfactory teachers. What's more, the researchers found that, based on the supply of available teachers and the benefit they serve, CPS could raise the cut line for unsatisfactory to 250 and still see improved outcomes.
One important caution that Sartain and Steinberg note is that previous research has found that teachers who are randomly assigned lower-performing students receive lower observation scores. In Chicago, Black teachers have been found, on average, to earn lower observation scores than their non-Black peers, and those differences can be attributed to differences in the schools where Black and non-Black teachers typically teach and not to their value-added contributions to student achievement. Because the overwhelming majority of REACH ratings are based on classroom observations, the researchers warn that in high-stakes systems, it is essential that school districts ensure they accurately measure teacher quality without bias.
Concern over bias in CPS observation scores was elevated this month when the Illinois General Assembly took up a bill that would give the state's Board of Education the ability to determine if racial (or other) factors skew teacher ratings. If bias is found, the Chicago Board of Education and the Chicago Teachers Union would be compelled to enter into negotiations to create a new evaluation system to be implemented by August 15, 2025.
Standing as the third largest school district in the country, Chicago Public Schools (CPS) implemented REACH teacher evaluations starting with the 2012-2013 academic year. REACH ratings are based on a combination of student growth measures and classroom observations, for which the latter serves as 70% of a teacher's final score. Teachers receive formal observation feedback (including their score) in the year the observations are completed, but final REACH ratings are not generated until a few months into the following academic year. It's the effect of this delay and the fact that teachers and the district possess knowledge of the lion's share of the final rating a year in advance that the researchers examined.
As is common practice across districts, non-tenured teachers in CPS receive no protections and can be removed at any point in their pre-tenure years, meaning principals can dismiss low-performing non-tenured teachers based on observation ratings alone. The result is that non-tenured teachers are about twice as likely to be replaced in the observation year than tenured colleagues who earn equivalent REACH ratings (when they become known the following year).
Tenured teachers are only subject to involuntary removal after REACH ratings are released the year following their observations. Tenured teachers who receive an "unsatisfactory" rating are given 90 days of remediation before receiving an updated observation score, which is used to recalculate their REACH rating. This means that unsatisfactory teachers will be well into the second semester of the year after receiving poor observation scores before receiving an updated and deterministic REACH rating. Only if the unsatisfactory rating persists is the teacher subject to dismissal.
When compared to the observation year, tenured teachers who earn unsatisfactory ratings are 50% more likely to leave the district in the year they receive their deterministic REACH rating (i.e., in the year after they received poor observation scores). In other words, even when low-performing teachers learn their observation scores (that account for 70% of the eventual REACH rating) in the current year, they frequently continue teaching until they are involuntarily removed from the district the following year.
Ultimately, this leaves two important questions: How do tenured teachers respond to remediation (the primary reason for giving them an additional year in the classroom), and does replacing low-performing teachers improve teacher quality? When the researchers examined the first question, they found that not only did teachers earning unsatisfactory ratings fail to improve; on average, they actually performed worse the following year. As a result, the timeline of REACH evaluations means CPS is ultimately assigning students to a teacher who is known to be low-performing and unlikely to improve.
To the more important question of what happens when low-performing teachers are removed, the study found that whether replacement teachers are transfers or new hires, their classroom observation scores, as well as reading and math student growth scores, are all significantly higher than the growth scores of those teachers rated unsatisfactory. Within the context of REACH ratings, which range from 100 to 400 points (with 210 being the line for the unsatisfactory rating), replacement teachers earn an average score of 304.6 compared to the 188.4 points earned by unsatisfactory teachers. What's more, the researchers found that, based on the supply of available teachers and the benefit they serve, CPS could raise the cut line for unsatisfactory to 250 and still see improved outcomes.
One important caution that Sartain and Steinberg note is that previous research has found that teachers who are randomly assigned lower-performing students receive lower observation scores. In Chicago, Black teachers have been found, on average, to earn lower observation scores than their non-Black peers, and those differences can be attributed to differences in the schools where Black and non-Black teachers typically teach and not to their value-added contributions to student achievement. Because the overwhelming majority of REACH ratings are based on classroom observations, the researchers warn that in high-stakes systems, it is essential that school districts ensure they accurately measure teacher quality without bias.
Concern over bias in CPS observation scores was elevated this month when the Illinois General Assembly took up a bill that would give the state's Board of Education the ability to determine if racial (or other) factors skew teacher ratings. If bias is found, the Chicago Board of Education and the Chicago Teachers Union would be compelled to enter into negotiations to create a new evaluation system to be implemented by August 15, 2025.