Today, we're announcing an update to our Teacher Prep Review. As a result of our appeals process, as well as due diligence we undertook on our own, we have changed 68 scores on our standards and 23 overall program ratings. We also dropped two programs from our sample because we learned that they did not lead to initial certification even though they were offered by a school of education. These changes are being made simultaneously on our website and that of U.S. News & World Report.

Our number one priority in evaluating teacher preparation programs is accuracy. Ultimately, the consumers of teacher preparation programs - aspiring teachers and school districts - will only turn to a source of information they know they can rely on. And considering that their reputations are at stake, programs have every right to expect that we will do our utmost to get our scores right.

Our obsession with accuracy means that for most standards we have two well-trained analysts independently evaluating programs. In cases of minor discrepancies between analysts, we give the program the benefit of the doubt and award it the higher of the two scores. In cases of more substantial disagreement, a third analyst reviews the evidence and resolves the matter. We constantly pore over our data to identify and address issues of analytical drift and variability. And we work to heed the recommendations of our expert Audit Panel, which gave its blessing to our methodology before we published the Review in June.

In July, we invited programs to submit appeals to us. 49 institutions took us up on our offer. Their objections, along with the evidence they provided, were posted on our Forum with our responses. In addition, we addressed issues that five programs posted on the website of the American Association of Colleges for Teacher Education (AACTE). 

We did a lot of double-checking as well. We discovered that we had not taken into account Oklahoma's unique elementary certification model which has a special concentration in math; this oversight impacted the scores of five programs on our Common Core Elementary Math Standard. We also found a flaw in the scoring algorithm for our Early Reading Standard that impacted 18 programs.

Having made over 16,000 scoring decisions over the course of the Review, the mistakes we have found amount to an error rate of 0.4 percent. But this is still too high. So we've strengthened our training and scoring protocols to address common sources of error, and we've intensified our quality control processes. A third of the errors in this edition resulted from our oversight of the Oklahoma math certification and the flawed Early Reading Scoring algorithm, and will not recur.

We regret every mistake we made and pledge to do better in future editions.

Here's a table detailing every scoring and rating change in this update: