Digging Deeper: Which types of institutions achieve excellence and equity for aspiring teachers of color?

The full breadth of what teachers need to know and be able to do is expansive, and content knowledge is one of many crucial components. This analysis examines efforts to measure and develop aspiring teachers' content knowledge because a broad base of content knowledge supports reading comprehension, another critical opportunity gap. Much as learning phonics helps students decipher the sound of words, learning about a breadth of subject areas helps students draw meaning from what they read. A review of decades of research confirms that

"higher levels of background knowledge enable children to better comprehend a text. Readers who have a strong knowledge of a particular topic, both in terms of quantity and quality of knowledge, are more able to comprehend a text than a similarly cohesive text for which they lack background knowledge."464

Tests of students' reading comprehension reveal that their knowledge of the topic predicts their comprehension more accurately than their reading ability does.465 Moreover, spending more class time on social studies is associated with improved reading ability, especially for students who are learning English and for those from low-income backgrounds.466

While there is more to learn, existing research confirms the views of the field:467 teachers teach a subject more effectively when they have knowledge of that subject (a view supported by leaders of teacher preparation programs and state education agencies).468

Additionally, disparities in coursework that emphasize a breadth of content is another source of inequities for students of color and students from low-income backgrounds. Learning core content builds the foundation for later grades and supports students' ability to enter postsecondary education. In a recent report on educational equity, the National Academies of Sciences identified "disparities in curricular breadth," in particular "availability and enrollment in coursework in the arts, social sciences, sciences, and technology" as a key indicator of educational inequities.469 Data from NAEP and other sources confirms a sizable opportunity gap in core content areas for students of color and students from low-income backgrounds.470

While more research in this area is warranted given the dozens of different tests in use by different states, the vast majority of research on this question finds that performance on a licensure test does correlate with teachers’ effectiveness in the classroom, as measured by student learning.

To explore the full breadth of research on licensure tests, NCTQ identified thousands of potentially relevant studies published over the last three decades.471 Of these, 15 were directly relevant to the question of whether licensure tests predict teacher effectiveness as measured by student learning outcomes. Of these, 11 found a statistically significant positive relationship between a teacher's test scores (or a binary measure of whether or not they passed the content licensure test) and that teacher's effectiveness in the classroom.472 The most recent study, examining MTEL tests in Massachusetts, found that not only did teachers' licensure test scores predict both teachers' value added scores and their evaluation ratings (and the latter was even more strongly predictive for teachers of color), but also that a one standard deviation increase in teachers' MTEL scores corresponded with an increase in teacher performance ratings equivalent to about two thirds of the improvement between a first- and second-year teacher.473

Among the four remaining studies, one found no statistically significant relationship between licensing test performance and effectiveness,474 although few in the study sample reported failing the licensure test on their first attempt. Two studies found a mixed effect.475 Hanushek's 1997 meta-analysis of research asserted that about a third of the 41 included studies found a positive relationship between teacher test scores (though not necessarily content licensure test scores) and student achievement.476 Goldhaber and Hansen’s 2010 study found a positive correlation between passing the licensure test and teacher effectiveness in math, but warned that this relationship existed for white teachers but that the benefits of Black students learning from Black teachers (regardless of their licensure test performance) was equivalent to those students having a white teacher who did well on the licensure test.477 The fourth study found no relationship in most models, and a negative relationship between passing California's CSET content exam on the first attempt and students' reading achievement, although all teachers in the study ultimately passed the licensure test.478

Of course, what is true of one licensing test may not hold for all tests or all contexts. For this reason, states should consider conducting an analysis of the predictive validity of their licensing tests, much as Massachusetts did,479 which could both strengthen the tests themselves and inform policy decisions around how to use them.

Licensure tests of content knowledge are meant as a guardrail, ensuring that those who enter the classroom have an adequate level of knowledge about the topics they will be teaching. However, this knowledge is necessary but not sufficient for teachers to be effective; content knowledge is only one strand of the tapestry of effective teaching. Because these tests do not capture the complete picture of what makes an effective teacher, one would not expect these tests to be fully predictive of teachers' effectiveness, nor is that their role. However, it would be alarming if ample evidence indicated a negative relationship between passing licensing tests and teacher effectiveness, perhaps suggesting that these tests were blocking large numbers of effective would-be teachers from earning licenses.

The research on licensure tests focuses almost entirely on student learning outcomes. However, research on the importance of teachers of color find that Black and Hispanic teachers have a positive correlation with academic achievement for students of color, but also are positively associated with many other outcomes (e.g., higher school attendance, lower suspension rates, more referrals to gifted and talented programs, and greater high school graduation and college matriculation rates).480 This research is sometimes used as evidence that licensure exam requirements that prevent aspiring teachers of color from entering the classroom should be dropped. However, there is ample evidence that many institutions across the country can successfully help aspiring teachers of color attain necessary content knowledge and pass these licensure tests, indicating that these options need not be framed as an “either/or” approach. Instead, institutions and the teacher preparation programs within them should take active steps to make up for gaps in aspiring teachers’ previous educational experiences, so that they enter the classroom equipped to bolster their students’ academic and non-academic outcomes. The alternative solution—dropping licensure tests as a guardrail—risks instead perpetuating these inequities.

In addition to the four primary subjects (English language arts, science, social studies, and mathematics), some states also test additional subjects, such as health, fine arts, or physical education.

Most states do not let anyone teach absent a passing score on the test, but there are a few exceptions. Some states allow candidates to substitute some other measure for having to pass a test, or allow delays in meeting this requirement. For example, North Carolina now only requires a math subtest (rather than testing all four subjects), and allows teachers to teach for up to three years before passing the math subtest. Arizona, Hawaii, and Oregon allow teachers to substitute a degree or major in the content area for passing the licensure test. Wisconsin accepts a content-based portfolio, and New Mexico just announced plans to allow a portfolio option as of July 2022.

Each state selects the licensure test that it believes best captures what its elementary teachers should know. While 23 states all used the same test (the Praxis Elementary Education: Multiple Subjects 5001 test) during the time period for which NCTQ collected data (September 2015 - August 2018), 20 other tests were also in use. Some states use multiple tests, allowing aspiring teachers to select from among them; other states changed tests during the study time period.

The structure of these many licensing tests varies. Most states (26) use a test that is divided into at least four parts or "subtests"—one for each subject.481 Test takers must pass all four subtests in order to pass the overall or "composite" test. Other states use tests that combine multiple subjects into each subtest (e.g., combining science and social studies into one subtest; or combining mathematics, science, the arts, health, and fitness into one subtest). (For more information about states' required tests and how they combine different subjects, see this list on page 6.)

Mostly, no. Almost all states allow anyone to take a licensure test, meaning that test takers do not have to be enrolled in a teacher preparation program. In some states (e.g., Iowa, North Dakota, Texas, and Wyoming), only someone who is enrolled in a program can take the test. The data included in this analysis includes all test takers, not only teacher candidates, with the exception of the few states that provided data only on teacher candidates. (For more about the data states provided, visit their dashboards here).

While it may not be possible to fully remove bias from any exam, test designers take steps to measure whether bias is present and to remove any questions that may lead to differences in outcomes based on anything other than one's understanding of the content.

These steps may include:

1. Establishing guidelines for writing assessment questions that identify terms, topics, and formats to avoid;
2. Inviting panels focused on equity and content to review test materials and ensuring diversity of membership on those panels;
3. Calculating statistical differences among groups on assessment items;
4. Investigating score differences among racial groups, men and women, test takers with disabilities, and English language learners (when sample size allows); and/or
5. Identifying test items that lead to large score differences among groups and removing them unless they can be justified to meet the content specifications for the assessment and the item has passed through an independent review.

Despite these steps, many remain concerned that bias still permeates licensing exams, given the lower pass rates for Black and Hispanic test takers compared with white test takers, at least on the most commonly required elementary content licensure test.482 However, it becomes difficult to disentangle whether these pass rates differ due to problems with the test itself or to systemic inequities in the quality of education available to different groups.483

Qualitative analysis has explored the "affective dimension" of the experience of taking a licensure test, finding that for test takers of color, the experiences surrounding taking the test can undermine test takers' confidence and set off stereotype threats or "identity threats."484 However, several studies note that preparation and support from the test takers' preparation program can mitigate some of the candidates’ nervousness that accompanies taking a licensure test and can generally support candidates in achieving greater success.485 When reviewing tests for bias and considering the overall testing experience, test publishers should consider not only the content of the test, but also the affective experience. This review could highlight the need for other changes, such as additional training for test proctors to make the testing experience more welcome and calming for test takers.

Concerns about the tests themselves remain.486 One study argues that small amounts of bias at the test item level may accrue into larger test-level biases; this "accretionary bias" could occur even if individual test items are not identifiably problematic.487 Several studies assert that the test is biased against test takers whose first language is not English.488

Some have also raised concerns about omissions in the content of licensure tests. Concerns have been raised that elementary curricula, and the teacher licensure tests designed to assess aspiring teachers’ knowledge of the content contained in those curricula, may lack diverse perspectives and content. As curriculum designers work to diversify their materials, so too may testing designers need to expand the scope of their exams.

The solution often offered to address concerns of bias is to drop licensure tests altogether. However, doing so in an attempt to solve one problem may create a much greater one. If states lose a guardrail ensuring that teachers enter the classroom knowing the content they will teach, they risk perpetuating opportunity gaps for the very students who most need teachers who can build their foundational knowledge–and threaten the knowledge base for all students, including those who may seek to become teachers in the future.

Generally, a “test taker of color” includes anyone who self-identifies as a race and ethnicity other than white, non-Hispanic. 

States were asked to provide data disaggregated by “white non-Hispanic test takers” and “Test takers who identify as a race/ethnicity other than white non-Hispanic.” (Pilot testing of these data requests found that requesting more disaggregated data resulted in the loss of data for many more institutions, as data was suppressed for institutions with small numbers of test takers of color to protect test taker privacy). Aggregating all test takers of color reduces this loss of data, while coming with the tradeoff of allowing for a deeper exploration of how test takers of specific races are performing. 

Some states provided data that disaggregated test takers into the groups: White, Two or more races, All other ethnicities, and No response. For the analysis of pass rate data, “Test takers of color” includes test takers who identify as “Two or more races” and “All other ethnicities.” All data on test takers’ race is self-reported.

This analysis uses one test for each state. In the event that a state has a single stand-alone test, that test is used. If a state has a composite test that incorporates multiple subtests, and it reports the composite pass rate, that composite pass rate is used. If the state has a composite test and does not report a composite pass rate, then the subtest with the lowest average first-attempt pass rate is used (as that is the subtest that presumably poses the greatest challenge to test takers). If a state requires or offers several different tests to fulfill the elementary content requirement, the test with the largest number of test takes is used.

When exploring these pass rates, there is an important limitation to the data. This data does not distinguish between teacher candidates formally enrolled in an institution's teacher preparation program as separate from those test takers affiliated with an institution but not enrolled in a teacher preparation program. So, for example, some test takers in this analysis might be enrolled in a teacher preparation program at X university, while others might attend X university, but are not enrolled in the teacher preparation program.

Gathering data limited to teacher candidates, rather than inclusive of all test takers, has proven difficult: Only a few states have developed data systems that can link pass rate data to teacher candidates and their institutions (some strong examples include Florida, Massachusetts, and Texas). In lieu of using their own data systems (which did not have the data), most states gathered licensure test data from their testing companies, which have not yet identified a straightforward way to supply pass rate data specific to teacher candidates matched to their preparation programs for these reports.489

Note that testing companies did not provide this data directly to NCTQ, but only to the states upon request. Data with too few test takers (e.g., fewer than five test takers at an institution) was suppressed to protect test taker privacy.

This private college located in the heart of Boston is notable for the success of all of its aspiring teachers, and especially its aspiring teachers of color. Test takers of color at Emmanuel College pass the Massachusetts Tests for Educator Licensure: General Curriculum (MTEL) test at a rate above the state average for all test takers, and pass at a rate on par with white test takers at the institution.

Associate Dean of the School of Education Sister Karen Hokanson, Ed.D., shares how her School has worked with department chairs across the college to ensure that candidates learn the core subjects they will be expected to teach. The preparation program faculty meets with liberal arts department chairs to align courses (e.g., Biology or World History) with the state curriculum frameworks that detail what aspiring teachers will be expected to teach. The faculty revisits this process annually, considering MTEL scores to target any additional content needs, and reviewing any changes to curriculum frameworks from the Massachusetts Department of Elementary and Secondary Education.

Through this process, the School of Education has also identified areas where candidates need additional instruction to learn content not included in their other coursework. For example, candidates tended to struggle in mathematics (evident from the math placement test taken during orientation, performance in the first foundational mathematics course, and MTEL math subtest scores), so the School of Education faculty worked with mathematics faculty to create three foundational mathematics courses for elementary candidates. Mathematics methods courses are then designed to build off these content courses, according to the School faculty. Sr. Hokanson shared that this process has been very helpful, and that “you see the results in objective measures like the teacher test.”

The School of Education leaders regularly track licensure test pass rate data. Sr. Hokanson receives MTEL scores every other Friday, which she shares with faculty to review at the following week’s faculty meeting. Several supports are available for students identified as struggling to pass the MTEL. First, the university-wide Academic Resource Center offers workshops to help counter test anxiety. The School of Education also offers MTEL prep courses in fall and spring semesters at no additional cost.

In response to candidates’ challenges in mathematics, the School established a weekly math lab course, and professors for Education and Mathematics make recommendations for the content of this lab based on this information. Students who do not pass the MTEL are required to participate in the math lab until they are able to pass the test. To defray costs of test fees for candidates, the School of Education uses their full allotment of vouchers from the state to help with licensure test fees.

The School of Education also provides support that goes beyond building academic skills and knowledge. For example, the School ensures that all students, and especially aspiring teachers of color, have role models from the education profession. The School hosts events that feature alumni from the School of Education who are now teachers, principals, or superintendents to share their story. This event “really resonates with the students and helps them be proud of the profession,” shares Sr. Hokanson. Sr. Hokanson continues, “we get to know [our students] well. We help guide them into areas where they feel they’ll be successful. We help them feel confident about making the decision to be a teacher and getting the skill set they need.”

This public university stands out for the success of its aspiring teachers of color. Test takers of color at Morehead State University pass the Praxis Elementary Education: Multiple Subjects exam at a rate above the state average for all test takers, and pass at a higher rate than white test takers at the institution.

Dean of the college of education Dr. April Miller attributes the institution's success to two steps the program has taken over the years.

First, the teacher prep program holds a "Data Day" each semester, in which program leads present candidates' pass rate data on the Praxis exams to the faculty. They use this information to explore what is working well in the program, and what needs to change. For example, the program has been able to identify that candidates are not performing as well in World History, and so is exploring options to shore up candidates' knowledge in that area. Dr. Miller shares, "Our data days are pretty important for our success."

Second, while the program does not put constraints around which general education courses candidates can take, they do offer strong recommendations. They further reinforce elementary candidates' content knowledge through methods courses in each core subject area (mathematics, English language arts, science, and social studies). These courses embed Praxis-style questions into exams or end-of-chapter activities, helping to solidify the content and preparing aspiring teachers to succeed on these tests.

This public university has helped its aspiring teachers, and especially its aspiring teachers of color, to achieve success on licensure tests. Test takers of color at Valley City State University pass the Praxis Elementary Education: Curriculum, Instruction and Assessment (5017) test at a rate above the state average for all test takers, and pass at a rate on par with white test takers at the institution.

Dean of the School of Education Dr. Allen Burgad credits actions by both the School of Education and his university in supporting aspiring teachers’ success.

The School uses several sources of data to track candidates’ success, including pass rate data on licensure tests, disaggregated by candidates’ race and ethnicity, home state, and other criteria. When candidates struggle on these assessments, they can use university supports such as the Resource Center, which has been especially important in bolstering candidates’ writing, an area where they tend to struggle. The School also uses a data-driven approach to monitor outcomes from clinical practice, surveying cooperating teachers and university supervisors to track candidates’ performance. The School of Education staff disaggregate this data by white candidates and candidates of color, looking at aggregate performance as well identifying any disparities between the groups.

Dr. Burgad also believes that the institution’s efforts to build an inclusive environment have contributed to the success of their aspiring teachers. He shared, “It’s not only what we do in our [preparation] program but also within our university. The culture within our university has the greatest impact.” The university has hired a Director of Diversity and Inclusion, and has fostered a sense of inclusion through community events and celebrations for diverse groups of students. The School of Education has supported this effort with professional development for trauma, social-emotional learning, and building inclusive classrooms. The School of Education also builds community through a cohort model: candidates are grouped into cohorts of 25 to 30 students, all of whom take classes together. While Dr. Burgad only has anecdotal evidence to measure this cohort approach, he believes that building this “family of learners” has helped his students thrive.