Teacher Preparation Program Accountability:
California

Analysis of California's policies

California's approval process for its traditional and alternate route teacher preparation programs does not hold programs accountable for the quality of the teachers they produce.

Most importantly, California does not collect or report data that connect student achievement gains to teacher preparation programs.Institutions must submit biennial reports containing aggregate candidate assessment data. Other types of data reflecting program efficacy that can be included in the reports are retention data and employer survey data. However, it does not appear that these data are mandatory. 

Further, California does not apply any transparent, measurable criteria for conferring program approval. In the past three years, just two programs in the state have been identified as low performing—an additional indicator that programs lack accountability.
The state's website does not include a report card that allows the public to review and compare program performance.

In California, the state maintains full authority over teacher preparation program approval. The state also conducts its own program reviews.

Citation

Accreditation http://www.ctc.ca.gov/educator-prep/program-accred.html Biennial Reports http://www.ctc.ca.gov/educator-prep/program-accred-biennial-reports.html Education Code 44374(f) Accreditation Framework http://www.ctc.ca.gov/educator-prep/accred-alignment.html

Recommendations for California

Collect data that connect student achievement gains to teacher preparation programs. 
As one way to measure whether programs are producing effective classroom teachers, California should consider the academic achievement gains of students taught by programs' graduates, averaged over the first three years of teaching. Data that are aggregated to the institution (e.g., combining elementary and secondary programs) rather than disaggregated to the specific preparation program are not useful for accountability purposes. Such aggregation can mask significant differences in performance among programs.

Gather other meaningful data that reflect program performance. 
Although measures of student growth are an important indicator of program effectiveness, they cannot be the sole measure of program quality for several reasons, including the fact that many programs may have graduates whose students do not take standardized tests. The accountability system must therefore include other objective measures that show how well programs are preparing teachers for the classroom, such as: 
1.    Evaluation results from the first and/or second year of teaching
2.    Satisfaction ratings by school principals and teacher supervisors of programs' student teachers, using a standardized form to permit program comparison
3.    Average raw scores of teacher candidates on licensing tests, including academic proficiency, subject matter and professional knowledge tests
4.    Number of times, on average, it takes teacher candidates to pass licensing tests
5.    Five-year retention rates of graduates in the teaching profession.

Establish the minimum standard of performance for each category of data. 
Merely collecting the types of data described above is insufficient for accountability purposes. The next and perhaps more critical step is for the state to establish precise minimum standards for teacher preparation program performance for each category of data. Programs should then be held accountable for meeting these standards, and there should be consequences for failing to do so, including loss of program approval.

Publish an annual report card on the state's website. 
California should produce an annual report card that shows all the data the state collects on individual teacher preparation programs, which should be published on the state's website at the program level for the sake of public transparency. Data should be presented in a manner that clearly conveys whether programs have met performance standards.

State response to our analysis

California's Commission on Teacher Credentialing (CTC) was helpful in providing NCTQ with facts that enhanced this analysis.

The CTC noted that the state holds all of its educator preparation programs accountable through its Accreditation System. California does not use or endorse value-added methodology; the state suspended K-12 testing for two years due to the transition to Common Core standards and new Smarter Balanced assessment implementation.

California does not analyze average raw scores of teacher candidates on licensing examination because these scores would have no meaning or significance. All of California's credentialing examinations are criterion-referenced, not norm-referenced, assessments. This means that candidates are not competing with or rated against each other. It does not matter to the determination of whether an individual met the competency requirement what the average score either passing or not passing is. In addition, for many of the Commission's examinations, including CBEST and CSET, most candidates chose to take these tests outside of a subject-matter preparation program, or instead of completing a subject-matter preparation program, and thus the scores do not relate in any respect to the quality of teacher preparation programs.

The CTC also noted that teacher preparation programs in California are at the graduate level and do not provide subject-matter preparation to candidates, as candidates have already completed their subject-matter preparation as undergraduates. Thus, whether or not a candidate passes the CBEST or the CSET examination, and how many times a candidate took a particular examination, provide no reflection on and have no relation to the teacher preparation program or its quality. Further, due to candidate privacy protections, candidates cannot be compelled to inform either the Commission or the teacher preparation program about scores or number of times the candidate took a particular examination. Candidates must submit evidence of having passed any examination(s) necessary to meet a credential requirement, but they are not required to provide any additional information about their test-taking history. This situation makes looking at average scores or number of attempts even more meaningless in context.

The CTC indicated that the data that must be included in the Biennial Report from all teacher preparation programs is clearly specified. Programs must report on these data, and this reporting is not optional but rather is mandatory for all programs accredited by the Commission. The data do include meaningful evidence of program efficacy, including standardized surveys of satisfaction from employers as well as from the mentor/master teachers, and also retention rates. California has embarked this past year on an extensive effort to strengthen and streamline its Accreditation System, as noted above. As part of this effort, a series of surveys have been developed and piloted for candidates, employers, mentor/master teachers, and others. In addition, specific data elements to be collected as part of public transparency regarding program and candidate outcomes are being defined for inclusion in a new data warehouse that will supply data for a variety of public data dashboards (equivalent to a "report card") to provide information for public access regarding performance of preparation programs. The state of California has authorized $5 million for this work in the state's 2015-2016 budget.

The CTC further noted that NCTQ faults California for having identified just two programs in the state as low performing, "an additional indicator that programs lack accountability." The Commission's viewpoint and practice is to intervene with programs early on that show a potential for becoming low performing to improve those programs rather than waiting until the programs would actually become low-performing. The Accreditation System provides early warning that a program may potentially be in trouble. Commission staff assigned to the particular accreditation cohort of that program work with program staff and program administration as needed to try to address the situation. California views this as a successful and positive practice to help candidates and programs improve and be successful rather than implementing a punitive and unhelpful approach of letting programs fail candidates and themselves in some way so that they can be publicly identified as "low performing," an approach that helps neither programs nor candidates in those programs. California has closed preparation programs that, despite all efforts to help them improve, have not sufficiently improved to warrant their continuance.

The CTC also noted that NCTQ inaccurately states that "California does not apply any transparent, measurable criteria for conferring program approval." California has rigorous program standards and requires all programs to provide comprehensive responses to each program standard in order to receive initial approval for the program. Trained California teacher preparation faculty, administrators, and others review each response to determine if the program should be allowed to serve candidates. Once a program is approved, it becomes part of the Accreditation System and has yearly activities it must accomplish to the Commission's satisfaction, whether through the Biennial Report, the Program Assessment review, the site visit, or other Accreditation-related activities. The Commission is in constant contact with programs during the entire seven-year cycle of accreditation, unlike other systems that rely only on the site visit after many years have passed between accreditation activities.

Lastly, the CTC stated that California retains full authority over the accreditation of programs providing preparation for California credentials.

Last word

While the CTC asserts that its accreditation process provides strong accountability for teacher preparation programs, its lack of setting transparent criteria for what constitutes a low-performing teacher preparation program does not make it clear how programs in the state are being held accountable. NCTQ understands the state's primary focus on program improvement over accountability and appreciates the emphasis on using early warning signs to offer support to programs; ultimately, however, it is the state's responsibility to maintain a high bar for teacher preparation program results so that teacher candidates are given the very best training.

While there is certainly room for differences of opinion on what criteria should be used to hold programs accountable, California's rejection of objective data puts the state in increasingly limited company across the country. Nearly all other states use student growth as part of a multiple measure system for teacher evaluation, and more states are doing the same for the programs that prepare them. It is clear from the state's response that the CTC is proud of its outlier status, but survey after survey shows the support of California's teachers moving in this direction, which is also certainly to the benefit of prospective teachers and, most importantly, to the state's students.