Once again, does teacher certification matter? The question has been asked and answered so many times for so long, that the debate has entered some kind of existential realm, disembodied from reason and logic. Believers will continue to believe and cynics will continue to question.

The journal of record for education, Education Week, came out this week with its latest rushed pronouncement that three new research studies find that certification matters. (The newspaper already featured an earlier metamorphosis of one of these studies four years ago but it's been a slow news millennium.) Before we all enroll in five-year ed programs and sign up for the NEA, let's take a look at the studies.

The first study is by Kristie J. Rowley, a graduate student at Vanderbilt's Peabody College. Using an important dataset and survey from the Early Childhood Longitudinal Study, Ms. Rowley finds that kindergarten students assigned to teachers with "full certification" made more math gains than students assigned to teachers on emergency licenses. Even though the dataset included reading and language scores, Rowley doesn't report these. Hmmm. She gives as her reason that "mathematics is the area most influenced by schooling." In kindergarten? She bungles the reporting of her statistics claiming that certified teachers in both elementary ed and early ed were the most effective, when in fact teachers with no certification (private school teachers) were more effective than early ed certified teachers.

A second study is from the American Institute for Research. This study looked at NAEP data to conclude that 8th grade students in Texas who had certified teachers did better on NAEP math. But here's the catch (as with all three studies.) The authors define anyone who is "provisional" as fully certified. So for example, a Teach For America teacher or any other alt cert teacher would have been defined, for the purposes of this study, as fully certified, leaving basically only those teachers at the bottom of the barrel to provide the comparison in quality. The study was only able to partially control for prior student achievement for about half of the students (most people theorize that high performing teachers gravitate to high performing students). Greenberg et al. warn: "the reader is advised to exercise caution to inferring any causality in the relationship between teacher qualifications and student achievement highlighted in the paper." Duly noted&by us anyway.

The third brief paper by Ed Fuller of the Dana Center and Celeste Alexander from the Southwestern Educational Development Laboratory, appears to have been rushed into some readable form to help wage the PR wars over the recent Texas decision to allow high school teachers in the classroom without ed coursework. (The authors say as much.) Fuller had collected similar data five years ago which made their way into an Ed Week headline but the study not only didn't get published, it was never written up. Anyway, Fuller and his colleague find here that students with certified teachers did better on the TAAS math assessment than students with non-certified teachers. But the authors jump to their conclusions after only one year of data, not just violating the rules for value-added studies but evoking William Sander's name in the process, a researcher who is adamant that three years of data are critical. Also, it's not clear how alt cert teachers were classified as fully certified or emergency certified.

While the two sides of this debate are not likely to converge, it would be good to agree about what would constitute a legitimate certification study.

First, (as Fuller's did) it would ideally look at groups of teachers within the same state so we don't have to argue over the state differences in certification requirements.

Second, it would look at student gains over a sufficient period of time to make sure the results aren't spurious and it would tie student gains to individual teachers. Without a good prior achievement measure, the minimal window of analysis needs to be three years and even this bar set by value-added guru, William Sanders, is questioned by some economists.

Third, it would use proper controls for prior student achievement, student socioeconomic status and other critical factors&and the paper would be published under a peer review process before newspapers write about it.

Fourth and most importantly, the important policy question here is whether teachers who go through regular certification routes are better than teachers who come from other routes with other talents. To compare certified teachers with uncertified teachers who don't have the basic smarts to pass a simple licensing test is just not very interesting or helpful (see Margaret Raymond's study of Houston teachers where many emergency certified teachers didn't even have college degrees.) The preparation that leads to certification is what this debate is supposed to be about, not whether states should be recruiting teachers from the fast food window at McDonalds.