Can AI help optimize instruction and increase student learning?

Mention AI in a school setting and most people think of ChatGPT and cheating scandals. But AI in school settings isn’t all bad. In fact, new research shows early indications it could be used to enhance teachers’ instruction by providing them with more regular feedback on how to improve.

We have long known that providing teachers with formative feedback strengthens their instruction and—as a result—students’ outcomes,¹ yet the average teacher in the U.S. has limited access to regular feedback given that it is resource intensive to provide.²

A new study by Demzsky et al. suggests AI presents an opportunity to make providing effective feedback more scalable and efficient. In this large scale, randomized control trial, researchers used automated tools to scan the transcripts of student-teacher interactions to provide specific suggestions to teachers for increasing teachers’ “uptake” of student ideas.

“Uptake” is defined as any time a teacher “acknowledges, revoices, and uses students’ ideas as a resource” as they are shared out loud during instruction. It often looks like elaborating on a student’s idea in real time or asking them a follow-up question to a comment they share. It is well-established as a core component of strong instruction³ and is included in widely used observation instruments like the Danielson Framework and Pianta’s CLASS.

To examine this, researchers followed a five-week online computer science course organized by Stanford University, looking across more than 900 instructors and 10,000 students. (Course materials were prepared by course organizers, so they were uniform across instructors.) Each class was recorded and automatically transcribed using AI. The researchers then deployed an automated tool to analyze transcripts for instances of “uptake.” They trained the machine to distinguish between random interactions (e.g., a student says, “I added 30 to 70,” and the teacher says, “OK, class—please turn to your partners”) and real instances of uptake (e.g. a student says, “I added 30 to 70,” and the teacher says, “Where did the 70 come from?”).

The researchers also created individual online dashboards for each teacher that displayed statistics on uptake and invited teachers to reflect on how they could incorporate more uptake into their next lesson. The dashboard (see Figure 1) included:

• High-level statistics on the frequency of uptake;
• Examples from their transcript;
• Open-ended reflection questions to help teachers set goals for themselves; and
• Links to other resources (like papers and blogs) on uptake and dialogic instruction.

Figure 1: Components of the Teacher Feedback Web Application

Half of the instructors received the “treatment,” which was a short weekly email pointing them to the dashboard as a resource. The researchers then tracked the instructors who accessed the dashboards and how it impacted their instruction over the 5-week period. Though the impact on student learning could not be determined because there were no assessments in the short online course, use of the dashboards increased instructors’ uptake of student ideas by 16% and provided suggestive evidence that students whose teachers looked at the feedback completed a greater percentage of assignments and were more satisfied with the course.

To be clear, there are limitations to the study: instructors were novice volunteers with much room for improvement and few resources to guide them (in contrast with K-12 teachers who are trained), not to mention technology could be a major limiting factor for scaling this. However, this research suggests AI may offer a low cost option for feedback (just $1 per session once the fixed costs of the system setup were paid), especially for areas where coaches and principals are unavailable or less skilled.