on AI in the classroom
I wanted to share this with the OMSCS TA Slack channel, but as I was writing it, it felt like it needed to be fleshed out a bit more first, and here is a good place for this kind of writing. So, dear reader, note that the original audience is fellow educators on a Slack channel.
Maybe this gets regularly expressed here, but I am feeling it today as I am grading some student reports. This program is not equipped to handle the use of Generative AI or LLMs in the classroom. I am not suggesting this is exclusive to the Institute, but programs that teach at scale do have certain unique challenges, and one of the biggest is evaluation.
Even with reasonable usage guidelines, some students will still use these tools unreasonably to complete their assignments. You could argue that there is always a certain percentage who cheat by “traditional” methods and will get away with it, and that is just something that will always persist no matter what measures are taken to combat it. But generative AI is different in how it impacts so many different processes of the academic experience, from research to coding to writing.
We know students aren’t adhering to basic AI guidelines about what materials they can share with LLMs. Because the academic material we teach is being used as training data for some of these models, they will only become more adept at completing student assignments. We have always known that some students cheat - we now know that it is substantially easier to do so. At the same time our ability to out-design assignments that cannot be solved by LLMs is shrinking.
This creates a self-defeating and potentially exhausting feedback loop: if we design around the models, then what we evaluate for becomes the new standard for what the next model should do. Our students then use the intermediary models to partially or completely solve the assignments, which may become training data for subsequent models.
If addressing the problem through changing evaluation is self-defeating, is it any better to address it through standards that cannot be enforced? We will look back at now as a transitional time. While it’s happening, though, it’s a challenge to maintain an equitable student experience. When one student uses a tool and gets away with it and another student doesn’t, then they are being evaluated on two very different assignments whether or not anyone even realizes it.
Link to original