How to use LLMs in lesson preparation?

LLMs have the potential, according to some, to ruin society. But they also have the ability to help. In some inspiring stories, they can level the playing field in education, allowing those who are in badly-resourced areas to receive the great equality: education.

So how do we make use of LLMs in lesson preparation as well as possible? The main thing I realized, after a day of playing around, is that your results depend on the LLM that you use. The results also probably depend a lot on the prompt-- a good prompt engineer might make up for a bad LLM.

I have in mind that for most teachers, the LLM is a human aid-- where you have the training to come up with the lesson plans yourself, but just would appreciate a little help from an LLM to get started. For some teachers who are really strapped for time or energy, or for students who are simply using an LLM to learn, the LLM might be everything.

At first, I used ChatGPT from OpenAI to try to lesson plan. I tried ChatGPT on lessons on biopolymers and the ideal chain model after making my own lesson plan. Then I tried it on an introductory physics lecture on angular momentum. Finally, I tried it on lesson plans for understanding Newton's Principia and the role it played in science history.

In short, it failed to help. The way it failed to help was interesting. The lesson plans shoved way too much material into way too short of a time, with very little in the way of depth. The interactive activities-- because the LLM was told to make the class interactive-- were sometimes dismal. Random walks were supposed to be simulated by giving students string to play with. As a demonstration, it's not a bad idea to use string to illustrate end-to-end distance and why this is or is not a good measure of "size". But I cannot imagine students at my colleges taking seriously a lesson in which they muck around with string for longer than 30 seconds and pretend that they're understanding polymers. (What happened to actual simulations? This activity seems like something to give a fourth-grader, not an undergraduate in college.) Formulas in the angular momentum lecture were oversimplified; the cross product lost a sin and was just mvr, as if American students couldn't handle the truth. Derivations were omitted in the biopolymer and angular momentum lectures. And, as usual, some material was just wrong. No, those were not the main points in the Principia.

Of course, prompt engineering is huge with LLMs. I tried hard to get ChatGPT to give me a biopolymers lesson I could use. But after all my effort, all I got was that it might be a good idea to bring a string or cable or something into class to illustrate the random walk model when we go 2D. And that's not at all what ChatGPT said to do with the string.

I then tried DeepSeek. Relatively speaking, it aced it. Actually, its lecture on biopolymers was quite close to what I wrote down all by myself without an LLM helping in any way, and I actually built two lesson plans around two activities suggested by DeepSeek for my Great Ideas in Science class. The key with DeepSeek was that there was less breadth and more depth for the 75-minute class. More relationships were derived. The in-class activities (group attempts to collect/synthesize information and debate with each other) meant more, and seemed designed for undergraduates rather than elementary schoolers. I was able to read through DeepSeek's lesson plans and simply steal ideas, and then spruce them up a bit. That's amazing.

It is interesting that an LLM trained on Chinese data does better at helping students retain facts than a model trained on American data, in my opinion. I take from this that America has a ways to go with education. In our classrooms, it would be great if we could: emphasize more derivations; do in-class activities that were less about feeling and more about collecting, synthesizing, and interpreting information; and take longer to go through each bit of material so that the class isn't whiplash. Perhaps there's a way to fix Masters in Education programs in America so that the course preparation data on which ChatGPT is trained leads to better lesson plans.