Six in the morning, or how agents learned by asking questions

It started with the question whether a model can learn a topic the way a person does, by asking questions, instead of lecturing the whole thing smoothly and in order right away. When I learn something, I ask about one word, then for an example, then “what if”, I get lost and come back, so I figured that since questions come in types, maybe it could run on its own: a student who knows nothing, an expert who answers, and a curator who makes sure it turns into learning and not a chat about everything (how it works is described in solutions).

The run I'll remember best is the one where the student, someone who had never programmed, was supposed to understand a short household budget program, two classes, a loop, a dictionary. After a few dozen questions it knew quite a bit about binary and tuple indexing, neither of which was in the program at all, and didn't know what a program is, because it never asked, and nobody turned it back from the side path :-) What helped was adding to the goals the basics someone like that has to start from.

Two things stay with me. Most of the work here is done by the curator, who knows what we're after and keeps the conversation from drifting, the expert pretty much just answers. And some rules are better kept in code than in a request to the model, because asked for restraint, it won't keep it anyway. I finished around six in the morning and went to bed half-conscious, but pleased.

Later, with drill-up, going from ready code upwards, I took apart a simplified program with drill-down alone in a separate article. The student asked about every line, including how many times the graph passes through its nodes and when it stops, so the article ended up with a precise description of that loop. A reviewer that gets only the text and runs nothing worked out from that description that with the default budget the graph needs 31 steps while LangGraph stops at 25 by default, so the example would have crashed in the eighth round. I missed it while simplifying the program, and the bug came out on its own, from the questions and the analysis, before anyone ran the code.