A Response to "A Metacognitive Approach to Learning and Performance in Human-AI Interaction"

A colleague and friend suggested I read A Metacognitive Approach to Learning and Performance in Human-AI Interaction.¹ I was glad to, because I was hoping it would offer a way out of a problem I keep running into. AI can make us more productive, but that productivity seems to come at the cost of skill formation for learners and skill retention for experts.

The paper takes this problem seriously and gathers a lot of research on it. I came away with questions about the proposed method rather than a solution. It is quite possible that I have misread parts of the paper, so I am writing my questions down in the hope that someone can point out what I am missing.

The Paper in Brief

The paper starts from a distinction between task performance and learning. Task performance is how well you complete a task right now, with whatever help you have. Learning is the lasting change in what you can do on your own, once the help is removed. The authors argue that AI pulls these two apart. With AI, a student can turn in an excellent essay or program without being able to produce one alone.

The paper explains this through cognitive offloading. Cognitive offloading means handing mental work to an external tool, the way we hand arithmetic to a calculator. AI lets learners offload a great deal of work, and the authors argue that learners often offload their metacognition along with it. Metacognition is thinking about your own thinking: setting goals, planning how to approach a task, monitoring whether it is going well, and evaluating the result. When learners hand all of this to AI, they produce good work while learning little. The paper calls this "autopilot performance."

The proposed fix is what the paper calls metacognitive onloading. The idea is to let AI take on some of the cognitive work while the learner keeps the metacognitive work. The learner sets the goals, states what a good answer looks like, checks the AI output against that, and judges whether the approach is working.

The paper says we can make this happen through scaffolding. Scaffolding is temporary support built into the learning environment. The paper's examples include a checklist the learner fills in before prompting the AI, a question in the middle of the task that asks the learner to predict what a good answer should include, and an exercise at the end where the learner reconstructs the solution without AI. The support is supposed to fade as the learner becomes more capable.

The authors put all of this into a two-by-two model they call SYNC. One axis is learning, which they tie to scaffolding. The other axis is performance, which they tie to AI support. The quadrant with high learning and high AI support is called "Synergetic Mastery," and it is the state the paper says we should aim for.

What I Found Useful

The paper is a clear warning not to confuse AI-assisted performance with learning, and the evidence it cites supports that warning. In one high school mathematics study, students who practiced with a general-purpose GPT interface did better during practice but scored lower than a no-AI control group on a later exam taken without AI. In an introductory computer science course, students did well on midterms and quizzes completed with AI but showed no advantage on a final exam completed without it.

I also found the discussion of four pitfalls useful. The paper describes how AI output can feel trustworthy because it is fluent, because it agrees with you, because it is coherent and well structured, or because it comes with a step-by-step explanation. None of these features make the output correct. Some of the remedies the paper suggests are practical. For example, it says an AI tool should by default give the minimum necessary help, such as one hint or one diagnostic question, and give more only when the learner asks for it.

My Questions

Can metacognitive work stand in for cognitive work?

I agree with the first part of the paper's argument. If you offload the cognitive work to AI, you will learn less. The second part is a stronger claim. It says that if you keep the metacognitive work for yourself, you can keep the productivity of AI and still learn. The paper makes this claim for learners, not only for experts.

I am not convinced. My view is that metacognitive work does not replace cognitive work. I do not think it does much to build a skill, and I think it does little to slow the loss of a skill you already have. I may be wrong about this, and I would like to see evidence either way. I am not even sure the approach works for experts who already have the skill. For learners who are still trying to acquire it, my experience makes me doubt that it works.

The evidence the paper cites does not settle this either. The same mathematics study had a second group that used a version of the tutor with teacher-designed safeguards and hints. That group also did better during practice, and on the exam without AI they scored about the same as the control group. So the safeguards removed the harm, but they did not produce more learning than working without AI.

Does planning to solve a problem teach you to solve it?

Planning, monitoring, and reflection are good for learning in a general sense. They help you organize your study, notice when you are confused, and use your time well. They make you a better and more efficient learner. What I do not see is why planning a problem set, while AI solves the problems, would develop your ability to solve problems. As far as I can tell, the ability to solve problems is built mainly by working through problems yourself.

There is also a dependence in the other direction. Take a student in a discrete math course who is learning to write proofs. Suppose the AI writes the proof and the student's job is to evaluate it. To judge a proof, the student needs definitions, inference rules, proof techniques, a sense of the common invalid moves, and some experience with proofs that did not work. That is the same domain knowledge they would have built by writing proofs. Without it, the instruction "check whether the proof is valid" does not tell them what to look for. A beginner will likely judge the proof by its surface: the notation, the length, a confident step-by-step layout. Those are exactly the features the paper warns can make AI output look more trustworthy than it is.

The paper acknowledges this. It says that learners who "lack the domain knowledge" for a new task struggle to plan, monitor, and evaluate. It also says that learners "lacking cognitive foundations and metacognitive abilities will find it difficult not to offload their own judgment and monitoring to AI." I agree with both statements. But I do not see where the method explains how learners get those foundations if the AI is doing the cognitive work.

Is verifying the work the new learning outcome?

There is one reading under which the approach could make sense. Perhaps in the future AI does most of the producing, and the human job is to verify what it produced. Then, instead of asking a student to write a proof or write a program, we would show them a proof or a program and ask whether it does what it claims. If that is where things are going, perhaps that is how we should teach these courses, and it may work.

I know that when students write code and write proofs, they also learn to read, understand, and verify code and proofs. I do not know whether it works the other way around. If students skip the writing, can they still learn to verify well? I think that is an open question, and I may be wrong to doubt it.

But the paper does not make this argument. It does not say that the learning outcome has changed from producing the work to verifying or explaining it. It describes the problem as students turning in work they cannot produce on their own. Then it offers a solution in which they check work that AI produced. Checking a proof is not the same as producing one, and recognizing good code is not the same as designing the program. If the goal has changed, I think the paper should say so and make the case for the new goal. If the goal has not changed, I do not see how checking AI output gets the student to the original goal.

What is the scaffolding doing?

This is the part of the paper I find hardest to follow, and I am not sure I can put my confusion into words well. Scaffolding is the answer at nearly every step. Learners may not plan, so we add scaffolds. Learners may not evaluate, so we add scaffolds. Learners may be misled by fluent or coherent output, so we add scaffolds. Learners may come to depend on the scaffolds, so the scaffolds should fade. The paper also describes the scaffolding in many ways: balanced, calibrated, contingent, just-in-time, just-enough, and fading.

My trouble is with what the scaffold is supporting. In the usual sense, a scaffold supports a learner while they do something they cannot yet do alone. When the scaffold is removed, they can do that thing alone. Training wheels help a child ride a bike, but the child is still the one pedaling. In the paper's Synergetic Mastery quadrant, the AI does the task, and the scaffold supports the learner's thinking about the task. So when the scaffold fades, what is the learner supposed to be able to do on their own? If the answer is "plan and check AI output," that is the different learning outcome I described in the previous section. If the answer is "do the task," I do not see where the learner practiced doing the task.

There is a related question for novices. The paper wants learners to set goals, decide what counts as success, and judge whether a strategy is working. A novice does not yet know how to do these things, which is part of what makes them a novice. So either the scaffold supplies this knowledge, or it does not. If it does, then the instructor or the tool is doing the metacognitive work, and the AI is doing the cognitive work, and it is not clear what work is left for the learner. If it does not, then the novice cannot do what is being asked. I suspect the paper means something in between. But it does not say where that point is, and the authors note that they do not yet have measures for telling whether onloading is happening.

I also notice that the scaffolds I find most convincing in the paper are the ones that ask the learner to make an independent attempt before using AI, or to reconstruct the solution without AI afterward. Those scaffolds give the cognitive work back to the learner. To me, that supports my concern more than the paper's claim. The parts of the method that look most likely to produce learning are the parts where the learner does the work.

Where I Land

I agree with the paper's diagnosis. I am not convinced by its treatment, at least for learners. For experts, there may be a case for keeping the planning and checking while handing off some of the production, mainly for productivity and accountability, though I am not sure it prevents skill loss. For learners, my current view is that doing the cognitive work of the domain is what builds the skill. I would want to protect that work rather than replace it with metacognitive tasks around AI output. To find out whether learning happened, I would assess students without AI present, as the studies the paper cites did.

I would be glad to be shown where I have misread the paper. My questions for the authors, or for anyone who finds the method convincing, are these:

1. What evidence shows that doing metacognitive work while AI does the task builds a skill as well as doing the task yourself?
2. If the AI is doing the cognitive work, how does a novice get the domain knowledge needed to plan, monitor, and evaluate?
3. Is the intended learning outcome still producing the work, or has it become verifying the work?
4. When the scaffolding fades, what should the learner be able to do on their own?