A Response to the AI Assessment Scale

I read the original paper introducing the AI Assessment Scale (AIAS),¹ the 2025 paper revising it,² and the v2.1 summary of the scale released separately in 2026.³ I read them with the same question I ask of other proposals for assessment in the presence of AI: how do we establish what a student knows or can do when AI can produce the work we ask them to submit?

Giving instructors and students a shared vocabulary for discussing AI use has real value. Clear expectations are better than ambiguous ones. The papers also acknowledge important problems: unreliable detection, unequal access, the difficulty of controlling AI use outside supervised settings, and the need to change assessments rather than simply attach new labels to them.

But the papers do not answer that question. What I found is a framework for describing the intended role of AI, together with examples of activities students might undertake. What the papers do not establish well enough is why the resulting work supports a particular conclusion about the student's ability.

This distinction matters because the revised paper presents the scale as more than a communication tool. It presents it as a framework for assessment redesign that strengthens validity. That is a stronger claim, and it needs a stronger answer than describing what students are expected to do.

Specifying the student's intended contribution does not establish that the student made that contribution.

The discussion below follows the five levels of the revised scale. Each section opens with the level as the v2.1 summary describes it. At Levels 3 and 4, the v2.1 summary adds conditions that the figure in the 2025 paper does not include, and I point those out where they come up.

1. No AI

The v2.1 summary: Assess students in a controlled environment designed to exclude AI, so that they demonstrate their knowledge and skills independently.

This is the clearest part of the framework. If independent performance is what we want to assess, observing independent performance provides relevant evidence.

The revision also makes an important correction. It explicitly rejects the idea that declaring a take-home task AI-free establishes that it was completed without AI. It calls for supervised settings and recognizes the need to accommodate assistive technologies. These are real decisions about the conditions under which evidence is collected. They are not just instructions about how students should behave.

Of course, supervision does not make every assessment well designed. We still need suitable tasks, appropriate marking criteria, and a clear account of what performance on those tasks demonstrates. But at least we can identify the connection between the intended ability and the observed performance.

What I do not understand is why this attention to the conditions of performance largely disappears when the scale turns to AI-assisted work. If we want to assess a student's ability to direct, evaluate, or work with AI, we still need evidence that the student exercised those abilities. Permitting AI changes the conditions we should observe. It does not remove the need to observe them.

2. AI Planning

The v2.1 summary: Allow AI to support exploration, outlining, research, and idea generation, while assessing planning and idea development.

The revision makes this more defensible than the original version. The original allowed AI-assisted brainstorming and structuring but prohibited AI-generated content in the final submission. That is a difficult boundary to interpret, let alone verify. An argument or structure can come from AI even when the student writes every sentence.

The revised paper instead says that the assessment itself should be a planning task. It acknowledges that AI use cannot technically be confined to planning and proposes making planning visible through rationales, notes, or storyboards. That makes it clearer what is being assigned.

But what makes the planning the student's?

AI can propose ideas. It can also refine them, compare them, criticize them, explain why one was selected, and generate a plausible account of how the plan developed. Asking for an account of the planning does not establish that the student did the planning.

The documentary example makes the gap especially clear. The revised paper describes a storyboard assignment worth 25% of the final grade. The instructor decides that students' planning and shot-sequencing abilities can be validly judged whether they use AI or not. But that is exactly the claim that needs to be established. The example asserts it and then lists tools students might use. It does not explain how the instructor distinguishes the student's planning ability from the tool's contribution.

A good storyboard establishes that a good storyboard was submitted. To conclude that the student can plan a documentary, we need additional evidence. To conclude that the student can plan a documentary with AI, we still need evidence of their performance with AI.

Making the plan visible is not the same as making the student's planning ability visible.

3. AI Collaboration

The v2.1 summary: Allow AI to help generate, draft, critique, and refine work, while assessing both the result and the student's evaluation, modification, and integration of AI output.

There are two problems here.

First, the v2.1 summary says that the task is designed so that AI alone will not reach the required standard. The figure in the 2025 paper does not include this condition. It is a claim about what AI can do. An instructor cannot simply specify it as a feature of the task.

Which AI? With which tools, instructions, resources, and opportunities to revise its work? If an instructor obtains a poor answer from one interaction, that does not establish that the task requires a student's contribution. Also, a limitation the instructor observes while writing the assignment does not establish that the limitation still holds when students do the work.

The revised paper itself describes increasing AI capability and the vulnerability of tasks previously thought resistant to AI. That makes resistance to AI a weak basis for an assessment framework.

Second, evaluating and modifying AI output are themselves tasks that can be delegated to AI. The student can ask for a critique, request revisions, compare alternatives, and obtain an explanation of the changes. A submitted evaluation does not become evidence of human judgment merely because the assignment calls for human judgment.

The original paper's Level 3 required students to submit their original work alongside the AI-edited version, claiming that this would establish the authenticity of their contributions. The revision removes that requirement because the original work cannot reliably be verified. I agree with that correction. But the same objection applies to the kinds of evidence the revised paper proposes elsewhere: prompt rationales, checking notes, prompt logs, and reflections.

If a draft the student says is original cannot establish independent authorship, I do not see why a reflection the student says is personal should establish independent evaluation.

The revised paper's science example also needs a more careful separation of abilities. Students conduct practical work and collect data before the assessment. The assessment itself is the write-up, in which AI helps analyze the data and draft the report. Even if the practical work had been observed, it would provide evidence of practical skills only. It does not establish the student's ability to analyze the results or construct the argument in the report. Those are separate claims, and they need their own evidence.

This can still be a useful learning activity. But a useful activity and a valid assessment of each ability involved are different things.

4. Full AI

The v2.1 summary: Expect students to use AI and assess the critical thinking and subject knowledge they demonstrate in directing it.

This is a legitimate learning objective. Working effectively with AI can involve specifying goals, decomposing problems, evaluating results, identifying failures, and deciding what to do next.

But the fact that those skills would be useful in completing a task does not establish that a student exercised them.

The v2.1 summary adds that the goal cannot be reached by AI or by a person working alone in the available time. Again, that is a big claim about what AI can do. Making a task larger does not establish that meaningful student reasoning is necessary to complete it. A student may delegate the decomposition, planning, execution, and checking as well as the final production.

We also need to define what counts as directing AI. If a student gives an agent the assignment and accepts its result, they have directed AI in a minimal sense. If that satisfies the objective, say so. If we intend to assess something more demanding, such as recognizing a faulty approach and correcting it, the assessment must elicit and establish that ability.

The revised paper gives a computer science example in which students receive access to coding tools and are assessed on solution design and their use of AI systems. But those are only what the instructor intends to assess. The example does not explain what evidence distinguishes a student's design decisions from decisions delegated to the tools.

Providing institutional access is helpful, but the example also allows students to use any available AI-assisted technology. That leaves another question: how much does a difference in performance reflect the student's ability, and how much does it reflect differences in the tools available to them? The paper acknowledges that equal access remains an imperfectly solved problem. But unequal access is not only a question of fairness. It also affects what the grade means.

If we want evidence of how students work with AI, bring them into a controlled setting with a common tool environment and give them tasks that require the relevant decisions. Observe what they do, including how they respond when the tool is wrong or the requirements change. The result would be evidence of performance with that tool under those conditions. That claim is defensible, and its scope is clear.

Simply permitting full AI use does not establish competence in using AI.

5. AI Exploration

The v2.1 summary: Use AI creatively to explore new problems, methods, and outputs, potentially with students and instructors co-designing the assessment.

There may be excellent educational reasons to do this. Students could learn a great deal by investigating an unfamiliar application, experimenting with different approaches, and discovering what does or does not work.

But novelty does not answer the question of who did the work. A student can produce a novel artifact by delegating most of the work. An ambitious project can leave us unsure about what the student understands. Co-designing the assignment does not, by itself, establish who performed the intellectual work within it.

The revised paper's example involves AI applications in dance education, such as systems providing feedback on movement or generating responsive lighting. The educator also invites external speakers from dance and AI to help students refine their ideas. These are interesting project ideas. They do not explain how individual learning will be established or how the grade will distinguish technical understanding, artistic judgment, tool capability, and assistance from other people.

The paper also presents this level as deliberately forward-looking, including technologies and applications that may not yet be fully feasible. Flexibility about future activities is reasonable. But leaving room for future applications does not establish the validity of future assessments.

What is the student expected to learn? What performance would demonstrate it? Under what conditions will that performance be observed? Those questions still need answers, no matter how innovative the project is.

Changing the Task Does Not Automatically Establish Validity

The revised paper directly addresses the criticism that rules and labels can be ignored. It says the scale should produce structural changes to assessment: different task mechanics, process evidence, staged checkpoints, and marking criteria. I agree that attaching a number to an unchanged assignment is not enough.

But even a real change to the assignment can leave the assessment problem unsolved.

For example, the implementation section proposes adding a short prompt log or reflection to an AI-assisted drafting task. That changes what students submit. It does not establish that the additional submission reliably reveals what they did or learned.

A sequence of deliverables may help students organize their work, a reflection may support learning, and a prompt log may be useful for a conversation. These are possible educational benefits. But they are not evidence that the assessment is valid unless someone explains why these observations support the intended conclusions.

The revised paper acknowledges that AI use cannot be controlled in take-home assessment. It places Levels 2 through 5 in a category of open assessments and recommends separate, supervised assessments without AI to establish individual learning. That is an important qualification. But it leaves a question: what do grades on the open assessments themselves establish about the student?

If the scale helps design learning activities while separate controlled assessments provide assurance of particular learning outcomes, say that plainly. The controlled assessments provide their own evidence. They do not automatically validate every grade assigned to an open task elsewhere in the program.

A Theory of Learning Is Not Yet an Argument for an Assessment

The revision grounds the framework in social constructivism and discusses AI as a source of support within the gap between what students can do independently and what they can do with assistance.

That may help explain why an AI-supported activity could promote learning. It does not tell us what a particular completed artifact establishes about the learner.

The paper itself acknowledges that AI support may remain available rather than being gradually withdrawn. If the intended outcome is independent ability, we need evidence that students can perform independently. If the intended outcome is effective performance with AI, we need evidence of that performance and clarity about the tool conditions.

We cannot infer either one just because the output is successful. A theory explaining how learning might occur does not establish that learning occurred in a particular case.

The same applies to trust and transparency. Honest disclosure matters, but even a perfectly honest statement that AI produced a solution does not tell us how much the student understands. The assessment problem remains even when no one has broken a rule.

What the Evidence Needs to Establish

The original paper explicitly acknowledges that it did not yet have empirical evidence of the scale's effectiveness in improving student outcomes or addressing misconduct. The revision cites a pilot implementation (Furze et al., 2024) as evidence of benefits. At one point it says the original scale "has been demonstrated to be effective in empirical studies for improving student outcomes, supporting thoughtful assessment redesign, and reducing misconduct." In its conclusion it says more cautiously that "early evidence suggests" these benefits, and it calls for further research on learning outcomes and assessment validity. It also reports broad adoption of the framework.

These are different claims, and I want to keep them apart. Adoption can establish that educators find a framework useful. Clearer expectations can be valuable. Reported improvements can justify further investigation. None of those, by itself, establishes that a particular assessment reliably distinguishes students who possess an intended ability from students whose tools supplied the relevant performance.

I am not claiming that the cited pilot is wrong. This response examines the two framework papers. It is not an independent evaluation of that study. My objection is that the revised paper's examples do not provide the missing argument for how their proposed evidence establishes individual ability.

For instance, higher grades on AI-assisted submissions would need to be distinguished from stronger demonstrated learning. A reduction in misconduct cases would need to be interpreted in light of changes to what use is permitted and how cases are identified. These are questions an evaluation needs to answer. We cannot infer the answers from summary phrases such as "improve student outcomes" or "reduce academic misconduct."

What the Levels Do Not Tell Us

The scale combines several different questions: whether AI is permitted, which part of the work it supports, whether its use is expected, and how exploratory the task should be. These may all be useful design choices. But choosing among them does not answer the question of evidence.

A level tells students how much AI they may use. It does not say which ability is being assessed. Independent problem-solving, understanding an existing solution, evaluating an output, and directing AI are different accomplishments, and two tasks at the same level can be assessing different ones. A level also does not say which tools and assistance are actually available or how those conditions are established. It does not say what the student actually demonstrates, or why that observation supports the intended claim. And it does not say what remains unestablished, even if the student performs well. Each proposed assessment needs to answer those questions, whichever level it is assigned.

The AIAS may help instructors discuss and design activities involving AI. That is a useful contribution, but a limited one. I am not convinced that its levels, examples, and proposed process evidence resolve the problem of establishing individual achievement in open assessment.

The papers keep doing the same thing. They say that students will plan, critique, refine, direct, or innovate, and then they describe an assignment that asks them to do so. What is missing is the reason to conclude that they did those things themselves, or demonstrated the intended competence in doing them with AI.

That is the question an assessment framework needs to answer. Giving the intended contribution a name does not make it observable.