Evidence · strength 1 of 5 · searched to 2 Nov 2026
Does AI feedback on students' work help them learn?
Not enough evidence yet to say whether AI feedback on students' work helps them learn. 0 of 2 studies found a clear gain in learning, tested without AI.
What trials found when students got AI-generated feedback on their essays or code: better work while the feedback was on, and what they learned once it was gone.
VerdictNot enough evidence yet
No good evidence
ImpactNo clear effect
0 of 2 studies found a clear gain in learning, tested without AI.
Evidence strength1 of 5
We have very low confidence in this estimate.
Cost per user a year$
Feedback ran on general AI models through the course platform; per-student costs are small but not reported.
Pilot rating under our v0.3 rules. The rules are published on our method page but not yet independently reviewed, so ratings may change as the method is checked and new studies arrive. Our method.
What the studies found
Learning (headline)
No clear effect- Escalante et al. (2023): −1.01 test points Pre-post gain on a 300-word academic writing test: GPT-4 feedback group gain minus human-tutor feedback group gain (5.848 v 6.86) (group x time interaction F = 3.094, p = 0.085; posttest d = -0.156, p = 0.591). Not statistically clear.
Both groups made similar progress in their academic writing over the six week long treatment, as indicated by the increase in mean scores for the EG (5.848) and CG (6.86).
- Zhou et al. (2025): +1.6 points Last-submission score after GPT-4 feedback was withdrawn (later half of each module) (coefficient 0.016 (SE 0.019), p = 0.402). Not statistically clear.
Last Submission Score (0-1 scale) ... Experiment 0.029 (0.014) 0.029 (0.014) 0.041* 0.016 (0.019) 0.015 (0.019) 0.402
Errors
Gain- Zhou et al. (2025): −2.5 points Share of tasks with 'wheel spinning' (repeated unproductive attempts) while feedback was on (coefficient -0.025 (SE 0.012), p = 0.032).
taking breaks (β = -0.035, p = 0.050, τ = -0.089), and wheel spinning (β = -0.025, p = 0.032, τ = -0.134) compared to the control group.
Quality
No clear effect- Meyer et al. (2024): +0.19 SD Revision performance on an argumentative essay (automated essay scoring), LLM feedback v no feedback. Not statistically clear.
The results showed that LLM-generated feedback increased revision performance (d = .19) and task motivation (d = 0.36).
- Zhou et al. (2025): +2.9 points Last-submission score on programming tasks (0-1 scale) while GPT-4 compiler-error feedback was on (coefficient 0.029 (SE 0.014), p = 0.041).
The experimental group, for tasks with GPT feedback enabled, showed significantly higher last submission scores (β = 0.029, p = 0.041, τ = 0.137)
Wellbeing
No clear effect- Meyer et al. (2024): +0.36 SD Task motivation, LLM feedback v no feedback. Self-reported. Not statistically clear.
LLM-generated feedback increased revision performance (d = .19) and task motivation (d = 0.36).
- Meyer et al. (2024): +0.34 SD Positive emotions, LLM feedback v no feedback. Self-reported. Not statistically clear.
Moreover, it increased positive emotions (d = 0.34) compared to revising without feedback.
A minus sign on time means the task took less time. Bands are provisional; see how impact is banded. Effects are not pooled yet: each band is set from one result per study, and where studies use different units we show only the direction.
How sure are we?
- Starting level2 of 5, from 3 studies graded A, B or C.
- Down one: old models0 of 3 studies show they used a model released since May 2025 (the 18-month window); the rest used older models or do not say which.
- Publication biasnot testable with fewer than 10 studies.
- Result1 of 5. We have very low confidence in this estimate.
Behind the average
None of these studies reports results for different groups of people or tasks.
How, where and why it works
Why it works
1 of 4. AI feedback improves the piece of work it comments on; whether students carry the skill forward is less clear.
For whom and where
0 of 4. Studies do not split results by student group.
What it takes
1 of 4. Feedback was built into the course platform; studies say little about teacher involvement.
The full cost
0 of 4. No study counts cost.
Four separate scores from 0 to 4, never added up: how much the studies tell us about why it works, for whom, what it takes and what it really costs (the EMMIE method).
What to do next
Measure your pilot →If you are trying it yourself
- Use AI feedback on a draft, then revise yourself.
- Check whether you can do the next task without it.
If you are rolling it out
- Test learning without AI, not just the quality of AI-assisted work.
- Keep teacher feedback in the loop.
What it takes
- Feedback was given inside the course platform, on the student's own draft.
What can go wrong
- In one trial, gains in task scores vanished once the AI feedback was withdrawn.
The studies
| Study | Grade | Design | People | AI model (released) | Company role |
|---|---|---|---|---|---|
| Using LLMs to bring evidence-based feedback into the classroom: AI-generated feedback increases secondary students' text revision, motivation, and positive emotions Jennifer Meyer, Thorben Jansen, Ronja Schiller, Lucas W. Liebenow, Marlene Steinbach, Andrea Horbach, Johanna Fleckenstein, 2024. Computers & Education: Artificial Intelligence 6, 100199 (journal) | C | RCT, realistic task | 459 upper-secondary students of English as a foreign language | GPT-3.5-turbo | none |
| AI-generated feedback on writing: insights into efficacy and ENL student preference Juan Escalante, Austin Pack, Alex Barrett, 2023. International Journal of Educational Technology in Higher Education 20:57 (journal) | C | other quasi-experiment, deployed work | Study 1: 48 university English-as-a-new-language students (23 GPT-4 feedback, 25 human-tutor feedback); Study 2 (preferences): 43 students | GPT-4 (via ChatGPT) | none |
| Impact of LLM Feedback on Learner Persistence in Programming Yiqiu Zhou, Maciej Pankiewicz, Luc Paquette, Ryan S. Baker, 2025. Proceedings of the 33rd International Conference on Computers in Education (journal) | C | RCT, deployed work | 257 first-semester computer science students (129 GPT-4 feedback, 128 control) at a European university; 141 C# programming tasks | OpenAI GPT-4 (gpt-4-0613) (Jun 2023) | none |
Grades: A, a well-run randomised trial in real work; B, a strong natural experiment in real work or a well-run trial on realistic tasks; C, a trial with clear limits or a matched comparison; D, before-and-after, surveys or case studies. How studies are graded.
Does the rating hold up?
- Run without company-run studiesNo change: no such studies.
- Run without working papers and preprintsNo change: no such studies.
How up to date is this evidence?
Method →Evidence searched to 2 Nov 2026. Studies cover models released Jun 2023 to Jun 2023. Rating computed 5 Nov 2026 under Module 15 v0.3 (pilot).
How this page was made: AI research agents searched for studies and read each paper; a second pass checked every number on this page against the paper itself, with the quote and where it appears. The rating is computed by fixed rules, not chosen. No person outside AI Adoption has reviewed it yet. Editorial owner: AI Adoption editors. Spot a mistake? Corrections.
Disclosure: AI Adoption uses AI tools, including Claude from Anthropic, in its research and editorial work. No company pays for or has any say in a rating.