Evidence · searched to 2 Nov 2026
What hasn't worked: when AI made work slower or worse
Studies where people using AI did worse than people without it, and why. Knowing where AI fails is as useful as knowing where it helps.
Every result comes from the study itself, with the authors' own words where they help. Most come from tasks that looked like a good fit for AI but were not, from people trusting the output without checking it, or from learning with an AI that gave the answers away.
Use cases rated mixed or against
Study results where AI made things worse
- Coding with AI, Becker et al. (2025): +19% Implementation time per issue (developer-logged time, before and after PR review), regression-adjusted for forecast difficulty.
we find that allowing AI actually increases completion time by 19%--AI tooling slowed developers down.
- Coding with AI, He et al. (2026): +30.26% Static analysis warnings (SonarQube), average post-adoption effect.
static analysis warnings increase significantly by 30.3%
- Coding with AI, He et al. (2026): +41.64% Code (cognitive) complexity, average post-adoption effect.
Code Complexity 0.3481 (0.0538) +41.64% (±7.62%)
- Coding with AI, Agarwal et al. (2026): +17.73% Static analysis warnings, agent-first repositories (IDE-first estimate +19.00% is not significant).
Static Analysis Warnings 0.163***/0.174 (0.048)/(0.114) 17.73 / 19.00
- Coding with AI, Agarwal et al. (2026): +39% Cognitive complexity, both groups.
static-analysis warnings rise by about 18% and cognitive complexity by roughly 39% (Table 2).
- Coding with AI, Agarwal et al. (2026): repositories already using AI IDEs before agents (IDE-first): Only a short bump in commits (+16-28% in months 0-2), then lines added about -61% and commits about -35% by month 6, while complexity still rises (section 4).
IF repositories show only a short-lived bump around adoption (commits +16-28% at τ=0-2) before estimates return near zero and eventually turn negative
- Data analysis with AI, Fitzpatrick et al. (2025): −12% Accuracy of answers to six questions on insurer reporting templates, 40 values in all, graded against ground truth (32.3 Gen AI vs 36.5 control out of 40).
the control group outperformed the treatment group with an average score of 36.5 out of 40 for control and 32.3 out of 40 for treatment
- Data analysis with AI, Fitzpatrick et al. (2025): Participants scoring under 100% on the baseline assessment (N=18): With Gen AI, the correct-answer rate fell from 79.63% on the assessment to 65.83% on the experiment task.
Gen AI use on the experiment task led to a decrease in percentage of correct answers from 79.63% on the assessment task to 65.83%
- Data analysis with AI, Fitzpatrick et al. (2025): Participants scoring 100% on the baseline assessment (N=49): With Gen AI, the correct-answer rate fell from 100% to 84.48%; the baseline task was too easy to separate people.
Gen AI use on the experiment task led to a decrease in percentage of correct answers from 100% to 84.48%
- Data analysis with AI, Brodeur et al. (2026): −0.95 major errors detected per team Major coding errors detected per team, AI-assisted vs human-only (0.74 vs 1.70).
human-only teams identified on average 1.00 minor and 1.70 major errors, compared with 1.40 minor and 0.74 major errors for AI-assisted teams
- Data analysis with AI, Brodeur et al. (2026): Teams where ChatGPT led the work (minimal human oversight): Reproduced 56.8 points fewer papers than human-only teams (37%) and took about 119% longer.
- AI in code review, Maddila et al. (2025): +5.5% TimeInReview: time a diff spent in review when AI patches were shown to reviewers (Expt. 1).
We see statistically significant regressions in the time a diff was in the review state (5.5%, p = .029)
- AI in code review, Cihan et al. (2024): +42% Average pull request closure time, before vs after the bot (5h52m to 8h20m; % computed).
The overall average pull request closure duration increased from five hours and 52 minutes before the introduction of CodeReviewBot to eight hours and 20 minutes
- AI in code review, Cihan et al. (2024): Project #1: Closure time rose from 2h48m to 4h38m (p < 0.001).
Project #1 showed a significant increase in closure duration, rising from two hours and 48 minutes to four hours and 38 minutes
- AI in code review, Cihan et al. (2024): Project #3: closure time rose from 20h22m to 30h51m (p < 0.001).
- Customer service with AI, Ni et al. (2026): top performers (Q5): Quality worsens: dissatisfaction up 0.070, ratings down 0.283 points; retrials up 0.009 (ITT).
reverse for Q5 agents, whose dissatisfaction rate increases by 0.070 (p<0.01) and whose ratings fall by 0.283 points (p<0.01).
- AI-written marketing content, Fang et al. (2026): −10.3% Ad clicks, AI-optimised Google Shopping ad titles.
- AI-written marketing content, Exner et al. (2025): AI-generated images that look more like AI: Click-through falls as perceived artificiality rises: interaction -0.5175 log-odds (SE 0.1764, p < 0.01).
the more an AI-generated image looks like AI, the lower its CTR (β AI generated image × looks like AI = −.5175, SE = .1764, p<.01)
- Writing with AI, Ju and Aral (2026): −0.136 points on a 7-point rating scale Human-rated ad image quality (7-point scale), human-AI vs human-human teams.
we found that human-AI teams produced higher quality ad text but lower quality ad images compared to human-human teams (Table 2)
- Writing with AI, Ju and Aral (2026): Output diversity: Human-AI teams produced more similar ads to each other (cosine distance -0.094), a homogenising effect.
human-AI teams produce outputs that are closer to their condition centroid than human-human teams (= -0.094, p < 0.001)
- Analysis and consulting with AI, Dell'Acqua et al. (2026): −19 points Correct solutions on a complex managerial task chosen to be outside the frontier.
for a complex managerial task selected to be outside the frontier, subjects using AI were 19% less likely to produce correct solutions compared with those without AI
- Legal work with AI, Schwarcz et al. (2026): Highest baseline skill (scores on tasks done without AI): AI access can lower quality for the strongest participants; o1-preview's quality gains were larger for lower-GPA students.
for those with the highest baseline skill levels (as measured by performance on tasks completed without AI), access to the AI tool can actually reduce overall performance quality.
- Legal work with AI, Autor et al. (2026): Junior lawyers, unassisted redlining after three months: No average gain (-0.03 SD, p = 0.89); scores split into more poor and more good results.
By contrast, the point estimate for junior lawyers is essentially zero (-0.03 SD, = 0.89).
- AI in financial analysis, Marsal and Perkowski (2025): Participants slower with AI (within-person comparison): 13 percent of participants were less efficient when they had the tool; 80 percent were more efficient.
80 percent of participants were more efficient when they had access to the tool, versus 13 percent who were less efficient.
- AI in financial analysis, Xue et al. (2026): −0.16 SD Relative earnings-forecast accuracy (scaled by the standard deviation of peer forecast errors), FactSet x Post.
post-integration FACTSET-associated reports exhibit 0.157 standard deviations lower relative forecast accuracy than the comparison group.
- AI in financial analysis, Xue et al. (2026): Analysts covering more firms than the median: Forecast accuracy fell by 0.174 SD (t = -2.40) for analysts with many firms, against +0.044 SD (not significant) for those with few; difference p = 0.029 (one-tailed).
GenAI use decreases accuracy and increases signal misalignment significantly when analysts face attention constraints, while the effects are largely insignificant when attention constraints are low.
- AI in financial analysis, Xue et al. (2026): Analysts on busy days (revising two or more firms the same day): Accuracy fell by 0.205 SD (t = -2.97) on busy days against -0.066 SD (not significant) otherwise.
The deterioration in accuracy and signal alignment is concentrated among analysts who cover more complex portfolios, work with smaller teams, or revise multiple firms' forecasts on the same day
- AI in financial analysis, Sheng et al. (2025): Asset managers that are not hedge funds: No clear benefit: weak evidence on FF4 alpha, and raw returns fell with adoption.
In contrast to their hedge fund peers, we find mixed evidence on whether these asset managers can generate superior performance
- AI in financial analysis, Sheng et al. (2025): Small, young and passive hedge funds: No significant return gain; gains confined to large, older and more active funds (top v bottom quintile differences significant for size, age, turnover).
large, older, and more active hedge fund companies are able to leverage generative AI to obtain significant returns, while small, younger, and more passive companies fail to do so.
- AI drafts of patient replies, Tai-Seale et al. (2024): +21.8% Time spent reading a patient message (includes reading the AI draft).
The estimated association of GenAI was a 21.8% increase in read time (95% CI, 5.2% to 41.0%; P = .008)
- AI drafts of patient replies, Jeanselme et al. (2026): +8.6% Message turnaround time (patient submission to first clinician reply) after a clinician adopts the tool.
Response tool adoption was associated with an 8.6% (95% CI, 4.6%-12.9%; P < .001) increase in overall message turnaround time.
- AI drafts of patient replies, Jeanselme et al. (2026): Messages received after hours: Tool use linked to 4.52 percentage points more after-hours replies.
for all messages received after hours, response tool use was associated with an increase in after-hours responses (4.52 [95% CI, 1.54-7.50] percentage points; P = .003)
- AI drafts of patient replies, Chen et al. (2024): Unedited AI drafts: 7.1% were judged to risk severe harm if sent as written.
- AI drafts of patient replies, Mandal et al. (2025): +7.3% Median time the message was open, with versus without the draft.
the median message open time increased by 7.27% (median [utilized]: 59 seconds, median [not utilized]: 55 seconds, difference: 4 seconds
- AI help with diagnosis and treatment, Goh et al. (2025): +119.3 seconds per case Time per case.
Physicians randomized to use the LLM spent 111.3 seconds more on each case (801.5 seconds versus 690.2, difference: 119.3 seconds, 95% CI: 17.4 to 221.2, p = 0.022)
- AI help with diagnosis and treatment, Everett et al. (2026): AI-as-second-opinion cases: Actionable-decision score fell after seeing the AI in 12 of 147 cases (rose in 52, unchanged in 83).
Comparatively, the actionable score decreased in 12 cases.
- Learning with an AI tutor, Bastani et al. (2025): −17% Unassisted closed-book exam score, GPT Base vs control.
students in the GPT Base arm perform statistically significantly worse than students in the control arm by 17%
- Learning with an AI tutor, Nie et al. (2025): −4.3 points Optional diagnostic exam participation, offered access vs control (intention-to-treat).
giving students access to and advertising GPT-4 led to a substantial, statistically significant decrease in their exam participation. The effect of access to GPT was negative 4.3 percentage points.
- Lesson planning with AI, Sungu et al. (2026): −0.11 SD Student intrinsic motivation (composite of interest, importance and enjoyment items), teachers given AI v business-as-usual control.
Providing teachers with the AI teaching tool significantly reduces their students' intrinsic motivation (β = -0.111, 95% CI [-0.199, -0.023], p = 0.015)
- AI in screening and interviews, Jabarian and Henkel (2026): Interview experience: AI-led interviews were rated significantly less natural by applicants (p = 0.014); other experience ratings were similar.
applicants rate the interview experience with the AI voice agent as significantly less natural (p = 0.014, t-test).
- AI in screening and interviews, Aka et al. (2026): Applicants invited to the AI interview: About 75% of invited applicants did not complete the 30-40 minute AI interview, moving screening cost onto applicants.
The participation cost falls on applicants as 75 percent of invited candidates do not complete the interview.
- AI in screening and interviews, Awuah et al. (2026): +13% Evaluator time to final grade per essay answer with AI assistance vs without (log seconds, with controls; mean 170 s).
AI assistance also increases grading time: essays take 13-17% longer to grade, driven by disagreement cases where grading time increases by 26%
- AI in screening and interviews, Awuah et al. (2026): Evaluators given AI suggestions: Evaluators overrode the AI in over 80% of cases where their grade differed, mostly when the AI suggested a higher grade.
Evaluators override the recommendation in over 80% of cases where their grade and the AI grade differ
- AI-written job posts and applications, Wiles and Horton (2026): −3.4 points Probability the employer made a hire within 14 days, conditional on posting a job.
treated employers who posted a job were 15% less likely to make a hire.
- AI-written job posts and applications, Wiles and Horton (2026): Employers less likely to edit the AI draft: Hiring fell 4.5 pp among employers with low predicted active use against 1.9 pp among high predicted use.
The treatment reduced hiring by 4.5 percentage points among employers with low predicted active use, compared with 1.9 percentage points among employers with high predicted active use.
- AI-written job posts and applications, Wiles and Horton (2026): Jobseekers: Treated posts drew about 5% more applications per post; the authors estimate jobseeker time costs about six times the employer time saved.
jobseeker application time costs are roughly six times larger than employer time savings per treated employer.
- AI-written job posts and applications, Cui et al. (2025): Employers (signal value): After launch, the link between cover-letter tailoring and callbacks fell by 51% and with offers by 79%; employers leaned more on review scores.
The correlation between cover-letter tailoring and receiving a callback fell by 51% after the launch of the AI tool
- AI in creative and design work, Fu et al. (2024): Native English speakers: Reported lower relaxation in the GenAI session (p < .05).
Native English speakers reported significantly lower relaxation in the GenAI-supported condition as compared to the human-only condition (p < .05).
- AI in creative and design work, Doshi and Hauser (2024): +10.7 Similarity of each story to the other stories in its condition (higher = less collective diversity), as a share of the human-only range.
the increase in similarity from having access to one or five generative AI ideas represents 10.7% and 8.9% of the total range, respectively.
- AI in systematic reviews, Gartlehner et al. (2025): One of the six reviews: AI-assisted extraction took up to 25 minutes longer per study than human-only; the other five were faster.
The range of time differences across the six reviews spanned from 78 minutes shorter to 25 minutes longer (Figure 3).
- Translation with AI, Martikainen et al. (2025): Individual translators: Speed change ranged from -2% (the translator with most segments) to +102%.
However, productivity varied notably between individuals, with improvements ranging from −2% to 102%.
- Business advice from AI, Otis et al. (2026): Low performers at baseline (below median): -0.08 SD (95% CI -0.13 to -0.03), about 9-10% lower revenue.
Subsample analyses show that low performers did nearly 10% worse because of the AI assistant, whereas high performers may have benefited by over 15%.
- Summarising with AI, Fitzpatrick et al. (2025): Questions needing information gathered from across the whole document: Gen AI group did worse on the business-model question, where control beat treatment.
the treatment group performed more poorly on questions where a high quality answer required collating information from throughout the document.
- Explaining rules with AI, Gosciak et al. (2026): −18 points Question-level accuracy after an INCORRECT suggestion vs control on the same questions (35 questions, control accuracy 38%).
an incorrect chatbot suggestion leads to a mean 18 percentage point decrease in caseworker accuracy relative to the control group
- Explaining rules with AI, Gosciak et al. (2026): Easy questions (control accuracy about 75% or higher) shown an incorrect suggestion: Accuracy fell by about two-thirds relative to control.
When shown incorrect chatbot suggestions on easy questions, caseworker accuracy falls by approximately two-thirds relative to the control group.
- Explaining rules with AI, Gosciak et al. (2026): Caseworkers generally, high-quality chatbot: Accuracy plateaued below what following every suggestion would give (the 'AI underreliance plateau').
AI underreliance plateau
- Requirements and design with AI, Broccia et al. (2026): Group that did the first session WITH ChatGPT: Improved about 6-7% between sessions vs about 12% for the group that started without ChatGPT; the authors read this as slower skill learning.
performing the task first with and later without LLMs reduced the learning effect
When AI goes wrong in the world
Studies measure averages. For individual failures, see AI incidents, and for dated events, The AI Record.