Evidence · strength 1 of 5 · searched to 2 Nov 2026
Do AI-drafted replies save clinicians time on patient messages?
Not enough evidence yet to say whether AI-drafted replies save clinicians time on patient messages. Across 4 studies, time on each message changed by −64.2% to +8.6%.
What studies found when clinicians answered patient portal messages starting from an AI draft: time spent, reply length and safety.
VerdictNot enough evidence yet
No good evidence
ImpactMixed
Across 4 studies, time on each message changed by −64.2% to +8.6%.
Evidence strength1 of 5
We have very low confidence in this estimate.
Cost per user a year?
Usually part of an electronic health record contract; prices are not published.
Pilot rating under our v0.3 rules. The rules are published on our method page but not yet independently reviewed, so ratings may change as the method is checked and new studies arrive. Our method.
What the studies found
Time (headline)
Mixed−64.2% to +8.6%- Tai-Seale et al. (2024): +21.8% Time spent reading a patient message (includes reading the AI draft) (95% CI 5.2% to 41.0%; P = .008).
The estimated association of GenAI was a 21.8% increase in read time (95% CI, 5.2% to 41.0%; P = .008)
- Tai-Seale et al. (2024): −5.9% Time spent writing the reply (95% CI -16.6% to 6.2%; P = .33). Not statistically clear.
a -5.9% change in reply time (95% CI, -16.6% to 6.2%; P = .33)
- Jeanselme et al. (2026): +8.6% Message turnaround time (patient submission to first clinician reply) after a clinician adopts the tool (95% CI 4.6% to 12.9%; P < .001).
Response tool adoption was associated with an 8.6% (95% CI, 4.6%-12.9%; P < .001) increase in overall message turnaround time.
- Jeanselme et al. (2026): +0.6 points Share of replies sent outside working hours when the tool is used (95% CI -1.02 to 2.24; P = .46). Not statistically clear.
Response tool use was not associated with an overall reduction of after-hours messaging burden (0.61 [95% CI, -1.02 to 2.24] percentage points; P = .46).
- Whitehouse et al. (2025): −64.2% Time to answer a simulated patient message with an AI draft versus a blank draft (P < .001).
Response time averaged 49 ± 67 seconds per question with artificial intelligence drafts vs 137 ± 115 seconds with a blank draft (P < .001).
- Mandal et al. (2025): −6.8% Median turnaround time, messages answered with versus without the draft (p < 0.001).
Responses utilizing GenAI drafts had a 6.76% shorter median turnaround time than those that did not utilize GenAI drafts (median [utilized]: 331 seconds, median [not utilized]: 355 seconds
- Mandal et al. (2025): +7.3% Median time the message was open, with versus without the draft (p < 0.001).
the median message open time increased by 7.27% (median [utilized]: 59 seconds, median [not utilized]: 55 seconds, difference: 4 seconds
Quality
No clear effect- Chen et al. (2024): +0.42 change in mean Cohen's kappa Agreement between physicians on clinical content, LLM-assisted versus manual. Not statistically clear.
Interphysician agreement in the clinical content of responses was poor for manual responses (mean Cohen’s kappa 0·10) but improved with LLM-assistance (mean Cohen’s kappa 0·52).
Wellbeing
Gain- Whitehouse et al. (2025): −1.2 points on raw NASA-TLX, AI draft minus blank Raw NASA Task Load Index (cognitive load) (P < .001). Self-reported.
Mean Task Load Index was lower with artificial intelligence drafts (3.2 ± 1.5 vs 4.4 ± 1.3, P < .001)
A minus sign on time means the task took less time. Bands are provisional; see how impact is banded. Effects are not pooled yet: each band is set from one result per study, and where studies use different units we show only the direction.
How sure are we?
- Starting level2 of 5, from 4 studies graded A, B or C.
- Down one: old models0 of 5 studies show they used a model released since May 2025 (the 18-month window); the rest used older models or do not say which.
- Publication biasnot testable with fewer than 10 studies.
- Result1 of 5. We have very low confidence in this estimate.
Behind the average
- Test-result queries received during working hours (Jeanselme et al. (2026)): Tool use linked to 3.67 percentage points fewer after-hours replies
- Messages received after hours (Jeanselme et al. (2026)): Tool use linked to 4.52 percentage points more after-hours replies
- Unedited AI drafts (Chen et al. (2024)): 7.1% were judged to risk severe harm if sent as written.
- Physicians vs administrative staff (Mandal et al. (2025)): Physicians started from the draft least often (mean 0.12 in the final period) while administrative support peaked at 0.32
How, where and why it works
Why it works
1 of 4. Drafts give a starting point, but in one trial clinicians spent longer reading each message, and replies got longer.
For whom and where
1 of 4. Studies rarely split results by role or message type; one found physicians started from the draft least often.
What it takes
2 of 4. Studies describe drafts built into the record system, with clinicians choosing whether to use each one.
The full cost
0 of 4. No study counts cost.
Four separate scores from 0 to 4, never added up: how much the studies tell us about why it works, for whom, what it takes and what it really costs (the EMMIE method).
What to do next
Measure your pilot →If you are trying it yourself
- Read every draft against the patient's record before sending.
- Note drafts that were wrong or unsafe and report them.
If you are rolling it out
- Measure time from audit logs, with a comparison group.
- Review a sample of sent replies for errors.
- Tell patients when replies are drafted with AI.
What it takes
- Drafts appear inside the record system; clinicians decide whether to start from them.
- Expect many drafts to go unused: in one health system, physicians started from the draft least often of all staff.
What can go wrong
- In one study, 7.1% of unedited drafts were judged to risk severe harm if sent as written.
- Replies written from drafts were longer, which can mean more for patients to read.
The studies
| Study | Grade | Design | People | AI model (released) | Company role |
|---|---|---|---|---|---|
| AI-Generated Draft Replies Integrated Into Health Records and Physicians' Electronic Communication Tai-Seale M, Baxter SL, Vaida F, Walker, Sitapati, Osborne, Diaz, Desai, Webb, Polston, Helsten, Gross, Thackaberry, Mandvi, Lillie, Li, Gin, Achar, Hofflich, Sharp, Millen, Longhurst (22 authors; surnames as on PubMed), 2024. JAMA Network Open 7 (journal) | C | field RCT, deployed work | 52 volunteer primary care physicians randomised (25 immediate, 27 delayed activation) plus 70 non-participating physicians as a contemporary control; 10,679 replies | Epic in-basket generative AI draft replies (underlying model not named in the paper) | none |
| AI-Generated Draft Response Tool, Message Turnaround Time, and After-Hours Replies Among Health Care Professionals Vincent Jeanselme, Catherine Austin, Jungmi Han, Rachel A. Lewis, Gregory W. Hruby, Karthik Natarajan, Shalmali Joshi, 2026. JAMA Network Open 9 (journal) | C | other quasi-experiment, deployed work | 983,154 first responses from 1,542 clinicians to 272,091 patients | Epic In Basket Augmented Response Technology using an OpenAI large language model (version not stated; authors had no access to it) | none |
| Easing the burden: A pilot study evaluating AI-generated In-Basket message drafts to streamline perioperative endocrine surgical care Whitehouse, Heslin, Desir, Raghunathan, Islam, Lallky, Philip, Reedy, Mehta, Parmer, Piersall, Oltmann, Dackiw, Maalouf, Sant (surnames as on PubMed), 2025. Surgery, article 109700 (journal) | C | RCT, realistic task | 11 health care providers, each answering 20 randomised simulated In-Basket messages (10 with GPT-4 drafts, 10 blank) | GPT-4 (Mar 2023) | none |
| The effect of using a large language model to respond to patient messages Chen, Guevara, Moningi, Hoebers, Elhalawani, Kann, Chipidza, Leeman, Aerts, Miller, Savova, Gallifant, Celi, Mak, Lustberg, Afshar, Bitterman (17 authors; surnames as on PubMed), 2024. The Lancet Digital Health (journal) | C | other quasi-experiment, realistic task | 6 attending radiation oncologists; 100 simulated cancer-patient messages (56 dual-annotated, 44 single-annotated) | GPT-4 (OpenAI API, temperature 0) (Mar 2023) | none |
| Utilization of Generative AI-drafted Responses for Managing Patient-Provider Communication Mandal, Wiesenfeld, Szerencsy, Small, Major, Richardson, Schoenthaler, Mann, Nov (surnames as on PubMed), 2025. npj Digital Medicine (journal) | D | cross-sectional, deployed work | 75 health care professionals (54 physicians); 11,313 patient messages answered, 10,475 with complete timestamps | GPT-4 via Epic InBasket drafts (Mar 2023) | none |
Grades: A, a well-run randomised trial in real work; B, a strong natural experiment in real work or a well-run trial on realistic tasks; C, a trial with clear limits or a matched comparison; D, before-and-after, surveys or case studies. How studies are graded.
Does the rating hold up?
- Run without company-run studiesNo change: no such studies.
- Run without working papers and preprintsNo change: no such studies.
How up to date is this evidence?
Method →Evidence searched to 2 Nov 2026. Studies cover models released Mar 2023 to Mar 2023. Rating computed 5 Nov 2026 under Module 15 v0.3 (pilot).
How this page was made: AI research agents searched for studies and read each paper; a second pass checked every number on this page against the paper itself, with the quote and where it appears. The rating is computed by fixed rules, not chosen. No person outside AI Adoption has reviewed it yet. This page is not yet in search engines: health, legal and finance pages wait for a named domain reviewer, and none is appointed yet. Editorial owner: AI Adoption editors. Spot a mistake? Corrections.
Disclosure: AI Adoption uses AI tools, including Claude from Anthropic, in its research and editorial work. No company pays for or has any say in a rating.