When NOT to Use an AI Scribe: Visits Where It Makes Things Worse

The honest counterpoint

There's a category of writing about AI scribes that treats them as universally beneficial. Just turn them on and reap the rewards. Some vendors sound like this. Many adopters do too.

But there are specific kinds of visits where AI scribes actively hurt: they reduce documentation quality, increase risk, or harm the patient relationship in ways the time savings don't compensate for. Knowing where the boundary is matters as much as knowing how to use the tool well within the boundary.

This article is about those visits. The ones where the better workflow is to type, type later, or document by hand.

High-acuity visits where attention should be on the patient

A visit where the patient is genuinely unstable (chest pain ruling out an MI, suicidal ideation actively being assessed, abdominal pain ruling out an acute abdomen) demands full cognitive presence from the provider. The AI scribe is supposed to free attention back to the patient, but in practice, providers who are using a tool they don't fully trust spend cognitive cycles wondering whether the tool is capturing things correctly.

In high-acuity visits, that wondering is wasted attention.

The pattern that works: use the AI scribe on the visit if you want to, but be willing to abandon it mid-visit. If you find yourself thinking about the AI instead of about the patient, the AI has become a distraction. Pause the recording, finish the visit by typing or by memory, and document fully afterward.

A few specific high-acuity patterns where many providers default to typing:

• The first 10 minutes of a code or pre-code situation. The AI scribe might capture useful documentation, but if your attention is being pulled to it, abandon it.
• A patient disclosing imminent suicidality. This is exactly the moment for the patient to feel fully attended to, not for the provider to be thinking about a tool.
• A patient with significant respiratory distress. The audio environment is bad (the patient is breathing hard), the visit is short, the documentation is dense. AI scribes underperform here.

Sensitive disclosures the patient doesn't want recorded

Some patients will not consent to recording, period. That's covered in how to handle patient refusals.

Some patients will consent to recording but will, during the visit, disclose something they want kept off the formal record. The AI is now capturing content the patient doesn't want documented.

The right response is to pause the recording when the patient signals (or when you recognize the disclosure type even if they haven't signaled), document the clinical information that's required (suicidality, child abuse, certain communicable diseases) in your own words at the appropriate level, and leave the rest in your memory and the therapeutic relationship.

The wrong response is to let the AI capture it because "the patient consented at the start." Consent is ongoing. The patient who consented at minute one has the right to revoke consent at minute fifteen, and the chart should reflect that revocation.

Documentation that requires a specific structure the AI doesn't produce

Some specialty documentation has required structures that AI scribes don't naturally generate:

Forensic evaluations. Court-ordered psychiatric evaluations, fitness-for-duty evaluations, disability determinations. These have specific structural requirements (specific sections, specific phrasings, sometimes specific length). AI scribes don't generate these well; the work is faster typed using a template.

Pre-operative anesthesia notes. ASA classification, airway assessment, specific risk factors. Many EHRs have a structured form; AI scribes generate prose that has to be transferred to the form.

Detailed dermatology lesion descriptions. Some dermatologists document each lesion with specific anatomic location, size, color, morphology. AI scribes generate prose; structured lesion documentation is faster in the EHR's structured fields.

Operative notes. AI scribes designed for office visits don't usually generate operative notes. Surgeons typically have their own dictation workflow for OR notes; the AI scribe isn't the right tool for that documentation.

Procedure notes that go in a separate document. Endoscopy reports, ophthalmology procedure notes, dermatology procedure notes. These often go in a structured procedure report, not the encounter note. AI scribes don't produce the procedure report; they produce the encounter note.

For these documentation types, the AI scribe is either irrelevant (the documentation lives elsewhere) or counterproductive (it generates prose that has to be reformatted). Type, or use the existing dictation or templating workflow.

Visits where the patient is too quiet or too distressed

The AI scribe needs to hear the patient. A patient who is whispering, crying, mumbling, or sedated may not produce audio the AI can transcribe reliably.

The signal that you're in this territory: you yourself are leaning in to hear the patient. If you can barely hear them, the AI can't either.

The chart for these visits often relies more on the provider's observation than on the patient's report. Documenting "patient appeared withdrawn, spoke in barely audible voice" is information you observed, not something the AI captured. Type the note.

Visits with audio you don't want stored anywhere

A small but important category: visits where the audio itself is sensitive in a way that recording is fundamentally inappropriate.

Patients in protective custody or witness protection. Their audio in any vendor's system, even briefly, is a security exposure.

Patients who are court-protected or whose location is confidential. Same principle.

Patients who have explicitly cited a fear of being recorded as the reason they're seeking healthcare. A patient who came in specifically because they're being stalked or harassed has a legitimate, deep aversion to being recorded.

High-profile patients whose audio could become news. Politicians, celebrities, or anyone whose unguarded statements in an exam room could be of interest to media or adversaries. The AI scribe vendor's security is probably fine; the risk-reward calculation may still favor no recording.

For these patients, the standard workflow assumption (turn on the AI scribe) is wrong. Type. The few extra minutes are bought protection.

Visits where the AI's diarization will scramble the chart

Multi-speaker visits with four or more speakers often degrade AI scribe accuracy enough that the time saved is less than the time spent fixing the chart. The boundary depends on the platform.

A general rule: more than three speakers and you should expect to spend substantial review time editing speaker attribution. If the review takes longer than typing would have, the AI scribe is net negative for this visit. Type.

Visits where you can't trust your own review

The AI scribe model assumes the provider does a thorough review of every note before signing. The review is the safety net.

There are conditions under which the review can't be trusted:

You're exhausted. A provider at the end of a 12-hour day reviewing notes from earlier visits may miss errors a fresh provider would catch. If you're at the point where you'd sign a note without really reading it, sign tomorrow morning instead, or type the note when it's fresh.

You're distracted by an unrelated crisis. A personal emergency, a parallel patient crisis, a system outage you're managing. The cognitive cycles for note review aren't available. Defer the review.

You're in a rush you can't escape. "I'll just sign these quickly so I can leave." Quickly-signed notes are how medication errors propagate. If you can't take the time to review properly, take the time after a short break.

The AI scribe's value collapses when the review is rushed. It's better to type a note thoughtfully than to AI-generate and sign without reading. The output of the second pattern can be worse than the output of typing.

Visits where consent is impossible to obtain meaningfully

Edge cases where the consent conversation can't happen meaningfully:

Patients with severe cognitive impairment. A patient with dementia cannot meaningfully consent to recording. If the patient has a power of attorney present, that person can consent on the patient's behalf, but many practices conclude that recording a cognitively impaired patient without their meaningful understanding is ethically uncomfortable, even when legally permissible.

Patients in delirium. Their consent isn't reliable. The visit should be documented; the recording is the wrong tool.

Patients under involuntary hold. The dynamics of consent under involuntary admission are complicated enough that adding a recording layer often isn't worth the friction.

Patients you're seeing without their knowledge (e.g., consultation about a patient's case without the patient present). The patient isn't there to consent. Family discussions about a patient's care without the patient may or may not be appropriate to record; the consent of the present participants is usually adequate but the absent patient's privacy should be respected.

The honest framing

AI scribes work well for the most common visit types: ambulatory primary care, most specialty office visits, follow-ups, telehealth in many cases, even most psychiatric and pediatric visits with appropriate adjustments.

They don't work well, or shouldn't be used at all, for:

• High-acuity unstable visits where attention should be on the patient
• Sensitive disclosures the patient wants off-record
• Forensic, pre-op, operative, and procedure documentation
• Visits with audio quality the AI can't handle
• Visits where the patient is protected and recording is a security exposure
• Visits with too many speakers
• Visits where review will be rushed or distracted
• Visits where meaningful consent can't be obtained

Recognizing the boundary protects the patient (their chart is more accurate, their consent is respected) and protects the provider (their notes are defensible, their tool is trustworthy where they use it).

A provider who knows when not to use the AI scribe is a more credible adopter than a provider who uses it on everything. The latter is signaling enthusiasm; the former is signaling judgment. Judgment is what makes the workflow safe long-term.

What this means for practice policy

Practices adopting AI scribes should have an explicit policy that includes the boundary, not just the enthusiasm:

"We use AI scribe technology for most routine outpatient visits with patient consent. We do not use it for [list]. We pause or stop recording when [list]. Patients can decline at any time without affecting their care."

The explicit list isn't bureaucratic overhead. It's the document that protects the practice if a question is ever raised about whether a specific visit was appropriately recorded. It also protects providers from feeling pressure to use the tool in situations where their judgment says not to.

---

For the contexts where AI scribes work well, see the broader article series. For the workflow for handling refusals that come up at the patient level, patients refuse AI scribe how to handle covers the in-visit conversation.