A woman in Australia agreed to have her specialist appointment transcribed by artificial intelligence. Months later she read the letter her specialist had sent to her GP and found a line saying she had been micro-dosing psychedelic mushrooms, offered as a possible explanation for bleeding around her kidneys. She had never taken them. Nobody had said it in the room. The ABC reported the case on 14 August 2026.

Software invented a clinical fact. That is the failure everyone talks about. Then the letter went to another doctor, and between the machine writing that line and it arriving in his inbox, no person read it against what actually happened in the consultation.

That gap is a job. It is most of what a healthcare documentation specialist does, and the reason the role is getting harder to automate rather than easier.

Transcription and summarisation tools fail in patterned, repeatable ways. Once you know the patterns, you can check for them deliberately instead of reading hopefully and trusting your eye to catch something.

The Royal Australian College of General Practitioners estimated last year that 40 per cent of GPs were regularly using AI scribes, and described that figure as conservative. So these patterns are now running through a large share of Australian primary care documentation.

Six failure modes cover almost everything that goes wrong.

None of these are exotic. Five of the six are things a trained medical transcriptionist has always checked for, because human dictation produces the same errors. What differs is the tone of the output. A tired human typist produces text that reads like it was produced by a tired human. A language model produces fluent, confident, plausible prose, and a wrong statement in fluent prose is much harder to notice than a wrong statement in garbled prose.

Speaking to the ABC, a Perth GP who chairs the RACGP’s digital health and innovation group and advises one of the scribe vendors put the practical version of it plainly: the classic error is getting the side of the body wrong, saying right when you said left, so doses and sides need close checking.

A single fabricated sentence matters this much because health records travel. The correspondence in the ABC’s report went from a specialist to a GP. In the ordinary course of care it would then inform the next consultation, the next referral, and any clinician who opens the file in five years without knowing where the line came from.

Records travel beyond clinical care as well. The patient in that report was receiving workers’ compensation at the time and worried the false drug reference could affect her claim, which is not far-fetched. Health information is read by insurers, by compensation schemes, by employers in some circumstances, and by patients themselves. A sentence that reads smoothly and sits in the right place in a letter carries the authority of the whole document.

So the cost of an unchecked draft is every decision made downstream of it.

The chief executive of the Consumers Health Forum of Australia, who is leading a research project on AI medical scribes, described the pattern this way: for the first few weeks of using AI, people check things pretty carefully, and then at a point they stop checking, because they assume it will be right.

That is a well-documented human tendency called automation bias, and it does not mean clinicians are careless. It means accuracy that depends on a busy person staying permanently sceptical about a tool that is usually correct is accuracy built on sand. As the tool gets better the error rate falls, and that falling error rate is precisely what erodes the vigilance that catches the errors that remain.

Australia’s regulator of medical practitioners, AHPRA, is clear that clinicians must check all output from an AI scribe for accuracy to meet their professional obligations. That obligation is not in doubt. What is in doubt is the structure behind it, because an obligation held by one distracted person at the end of a long clinic is not a system.

What fixes this is dull and effective. Someone whose actual job is the document, trained in what a report of that type must contain, reads it against a template and treats the AI draft as a draft rather than as a record.

Consent is the other half of the picture. The Digital Rights Watch report found large differences in how consent is obtained, from a sign in the waiting room, which does not meet privacy law requirements, through to patients being refused appointments when they decline. The same report noted people may hold back on sensitive matters, including family violence, when they know AI is listening. Whatever a practice decides, someone has to write that process down, keep it current and make sure the front desk applies it consistently.

Eleven days before the ABC report, at the Australian Institute of Digital Health HIC2026 conference in Sydney on 3 August 2026, the Therapeutic Goods Administration said its review of digital scribes had moved into compliance action. Tracey Duffy, who heads the TGA’s product quality division, told the conference the regulator had spent twelve months engaging with vendors and was shifting to enforcement against organisations that had deployed a scribe operating as a medical device without seeking medical device approval.

Her findings were specific: scope creep, AI functions influencing the decisions of clinicians, a general lack of transparency about how products are managed once they are out in the field, and gaps in monitoring controls for foreseeable risks. She said action could go out over the following twelve months, and that the TGA would publish the review outcomes and updated guidance for developers.

Very little about the rules themselves is changing. The TGA’s broader review found the existing legislative framework fit for purpose, needing refinements rather than new laws. What has changed is that the line those rules draw is being enforced.

The line is narrow. Software that only listens and drafts sits outside the medical device framework. Software that starts shaping a clinical decision crosses into it. We covered where the line sits in our earlier piece on AI scribes and the future of documentation work. What has changed since is that the regulator has looked, found non-compliance, and said so publicly.

For anyone working in health information, the human review step is now the thing the regulator checks for. A busy practice can no longer quietly drop it.

AI now drafts the note. A qualified human checks it. That sentence is close to a job description for a healthcare documentation specialist, and it is a different job from the one the role used to be sold as.

Typing was never the whole of it, but typing is the part the software has taken. What remains needs judgement. A documentation specialist reads a draft against its source, knows what a document of that type has to contain, decides what belongs in a record and what does not, and knows how to raise and correct an error once a document has already been sent.

Where you sit matters as much as what you know. Someone in that role reads across a whole run of correspondence rather than sitting inside one consultation, so a contradiction between two letters is visible to them in a way it is not to the clinician who dictated either one. Someone reading only the note in front of them has no reason to notice that last month’s letter said the other side.

That is also the answer to whether the work is going away. Software can draft a document. It cannot hold professional accountability for what the document says, cannot be named when a record is queried, and cannot go back to the treating clinician to ask what was actually meant. Our earlier piece on medical editing and proofreading versus transcription sets out how that shift has already changed the day-to-day work.

“Check the AI output” sounds like proofreading, and treating it as proofreading is how errors survive. Proofreading asks whether the text is well formed. Checking a clinical document asks whether it is true, complete and internally consistent, which needs knowledge the text itself does not contain.

All of that is teachable. It is a body of knowledge applied methodically, which is what a qualification is for, rather than a talent for spotting mistakes.

The 11288NAT Diploma of Healthcare Documentation is built around producing, checking and safeguarding health information. Study covers the Australian health system and how records move through it, privacy and medico-legal obligations, advanced clinical terminology, producing and editing healthcare documents, AI in healthcare documentation, and professional practice. Graduates work in roles including medical records and health information officer positions, AI documentation review, documentation and transcription services, in-house roles in practices and hospitals, and independent contracting from home. Transcription sits inside that as one applied skill the course builds, rather than the whole of the job.

In the ABC report, the patient found the error only because she read the post-operative letter her specialist sent to her GP, and she was worried about what a false drug reference might do to a workers’ compensation claim. She said afterwards that she loves AI, but that the essential ingredient is the human, and that what happened came down to a lack of human involvement.

Three practical things follow from that.

Technology is going to keep improving, and it should. Scribes give clinicians their attention back, which is worth having. But a health record is a legal document, a clinical handover and a funding instrument at the same time, and someone has to read the draft against reality before it becomes all three.

Interested in the work of making a health record trustworthy? Explore the 11288NAT Diploma of Healthcare Documentation, delivered 100% online and self-paced with daily intakes, or read our healthcare documentation career guide.

TalentMed Pty Ltd, RTO 22151. Nationally recognised training delivered online across Australia. This article discusses publicly reported material about AI scribes in Australian healthcare, including ABC News reporting of 14 August 2026 and the Digital Rights Watch report, and is general information rather than clinical, legal or privacy advice.