An ongoing dialogue on HIV/AIDS, infectious diseases,
November 18th, 2025
When AI Gets the Medical Advice Wrong — and Right
National Foundation for Infectious Diseases.
A journalist recently reached out to ask about the shingles vaccine. We mostly talked through the usual topics — how common shingles is, why the vaccine works so well, and side effects. Plus, the whole topic of zoster vaccination has been much in the news recently given studies associating receipt of the vaccine with a reduced risk of dementia.
Before we wrapped up, I suggested that hearing from someone who’d had a tough time with shingles might strengthen the piece. A colleague of mine, Lisa Junker (who is Senior Director of Publishing at the Infectious Diseases Society of America), had a serious case involving her ear several years ago.
It was a classic case of herpes zoster oticus, with severe pain, vertigo, and hearing loss. What she memorably said about it when she came back to work: “Shingles: Do not recommend!” She graciously agreed to speak with the reporter for her WebMD piece.
The article came out, and it’s excellent. I noticed, however, that Lisa had received some wrong counseling from the ENT who cared for her:
Her ENT warned that if shingles returns, it will likely strike the same spot.
This misconception is surprisingly common. It’s right up there with the patient who’s told that waxing and waning symptoms in the same dermatome after an episode represents “recurrent zoster.” It almost never is — which is why this consult is one of the more common outpatient questions we ID doctors receive, and why I wrote about it several years ago. (It’s rarely covered clearly in textbooks or journal articles.)
In short, herpes simplex frequently recurs in the same location, but herpes zoster does not. Recurrent zoster usually (but not always) recurs in a different dermatome altogether, usually several years after the first episode, if it recurs at all — one study found that roughly 10% of people had another episode within 10 years, though this may be an overestimate since there are lots of cases of herpes simplex (and other things) misdiagnosed as herpes zoster.
That’s one of the things about this unpleasant disease. It’s unpredictable. And the ENT’s confident statement that it’s going to come back in the same “spot” — delivered as clinical fact — simply wasn’t correct. I let the reporter know, and she promptly removed the sentence.
And then curiosity got the better of me. Motivated by this New York Times piece that highlighted that patients used AI to get medical advice, I plugged this prompt into two AI tools, OpenEvidence and ChatGPT:
When people get shingles again, is it usually in the same location?
I deliberately left off medical terms such as “dermatome”, “reactivation”, or “recurrent.”
Here’s what happened: OpenEvidence hit the ball out of the park:
Recurrent shingles (herpes zoster) typically does not occur in the same location; it most often affects a different dermatome during subsequent episodes.
With unlimited queries limited to healthcare professionals and medical students, OpenEvidence is currently the best AI tool for medical information. I’ve written about it elsewhere as the “Google for Doctors — and nurses and PAs and anyone with an NPI.” Not only did it provide the correct answer, it offered several highly relevant citations. (Disclosure: NEJM Group, including NEJM Journal Watch, is among the publishers providing content to OpenEvidence.)
ChatGPT? Let’s stick with the same metaphor (can you tell I miss baseball?): it was more Charlie Brown than Aaron Judge, a big whiff:
When someone has more than one episode of shingles, the new episode most commonly appears in the same dermatome as the prior one. This fits the biology: the varicella-zoster virus remains latent in a specific dorsal root (or cranial nerve) ganglion, and reactivates from that same reservoir.
Not just wrong, but wrong with a pathophysiologic justification! Brilliant. This confident but incorrect answer echoed the ENT’s advice and underlies the very misconception that leads to clinical referrals asking about “recurrent” zoster.
When I corrected the model, it “thought” for 35 seconds, then acknowledged the error (“You’re right to push back on that”) and gave the correct answer. It even cited my 2014 blog post! So proud.
But the comparison was a useful reminder of how these systems work: highly fluent, highly confident, occasionally wrong.
And so, in the span of a single afternoon, the same misconception surfaced from three different sources: a clinician, a widely read health article citing that clinician, and an AI model. What can we take from this?
- Authoritative tone ≠ accuracy. Clinical confidence and AI assurance can all sound convincing, even when the underlying fact is off.
- AI varies depending on design. Tools that constrain themselves to curated evidence bases (like OpenEvidence) will give more reliable medical answers at baseline; generative models need supervision.
- Clinician expertise still matters. We’re the interpretive safety net, especially for edge cases, clinical nuance, and information not readily available through standard searches.
In the end, the article was corrected, Lisa is now happily vaccinated, and the real clinical pearl remains the same: If shingles returns, it rarely looks like the first episode.
And it’s definitely worth preventing — both for the first time, and for a recurrence.
9 Responses to “When AI Gets the Medical Advice Wrong — and Right”
Leave a Reply
FROM NEJM — Recent Infectious Disease Articles- Stability of Avian Influenza A(H5N1) Virus in Milk from Infected Cows and Virus-Spiked Milk December 4, 2025Highly pathogenic avian influenza A(H5N1) virus has spread among dairy cattle, with virus detected in milk samples. In this report, virus viability after cold storage or pasteurization is assessed.
- Updated Evidence for Covid-19, RSV, and Influenza Vaccines for 2025–2026 December 4, 2025Results from recently published, peer-reviewed studies support the safety and effectiveness of immunizations against Covid-19, RSV, and influenza.
- Glucocorticoids for Pneumonia in Africa — Old Therapy, New Context December 4, 2025The African continent is home to 1.55 billion people, one fifth of the world’s population, spread over an area larger than the United States, China, and India combined. The disease burden is enormous,1 and health care resources are severely constrained. In one multinational survey, one in eight hospitalized...
- The New U.S. Global Health Strategy — A Reset of America’s Health Cooperation December 4, 2025The U.S. administration recently unveiled its new approach to global health. A retreat from multilateralism, a geopolitical emphasis, and rushed transitions could threaten progress.
- Empyema Necessitans December 4, 2025A 66-year-old man presented with 2 weeks of shortness of breath and cough, 5 days of left flank pain, and 2 days of a rapidly expanding mass on his left side.
- Stability of Avian Influenza A(H5N1) Virus in Milk from Infected Cows and Virus-Spiked Milk December 4, 2025
Conan once described haning shingles as “feeling like someone was prying my eyeball out with a rusty spoon.”
I got vaccinated shortly afterward and strongly encouraged everyone I knew to do the same.
Brilliant!
I was surprised to learn that zoster does not often recur in the same area.I had it in my right flank around age 31,around to the lower right abdomen,a real nagging pain for a few weeks.But I hsve had 3 recurrences,same area each time,but mild.As a retired Internist,I was puzzled each time,but kept looking ,and my wife confirmed a small vesicle was visible each time.decades apart but enough to get s CT twice,as I have a kidney stone on that side.I think zoster is hiding out in 1 nerve root..so,it can happen that way.I had 2 patients feel they had 3 episodes,but don’t know they were correct
The key test on that “small vesicle” would be a VZV PCR. If negative, it could be one of the non-specific dermatologic reactions that occur in a previously involved dermatome, and not viral reactivation.
More here.
–Paul
As I was reading this post, I wondered something I never had before. Why is this disease called “shingles”? It certainly doesn’t look like anything on my roof.
ChatGPT says it comes from the Latin “cingulum” meaning “girdle” or “belt.” Makes sense!
In French, it is called “zona” and in Spanish it is “culebrilla”–which I assumed meant serpent, but I just checked–culebra means snake–so it must mean something like akin to snake which is much more evocative.
Agree about shingles–nothing like roof shingles!
I love your post–apolitical but fundamentally a comment on the new truth. Uncritical truths that were patently false are not unique to the AI era. Wasn’t the earth the center of the universe? Gallileo ws nearly burned by the Catholic Church as a heretic. AI masquerades as an arbiter of truth and public is seduced into adherence to a new religion. The effort of thinking makes the lure of easy truth so attractive. Great scientific breakthroughs were epiphany driven and proven through experiments–Newton’s laws of gravitation, Einstein’s relativity. Maybe we in the hallowed medical profession have ourselves to blame for foisting our belief in the randomized controlled trials as the arbiter of truth–sort of like AI culling some pattern out of whatever data it is fed. Back to COVID trauma–>90% of “high risk” folks with covid did just fine without paxlovid in the RCT–but still, paxlovid got blanket approval even for though the majority did not benefit–yet it is “the treatment” for “high risk”, right?
Why is OpenEvidence restricted to healthcare providers? I am a retired scientist and writer, and I’m disappointed that I can’t have access. It sounds really useful.
I had very different responses from a variety of LLMs than yours and they ALL say shingles, if it returns, usually does so in different places.
Claude: Shingles recurrence doesn’t consistently happen in the same location. While it can reoccur in the same dermatome (the area of skin served by a specific nerve), it can just as easily appear in a completely different part of the body.
This happens because the varicella-zoster virus lies dormant in nerve roots throughout your spine and can reactivate in any of those locations. Each time the virus reactivates, it’s somewhat random which nerve root is affected.
That said, recurrent shingles is relatively uncommon overall
Gemini: No, when shingles recurs, it is more likely to appear in a different location (a different dermatome) than the original outbreak.
Here are a few key points:
Different Location is More Common: The virus that causes shingles (varicella-zoster virus) remains dormant in nerve cells throughout the body, not just in one place. Therefore, when it reactivates, it often travels down a different nerve path, resulting in a rash on a new area of the body.
A Small Percentage Recur in the Same Spot: One study found that only about 16% of people who experienced a recurrence of shingles had it in the exact same dermatome (area of skin supplied by a single nerve) as before.
Exception: Shingles near the Eye:
ChatGPT: Usually no. Shingles tends to return in a different nerve distribution, though the body behaves like a stubborn oracle that sometimes repeats itself.
Most recurrences stick to one rule: the virus reactivates along a sensory nerve root, and it often chooses a new one the next time. About a quarter of recurrences do show up in the same spot, but the majority wander.