A few days ago, I was discussing with a doctor who had built a telemedicine platform, and I asked a question that seemed simple at first: if a patient has a virtual consultation and something goes wrong afterward, who’s responsible?
Her answer came fast: the doctor. The consultation happened over a screen instead of in a clinic, but clinical responsibility didn’t change.
That got me thinking about a harder version of the same question: what happens when AI is the one making the call?
Is it the founder who built the product? The developers behind the AI? The company that built the underlying model? Or the hospital or clinic using it?
Unlike telemedicine, where accountability is fairly clear, health AI adds layers. And as AI takes on a bigger role in care, founders can’t afford to leave this question unanswered.
We’ve already seen why this matters
One of the clearest examples is Babylon Health, once one of the world’s leading digital health companies. Its chatbot asked patients about their symptoms and told them what level of care to seek.
Over time, clinicians and patient safety experts raised concerns about the chatbot missing symptoms of a heart attack and a blood clot. In one reported case, it correctly flagged heart attack symptoms in a male patient, then labeled the same symptoms a panic attack when tested on a female patient.
The debate stopped being about whether AI belongs in healthcare. It became about whether there was enough clinical oversight behind it.
The lesson isn’t “never use AI in healthcare.” It’s simpler: AI can be wrong, and when it is, someone has to answer for it.
Why disclaimers aren’t enough
Most health AI products carry a familiar line: “This tool is not a substitute for professional medical advice.”
Legally, that sentence matters. Clinically, it barely moves the needle.
Picture a woman in early pregnancy with mild, one-sided abdominal pain for a couple of days. She checks a symptom app, which tells her some cramping is normal early on and to rest and monitor it. A week later, she’s in the ER with a ruptured ectopic pregnancy.
From her side, she didn’t ignore a disclaimer. She followed advice from a product she trusted about something that felt too minor to question. Whether the company meant to give medical advice or not, the product still shaped a real healthcare decision. That’s why accountability can’t stop at the terms and conditions.
So who’s actually responsible?
Honestly, there’s no single answer. Responsibility is shared, just not equally.
The AI itself is never responsible. It generates outputs based on the data, models, and instructions provided by people. How those outputs are used in healthcare is a human responsibility.
The company behind the underlying model is responsible for that model’s capabilities and limits, not for the clinical claims you build on top of it or how you use it.
That part is on the founder. If your AI assesses symptoms, triages patients, or tells someone what to do next, that’s a clinical product decision. You’re responsible for making sure it’s evidence-based, properly validated, and clinically overseen.
If clinicians help develop or approve the clinical content, they carry professional responsibility for that piece too, the same as they would developing any clinical guideline.
And healthcare organizations that deploy the tool have their own responsibility: training staff, making sure they understand its limits, and using it appropriately in real workflows.
Accountability doesn’t live in one place. It follows the decisions each party made while building, approving, and deploying the system.
Accountability is designed long before launch
Clinical input isn’t just a doctor reviewing content before launch. Clinicians should be asking: What happens when the AI is uncertain? When should it stop and send someone to emergency care? Who reviews errors after launch, and how does new evidence get folded back in?
These are governance questions; they decide whether a product stays safe once real people use it.
Because most conversations about health AI stop at capability: can it diagnose accurately? Can it triage well? But there’s an equally important question: if the AI gets something wrong, what have we built around that mistake?
Clinical credibility isn’t about slowing innovation down but about keeping it safe once it reaches the people it was built for.
Better Woman Health is published weekly.
Subscribe at betterwomanhealth.com to get it directly in your inbox.
Clinical Credibility Toolkit
The Clinical Credibility Toolkit is a free resource designed for FemTech founders building digital tools for women’s health.
It includes the Clinical Credibility Scorecard, a 28-question assessment that helps you identify gaps in your clinical foundation, evidence strategy, safety architecture, and investor readiness, and the Red Flag Detection Checklist, a 25-question assessment that tells you whether your symptom-tracking app can actually detect and act on medical red flags, not just log them.
If you are preparing to pitch, pursuing healthcare partnerships, or simply want to know where your product stands clinically, start here.
Access the toolkit → app.ayomide.me
Thanks for reading. See you soon!
Dr. Ayomide O.
Clinical Strategist & African Market Advisor
Find me on LinkedIn or Book a 1:1 Call
