A healthcare organization bought a chatbot.

Great demo. Slick interface. The vendor showed numbers that would make any board presentation sing.

Six months later, clinicians refused to use it. Nurses worked around it. The tool sat there, technically functional, practically useless.

Nobody had asked the people on the floor whether they'd actually use it.

I've watched variations of this story play out more times than I'd like to admit. The technology works. The implementation fails. And the failure almost always traces back to questions that were never asked during the buying process.

Here are the 5 questions I now treat as non-negotiable before any healthcare AI tool gets past initial evaluation.

Question 1: Where does my data live — and can you guarantee regional residency?

This is the question that catches the most vendors off guard.

When we evaluate AI solutions in Canada, regional data residency isn't optional. Patient data needs to stay within Canadian borders. Full stop. Not "we're working on a Canadian region." Not "our encryption makes it equivalent." The data needs to physically reside in-country.

I've seen promising tools get eliminated in the first conversation because the vendor couldn't guarantee Canadian data residency. Their product was excellent. Their infrastructure didn't meet the requirement.

What to listen for: A clear, immediate answer with specifics — which cloud region, which data centres, and whether processing (not just storage) happens within your jurisdiction.

Red flag: Any hesitation, any "let me get back to you," or any attempt to argue that encryption makes residency irrelevant.

Question 2: What happens when your model is wrong — and how do clinicians override it?

Every AI model makes mistakes. The question isn't whether it will be wrong. The question is what happens when it is.

In healthcare, a confident wrong answer isn't just an inconvenience. It can affect patient care. So you need to know: How fast can a clinician override the AI's recommendation? Is there a clear fallback workflow? Does the system log overrides so the model can learn?

What to listen for: A documented override process that takes seconds, not minutes. An error logging system that feeds back into model improvement. Clear documentation on known failure modes.

Red flag: "Our accuracy is 97%, so overrides are rarely needed." That's not an answer. That's a deflection. The 3% matters more than the 97% when it's your patient.

Question 3: Does this support FHIR standards — and how does it integrate with our existing EHR?

This is the question I've seen make the most vendors uncomfortable.

Interoperability isn't a nice-to-have. If an AI tool can't talk to your existing electronic health record system through standardized protocols, you're signing up for a custom integration project that will cost more and take longer than the tool itself.

I've learned to ask this question early because the answer tells you everything about a vendor's maturity. A vendor that has FHIR-native integration has done the hard work of understanding how healthcare actually operates. A vendor that says "we can build a custom API" is asking you to fund their product development.

What to listen for: Native FHIR R4 support. Demonstrated EHR integrations with your specific system (Epic, Cerner, MEDITECH — not just "any EHR"). A realistic timeline that isn't "4-6 weeks."

Red flag: "Custom API development required." That translates to: we've never actually integrated with a live health system at scale.

Question 4: Show me exactly how you handle anonymization — not just that you do it.

Every vendor says they anonymize data. Very few can show you the depth of how they do it.

Here's what I've learned: basic anonymization — removing names, swapping IDs — is table stakes. But patterns can still leak. A combination of age, postal code, admission date, and diagnosis can re-identify a patient in a small community hospital. A radiology report with specific measurements tied to a timestamp can be traced back.

The question isn't "do you anonymize?" It's "show me your de-identification methodology, and show me what residual re-identification risk remains."

What to listen for: A documented de-identification framework (ideally aligned with Safe Harbor or Expert Determination methods). A risk assessment for re-identification. A clear policy on what happens when their model is trained — does any patient-level data, even anonymized, leave your security perimeter?

Red flag: "We follow HIPAA/PIPEDA guidelines" as a complete answer. That's a compliance checkbox, not a methodology.

Question 5: Can I talk to a clinician who's been using this in production for 6+ months?

Not a case study PDF. Not a logo wall. Not a Chief Innovation Officer who approved the purchase. A clinician. Someone who uses this tool in their daily workflow, in a live clinical environment, and has been doing so long enough to have an honest opinion.

This is the question that separates real solutions from slide decks.

The chatbot story I opened with? The vendor had impressive case studies. They had logos. What they didn't have was a single nurse or physician who could say "yes, this actually changed how I work, and I'd be upset if you took it away."

Clinician adoption is the single biggest predictor of whether a healthcare AI deployment succeeds or fails. A tool can be technically brilliant. If the people on the floor don't use it — because it doesn't fit their workflow, because it adds 30 seconds they don't have, because nobody asked them what they needed — it's shelfware.

What to listen for: A vendor who can connect you with a frontline clinician within a week. Someone who talks about workflow integration, not just features. Honest feedback about what didn't work initially and how they iterated.

Red flag: "We can arrange a call with our Customer Success team." That's not what you asked for. You asked to talk to a clinician.

The Scoring Framework

Before your next evaluation, score every vendor on these 5 questions:

Results to look out for :-

The hard truth: most healthcare AI purchases fail not because the technology is bad, but because nobody asked the right questions early enough. These five questions would have saved every failed deployment I've seen.

Ask them. Demand clear answers. And if a vendor can't answer all five — they're selling you a future, not a solution.

— Guryash

P.S. What questions would you add to this list? Reply and tell me — I'm building a community evaluation guide and your experience matters.