HIPAA and AI APIs: What Zero-Data-Retention Actually Buys You

Healthcare builders keep asking the same question: can I use a third-party AI API for PHI without a full HIPAA infrastructure build-out? The honest answer is nuanced — but zero-data-retention changes the conversation significantly. Here's the practical view.

The BAA is the starting point, not the finish line

Under HIPAA, any vendor handling protected health information (PHI) on your behalf is a business associate and needs a signed Business Associate Agreement (BAA). That's table stakes. But a BAA is a legal document — it doesn't change what the vendor technically does with your data. You need both the contract and the architecture to line up.

What zero-retention actually changes

Most AI providers retain prompts for 30 days or more. If PHI sits on their disk, that's a data store you now have to account for in a breach scenario. A zero-retention backend changes the risk profile:

• No persistent PHI store — nothing to breach, nothing to exfiltrate.
• No training on your data — the biggest silent risk with consumer-tier APIs.
• Shorter retention argument — data that never persists is dramatically easier to defend in an audit.

Zero-retention doesn't automatically make an API HIPAA-compliant — you still need the BAA and a proper security posture — but it removes the single biggest reason most AI APIs can't clear the bar at all.

How to evaluate a provider for regulated workloads

• Will they sign a BAA? If not, stop. That's the gate.
• Is retention zero, in writing, technically enforced? Ask about the persistence layer, not the marketing page.
• Is there a no-training commitment on your tier? This is separate from logging and equally important.
• Where is the data processed? Residency matters for healthcare even beyond HIPAA's letter.
• Is the endpoint a true drop-in? If adopting the compliant backend means re-architecting, the project may die before it ships.

The bottom line

HIPAA compliance for an AI API is a stack: BAA + zero-retention architecture + clear residency + no-training. Zero-retention is the part that makes the rest of it achievable at all. If a provider can't demonstrate that their inference path has no persistence layer, they can't handle your PHI — no matter what their pricing page says.