HIPAA and AI APIs: What Zero-Data-Retention Actually Buys You

Healthcare builders keep asking the same question: can I use a third-party AI API for PHI without a full HIPAA infrastructure build-out? The honest answer is nuanced — but zero-data-retention changes the conversation significantly. Here's the practical view.

The BAA is the starting point, not the finish line

Under HIPAA, any vendor handling protected health information (PHI) on your behalf is a business associate and needs a signed Business Associate Agreement (BAA). That's table stakes. But a BAA is a legal document — it doesn't change what the vendor technically does with your data. You need both the contract and the architecture to line up.

What zero-retention actually changes

Some AI providers retain prompts for defined periods. If PHI sits on their disk, that's a data store you now have to account for in a breach scenario. A gateway that does not retain prompts changes part of the risk profile, but upstream processing terms remain relevant:

• No persistent PHI store at the gateway — confirm the equivalent upstream controls before treating this as an end-to-end outcome.
• No training terms — confirm them with every processor handling the request.
• A narrower gateway retention argument — useful evidence, but not a substitute for an audit of the full path.

A gateway that does not retain prompts doesn't automatically make an API HIPAA-compliant — you still need the BAA, a proper security posture, and verified upstream controls.

How to evaluate a provider for regulated workloads

• Will they sign a BAA? If not, stop. That's the gate.
• What are the gateway and upstream retention terms, in writing? Ask about the persistence layer, not the marketing page.
• Is there a no-training commitment on your tier? This is separate from logging and equally important.
• Where is the data processed? Residency matters for healthcare even beyond HIPAA's letter.
• Is the endpoint a true drop-in? If adopting the compliant backend means re-architecting, the project may die before it ships.

The bottom line

HIPAA compliance for an AI API is a stack: BAA + documented retention controls + clear residency + no-training terms. Verify each processor in the inference path before sending PHI — pricing claims are not compliance evidence.