Can You Use ChatGPT (or Any LLM) With PHI? HIPAA Rules for Healthcare AI
Not with the consumer ChatGPT app — it has no Business Associate Agreement, so entering protected health information into it violates HIPAA. You can use an LLM with PHI when you sign a BAA with the provider and configure the deployment correctly: training disabled, access controlled, activity logged. The product tier and the architecture decide compliance, not the model.
Is ChatGPT HIPAA compliant?
No. The consumer ChatGPT product — Free, Plus, and Pro — is not HIPAA compliant, because OpenAI does not offer a Business Associate Agreement for it. Without a BAA, any PHI you type into the box is disclosed to a vendor that has no HIPAA obligation to protect it, which is an impermissible disclosure regardless of how the data is encrypted in transit. The compliant paths to the same underlying models exist, but they are different products: the OpenAI API and ChatGPT Enterprise, both of which can be covered by a BAA.
A Business Associate Agreement (BAA) is the HIPAA-required contract under which a vendor that creates, receives, or processes protected health information on your behalf legally commits to safeguard it. Without one in place, sharing PHI with that vendor is an impermissible disclosure on its own — independent of the technology.
Does a BAA make an LLM HIPAA compliant?
A BAA is necessary but not sufficient. It establishes the legal obligation; it does not configure the system. A signed BAA paired with a deployment that still logs prompts to a training pipeline, or that lets every user query every patient's data, is not compliant — it is a contract sitting on top of a non-compliant architecture. Compliance is the BAA plus the configuration: training and retention disabled, access scoped to minimum necessary, audit logging on, and a clean data-flow design. Treat the BAA as the entry ticket, not the finish line.
Which LLM platforms will sign a BAA?
Most enterprise and cloud AI platforms will; consumer apps will not. The table below summarizes the common options. BAA availability and the list of covered services are tier- and configuration-dependent, so confirm the specific service is in scope before any PHI flows to it.
| Platform | BAA available? | Trains on your data by default? | Notes |
|---|---|---|---|
| Consumer ChatGPT (Free / Plus / Pro) | No | Historically yes, unless you disable it | Never appropriate for PHI |
| OpenAI API / ChatGPT Enterprise | Yes, for eligible accounts | No — API and enterprise data is not used for training | BAA plus zero-retention options on request |
| Azure OpenAI Service | Yes, under the Microsoft BAA for in-scope services | No — your data stays in your tenant | Common enterprise healthcare path |
| AWS Bedrock | Yes — HIPAA-eligible under the AWS BAA | No — not used to train base models | Model-agnostic; data stays in your AWS account |
| Google Vertex AI | Yes, under the Google Cloud BAA | No | GCP covered-service list applies |
| Anthropic API (Claude) | Yes, for commercial customers; also via Bedrock and Vertex | No — API inputs and outputs are not used for training | Available directly and through cloud platforms |
The pattern is consistent: the same model family is non-compliant in the consumer app and compliant through the API or cloud platform, because only the latter come with a BAA and an enterprise data-handling posture. The decision is rarely which model — it is which product tier and how it is wired up.
What do you need besides a BAA?
The BAA covers the legal relationship. The deployment still has to enforce HIPAA's Security and Privacy Rules in practice. Five controls do most of the work:
- Training and retention disabled — confirm in writing that prompts and outputs are excluded from model training, and use zero-data-retention where the provider offers it.
- Access controls and minimum necessary — role-based access so a user reaches only the PHI their job requires, not the whole corpus by default.
- Audit logging — a record of who accessed what and when, retained and reviewable, because the Security Rule expects it and breach investigations require it.
- De-identification where the use case allows — if the task does not truly need identifiers, strip them; de-identified data is no longer PHI and widens your tool options.
- BAAs all the way down — every subprocessor in the chain that touches PHI needs to be under a BAA too, not just the front-door vendor.
What happens if you use PHI without a BAA?
It is an impermissible disclosure under the HIPAA Privacy Rule, and it can trigger breach-notification obligations to patients, HHS, and in some cases the media. Enforcement runs through the HHS Office for Civil Rights, with tiered civil monetary penalties that scale with culpability — from roughly $100 per violation at the low end to more than $50,000 per violation at the highest tier, subject to annual caps — on top of the reputational damage and lost trust. The "conduit exception" that exempts pure transmission services does not cover an LLM vendor, because the vendor processes the data rather than merely passing it through.
Where teams actually get this wrong
In the healthcare billing environments I have worked in, the BAA is rarely the thing that fails — it is the configuration around it. Two gaps recur. The first is a default setting that quietly leaves prompt data eligible for retention or training, so the contract says one thing and the deployment does another. The second is access that is all-or-nothing because nobody scoped roles to minimum necessary, which turns a single compromised login into a whole-database exposure. Both are cheap to fix at design time and expensive to discover during a breach review. The right move is to design for the BAA and the controls from the first architecture diagram, not to retrofit compliance onto a prototype that already works.
Frequently asked questions
Is the consumer ChatGPT app ever OK for PHI?
No. Without a BAA, entering PHI into the Free, Plus, or Pro app is an impermissible disclosure under HIPAA, even for a quick one-off question. Use an API or enterprise tier with a signed BAA and retention disabled, or de-identify the data before it ever reaches the tool.
Does signing a BAA alone make us HIPAA compliant?
No. A BAA creates the legal obligation but configures nothing. You still need training and retention disabled, role-based access, audit logging, minimum-necessary data flows, and BAAs with every subprocessor that touches PHI. Compliance is the contract and the architecture together, not either one alone.
Is de-identified data still PHI?
No. Data de-identified under HIPAA's Safe Harbor method (18 identifiers removed) or Expert Determination is no longer PHI and can be used with tools that lack a BAA. Re-identification risk and contractual limits still apply, so the de-identification method should be validated rather than assumed.
Can we use the OpenAI or Anthropic API directly with PHI?
Yes, with a signed BAA and the right configuration. Both providers offer BAAs for qualifying API customers, and API inputs and outputs are not used to train their models. You still own the access controls, audit logging, and minimum-necessary design on your side of the integration.
Is pasting PHI into a prompt a HIPAA violation?
It depends on the destination. Pasted into a tool with no BAA, such as consumer ChatGPT, yes — it is a disclosure to an unbound vendor. Pasted into a BAA-covered, properly configured deployment, no — that is a permitted disclosure to a business associate acting on your behalf.
Planning an LLM deployment on regulated data?
We design HIPAA-aware LLM workflows from the first diagram — BAA coverage, no-training configuration, access controls, and audit logging — and ship them to production. Bring the workflow; leave with an architecture and an estimate the same business day.
Book a 30-minute intro call Prefer email? clayton@quantsolvent.co