Using AI with patient data: a practical checklist for agencies
The fastest way to turn an AI project into a compliance problem is to paste patient information into a tool nobody vetted. The good news is that using AI safely with protected health information (PHI) is very achievable. It just takes discipline up front.
Here's the checklist I use before any AI tool touches patient data. It's practical guidance, not legal advice, so involve your compliance officer and counsel.
Before you start
- Is there a Business Associate Agreement? Any vendor that processes PHI on your behalf needs a BAA. Many cloud and AI providers offer BAAs, but often only for specific services and configurations. Confirm the exact service you'll use is covered.
- Where does the data go? Know which region it's processed in, whether it's stored, and for how long.
- Is it used for training? Get it in writing that your data won't be used to train the vendor's models.
- Who on staff is using consumer tools? Free chat tools are rarely covered by a BAA. Set a clear policy and give staff an approved alternative.
When you design the workflow
- Minimum necessary. Send the agent only the fields it needs. An eligibility check needs member ID and date of birth, not the visit notes.
- Keep it in your tenant. Where possible, run AI services inside your own cloud subscription, behind your own identity and network controls.
- Put a person in the loop. Decide which outputs need human approval. Anything clinical, coding-related or billing-related should.
- Log everything. Record what went in, what came out, and who approved it. You'll need this for audits and for improving the system.
Once it's running
- Measure accuracy. Spot-check outputs regularly against what a person would have done.
- Review access. Include AI service accounts and keys in your regular access reviews.
- Have an off switch. Every agent should be easy to pause without breaking the underlying workflow.