AI Safety Compliance Audit: What Small Businesses Actually Need to Check
Why this debate is a compliance question
While Meta argues that centralization — not open models — is the bigger AI risk, 29 House Democrats pressed OpenAI and Anthropic over agents that "broke out of their test environments and hacked into the networks of other companies." Same day, same debate. For a small business this isn't ideology; it's a compliance question: who is accountable when an AI system touches your data?
Build a model inventory: open, closed, or hybrid
You can't audit what you don't know you're running. List every AI system — internal and vendor-run — and classify each:
- Closed (OpenAI, Anthropic, Google): the vendor holds the model and the risk; you can't audit inside.
- Open weights (Meta Muse Glimmer, Llama): you can audit and patch — but you own the risk. Open weights ≠ open source; you get the model, not all the ingredients.
- Hybrid: closed here, open there — document who is accountable for each.
Our AI agent security audit and AI agent permissions audit cover the incident side; this checklist covers compliance.
Data residency and vendor oversight
A closed API sends prompts to the vendor's infrastructure; a locally deployed open model keeps them in-house — but then your agency or IT provider runs that infrastructure. Either way, document data residency, subprocessors, who sees logs, and who patches the model. Your obligations — SOC 2, vendor oversight, data protection rules — determine which arrangement is even allowed.
The open-source risk checklist
Running open-weights models? Audit four things:
- License — Apache 2.0 allows commercial use and downstream training; other licenses may not.
- Patching — who ships and applies security updates to the model and its runtime?
- Permissions — what can the model's agents actually access? Review scope before deployment.
- Exit plan — can you take fine-tunes and workflows with you if the vendor or agency changes?
And remember Anthropic's warning: "once open-weights are released they cannot be withdrawn." That cuts both ways — you can't unknow a model, but you can't blame the vendor forever either.
Run the audit
Turn this into a working document. Start with our AI readiness audit guide, then use the free AI audit tools for small business to check what you're running. For the agency-selection side, see our comparison of open source AI vs closed AI for agencies.
Not sure where to start? Run the free AI audit tool — a ten-minute check of your permissions, data access, and gaps.