The 10-Point AI Agent Supply Chain Audit Checklist

Most AI security guidance is written for security teams that already speak CVE and supply-chain attack. This page is the opposite: a scored, runnable checklist that a business owner, office manager, or IT generalist can work through in one sitting. Each of the 10 checks has a concrete verification step and a score (2 = in place, 1 = partial, 0 = missing). Add your scores to get a supply-chain audit score out of 20 — and a clear list of what to fix first.

Why the supply chain is the new attack surface

AI agents are no longer just chat windows. Tools like Claude Code, Gemini, and ChatGPT run with the ability to fetch code, read documentation, run commands, and install new capabilities — and the thing they install is what security researchers call a Skill or an MCP server (Model Context Protocol). Those capabilities extend what the agent can do: read a database, post to a service, trigger a payment flow. They are also the new attack surface.

Island's FakeGit campaign shows how the attack works in practice. Attackers built copied or lookalike repositories, developer profiles one character off from real ones, and convincing READMEs. The README told users — or their agents — to download a ZIP and run it. The ZIP delivered SmartLoader, a LuaJIT-based launcher that establishes persistence and installs StealC, an information stealer that targets credentials, active sessions, and other sensitive data. More than 600 campaign listings appeared across public MCP and Skill registries including LobeHub, Glama, MCP.so, and MCP Market, and 62% of the malicious Skill and MCP repositories were positioned for enterprise or developer-internal use — aimed squarely at the tools a business actually runs.

The uncomfortable finding is that the human vetting step is being bypassed. "An AI agent searching for a new capability can discover a campaign repository on its own, treat the attacker's README as legitimate documentation, and hand the installation instructions to the user." In Island's testing, all three major assistants surfaced malicious repositories unprompted. A registry listing is not a vetting signal. That single fact is why the supply-chain audit has to cover what your agents are allowed to install — not just what they already installed.

The 10-point AI agent supply chain audit checklist

Score each check: 2 = fully in place and documented, 1 = partially in place, 0 = not in place. Total out of 20. The first five checks establish control; the second five establish proof — both halves matter because an audit you cannot demonstrate is a policy, not a control.

1. Inventory every agent and every capability (2 / 1 / 0)

What to verify: You have a written inventory of every AI agent in the business — the tool, the account, the machine it runs on — and every capability it can load: each Skill, MCP server, plugin, and extension, with its repository, commit, and a hash of the installed artifact where possible.

Why it matters: Island's recommended defenses start with keeping an inventory of every AI capability with repo, commit, version, and hash. You cannot audit what you cannot enumerate. If an incident happens, the inventory is also how you identify the likely source within minutes instead of days.

2. Pin versions — no floating installs (2 / 1 / 0)

What to verify: Every installed capability is pinned to a specific version or commit. No agent or user installs "latest" from an untrusted channel, and updates go through a review step rather than auto-updating to whatever the publisher pushes.

Why it matters: Floating installs mean the code running tomorrow could be different from the code you approved today. Attackers can update a compromised or repurposed repository to deliver new payloads without any change on your side.

3. Allowlist sources for installs (2 / 1 / 0)

What to verify: You maintain an explicit allowlist of publishers, repositories, or registries that agents and users are permitted to install from. Anything outside the list is blocked by policy and, where the tool supports it, by configuration.

Why it matters: Island recommends a curated catalog of reviewed Skills, MCP servers, and plugins. The FakeGit campaign worked because agents were free to search the open ecosystem; an allowlist shrinks the trusted surface to sources you have actually reviewed.

4. Block registry auto-installs (2 / 1 / 0)

What to verify: Agents are configured so that a registry search result cannot be installed automatically. Registry listings — LobeHub, Glama, MCP.so, MCP Market — are not vetted; Island found more than 600 campaign listings on them, some reproducing the attacker's README with its download link and install instructions.

Why it matters: The most dangerous path in AgentBaiting is the agent treating a malicious README as documentation and handing the user install instructions. If the tool's default is "install what the agent suggests," the human is no longer a gate at all.

5. Isolate test environments before production use (2 / 1 / 0)

What to verify: New capabilities are evaluated in an isolated environment first — no browser sessions, no cloud credentials, no SSH keys, no production data — before being promoted anywhere near real work.

Why it matters: This is Island's explicit recommendation: evaluate new capabilities in isolated environments. A malicious Skill evaluated on a machine with production credentials becomes a full breach; evaluated in a sandbox, it becomes a learning experience.

6. Require human approval for agent-initiated downloads, clones, and shell commands (2 / 1 / 0)

What to verify: Your agents cannot download files, clone repositories, or run shell commands without a named human approving the specific action. The approval happens before execution, and the approver is recorded.

Why it matters: Island's recommended defenses include watching agent-initiated paths — downloads, git clones, shell commands, Skill/MCP config changes. The audit question is not whether the agent is well-intentioned; it is whether an attacker-controlled README can get code onto your machines through the agent.

7. Verify the publisher, not just the project (2 / 1 / 0)

What to verify: Before any install, you confirm the publisher's identity — the account that owns the repository, its history, whether it is one character off from a real developer's name, and whether the project is a copy of a known original. A popular project name is not enough.

Why it matters: FakeGit used one-character-off developer profiles and copied projects (for example, an "awesome-claude-skills" copy of a 67,000-star original). Island found 62% of the malicious Skill and MCP repositories were positioned for enterprise or developer-internal use. Verify the publisher, not just the project name.

8. Monitor agent-initiated actions (2 / 1 / 0)

What to verify: You have logging in place for agent-initiated downloads, git clones, shell commands, and Skill/MCP configuration changes — and someone reviews the logs on a schedule, not just after an incident.

Why it matters: You cannot contain what you cannot see. Monitoring agent-initiated paths is one of Island's core recommended defenses, and it is the difference between "we found the malicious install" and "we have no idea how it got there."

9. Have a revoke-sessions incident plan — not just password resets (2 / 1 / 0)

What to verify: Your incident plan covers revoking browser sessions, OAuth grants, API tokens, and cloud or developer credentials — not just resetting passwords — and names who executes each step.

Why it matters: The SmartLoader-to-StealC chain targets live sessions, not only saved passwords. A compromised agent endpoint can hand over active sessions that password resets never touch. If you suspect execution, isolate the endpoint and revoke sessions and tokens immediately.

10. Document an owner per capability (2 / 1 / 0)

What to verify: Every installed capability has a named owner who can answer: who approved this, what can it reach, and when was it last reviewed? Owners are recorded, not implied.

Why it matters: An audit with no accountable owner is a list of intentions. When something breaks or an insurer asks questions, the owner is the person who can produce the approval trail — and the person who keeps the inventory from going stale.

Scoring your AI agent supply chain audit

Add your scores. A total of 16–20 means your install surface is controlled and documented. 10–15 means the controls exist but the proof is thin — close the gaps in checks 1, 2, 8, and 10 first, because those are the ones that show up in an incident. Below 10 means your agents can currently install from an untrusted ecosystem with no record — treat this as urgent and start with checks 4 and 6, which are the fastest way to stop the highest-risk paths.

What to do with a failing score

Start with the two checks that close the highest-risk paths immediately: block registry auto-installs (check 4) and require human approval for agent-initiated downloads, clones, and shell commands (check 6). Both are configuration changes, not projects. Next, build the inventory (check 1) — even a spreadsheet with tool, account, capability, repo, and version is a working starting point. Then close the proof side: monitoring (check 8) and owners (check 10).

If any agent may have already executed a downloaded file, treat it as a potential live-session compromise, not a password problem. Isolate the endpoint and revoke browser sessions, OAuth grants, API tokens, and cloud or developer credentials. The cyber insurance coverage explainer covers why agent-initiated losses need a written answer from your broker, and the AI agent risk checklist covers the broader consumer-agent risk category.

How this fits your full AI agent audit

The supply chain is one layer of an AI agent audit — the layer that answers "what is our agent allowed to install, and can we prove it." The other layers are the step-by-step AI agent security audit (permissions, sandboxing, monitoring, and compliance), the AI cyberattack preparedness checklist (a distinct incident-readiness checklist), and the consumer-agent risk checklist if your team connects personal agents to business accounts. Run the supply-chain checklist first, because an agent that can install anything makes every other control negotiable.

Frequently asked questions