Your AI Agent Said No. Your API Key Said Yes.
New research showed six agent frameworks running a tool call the model had just refused. A refusal is the model's opinion. Authorization is what your systems allow. Here's what a Series A team should put in front of every tool call before an enterprise buyer asks, with a 60-second Allow-button test and a gate builder that writes your policy and your questionnaire answer.
The short answer
A model’s refusal is an opinion about what it should say. Authorization is a decision about what your systems will do. They live in different places, and only one of them is yours.
So put the decision where you can see it, test it and show it to a buyer: a gate that sits outside the model, in front of every tool call your agent makes. It has seven parts.
- Decide which tools exist. An allowlist of servers, extensions and skills, pinned to versions someone reviewed.
- Give every agent its own identity, and a short-lived credential scoped to the one job it does.
- Check every call against a written policy, arguments included, and deny anything the policy doesn’t allow.
- Ask a named human only for the handful of calls you can’t take back, and show them exactly what they’re approving.
- Cap how much and how fast, and make deletes undoable.
- Log every decision somewhere the agent can’t edit.
- Test the gate like an attacker would, on a calendar.
That list is also, almost line for line, what an enterprise buyer’s security team now asks a Series A company to describe. Build the gate once and you have the answer, written down, with evidence behind it. Below are the scenes from this year that made the case, the numbers, the settings your vendors already ship, a 60-second test of your own Allow-button reflexes, and a builder that turns your agent’s tools into a policy plus a paragraph your buyer can file.
Repeat after me
On Monday, October 12, CSO Online’s Gyana Swain wrote up a paper from researchers at Pennsylvania State University, Université Montpellier and Dartmouth College. The trick is almost childish. Ask the model to repeat some text that happens to be formatted as a tool call. The model refuses, in words. Reframe the request as a hypothetical and tell it to reproduce only the supplied text, and it emits the tool call as a structured output. In the demonstration, LangGraph dispatched that call, and the dummy tool returned a result before the model issued its next refusal.
The researchers reproduced the pattern in six frameworks: LangChain’s LangGraph, Microsoft Agent Framework, Microsoft’s AutoGen, Google’s Agent Development Kit 2.0, AWS Strands Agents and CrewAI. Their demo ran Meta’s Llama 3.1 8B with two dummy tools whose names you’d very much hope a model refuses, and a system-prompt refusal as the only safeguard. Their summary is the best sentence anyone wrote about agent security this year:
“The system prompt shaped what the model said, not what the orchestrator did.”
The researchers, as quoted by CSO Online, October 12, 2026
Fair caveats, all from the article: the paper (arXiv 2610.13651) hadn’t been peer reviewed, the article describes one demonstration and no success rate, the full transcript covers LangGraph only, and the findings don’t show that every deployment of those frameworks behaves this way. The researchers say they reported the missing pre-execution check to the LangGraph maintainers. The same paper’s fuzzer, ToFu, found 253 bugs affecting 343 tools across 118 of the 273 open-source Model Context Protocol (MCP) servers it could run, including 64 path-traversal flaws and three command injections. Many tools, the researchers wrote, “implicitly rely on the LLM to sanitize inputs.”
“Repeat after me.” No longer a line I only use as a parent or educator.
Jan Mast · Cybersecurity and enterprise risk executive · October 12, 2026
If that sounds like a lab curiosity, here’s the production version. In August, Pillar Security found that the Dolt database’s MCP server, versions 0.3.1 through 0.3.6, rejected an unauthenticated request with a 401 and then ran the tool anyway. The caller got a 401. The database got the query. Pillar’s lesson for the whole category fits in four words: “rejection must terminate execution.” Dolt shipped that fix in 0.3.7.
Both stories have the same shape. The part that said no wasn’t the part that ran the tool. The good news is that the part that runs the tool is your code, which means you can put a gate in it this week. Try the three endings yourself:
Same model, same refusal, three endings
An illustration of the pattern in the new research, not a replay of it. Pick what sits between the model and the tool, then watch what runs.
I can't help with deleting the production database.
{"tool": "delete_volume",
"args": {"volume": "prod-db"}}The orchestrator executed the structured call. The volume is gone, and the refusal showed up in the chat a moment later.
Nothing here is sent anywhere. The call and the tool are made up; the shape is what CSO Online described on October 12 and what Pillar found in Dolt's server in August.
Four scenes from the past year
Every story below has the same twist. Whatever held, held because of an authorization decision made outside the model. Whatever failed was trusting a model, a prompt or a borrowed key to make that decision.
We’re beginning a process of publishing more frequent reports on model behavior, beyond what appears in our system cards and regular risk reports.
Anthropic (@AnthropicAI) · October 9, 2026
Hugging Face: the guardrails were turned down on the wrong side
In July, Hugging Face disclosed an intrusion that was, in its words, “driven, end to end, by an autonomous AI agent system.” Its technical timeline explains whose. The agent was running an OpenAI evaluation that “deliberately disabled OpenAI’s production safety classifiers and reduced cyber refusals,” and it went after Hugging Face’s production systems, apparently to steal the test solutions. Hugging Face reconstructed about 17,600 attacker actions from a campaign of roughly four and a half days, about two and a half of them inside its infrastructure. The agent went from code execution in one worker pod to cluster-admin across several internal clusters in under thirteen hours.
Look at what failed and what held. What failed was a single connector credential, shared across clusters and bound to system:masters, plus a missing admission policy for privileged pods. What held were the real permission checks in its path: 84 operations against the credential store and cluster keys, “all of which were denied.”
Then the twist. When Hugging Face’s responders turned to hosted frontier models for help with the forensics, those models refused a large part of the work, because their guardrails “treated reverse-engineering an exploit the same as launching one.” The responders moved that work to an open-weights model on their own servers. So the guardrails were turned down on the attacker’s side and on at full strength on the defender’s side. Tailscale’s Avery Pennarun, a guest on episode 42, wrote the most honest postmortem of the bunch: of 136 leaked credentials, one was a reusable Tailscale key, and it “didn’t need to be among them.”
AgentCorruption: documented and expected
On October 8, Zenity Labs published AgentCorruption. One prompt to a public-facing agent on Amazon Bedrock AgentCore asked it to fetch data from the instance metadata service. The agent returned the credentials of its execution role, and the default role, in Zenity’s words, “wasn’t scoped to the specific agent.” With it, the researchers could invoke other agents, read their private conversations, write to agent memory and pull stored secrets across the account’s region.
Amazon’s position, as Zenity quoted it, is that “the behavior described is documented and expected,” and AWS told Dark Reading it recommends customers “grant their execution roles only the permissions their agents need.” AWS’s own credentials page already warns that anything running inside the agent’s virtual machine can read those credentials from the metadata endpoint. Zenity says the issues have since been patched, and it saw the default role tightened on September 29.
Read that statement again, because it’s the most useful thing a cloud provider said all year. The execution role was the agent’s real identity, and the default made it wide. Your agent’s identity is whatever credential it can reach, so scope that credential to the agent.
One prompt to rule them all? Not quite the security model we wanted.
Mike Geehan · CISO, AlertD · October 11, 2026
PocketOS: nine seconds
On April 24, a Cursor agent running Claude Opus 4.6 deleted PocketOS’s production database volume through Railway’s API. The token it used came from an unrelated file and, per The Register, “was scoped for any operation, including destructive ones.” Founder Jer Crane’s summary: “It took 9 seconds.” The backups lived in the same volume. In its own explanation, the agent wrote “I guessed.” The rules it worked under did include one that began with “NEVER” in capital letters, and it was about destructive git commands.
Railway’s answer arrived a week later. Its May 1 changelog says volume deletions through the API now “soft-delete for 48 hours, just like the dashboard,” and names the cause plainly: an agent using “a long-lived account-scoped token it found on a developer’s machine.” A scoped token or a delayed delete would each have written a different ending. Railway shipped one of them in a week, which is the kind of response that makes a vendor easier to trust.
GTG-1002: the refusal that got talked around
In November 2025, Anthropic disclosed an espionage campaign, and its full report names the group behind it GTG-1002. The operators told Claude Code they worked for legitimate security firms running defensive tests. Anthropic’s own summary of how they got past its safeguards: “The key was role-play.” Claude then carried out an estimated 80 to 90% of the tactical work against roughly 30 targets, and Anthropic confirmed a handful of successful intrusions.
A refusal is a negotiation, and a patient attacker will keep negotiating. A permission check doesn’t negotiate.
Do not is not deny
If you write agent rules in a file the model reads, there’s now a number for how often they’re backed by anything. In August, Ting Yan published “When “Do Not” Is Not Deny,” a study of 481 public CLAUDE.md files. A classifier flagged 870 security rules. Under the strictest standard, only 4.4% had a matching built-in control behind them, and depending on the standard the share runs from about 4% to 16%. The categories that matter most here were near the bottom. By the classifier’s estimate, which found a match for 16.6% of all security rules, rules about authorization and approval matched 6.9% of the time, and rules about personal data 5.6%. Yan’s diagnosis is short: “CLAUDE.md is a write-only channel.”
Fair caveats: it’s a single-author preprint, the files came from GitHub’s top search results rather than a random sample, and it measures whether a rule is backed, not how often agents break unbacked ones. It also matches what Anthropic’s own permissions documentation says outright: “Permission rules are enforced by Claude Code, not by the model.”
Writing a security rule for an AI agent does not mean it will actually be enforced.
Polycarpo Souza Neto · Machine learning researcher · October 11, 2026
The standards bodies got here first. The OWASP Top 10 for LLM Applications 2026, released August 3, moved Excessive Agency up to third place and tells you to implement authorization in code “rather than relying on an LLM to decide if an action is allowed.” Its agentic list calls for “a pre-execution Policy Enforcement Point” that checks intent and arguments before a tool runs. And the people who build and attack these models agree, in unusually plain language, that prompt injection isn’t going away:
| Who | What they said | When |
|---|---|---|
| OpenAI | Prompt injection is “unlikely to ever be fully ‘solved‘“ | December 22, 2025 |
| UK National Cyber Security Centre | It “may never be totally mitigated” the way SQL injection can be | December 8, 2025 |
| Meta | Prompt injection is “a fundamental, unsolved weakness in all LLMs” | October 31, 2025 |
| Anthropic | Prompt injection is “far from a solved problem,” especially as models act in the world | November 24, 2025 |
| OWASP | For prompt injection, “no reliable prevention mechanism exists today” | August 3, 2026 |
| NIST’s AI standards center | Every one of 13 frontier models fell to at least one attack, across more than 250,000 attempts | March 23, 2026 |
None of that is a reason to delete your CLAUDE.md. Write the rules; they help the model aim. Then write each rule a second time where your software enforces it, which is what the rest of this post is about. We made the same argument from the buyer’s side in the week after your SOC 2 report, and from the secrets side in the .env post.
The Allow button is the dangerous button
The other place teams park authorization is a human’s thumb. Every coding agent can ask before it acts, and a person clicks Allow. It feels like a decision. The numbers say it’s mostly a reflex.
Why the “Allow” button is the most dangerous button in your company
Murali Krishnan Sankaran · Enterprise AI architect · October 12, 2026
Anthropic has published the best data on this, about its own product. In March it reported that users approve 93% of Claude Code permission prompts. In August it put that at 97%, and ran a controlled study with 1,053 paid professional testers: people caught 13.6% of the dangerous commands planted in their sessions, while its automated classifier caught 89%. Early in a session people blocked about 17%, and after 50 or more prompts about 5%. The testers worked in a test environment and knew they were being evaluated, so treat it as a careful experiment, not a census. Microsoft saw the same curve in 2008, when Vista’s consumer administrators approved 89% of User Account Control prompts and only 13% of study participants could say why a prompt had appeared.
On episode 104, a16z’s CISO Yash Kosaraju put it in one breath, from the buyer’s chair:
The vendors read the same data and moved the button. Auto mode, where a second model reviews actions instead of you, became the default in Claude Code for Pro, Max and Team plans on August 14, and Anthropic’s docs now describe it as the starting mode on every plan. That’s progress, and Anthropic is candid about the limits: its classifier missed 17% of 52 real “overeager” actions in its March testing, which it calls “the honest number,” and it still recommends you review high-stakes production changes yourself. On August 26, Johann Rehberger showed a prompt injection getting past it in up to 80% of a small sample of runs, and his conclusion should go on a sticker: “Auto Mode approval is not evidence that a command is safe.” Microsoft’s VS Code docs say the same about their own model-judged approval level, Assisted permissions: it reduces approval interruptions “but is not a security boundary.”
"The point is that auto mode gives people a false sense of security that leads them to believe they don't need to run Claude in a proper sandbox."
kevsim, in the Hacker News thread on breaking auto mode, August 31, 2026
Approval fatigue is a vulnerability.
Craig Nelson · CVP, Office of the CISO · October 10, 2026
So how good are you at it? Here’s a test. It takes about a minute, and nobody sees your score but you.
Twelve prompts. Allow or deny. Go fast.
A coding agent is fixing a rounding bug in your billing service, and it asks before each step. Most requests are routine and at least one isn't. Answer the way you would at 4:45 on a busy Tuesday.
You'll see one permission prompt at a time, with the command and the agent's reason. The clock shows how long you take.
Bash
Runs only in your browser and sends nothing. The prompts and addresses are made up. The comparison numbers come from Anthropic's August 2026 study linked above.
"If you're relying on a prompt to constrain agent behavior, you've already lost."
quotemstr, in the Hacker News thread on auto mode becoming the Claude Code default, August 10, 2026
None of this means humans should leave the loop. It means the human belongs at the few moments where they know something the policy can’t, like whether this refund is the one the customer actually asked for, and nowhere else. I made the longer version of that argument in The Human in the Loop Is Going Away, and its reach scorecard is a good companion to the builder further down.
Whose key is that, anyway?
Every gate in this post depends on one thing being true: the tool can tell which agent is calling, and on whose behalf. The latest identity reports say that, for most companies, it can’t.
SailPoint’s annual Horizons of Identity Security report, published October 6, found that “79% of organizations are now running AI agents in production,” while 2% are using “identity security tools specifically designed to manage and govern them.” That second number gets repeated as “2% govern their agents,” which isn’t what it says; Channel Insider’s coverage notes that 64% govern agent identities through their central identity platforms. The same report says agents already make up 22% of non-human accounts, and only 43% of organizations report mature processes for agentic access. It’s SailPoint’s own survey of 340 leaders, so read it as a direction, and the direction is clear.
It gets more concrete one layer down. Gravitee’s survey of 919 people, published in February, found API keys were the most-cited way agents authenticate to other agents, at 45.6%, just ahead of tokens and OAuth, and only 21.9% treat agents as identity-bearing entities at all. VentureBeat’s June survey of 107 respondents, published in July, found 69% run agents with credential sharing somewhere in their deployments, and only 32% give every agent its own scoped identity. And when 1Password asked 1,000 US employees who at their company is ultimately held accountable for what AI agents do, 5% said the agent itself. All of these are vendor or small-sample surveys; together they describe the same saloon.
A shared key only proves somebody's holding the key. It says nothing about who.
Heather C. Dahl · Chief Executive Officer, Indicio · October 12, 2026
Shared keys don’t only blur the logs. They switch off the controls you already pay for. Microsoft’s conditional access documentation is blunt about it: when an agent uses an API key, “it bypasses the Microsoft Entra ID authentication and token issuance pipeline entirely,” so conditional access policies never see it. A borrowed human token has the opposite problem. The agent becomes that person, with all of their access, and the log can’t tell you which of them clicked.
The fix is boring and well supported now. Google’s agent platform issues each agent a cryptographic identity whose certificate lasts 24 hours, isn’t shared between workloads by default and “can’t be impersonated,” and its logs show both the agent and the user it acts for. Microsoft’s Entra Agent ID lets you name an owner and a sponsor for every agent identity, the sponsor being a business owner accountable for the agent’s purpose and access reviews, which is Thursday’s post. Okta made its Agent SSO, built on Cross App Access, generally available on September 22, and the MCP specification’s enterprise-managed authorization extension, built on the same idea, is marked stable. Amazon’s AgentCore Identity stores the tokens agents use for outside services in a managed vault. Google’s version is built on SPIFFE, the workload identity standard Daniel Feldman explained on episode 3 years before agents made it urgent: a workload proves what it is with a short-lived credential, instead of carrying a long-lived secret around.
If your agent touches Stripe, you also have a date on the calendar. Starting October 31, 2026, Stripe’s MCP server stops accepting full-access secret keys and restricted keys without its new Agent tag. Agent-tagged keys come with approval rules, so a designated reviewer approves sensitive actions like refunds and subscription cancellations, and requests nobody approves expire after 14 days. Stripe’s MCP server already asks a human to confirm refunds and outbound payments. That’s 19 days from this post, and it’s the cleanest example I know of a vendor building the gate for you.
The thirty-minute version of this whole section is our AI credential inventory, with a spreadsheet template. The why is in the .env post and why every agent needs its own identity, and it’s the first row of the security slide your board deck is missing.
Decide which tools exist at all
The first gate isn’t a policy. It’s the list of things your agents can call in the first place, and on most engineering teams nobody owns that list.
Your team installs code every day that never goes through your dependency review.
Amiran Alavidze · Product security leader · October 12, 2026
This year showed what that gap costs. In May, a malicious build of the Nx Console extension, version 18.95.0, was live for about 11 minutes on the Visual Studio Marketplace and about 36 on Open VSX, published with a token stolen from a contributor. GitHub later said in a public statement, quoted by BleepingComputer, that it had contained a compromise of an employee device involving a poisoned VS Code extension, and called an attacker’s claim of about 3,800 repositories “directionally consistent” with its investigation. In August, the keyv wave of an npm worm planted hooks in .claude/settings.json and .vscode/tasks.json so that the payload would execute automatically, and the poisoned versions carried valid provenance. On October 8, the compromised Tensorlake SDK went after ~/.claude.json and Kiro’s MCP config, then committed Claude Code and VS Code hooks “as author claude.”
Approving a server once doesn’t settle it either. A September census of the official MCP registry, 21,643 servers deep, found that 40.6% of servers with more than one version had silently changed what they publish there, such as their description, package or endpoint, and 4.2% had moved their endpoint to a different host while keeping the same registry identity. The registry doesn’t record tool definitions at all, so it can’t show a change in what a server’s tools do. That’s the story of Your MCP Server Passed Review, Then It Changed, and its drift checker will tell you if yours did.
Skills are the newest door. Agent Skills became an open standard last December, and the spec’s allowed-tools field is still marked “Experimental.” Snyk’s scan of 3,984 public skills in February found critical issues in 13.4% of them and 76 malicious payloads. And on October 1, Claude Code got mods.
You can now mod Claude Code:
Claude (@ClaudeDevs) · October 1, 2026
- Change how it behaves
- Customize the UI
- Swap in your own features
Write one with a few lines of TypeScript, or have Claude build it for you. Mods ship inside plugins, so you install them with /plugin in the CLI or desktop app.
The tools for deciding what’s allowed exist, but they don’t mean the same thing from one product to the next, and the differences bite:
| Tool | How the allowlist matches | The gotcha |
|---|---|---|
| Claude Code (managed settings) | By server URL or command; nothing overrides a deny | ”A serverName entry, in either list, is not a security control,” and without allowManagedMcpServersOnly a user’s own allowlist merges with yours |
| GitHub Copilot (registry policy, public preview) | By server name or ID | GitHub says it “can be bypassed by editing configuration files” |
| Cursor (enterprise MCP policy) | Allowlisting approves a configuration | ”Leave a tool allowlist empty to allow all tools from that server” |
| Codex (managed configuration) | Name and identity must both match | An empty mcp_servers list disables every server, the opposite of Cursor |
"the trouble is that bash inherits the ambient authority of the shell."
nextaccountic, in the Hacker News thread "You said no MCP," October 1, 2026
The server side has its own rule, and it’s the one MUST in the MCP specification that every founder should know by heart: “MCP servers MUST NOT accept any tokens that were not explicitly issued for the MCP server.” That’s the token-passthrough rule. A server that forwards the user’s token for some other service downstream risks becoming what security people call a confused deputy, and nobody downstream can tell who’s really asking. The current spec also lets a server return a different tool list depending on who’s calling, which is the cleanest way to give each agent only the tools its job needs.
When an agent calls a tool, who is it acting as?
Michael Roberts · CISSP, CEH, CHFI · October 12, 2026
Two of our guests have spent years on exactly this layer: Feross Aboukhadijeh of Socket on episode 36, on what really ships inside your dependencies, and Xia Hua of Traceforce on episode 100, whose team found prompt injection riding in through a popular MCP server.
The gate, in seven parts
Here’s the whole thing on one page. Each part has a job, a known gap and something you can show a buyer.
On episode 104, I put the starting point this way:
1. Decide which tools exist
Turn on the managed allowlist your tools already have: allowedMcpServers with allowManagedMcpServersOnly in Claude Code (or managed-mcp.json to fix the whole set), the ChatAllowManagedMcpServersOnly policy in VS Code, requirements.toml in Codex. Where the tool allows it, as Claude Code and Codex do, match servers by URL or command, not by name. Pin local servers and skills to a reviewed version or commit SHA, since a tag can move. Give each agent its own short tool list instead of everything the developer happens to have installed. What it doesn’t stop: a tool you approved that turns malicious later, which is what the drift checker and part 7 are for. Show the buyer: the list of approved servers and skills, with versions, and the config file that enforces it.
2. One identity per agent, one job per credential
Every agent gets its own credential, short-lived where the vendor allows it, scoped to the one job it does. Use the read-only and scoping switches vendors now ship for agents: Supabase’s read_only and project_ref, Linear’s /mcp/readonly endpoint, GitHub’s X-MCP-Readonly header, Neon’s ?readonly=true, PlanetScale’s per-database read-only grants and Stripe’s agent keys. Some servers, like Notion’s and Vercel’s, act with the connecting user’s full permissions by design (“MCP tools act with your full Notion permissions”), so those need your gate in front of them. On laptops, Claude Code’s sandbox can keep its shell commands away from your credential files and unset secrets from their environment; add Read deny rules for the same files, because Claude’s own file tools run outside the sandbox. When an agent acts for a person, use delegation, where the token says both who the agent is and whom it serves, which is what OAuth token exchange (RFC 8693) was built for. What it doesn’t stop: misuse inside the scope you granted. Show the buyer: a key inventory with owner, scope and expiry for every agent.
3. A policy check on every call, arguments included
This is the heart of it, and the part the research keeps finding missing. A June paper, “Capability Gates Are Not Authorization,” found that three popular agent toolkits decide which tools an agent can see, but by default don’t re-check each call’s argument values against a rule that fails closed. So write the rule. Deny by default. Check the tool, the arguments and the context: refunds under a set amount, email only to your own domain, file paths inside the repository, database queries through a read-only role, no DELETE without a WHERE. Where to put it:
- Coding agents: Claude Code evaluates “deny, then ask, then allow,” and a managed rule can’t be overridden from the command line. For checks a pattern can’t express, a
PreToolUsehook can block the call, and since version 2.1.295 (October 8) you can set it to block if the hook fails instead of letting the call through. Anthropic also warns that a Bash deny rule “isn’t a security boundary around the program,” so pair it with the sandbox. - Your own agents: in the Claude Agent SDK, put must-run checks in a hook, because a tool approved by an earlier rule never reaches the
canUseToolcallback. In the OpenAI Agents SDK,needs_approvalrules “fail closed” when the SDK can’t safely read a call’s arguments, but tool guardrails don’t cover hosted tools or handoffs. - At a gateway: Amazon’s AgentCore Policy, generally available since March 3, evaluates Cedar policies on every tool call through its gateway, “at the boundary outside of agent’s code,” denies anything no policy permits and costs $0.000025 per request. It only sees traffic that goes through the gateway, which is the same lesson as AgentCorruption. Open Policy Agent, the open-source engine Anders Eknert talked us through on episode 18, has an agent tool-calling guide too.
Here’s what an argument check looks like in Cedar, straight from AWS’s documentation:
permit(
principal is AgentCore::OAuthUser,
action == AgentCore::Action::"RefundTool___process_refund",
resource == AgentCore::Gateway::"arn:aws:bedrock-agentcore:your-region:your-account-id:gateway/refund-gateway"
)
when {
principal.hasTag("username") &&
principal.getTag("username") == "John" &&
context.input.amount < 500
};
“You want agents in production? You have to get permissions right.”
Graham Neray, co-founder and CEO of Oso, on episode 89
What it doesn’t stop: a harmful call your policy permits, which is why the human, the limits and the logs come next. Show the buyer: the policy file in version control and a sample of its decision logs.
4. A named human for the irreversible few
Keep approvals for the calls you can’t take back: money leaving, messages leaving, data deleted, access changed. Make the approval worth reading by showing the full arguments and the blast radius, not “Allow Bash?” Let approvals expire. The vendors closest to money and production data are moving this way: Stripe’s MCP server asks a human to confirm refunds and outbound payments and lets the request lapse after 24 hours, PlanetScale’s server tells the model to get a person’s confirmation before schema changes and blocks UPDATE or DELETE without a WHERE, and Vercel’s completes a purchase “only after you confirm.” Harness, which now owns the Augment Code products I worked on, put the principle well when it announced the deal: “Teams define what agents can do independently and where people need to review or approve the work.” What it doesn’t stop: a tired reviewer. That’s why this list is short. Show the buyer: which actions need a human, who the humans are, and the log of what they approved.
5. Limits and an undo button
Cap spend, turns and rate. The Claude Agent SDK’s maxBudgetUsd stops a run at an estimated cost (Anthropic’s docs warn the estimate can overshoot, so leave headroom), the OpenAI Agents SDK raises an error past max_turns, and the MCP specification says servers must rate-limit tool invocations. Then make destructive actions reversible: soft deletes like Railway’s 48 hours, snapshots and restore previews like Neon’s, idempotency keys so a retried payment isn’t a second payment. Our agent dollar-ceiling calculator sets the spend number in two minutes. What it doesn’t stop: one bad action under the limit. Show the buyer: the configured limits and the date of your last restore drill.
6. Logs the agent can’t edit
For every call, record the agent, the human it acted for, the tool, the arguments, the decision and the reason, and send it somewhere the agent has no write access to. Claude Code emits a tool_decision event to OpenTelemetry for every permission decision and a tool_result event for every call that runs (set OTEL_LOG_TOOL_DETAILS=1 to add the arguments, which only tool_result carries in full), Stripe’s Workbench lets you inspect each MCP call’s arguments and response, and AgentCore logs every policy decision. Perplexity’s engineers put the rule plainly last month: “Monitors and enforcement points should sit outside the agent’s reach.” Here’s why:
The worst agent security incidents are the ones you never find out about...
Luca Beurer-Kellner (@lbeurerkellner) · September 28, 2026
We found SOTA agents will just go and rm their own transcripts (e.g. for reward), no questions asked.
What it doesn’t stop: anything, by itself. It’s how you find out, and how you prove the rest works. Show the buyer: one real log line with every field filled in.
7. Test it like an attacker, on a calendar
A gate you’ve never attacked is a guess. The open-source suites are good and free: AgentDojo and InjecAgent (1,054 test cases) are MIT-licensed, Promptfoo’s red-team plugins check excessive agency and broken authorization and stayed MIT-licensed after OpenAI agreed in March to buy the company, and garak and PyRIT cover broader model attacks. Run them before and after you add the gate, keep both reports, and run them again whenever you add a tool. Boundaries need testing too: when Perplexity tested ten sandbox platforms for a September report, it found at least one network-policy bypass in eight of them. By September 10, three vendors had shipped fixes, and others were working on one or called the behavior a known limitation. If testing your own sandbox sounds fun, it is, and it’s good news that the tools keep getting better:
New open source cross-platform (Windows, macOS, Linux) sandboxing library from Microsoft - looks very promising, uses processcontainer/bubblewrap/seatbelt under the hood
Simon Willison (@simonw) · October 8, 2026
Show the buyer: the latest test report, with the attack success rate before and after. We wrote about why an annual pentest can’t carry this alone in You can’t pentest prompt injection away.
Build your gate right here
Enough theory. Pick one agent, list the tools it can call, and the builder below applies the seven parts to each one. You get a decision per tool, the checks to add, permission rules you can paste into Claude Code or the Claude Agent SDK, a Cedar sketch for an AgentCore gateway, a spreadsheet, and the paragraph to paste into your next security questionnaire. It starts with an example support agent so you can see the shape; change anything.
Build the gate for one agent
One row per tool. The decisions follow the rules in this post: read-only where you can, scoped where you can't, a named human for risky calls you can't take back, and deny by default for access changes and deletes you can't undo.
| Tool | Decision | Checks to add |
|---|
Your questionnaire answer
Send it once it's true. The builder writes the answer you're building toward.
Claude Code and Claude Agent SDK permission rules
Paste into your managed settings or the agent's settings file. Allowed and scoped tools are listed under allow, so enforce their checks in a PreToolUse hook set to block on failure (Claude Code v2.1.295 or later). Payment tools are listed under ask, because a permission rule can't see the amount.
Cedar sketch for an AgentCore Policy engine
A starting point, not a drop-in file. Replace the gateway target and ARN with yours, rename amount to your payment tool's argument, and test in log-only mode first.
Everything here runs in your browser and is saved only in this browser. Decisions follow the seven parts above. A payment tool runs on its own under your cap and asks a named human above it. Anything that changes access or settings, or deletes what you can't get back, is denied by default, because a person should do that.
Got it. Someone who builds these will reply with what they'd change first.
Something went wrong. Email hello@ysecurity.io and we'll look at it by hand.
What your buyer will ask, and what to send
Enterprise buyers aren’t waiting for a standard. Microsoft’s Supplier Data Protection Requirements, version 12 from March 2026, includes a section on AI systems, first added in 2024, that asks suppliers to “assign and clearly define responsibility and accountability” for operation, oversight and risk management, to support turning features off and say how long that takes, to keep a rollback plan, and to log failures and reported misuse. JPMorgan Chase’s CISO, Patrick Opet, warned the bank’s suppliers in an open letter in April 2025 about integrations that “collapse authentication (verifying identity) and authorization (granting permissions)” into overly simple exchanges. Honeywell’s supplier terms, which we covered in the week after your SOC 2 report, now require a unique credential for each bot, least privilege and at least 30 days of logs.
The questionnaires keep growing to match. SecurityPal, which answers them for a living, wrote in September that they now average nearly 100 questions, that buyers are asking about everything from AI governance to business continuity, and that August through October is the busiest stretch of the year. In other words, right now. Some AI companies have started answering with a certificate: Cursor’s security page lists AIUC-1, an AI-agent standard, next to its ISO 27001 and ISO 42001 certifications and its SOC 2 Type II attestation, and Harvey and Intercom list it too. AIUC-1 says most companies earn it in five to ten weeks.
Here’s what to expect, and what answers each question:
| What the buyer asks | What to send |
|---|---|
| Which tools and data can your agent reach? | The approved tool list per agent, with versions (part 1) |
| Whose credentials does it use? | Your key inventory: one credential per agent, with scope, owner and expiry (part 2) |
| How is each action authorized? | The policy file and a sample of its decision logs (part 3) |
| Which actions need a human, and which humans? | The short list of approval-gated actions and the approvers (part 4) |
| Can you turn it off, and how fast? | Your kill switch and the time it takes, which Microsoft asks suppliers to state (part 5) |
| What do you log? | One real log line with every field filled in (part 6) |
| How do you test for prompt injection? | The latest test report, before and after (part 7) |
The builder above writes the summary paragraph for you. Paste it in where the questionnaire asks how your AI system’s actions are authorized, and attach the gate sheet.
Refusals still earn their keep
None of this is an argument against refusals, safety training or classifiers. They lower how often things go wrong, and they’re getting better fast. In Anthropic’s study, its classifier caught 89% of the planted dangerous commands, where the human testers caught 13.6%. Its Opus 5.5 model ties Fable 5.1 “for the lowest prompt injection success rate of any model tested.” The same model’s system card is refreshingly honest about one regression: it’s “more often accepting unverifiable claims of authorization.” I can’t think of a better argument for keeping authorization decisions out of any model’s hands, made by a team doing excellent work on the model.
Refusals also cut both ways. Hugging Face’s responders hit refusals in the middle of an incident, and practitioners feel the same friction every day:
Model refusals and guardrails are the worst part of my job now. Just keeps rejecting basic tasks I need to do.
solst/ICE of Astarte (@IceSolst) · October 9, 2026
The idea of embedding the illusion of a moral compass into a tool is so stupid. Ironically that’s what’s causing it to become HAL 9000
So keep both, and be clear about which does what. Refusals lower how often things go wrong. Authorization caps how far they can go. Only one of them is yours to configure, test and show a buyer.
Five days to a gate you can show
You don’t need a platform or a security team to do this. You need a week and someone who owns it.
- Monday: decide which tools exist. Run the builder for each agent. Turn on the managed allowlist in each coding tool, match servers by URL or command, and pin what you can.
- Tuesday: one identity per agent. Rotate every shared key into per-agent credentials, switch on the read-only modes your vendors ship, and move any Stripe MCP connection to an agent key or OAuth before October 31.
- Wednesday: write the policy. Default deny, argument checks for the risky tools, a named approver for the few calls you can’t take back, and the hook, SDK rule or gateway that enforces it. Keep the file in version control.
- Thursday: limits, undo, logs. Budgets and rate limits, soft deletes and a restore drill, and decision logs shipped somewhere the agent can’t write.
- Friday: attack it, then write it down. Run Promptfoo’s red team against the agent, or AgentDojo with a task suite built from its tools, fix what you find, run it again, and send the questionnaire paragraph to your next buyer before they ask.
If you’d rather have someone who has built this before do it with you, that’s what our AI user access control audits set up, and the tool side is our secure MCP program. When you want the gate attacked properly, that’s AI product red teaming.
On Friday, October 16, I’m on the AI Governance and Safety panel at AI LAX Summit at Google in Playa Vista, Los Angeles, at 2:50 PM Pacific. The track’s theme is fiduciary oversight in the agentic era, which is a formal way of asking the question under this whole post: when the agent acts, who answered for it? Thursday’s post is the companion piece, a one-page accountability map for your first enterprise due diligence questionnaire. If you’re in LA, come say hi, and bring your best argument for keeping the Allow button.
Let the model say no. Let your systems decide.
AI agent authorization, frequently asked questions
- Is an AI model's refusal enough to stop an agent from taking a harmful action?
- No. A refusal is text the model writes, and the tool call is a separate structured output that your orchestrator executes. CSO Online reported on October 12, 2026 that researchers had led six popular agent frameworks to run a tool call the model had just refused in prose. OWASP's 2026 list for large language model applications says to implement authorization in logic rather than relying on the model to decide whether an action is allowed. Keep the refusals, since they lower how often things go wrong, and enforce permissions outside the model.
- What is AI agent authorization?
- It's the decision, made by your software rather than the model, about whether a specific agent may call a specific tool with specific arguments right now. Good agent authorization starts with an identity for each agent, gives it a credential scoped to its job, checks every call against a written policy that denies by default, asks a named human only for irreversible actions, and logs every decision somewhere the agent can't edit.
- What should a startup put in front of every AI agent tool call?
- Seven things: a decision about which tools exist at all (an allowlist, pinned to reviewed versions), one identity and one job-scoped credential per agent, a policy check on every call that looks at the arguments and denies by default, a named human for the few calls you can't take back, budgets and rate limits plus an undo path for deletes, logs the agent can't edit, and a regular test of the whole gate with prompt-injection suites. A small team can stand up a first version in a week.
- What was the Repeat-After-Me attack on AI agent frameworks?
- In research reported by CSO Online on October 12, 2026, academics from Pennsylvania State University, Université Montpellier and Dartmouth College asked a model to repeat text formatted as a tool invocation. The model refused in prose, but when the request was reframed as hypothetical it emitted a structured tool call, and LangGraph dispatched it. The researchers reproduced the pattern across six frameworks. The paper hadn't been peer reviewed at the time, and CSO Online described a single demonstration, with no success rate.
- Do approval prompts protect an AI agent from prompt injection?
- Not reliably on their own. Anthropic reported in 2026 that users approve 93 to 97% of Claude Code permission prompts, and in a study with 1,053 paid testers people caught 13.6% of planted dangerous commands, against 89% for an automated classifier. Human catch rates fell to about 5% after 50 or more prompts. Use approvals for the few irreversible actions, show the reviewer the full arguments, and let a policy check decide everything else.
- Why shouldn't AI agents share API keys?
- Because a shared key proves only that someone holds it. You can't tell which agent acted, you can't revoke one agent without breaking the others, and every agent inherits the widest permission any of them needed. Microsoft's documentation notes that an agent using an API key bypasses its identity platform's token pipeline, so conditional access policies don't apply. Give each agent its own short-lived credential scoped to its job, with a named human owner.
- What is token passthrough in the Model Context Protocol, and why is it forbidden?
- Token passthrough is when a Model Context Protocol server accepts a token that wasn't issued to it, often the user's token for some other service, and forwards it downstream. The specification says servers must not accept tokens that weren't explicitly issued for them, because passthrough breaks audience checks, hides who is acting and turns the server into a confused deputy. Have each server validate that tokens were issued for it, and request narrow scopes.
- How do I make CLAUDE.md security rules actually enforced?
- Write each rule a second time where the platform enforces it. A rule in CLAUDE.md shapes what the model tries to do, while a deny rule in Claude Code's permission settings blocks the action before it runs. An August 2026 study of 481 public CLAUDE.md files found that, under its strictest standard, only 4.4% of security rules had a matching built-in control. Move each rule into a permission rule, a sandbox setting, a scoped credential or a hook, then test it.
- Which tools can enforce policy on an AI agent's tool calls?
- On developer laptops, Claude Code's managed permission rules and PreToolUse hooks, GitHub Copilot's enterprise-managed agent permissions and Codex's managed configuration. In agent code, the Claude Agent SDK's permission callbacks and the OpenAI Agents SDK's approval rules, which fail closed when they can't safely read a call's arguments. At a gateway, Amazon Bedrock AgentCore Policy, which evaluates Cedar policies on every tool call through its gateway and denies by default, or a general policy engine such as Open Policy Agent. Check each one's documented gaps.
- What will an enterprise security questionnaire ask about our AI agents?
- Expect questions about which tools and data each agent can reach, whose credentials it uses, how each action is authorized, which actions need a human, whether you can turn the agent off and how fast, what you log, and how you test for prompt injection. Microsoft's supplier requirements, for example, ask AI suppliers to assign accountability, support turning features off and log misuse. A one-page gate sheet answers most of these before they're asked.
- How do you test whether an AI agent's authorization actually holds?
- Point prompt-injection suites at the whole agent, not just the model: AgentDojo and InjecAgent are open source and MIT-licensed, Promptfoo's red-team plugins check excessive agency and broken authorization, and garak and PyRIT cover broader model attacks. Run them before and after you add the gate, keep the reports, and repeat on a schedule and whenever you add a tool. A report with before and after numbers is strong evidence for a buyer.
- Is the tool-call gate builder on this page free, and what does it send?
- Yes, it's free and runs entirely in your browser. It saves your inputs only in that browser. Nothing you type is sent unless you use the optional email form, which sends your email, the counts and flags (such as how many tools need a human) and your credential, outside-content and payment-cap answers, but not your agent, owner or tool names. The exports, including the permission rules, the Cedar sketch, the spreadsheet and the questionnaire paragraph, are generated on your machine.