The best story in this episode took less than fifteen minutes to happen.
“And about a year ago, [a] challenge was brought to us, and they said, well, you know, we were looking for a way to see what would happen if an agent could spend money. And when we put it in a test environment and said, hey, let’s see what happens, in less than 15 minutes, all the money was spent.”
Eric Olden, at 3:04
Eric Olden has been solving identity problems since the web was a bunch of brochures. He founded Securant Technologies in 1995 while a student at UC Berkeley, built ClearTrust, one of the first commercial web access management products, co-authored the proposal that became SAML, and sold Securant to RSA Security in 2001. He started Symplified, one of the first identity services in the cloud, ran Oracle’s cloud security and identity business, and in 2019 co-founded Strata Identity in Boulder with Topher Marie and Eric Leach. When Sasha and I recorded with him, Strata was turning its identity orchestration into guardrails for AI agents. We published this on November 4, 2025, and the show notes were 41 words. This page is the rest of it: the fifteen-minute test, why agents in production are still in the low single digits, the audit model of intent, context and outcome, why agents are people too, the chatbot ruling he half-remembered and what the tribunal actually said, the flight simulator, the three missing guardrails of the early web, standards first, passkeys and the open policy layer, the bar at hyperscale, an apology at Oracle, the hardest days of a founder, and what Strata did in the year after we hit stop, which ended with Rubrik buying the company. There are four things to try along the way: a quiz, a scorecard for whether an agent is ready for production, a calculator for how many test runs it takes before you can sign off, and a timeline you can filter.
Fifteen minutes and the money was gone
Strata builds its products with design partners, a lot of them big financial institutions and insurers, and nobody on the tape names this one or the amount. What matters is what they expected. They asked Eric what Strata would do to control an agent like that, and he asked them what they had thought would happen.
“They said, well, we thought we could treat the AI like a service account and, like, put a, for lack of a better term, like a straitjacket on a piece of software and control it. And I said, oh, I can tell you that’s not going to work, because the only one who could do that would be the model builder themselves.”
Eric Olden, at 3:04
If OpenAI or Anthropic wanted to change how their model behaves, they could. Everyone else is a customer of the model, and a customer cannot put a straitjacket on it. What a customer can do is quieter and works every time: “What we could do is to take away the ability to press the API button to spend the money.” You control the AI, Eric said, but not in the way that seems natural. “You do it indirectly, through a control plane.” In Strata’s software that became guardrails, set by policy, for how the AIs, the agents and the humans authenticate, how they get access, how they are registered and provisioned just in time, “and all of the A’s that we do in identity, but you do them in a different way at machine scale.”
This is the most useful sentence in the episode for anyone shipping an agent this quarter, and it is the same idea behind our post on why an agent saying no is not authorization: the model’s judgment is not a control, and a refusal in the prompt is not a policy. The control lives between the agent and the thing it can break, where you can see it, test it and change it without retraining anything. It is also why the spending story belongs next to the agent that costs $50,000 an hour. An agent with a credit card and no limit outside itself is a budget with a fifteen-minute half-life.
Low single digits in production
Sasha’s first question was where the biggest opportunity is in managing agents, and Eric said there were two answers tied for first. The guardrails were one. The other was what he hears every time he asks the people on Strata’s wait list a single question.
“And when you ask them, what percent of your agents are in production? No, spoiler, it’s in the low single digits.”
Eric Olden, at 3:04
The two answers are tied because each one causes the other. Companies have to get agents into production, “otherwise you’re spending all this money on an investment that won’t give us any return,” and they cannot put them into production because they do not have the guardrails. Over three hundred and fifty people and organizations had signed up for Strata’s sandbox wait list when we recorded, split, he said, rather evenly between engineers with hands on keyboards and less technical people, program managers and business analysts. Low single digits is his read of that list, not a survey. The surveys since point the same way. Gartner predicted in June 2025 that more than 40 percent of agentic AI projects will be canceled by the end of 2027, because of “escalating costs and unclear business value,” and estimated that only about 130 of the thousands of vendors selling agentic AI are the real thing (The Star, from Reuters). In February 2026 a Cloud Security Alliance survey of 285 IT and security professionals, commissioned by Strata, found that only 18 percent were highly confident their identity systems could manage agent identities, only 21 percent kept a real-time inventory of their agents, 44 percent used or planned to use static API keys for them, and 84 percent doubted they could pass a compliance audit focused on agent behavior or access controls (Cloud Security Alliance).
Eric put the survey’s other finding in one LinkedIn post this year: “only 17% enforce runtime access controls consistently, and 45% cite credential over-provisioning as a top concern,” because traditional least privilege assumes you can design access in advance, and “That assumption breaks the moment you introduce agents that decide what to do at runtime” (Eric Olden on LinkedIn). That is the argument of our post on why agents need least privilege more than humans ever did, and the 44 percent on static keys is the whole subject of your agents share one API key, which we wrote as the companion to this episode.
Intent, context, outcome
Sasha asked whether guardrails were the whole answer or whether observability belonged next to them, the full stack trace of what an agent did. Eric said she was absolutely right, and then described the part of Strata’s design I have repeated most since.
“And so what’s different in agents and audit that we didn’t have to think about with humans, that are deterministic, more or less, but we never really had to ask and record intent. And so that’s what’s changed in observability, is what now our audit model has three components.”
Eric Olden, at 6:18
The first component, intent, is generally the prompt. The second is context, “which are typically the attributes that were used,” the environmental details, the facts. The last is the outcome: “Like, what happened, right? Did something get purchased? Did something get promoted to production, whatever the case may be?” A log that records who called which API answers the question we asked about people. It does not answer the question we now have to ask about agents, which is why it did what it did. Strata’s AI Identity Gateway, which shipped two weeks after the episode, logs agent requests, responses and token exchanges at each step for exactly this reason (Help Net Security).
Then he went further, with a nerd alert. Guardrails and observability are not two sides of one coin, he said, “it’s actually more like one of the, nerd alert, D&D dice, right? Six-sided die. So it’s not just the one, it’s how the one affects the other, which affects the other.” The third face is identity governance, the life cycle of everything: who owns the agent, what it may still do, and when it goes away. You cannot observe what no guardrail enforced, and you cannot govern what you never observed. Our agent offboarding checklist is the governance face of the die for the day an agent is retired.
Agents are people too
The governance question is where Eric reached for Stranger Things. “Remember the Upside Down?” I said it was kind of the analog, a different area, and he agreed: “it’s different, and it rhymes.” Everything we are used to doing with people has to be flipped around to work with agents, and the flip is smaller than the industry has made it sound.
“And so I think of that as agents are people too. And that’s your decoder ring about how do you make all this stuff work. Guess what? You don’t have to reinvent it. It’s not a non-human identity problem. It’s the same thing as treating a human.”
Eric Olden, at 8:09
Sasha took the analogy all the way. There is a unit of work, she said, that used to rely on human labor and can now be done by agentic flows, and “just like with humans, you have to do the background checks, you have to ensure that proper access controls are in place in order for this workforce not to venture out in places where it shouldn’t.” Joiner, mover, leaver. An owner, a manager, a role, access that matches the job, a review when the job changes, and an exit when it ends. The industry calls agents non-human identities, and Eric’s point is that the name hides the answer, because the human methodology and structures we are already good at carry straight over once you treat humans and agents as peers. The survey found that nearly half of organizations already extend their human identity models to govern agents. The difference is speed and volume: the same A’s, at machine scale. Our posts on who signs for the agent and agent governance at onboarding are the human resources department for this workforce.
Who is saying no
Sasha asked who blocks agents from production: compliance, security, or the whole organization deciding it is not ready. Eric’s answer was not a department.
“The thing that’s, I think, holding people back is a lack of confidence.”
Eric Olden, at 10:09
Enterprises have a ritual for pushing anything into production, a checklist, someone who signs off, blue-green testing, smoke tests. “And I think what really gets people shook is when you run the same prompt and the AI does something different every time you do it. And that really brings that fear of like, I don’t want to be a headline.” The cure he offered is the oldest one in security, “trust, but verify, like, never been more true than right now,” and a rule of thumb I have used in every agent review since.
“And the way I believe anybody should get confident is like, see it, do it, and then do it five more times. And if you can see predictable responses and outcomes, then you’re ready to go to production. I’d sign off on that.”
Eric Olden, at 10:09
Six runs is a good habit and a bad statistic, which is the point of the calculator further down: a failure that shows up one run in twenty will hide in six clean runs most of the time. The habit is still right. Sasha added the part that makes it hard, that the same input gives a different output when the context changes, so “these are not simple unit tests, where these are integration tests, and you’ve got a lot of moving parts.” The answer to that is the flight simulator.
The chatbot that gave away a ticket
Eric told a cautionary tale from memory, and the record is better than his version.
“And I think one of the airlines, forget which one, the, the airline, the chatbot said, oh, you get a free ticket. And they’re like, well, no, no, we didn’t mean for you to get a free, whatever, $1,000 ticket. And the court says, hey, that’s on you, airline. Give them the ticket, because you want to use the agent to do the work of a person. If it was a person who said that, we would hold you to that as well.”
Eric Olden, at 10:09
The airline was Air Canada, and it was not a free ticket. In 2022 Jake Moffatt asked the chatbot on Air Canada’s website about bereavement fares after his grandmother died, and it told him he could apply for the discount after buying a full-price ticket, which the airline’s actual policy did not allow. Air Canada argued before British Columbia’s Civil Resolution Tribunal that the chatbot was “a separate legal entity that is responsible for its own actions.” Tribunal member Christopher Rivers called that “a remarkable submission,” wrote that “It should be obvious to Air Canada that it is responsible for all the information on its website,” and ordered it to pay $812 Canadian, against the $1,630.36 he had paid for full-price tickets (CBC News). Eric’s conclusion survives the correction intact. A company that uses an agent to do the work of a person owns what the agent says, the same way it owns what the person says. The Hacker News thread put the legal theory in one line.
"Quite a sensible judgement, it seems. Pretty outrageous for the airline to try to claim the chatbot was its own legal entity."
Arrath, on Hacker News, February 2024
“So I don’t think anybody wants to be having to give out tickets, because that’s not anyone’s business model.” An $812 judgment in a small claims tribunal is a cheap lesson. The fifteen-minute test was a cheaper one, because it happened in a test environment, which is the whole point of the next section.
Don’t learn to fly on the plane to Hawaii
Strata does a lot of work in aerospace and defense, and Eric flies a lot, so his explanation of the sandbox starts at the airport.
“If you were to say, hey, the three of us, we want to learn to fly, what we would not do is, we would not go, hey, go to the airport and get behind the cockpit of a plane loaded with passengers going to fly across the Pacific to Hawaii. It would be a very bad way to learn.”
Eric Olden, at 13:57
People do fly across the Pacific, so the question is how you get from not knowing how to being confident, and the answer is simulation. You learn to take off and land. Then the simulator takes away the right engine and tells you to deal with it, pilot, and if you crash, “You just restart it and put another quarter in the machine, and Bob’s your uncle.” Day one with an agent is the same: get one transaction working and learn how it works. Day two is where it gets useful. “Let’s change the identity system. Let’s change the AI model. Let’s change the MCP application, or the API behind the MCP.” Then you break the agent on purpose: give it too much permission and see what it does, peel back the standing privileges, and ask whether the transaction still works “with a super tightly scoped scope.” Those are the experiments you don’t want to run in production, “because you don’t want to end up paying that $1,000 ticket, or worse.”
He called it a safe-to-fail mindset, and he was candid that the first version of anything is usually pretty rough. Two weeks after the episode Strata shipped the sandbox he was describing, the Maverics Sandbox for Agentic Identity, with identity providers, MCP servers and APIs already wired in and live policy enforcement, which Eric had promised would be free to start, alongside the AI Identity Gateway that enforces least-privilege policies written in OPA’s Rego language and validates proof of possession on tokens. Eric’s line in the release: “AI agents are the new digital workforce, but without identity they’re invisible and ungoverned” (Help Net Security). If your MCP servers are part of what changes on day two, our post on the MCP server that passed review and then changed is the test plan for that engine.
Three missing guardrails of the early web
I asked about the first time Eric tackled an open identity problem, and the answer went back to graduating in San Francisco in the mid-nineties, when people said you couldn’t use the web for anything commercial. “And I was like, yeah, that’s going to last.” The web servers of the day, the early one from the University of Illinois team behind Mosaic and then Netscape’s, were great at serving brochures, “HTML and images,” and what everyone actually wanted was e-commerce. What held it up, he said, was “Another missing guardrail,” and it took three things to fix.
The first was SSL. “The first, give credit to Netscape and Taher Elgamal and his team for inventing SSL,” which is how people became willing to type a credit card number into a browser. Taher was our guest on episode 61, where he explained why SSL made the client certificate optional and the password filled the gap. The second was scale. A web server’s access control lists worked for a couple dozen people, not for e-commerce sites with millions of them.
“And that became the whole realm of web access management. And I was the founder and CTO of a company, Securant, that solved that problem. And we went from two guys in a garage in Berkeley to a three-hundred-person company in a very short amount of time, because the right timing matters.”
Eric Olden, at 17:03
Securant’s product was ClearTrust, which his LinkedIn profile describes as “the first commercial WAM product that managed access and SSO to Web apps” (LinkedIn). RSA Security completed its purchase of Securant on September 5, 2001, for $134.5 million in cash, and Art Coviello’s line in the release could open a talk on agents today: “Authorization plays a critical role in RSA Security’s mission to enable secure e-business” (RSA Security, Form 8-K). The third missing guardrail was trust between strangers, “Which in the late ’90s we called federation.” It needed a standard, and Eric said he was one of the four co-authors of what became SAML. The record matches the number: when the OASIS Security Services Technical Committee first met in January 2001, it took in four contributions, S2ML from Netegrity, AuthXML from Securant, X-TASS from VeriSign and ITML from Jamcracker, and merged them into the Security Assertion Markup Language, an OASIS Standard in November 2002 and, as SAML 2.0, in March 2005 (Wikipedia). “And then here we are, 2025, and I think SAML [is] very much here to stay.”
The pattern is the reason to tell the story. Every wave of computing has had a missing guardrail that held back the commercial version, and each time the fix was identity: who is this, what may they do, and can a stranger trust the answer. Agents are the fourth wave of the same question, and Eric’s mindset has not changed in thirty years: “Because the world will tell you what you can’t do. And I love the mindset of, don’t tell me that. Tell me what you can do.”
Standards first
When people ask Eric how he would solve the agent problem, he said, he doesn’t start with the technical implementation.
“First word was standards. Use standards, because if there’s anything that you want to do quickly, you gotta do it in a standardized way. Otherwise, everything becomes bespoke, and things just don’t move fast enough.”
Eric Olden, at 21:00
This is a distributed computing problem, and the standards already exist: OAuth, OIDC, SAML, PKCE, SPIFFE. Some vendors, he said, talk about solving it with a future version of OAuth or a new app-to-app integration three years from now. “Okay, well, [last] I checked, people are trying to do it today.” He paraphrased Voltaire, “don’t let perfect be the enemy of good,” and the attribution holds up: Voltaire quoted the Italian proverb “Il meglio è l’inimico del bene” in 1770 and put “le mieux est l’ennemi du bien” in his poem La Bégueule (Wikipedia). The good he wants today is OAuth bootstrapped and pushed to the edge, with real implementations of things like DPoP, “and be able to provide ways that you don’t just recycle and replay tokens.” DPoP is RFC 9449, published in September 2023, and its abstract says what he means in one sentence: “This mechanism allows for the detection of replay attacks with access and refresh tokens” (IETF). A token bound to a key the agent holds is useless to anyone who copies it, which is the same lesson as the garage door on episode 29, with a token instead of a radio code.
And then the dream, the small part he hopes to play in all of this: “I’m doing everything I can to get rid of passwords.” That is exactly what Taher said when he was here, I told him, which is a good sign for both of them.
Passkeys, AuthZEN and the best-kept secret in authorization
Sasha asked where the community actually works together on agent security, and Eric named two places. The first was the FIDO Alliance and passkeys.
“I hate passwords. I know they’re cheap-ish, but they’re way more expensive when things go wrong. But I love that, again, some big players got together, Apple and Google and Microsoft, amongst others, and they said, hey, let’s [make] this investment. We all agree on phishing-resistant passkeys.”
Eric Olden, at 23:30
The investment was announced on May 5, 2022, when Apple, Google and Microsoft committed to expanded support for the FIDO standard (Apple), and in October 2026 the FIDO Alliance counted seven billion passkeys (FIDO Alliance). “I just put my face in front of my phone and I log in.” Then he described AuthZEN, which Strata helped build, a way for “different enforcement points by different vendors” to execute the same policy. On the tape he places it in the FIDO Alliance. It is the OpenID Foundation’s AuthZEN Working Group, and its Authorization API 1.0 was approved as a Final Specification on January 12, 2026 (OpenID Foundation). The second place was the Cloud Native Computing Foundation, home of Kubernetes, and of the Open Policy Agent, which graduated there in February 2021 (CNCF). “And I love OPA,” he said, and I couldn’t resist.
“It’s the best-kept little secret in the authZ space ever.”
Jon McLachlan, at 25:10
Anders Eknert of Styra, the company behind OPA, made the case for it on episode 18. Strata contributed Hexa to the same foundation, an open source tool for policy interoperability built around IDQL, the identity query language. If you run Amazon and Azure and want one policy in both, Hexa “will basically point it at system A. It’ll extract all your policies out of A, convert it into IDQL format, and then you point it at system B, and then it pushes it into the syntax of system B.” The CNCF accepted Hexa as a sandbox project in September 2022, with Eric’s line “IDQL and Hexa eliminate identity silos” (Business Wire). Authorization is the layer most companies still build by hand, which is the argument of authorization is the last layer, and the agent era makes that more expensive every month.
The bar at hyperscale
Before Strata, Eric ran Oracle’s security and identity division as senior vice president and general manager of cloud security and identity management, from October 2017 to July 2019 (LinkedIn). That is where he learned what the stakes look like from a hyperscaler’s side of the table.
“And when you’re dealing with millions and millions of customers, and downtime is in the tens of millions of dollars per hour, right? You think about things on a whole nother level.”
Eric Olden, at 26:29
Strata’s Department of Defense work raised the bar again: in certain circumstances, he said, an outage “truly is life or death.” So he trained his whole career to be the person who can say, “I know this will work,” and he said he hopes that isn’t arrogance, “but it’s evidence that makes me confident.” I added the part that only comes from being on the call: you have to be on the on-call rotations and see what happens during an incident before you understand where the bar is, and the word I like to use is bulletproof. I also said I think of innovation as only the parts that are easy to adopt. If it’s not easy to adopt, it won’t get used, and then it’s a nice side project or a cool demo. Eric agreed and pointed at vibe coding. He is encouraged that AI tools let more people build things and solve problems, “because that’s [to] me what software is all about,” and he is clear about the other side of it: “sometimes that expression, I know enough to be dangerous, this is really showing itself.” More people in the game, and the bar kept high on QA, with nothing going straight from a first prompt into production.
The apology at Oracle
Sasha asked for Eric’s proudest moment, and he distilled a couple of years at Oracle into one dinner. He was brought in as a startup guy to make his division build software like a startup, agile and rapid iteration, concepts that were known but not practiced. The division was a bell curve: a few eager people, a skeptical middle, and a tail that said it would never work. In his last month there, after the final training class finished, a man walked up to him at the graduation dinner.
“And he said, Eric, I owe you an apology. And I said, oh, I don’t understand, but tell me more. And he says, well, a year and a half ago, [you] got in front of the whole division, and you said, we’re going to do this crazy hard thing. I said at the time to my friend, I said, this guy is not going to last. He does not know what he’s trying to do. He’s never been at a company like Oracle.”
Eric Olden, at 30:28
He had waited in the last class for the next guy to show up, and he was apologizing because Eric had been right. Eric was quick to say it wasn’t him teaching everything, he had a great team that did it around the world. What he took from it is the lesson for anyone rolling agents out to a skeptical organization: give people a real opportunity, don’t prejudge them, and accept that they may not take it the first time around. “And even the most cynical, skeptical people, through their own doing, can show that this can work.” The agent rollout that wins is the one the skeptics end up running.
Not freezing
I asked for the most difficult day, and Eric answered with a habit instead of a date.
“So the most difficult thing, I think, is not freezing when the inevitable thing happens. And it doesn’t matter where, it could be customer, it could be partner, it could be financing, could be God knows what it’s gonna be, but it’s gonna happen.”
Eric Olden, at 33:52
He learned to meditate a lot as a founder, because in your head you can slay dragons that would beat you if you tried to fight them for real. Be ready for change, flow through it, hope for the best, “Get back on that horse. It’s not a failure until you stop trying.” When I asked about legacy, he talked about his two kids and the people he has worked with for ten, twenty and thirty years, and a friend who once called him pathologically optimistic, which I told him is the biggest compliment you can receive. Then a run of founder proverbs: “you go to war with the army that you got,” Mike Tyson’s line about everyone having a plan until they get punched in the mouth, and the one I would put on Strata’s wall, “Don’t tell me what you can’t do. Find a way, or make one.” His words of wisdom at the end were three short ones: go standards, please don’t reinvent the wheel, “And then, don’t test in production.” I added the corollary. Don’t test on my airplane.
What happened after the tape
The year after we hit stop is the strongest evidence for everything Eric said. Strata had already introduced Identity Orchestration for AI Agents in July 2025, with short-lived scoped credentials, policy-as-code authorization with human approval for sensitive actions, just-in-time provisioning of agent identities and an MCP-native proxy, and Eric’s framing was that “Autonomous AI agents now act as users in enterprise systems” (Help Net Security). His keynote from that launch is the best twenty-nine minutes on the subject, and it starts with trying to buy concert tickets through ChatGPT.
Two days after this episode, Outshift by Cisco posted his panel on multi-agent foundations, agent discovery, identity and trust, and on November 19 Strata shipped the AI Identity Gateway and the free sandbox. In February 2026 the survey with the Cloud Security Alliance put numbers on the lack of confidence he described, with his line “The future of identity must be orchestrated, contextual, and continuous” (Cloud Security Alliance).
Then, on June 9, 2026, at its FORWARD conference in Las Vegas, Rubrik announced that it had acquired Strata. Terms were not disclosed, and the release built Rubrik’s new Identity Continuity capability on Strata’s technology, failing over to a secondary identity provider when the primary one is compromised or goes down, alongside Identity Roll Forward for Active Directory, with Rubrik’s chief business officer Mike Tornincasa saying “Strata closes the most critical gap: Identity Continuity” (Rubrik). Eric is now Rubrik’s chief identity architect (LinkedIn), and a little over two weeks after the announcement he wrote the thank-you note every founder hopes to write one day.
Now that Strata is officially part of Rubrik, I want to thank the people who got us here.
Eric Olden · June 26, 2026
When Topher Marie, Eric Leach and I started Strata in 2019, identity was stuck. Every enterprise we met was locked into a single provider, unable to move, unable to modernize, and exposed the moment that provider went down. We set out to fix that.
Over nearly seven years the team built three things that did not exist before. Identity orchestration, so enterprises could run any identity provider and switch on their own terms. Identity continuity and failover, so identity keeps working even when a provider does not. And in 2025, we shipped the world’s first agentic identity platform, built for a world where millions of AI agents need a control plane to stay under more predictable human control.
“a control plane to stay under more predictable human control” is the lesson of the fifteen-minute story, a year and a half later, with a company sold on it.
Did you catch it? A six-question quiz
Interactive · six questions
Did you catch it?
Six questions on what Eric actually said, with the timestamp for each answer. The answers are under each question for anyone reading without JavaScript.
-
A design partner let an agent spend money in a test environment. How long did the money last?
- About a day
- Less than 15 minutes
- Until someone noticed at the end of the month
b. "in less than 15 minutes, all the money was spent." Transcript, 3:04.
-
How does Eric say you control an AI agent?
- Put a straitjacket on the model
- Write a stricter system prompt
- Indirectly, through a control plane that takes away the ability to press the API button
c. "You do it indirectly, through a control plane." Only the model builder could put a straitjacket on the model. Transcript, 3:04.
-
What are the three components of Strata's audit model for agents?
- Intent, context and outcome
- Who, what and when
- Request, response and latency
a. Intent, generally the prompt, context, the attributes and facts, and the outcome. "we never really had to ask and record intent." Transcript, 6:18.
-
What does Eric call his decoder ring for agent identity?
- Zero trust for machines
- Agents are people too
- Non-human identity first
b. "It's not a non-human identity problem. It's the same thing as treating a human." Transcript, 8:09.
-
How should anybody get confident about an agent before production, in Eric's words?
- Read the model card
- Wait for a vendor's certification
- See it, do it, and then do it five more times
c. "see it, do it, and then do it five more times," and look for predictable responses and outcomes. Transcript, 10:09.
-
What was the name of the company Eric founded in a garage in Berkeley?
- Securant
- Symplified
- Netegrity
a. "I was the founder and CTO of a company, Securant." RSA Security bought it in 2001. Transcript, 17:03.
Is your agent ready for production?
Eric’s checklist is the one an enterprise already uses for any software, plus the parts agents change. This scorecard asks eight questions about one agent you are about to ship, in the order the episode raises them, and tells you whether it belongs in the sandbox, in a pilot, or on the plane with passengers.
Scorecard · eight questions
Is your agent ready for production?
Answer for the agent with the most power: the one that can spend money, change data or touch production. Answer for how it runs today, not the design document.
-
Does the agent have its own identity, separate from the person who launched it and from every other agent?
- It uses a shared service account or one API key
- It has its own credential, but nobody owns it
- Its own identity, a named owner, and an entry in a registry
"agents are people too." A person gets a badge, a manager and a role. So does an agent. Your agents share one API key is the afternoon fix.
-
When the agent tries to spend money, delete data or change production, what actually stops it?
- The prompt, or the model's own judgment
- A person approves some actions by hand
- A policy outside the model decides every action, with limits
The straitjacket goes on the model, which you don't own. The control plane goes on the API button, which you do. An agent saying no is not authorization.
-
What credential does the agent hold while it works?
- A long-lived key, or a person's password
- Tokens that expire, but broadly scoped
- Short-lived tokens scoped to the task and bound to the agent, so a copied one fails
"you don't just recycle and replay tokens." DPoP, RFC 9449, binds a token to a key the agent holds.
-
What does your log record for each action the agent takes?
- Nothing beyond the downstream API's own logs
- Which agent called what, and when
- Intent, context and outcome, traceable to a person or system
Only 28 percent of the organizations in the 2026 survey could trace an agent's action to a human or system everywhere.
-
Where was the agent tested before it reached real users?
- In production, by its first users
- In a demo that worked once
- In a sandbox, with the model, identity system and tools swapped, and privileges peeled back
Day two is the useful day: change the identity system, the model, the MCP server, then give it too much permission and watch. The MCP server that changed is one engine to fail on purpose.
-
How many times have you run the same task and checked the outcome?
- Once
- A few times, and the results varied
- Six times or more, with predictable responses and outcomes, and more for anything that spends money
"see it, do it, and then do it five more times." The calculator below shows why the riskiest actions need more than six.
-
Does a named person sign off before the agent goes to production, against a written checklist?
- No
- Informally
- Yes, against the same checklist as any release, plus the agent items
"someone signs off on it." Blue-green, smoke tests, and now the audit fields and the scopes. Who signs for the agent is the one-page map.
-
When the project ends or the owner leaves, what happens to the agent and its credentials?
- It keeps running with its keys
- Someone remembers to turn it off, eventually
- Its life cycle is governed: reviewed, re-scoped, and retired with its credentials
The governance face of the die. The agent offboarding checklist is the exit interview.
The agent shares an identity, the only thing between it and the money is its own judgment, and nobody could reconstruct why it did what it did. That is the fifteen-minute test waiting to happen in production. Give it its own identity and owner this week, then put one policy in front of the one API that spends money or deletes data.
The agent has its own identity and some limits, but the tokens live too long, the log records the call and not the intent, or the testing stopped at a demo. Bind the tokens, add intent, context and outcome to the log, and run day two in the simulator: swap the model, swap the identity provider, over-permission it and watch.
Its own identity, a policy outside the model on every consequential action, short-lived bound tokens, an audit record a reviewer can read, repeated runs with predictable outcomes, a signature and a retirement plan. What is left is to keep it that way: rerun the simulator whenever the model, the tools or the scopes change.
How many runs before you sign off?
“See it, do it, and then do it five more times” is the right habit, and six runs is a small sample. This calculator shows what six clean runs can and cannot tell you about a failure that only shows up some of the time, how many clean runs it takes to be confident the failure rate is low, and what that rate means at production volume. Change the numbers to your agent’s.
Calculator
How many runs before you sign off?
An agent that does the wrong thing on some fraction of runs, a number of clean-looking sandbox runs before sign-off, and the volume and stakes in production.
If a run goes wrong 5 percent of the time, six sandbox runs catch it about 26 percent of the time and all look fine about 74 percent of the time. To be 95 percent sure the rate is below 5 percent, you need about 60 clean runs in a row. In production at 1,000 runs a day, the same rate is 50 wrong runs a day, which is why the limit has to live in a policy outside the model, not in the hope that testing found everything.
The 95 percent figure uses the statistician's rule of three: after n runs with no failures, the failure rate is below about 3/n with 95 percent confidence. It assumes the runs are independent and the sandbox matches production, which is the reason Eric's day two swaps the model, the identity system and the tools. The 5 percent, 1,000 runs and $50 are placeholders, not research.
Thirty years of identity on one line
Everything on this page sits on one timeline, from the browser that made the web visual to the week this page was written. Filter it by thread.
Timeline · 1995 to 2026
Thirty years of identity on one line
Filter by thread. Dates are from the public record except where marked as Eric's own account on the tape.
- SSL 2.0 ships
In Netscape Navigator, the first guardrail of e-commerce (Wikipedia).
- Securant Technologies
Founded while a student at UC Berkeley. ClearTrust, web access management (LinkedIn).
- Two guys in a garage in Berkeley
To a three-hundred-person company, "because the right timing matters."
- Four proposals, one standard
S2ML, Securant's AuthXML, X-TASS and ITML go into the OASIS committee that writes SAML (Wikipedia).
- RSA Security buys Securant
For $134.5 million in cash (SEC).
- SAML 1.0
An OASIS Standard, with SAML 2.0 following in March 2005 (Wikipedia).
- Symplified
Identity as a cloud service, from Boulder, through June 2012 (LinkedIn).
- Oracle
Senior vice president and general manager, cloud security and identity management, through July 2019 (LinkedIn).
- Strata Identity
With Topher Marie and Eric Leach, in Boulder: identity orchestration (Wikipedia).
- OPA graduates
The Open Policy Agent becomes a graduated CNCF project (CNCF).
- Passkeys, together
Apple, Google and Microsoft commit to expanded support for the FIDO standard (Apple).
- Hexa joins the CNCF
As a sandbox project, with IDQL. "IDQL and Hexa eliminate identity silos" (Business Wire).
- $26 million Series B
Led by Telstra Ventures, $42 million raised in all, and a 380 percent increase in revenue in a year (SecurityWeek).
- DPoP
RFC 9449 lets a server detect replayed access and refresh tokens (IETF).
- Moffatt v. Air Canada
The chatbot is not "a separate legal entity," and the airline pays $812 (CBC News).
- Fifteen minutes
A design partner's test agent spends all the money, by Eric's account.
- Gartner on agentic AI
Over 40 percent of projects canceled by the end of 2027 (The Star).
- Identity orchestration for AI agents
Eric's keynote and the launch: scoped credentials, policy as code, an MCP-native proxy (Help Net Security).
- Episode 81 airs
This conversation. "agents are people too."
- Multi-agent foundations
Eric on an Outshift by Cisco panel on agent discovery, identity and trust (YouTube).
- The AI Identity Gateway and a free sandbox
Runtime control for agent actions, and the simulator Eric described (Help Net Security).
- AuthZEN 1.0
The OpenID Foundation approves the Authorization API as a Final Specification (OpenID Foundation).
- The time-to-trust survey
285 professionals: 18 percent highly confident, 84 percent doubt an agent audit (Cloud Security Alliance).
- Rubrik acquires Strata
At FORWARD in Las Vegas, for identity continuity and resilience (Rubrik).
- Chief identity architect
Eric joins Rubrik and thanks the people who got Strata there (LinkedIn).
- Seven billion passkeys
The FIDO Alliance's count, four years after the joint commitment (FIDO Alliance).
What to do this week
Four things, in the order Eric would do them.
- Give the agent its own identity and an owner. List every agent and the credential it uses. Anything on a shared service account or a person’s key gets its own identity, a named owner and an entry in a register. The 30-minute credential inventory is the first pass.
- Put a policy in front of the button. Find the one API the agent can call that spends money, deletes data or changes production, and put a policy outside the model between them, with a limit and short-lived, scoped tokens. Don’t rely on the prompt.
- Log intent, context and outcome. For every consequential action, record the prompt, the attributes and facts it used, and what actually happened, traceable to a person or system.
- Build the simulator before the flight. Run the task in a sandbox until the outcomes are predictable. Then swap the model, swap the identity provider, change the MCP server, over-permission the agent and watch. Sign off only after that, against a written checklist.