Most of the AI projects that disappoint people were decided before anyone looked at the plumbing. Jacob Andra and Stephen Karafiath built a consulting firm around looking at the plumbing first.
“And often, these two circles don’t overlap. They’re two different skill sets, and we sit right at the intersection to kind of bring both to bear to help companies modernize and adopt technology.”
Jacob Andra, at 1:59
Jacob is the chief executive of Talbot West, which he started in Salt Lake City in April 2024, and Stephen, who joined him that September, leads its solutions architecture practice as senior vice president of AI solutions architecture (Jacob on LinkedIn, Stephen on LinkedIn). Stephen spent almost 17 years at Oracle, from 2006 to 2023, the last four as director of its Developer Innovations Team. Jacob ran a paint and plaster company for 11 years, then marketing at a string of companies, then his own content firm, before Talbot West. The firm sells no software: Jacob founded it, in its own words, with “no software to resell and no platform commissions to collect” (Talbot West). Both sit on the AI advisory board of 47G, the Utah-based aerospace, defense and hard tech industry group (47G), and Jacob hosts a podcast of his own, The Applied AI Podcast (appliedaipod.com).
Where we sit, so you can weigh what follows: YSecurity has no business relationship with Talbot West that I know of. We work on the same question from the other side. They map an organization before it picks its AI, and we secure what it picks, from the governance program to the compliance work that regulated buyers ask for. Sasha and I hosted. It aired on April 21, 2026, and the tape dates itself to the second half of 2025: I mention Windsurf’s sale as something that “just happened,” and Cognition signed its deal for Windsurf in July 2025 (Cognition), and Jacob cites the MIT report on AI returns, which came out that summer.
Two circles that rarely overlap
Jacob’s description of the firm on the tape is short: “Talbot West is an AI-first digital transformation.” Then he draws it. One circle is traditional business process consulting, “deep understanding of business systems and processes.” The other is AI expertise. In his experience the two rarely sit in the same firm, and Talbot West’s whole pitch is the overlap.
The firm’s own pages say the same thing more formally today. Its about page describes senior expertise across strategy, architecture, engineering, AI, data, governance and operations, and says the firm picks the mechanism after it defines the outcome: “We do not resell software or depend on a particular technology being selected” (Talbot West). AI is one valid answer on that page. So are conventional software, process redesign, an existing platform, delay and, in its words, non-consumption. The name comes from the Talbot, a hunting hound known for holding a scent across rough ground, and the West stands for working at the frontier, where technology changes faster than investment cycles.
Stephen added one line before he handed the question back: “I mean, I think that’s a really good kind of overview of the bridge between the, uh, technical and the business.” Then he turned the interview around and asked about us, which is how I ended up describing YSecurity as a team that steps in “really early in a company’s life cycle before, you know, usually you even hire your first your first security person.” Stephen saw the niche from his Oracle years, where security came with million-dollar contracts, and from startups, where it often doesn’t:
“Then moving into the startup world, it’s like nobody thinks about security because we don’t have the budget for it and everything’s shoestring architecture and all of that. And then, you know, decisions that could have been made early aren’t made until later and it’s very painful.”
Stephen Karafiath, at 3:21
Jacob’s reply was four words, “Might as well bake the best practices in early, right?” Mine was that the trick is focus: “Can you focus on how security can be part of your go-to-market engine and really use it as a market distinguisher, which leads to amazing outcomes like closing deals faster, closing deals in new regulated markets, maybe even leading to more attractive M&A terms.” Then Sasha joined us, a little late and right on time for the first real question.
Friends first, co-founders later
Plenty of founders who listen to the show ask us how to find a co-founder, so I asked how these two found each other. Jacob’s answer is the one most people don’t want to hear, because it can’t be hurried:
“We’ve been friends for a long time on a personal level before ever doing business together. And Steve’s watched me build other businesses.”
Jacob Andra, at 11:36
The record behind “other businesses” is long. Jacob owned and ran TerraFirma Paint & Plaster from 2003 to 2014, then ran marketing at Advanced Composites, U.S. Translation Company and One Click Retail, was chief marketing officer of SOMA in Helsinki, and from 2019 ran CustomerFaucet.com, a copywriting and site architecture firm, before starting Talbot West in April 2024 (LinkedIn). He credits Stephen “hugely for even the launch of Talbot West,” and says he eventually “poached” him as a co-founder.
Stephen’s side of the story starts at the top of a big company. He had worked his way through Oracle “from starting as a developer into consulting into product development, finally landing in sales engineering,” and ran the Developer Innovations Team, which built minimum viable products for Fortune 500 customers without having to carry them into production. “It felt like I’d made it,” he said. What moved him was watching the company’s pace on AI, and the experience of a startup afterwards:
“And then trying out the startup ecosystem, like you guys are allergic to. One of my friends who went and managed a big team at Google and some of my other friends from Oracle. We kind of banded together and made some pretty incredible strides even before ChatGPT 3, on some RAG-based architectures when it was new, but it quickly became apparent: wait, like, SaaS isn’t where it’s at anymore.”
Stephen Karafiath, at 13:07
His LinkedIn shows the startup years: he was vice president of AI solution engineering at Morfius from April 2023 to October 2024, and before Oracle he led a developer team at Global Exchange Services from 2002 (LinkedIn). The models to scale what they built, he said, kept turning into bespoke consulting for the big customers they were selling to, which was exactly the work Jacob was doing. He called it “a hand in glove fit,” and he described his co-founder as someone who can “intuitively load a huge amount of context in his head, almost like an AI.”
Two of their own conversations are worth watching next to ours. On the first episode of The Applied AI Podcast, Jacob and Stephen talked through how the firm approaches digital transformation, and a few months later they went deep on large language models and what they keep getting wrong.
We are not security experts
I asked whether they think of themselves as security people. Jacob didn’t hedge: “We are not security experts, so we do love to bring in people like yourselves for specific security stuff.” Stephen’s answer was more interesting, because it described a career of security as a default setting rather than a job title:
“Yeah, I think to answer your question, having spent decades at Oracle, secure by default, encrypting data at rest, data [in] transport, all of that, those concerns were always paramount. So I’ve always been security adjacent. I’ve run security related teams, but it’s never been my focus.”
Stephen Karafiath, at 5:20
He had spent most of his career steering clear of public sector work and federal security rules, he said, until Jacob pulled him into a digital transformation for a company that does “a lot of high-tech aerospace contracting to the DoD.” That meant dusting off what he knew about Level 2 of the Cybersecurity Maturity Model Certification (CMMC), the Defense Department’s program for contractors that handle federal contract information or controlled unclassified information, and about secure enclaves and cloud regions authorized under FedRAMP, the federal program for cloud services. His own conclusion was the one we’d give any client: “I would prefer to let experts handle it the same way as I could do my own taxes, but I think security expert or tax expert will do it better than I would.”
That’s a healthy instinct, and it’s also the reason this conversation belongs on a security show. The decisions Talbot West maps are the ones that decide what security work a company will need later.
Forking pathways
With the aerospace client, Jacob said, the job isn’t to solve the security questions. It’s to show where they sit:
“We’re trying to shine a strong spotlight on the decision-making pathways, the dependencies and which ways those dependencies flow and how all of this stuff ties together because it it turns into such a tangled rat’s nest.”
Jacob Andra, at 6:28
One of their recommendations was to bring in “a firm that’s an expert in CMMC 2 compliance and security issues.” The rest was a map of forks: many of the company’s decisions, he said, would be “gated by whether they’re going to do a GovCloud or on-prem,” and Talbot West’s role was to show those forks, not to answer them. Stephen had described the same fork from the other end. The client might have been saving money with on-prem deployments, “but really they need like a FedRAMP-certified GovCloud.”
This is the cheapest kind of security work there is, and the one most companies skip. Once the hosting decision is made, half of the compliance plan is made with it: which tools are allowed, which vendors can be in the loop, which audit evidence you’ll need and who can see the data. A map that shows the forks before the commitment hardens is worth more than a remediation plan after it. It’s the same reason we tell defense startups that compliance is both the barrier to entry and the moat, in our post on building a defense tech startup.
Sasha asked the follow-up every regulated company asks, how a modern technology stack helps with the moats in organizations that have been around a long time. Jacob handed it to Stephen (“Why don’t you take that one, Steve?”), and Stephen’s answer was that the news is better than it used to be. So much now comes “on-demand as a service,” already certified in a FedRAMP environment, that a company can outsource more of the legwork, “as long as you’re careful to not run afoul of the regulations.” His general rule: “instead of trying to reinvent the wheel, go out and figure out the best practices, see what other companies have done, see what other offerings are new to the market.”
One or two degrees
Jacob took the other half of Sasha’s question, the inefficiency inside big organizations. “The bigger the organization, the more inefficiency,” he said, government and private sector alike, and no one gets it to zero. The point is that at that size you don’t need to:
“On a large organization, if you can even move the needle one or two degrees, you’re talking massive impact on bottom line and profitability and all of that.”
Jacob Andra, at 9:02
What he described next is the method, and it’s the heart of the episode. Talbot West looks “in a very holistic system of systems way, not just plugging an isolated solution in and thinking that’s going to solve it.”
“Because again, back to this idea of dependencies, you’ve got to map it. You’ve got to showcase the impact of how this depends on that. You got to have your data sources right, address those at the root level and all of this.”
Jacob Andra, at 9:26
The order matters. Most of the time goes into the evaluation, and only “once we have clear pathways established and can show ROI” does a specific initiative go ahead, one that solves a specific pain point “now that we understand how it relates to all these other things.” It’s the discipline the MIT numbers reward, and we wrote it up at length in why 95% of enterprise AI projects fail. Talbot West has since published its own version, a guide to the dependencies that have to be true before an AI initiative can work (Talbot West).
If it ain’t broke
I asked what inefficiency they run into first, almost like clockwork, and whether AI had changed it. Stephen’s answer spanned his whole career. The biggest inefficiencies “as, um, organizations tend to get larger are the growing pains,” from emailing a spreadsheet back and forth, to a first SaaS-based CRM, to outgrowing that. Then the legacy processes nobody updated and “the technical debt that has come from previous implementations.” The older the company, the bigger the pile:
“I mean, when we work with some of these large telecoms that have been around for like 70 years, it’s like their internal system is like a web portal, the best technology that like 1992 had to offer.”
Stephen Karafiath, at 10:37
The reason it survives is human, not technical, and it’s the line from this episode I hear in most security engagements too:
“Like if it ain’t broke, don’t fix it. Even if it’s taking literally 100x the effort that like some of the modern AI systems that Jacob is helping these guys realize they need, there doesn’t seem to be that motivation there until you really feel the pain.”
Stephen Karafiath, at 11:07
In security the pain usually arrives as an incident or a lost deal. The cheaper way to feel it is on purpose, before either: a penetration test that breaks the old portal while nobody’s depending on it being whole, or an enterprise security questionnaire answered honestly in a draft before it’s answered for real.
One of eight shapes
Sasha asked how they decide which pain point deserves a solution when every client sees the world from a different angle. Stephen’s answer comes from decades of Fortune 500 meetings at Oracle and the consulting since:
“Every customer, especially these big guys, they all think that they’re a unique snowflake with their pain points. They’re like, oh, you need to understand me.”
Stephen Karafiath, at 17:36
Once you’ve seen a hundred companies, he said, the hundred and first arrives with its unique pain points, and some really are unique. “But a lot of these things that you think are unique to you are not actually unique to you. You’re like literally like one of eight different shapes.”
We see the same thing in security programs, which is why a startup’s first SOC 2 doesn’t have to be invented from scratch. The shapes repeat. What doesn’t repeat is the detail of a company’s data, vendors and people, and that’s what the sitting down is for. Stephen said this kind of deep snapshot of a customer “would not have been possible before AI came out,” and handed it to Jacob to explain the system they built for it.
APEX: three rounds to one first move
Jacob’s clients, he said, often have “500 different directions they want to run,” initiatives they’ve started around AI and ones that have stalled out. It’s overwhelming, and the answer Talbot West built is a method with a name:
“It’s called APEX, that stands for AI Prioritization and Execution. And it essentially maps, it maps their entire business, all their systems and processes, and then it maps all the ways AI could be applied within that ecosystem. And it prioritizes them against five key metrics.”
Jacob Andra, at 18:55
On the tape he named three of the five, highest return on investment, technical feasibility and cost, and pointed to the website for the rest. Talbot West’s published version lists the five dimensions as stakeholder urgency, revenue impact, technical feasibility, implementation complexity and strategic alignment, scored through three rounds of competitive elimination, and says the first round is the most important because it removes the exciting ideas that score poorly on feasibility or alignment (Talbot West). The rounds work like a tournament:
“So one round is more of a shallow dive across all these, ranking them across these five criteria. Then there’s kind of almost a semi-final round where you have just a small handful and out of that there’s a clear winner and you do a really deep dive on that and provide a pretty comprehensive roadmap on what that one would look like to implement, what it would drive in ROI and all that. And it’s very obvious where you should start.”
Jacob Andra, at 19:25
The part I’d steal is what happens to the runners-up. After the winner, there’s “your second-round bench,” and the candidates are “tied together in a kind of stepping-stone format.” Jacob made the same point on LinkedIn in August 2026, in a sentence worth pinning above any AI roadmap.
The highest-ROI investment may not be the best first one. Every company feels the pull to modernize. Run a model showing the ROI of various initiatives and pick the highest-scoring one, right? Wrong. Projects aren't independent. Some unlock others.
Jacob Andra · August 18, 2026
Security work sequences the same way. The first control you put in should be the one that makes the next three cheaper: an identity provider before access reviews, an inventory before shadow AI detection, a logging pipeline before anyone promises an auditor evidence.
The 2030 thesis
The stepping stones lead somewhere, Jacob said, and the firm has a name for where:
“We actually have a thesis of where we think this is going. It’s called our 2030 thesis. And we think that the industry and businesses are moving toward more of an integrated total organizational intelligence, and that in the future that’ll drive tremendous competitive advantage.”
Jacob Andra, at 20:02
Today’s landscape, in his words, is “very siloed and very fragmented.” He thinks competitive pressure and technology “make that an inevitability,” so each project should be a step toward it, and the steps start “bolting together” over a few years. He sees Talbot West as the quarterback. When I put it back to him as a field that’s changing, with AI driving it, he widened it: “it’s not just AI, it’s a lot of other things,” data orchestration layers among them, and the exact year matters less than direction. “The point is companies that work with us are going to be much closer to that through working with us than they would have been on their own.”
Stephen’s LinkedIn describes the same idea as “a central nervous system for your company,” a system of systems with subsystems in every role and department (LinkedIn). Talbot West’s architecture discipline for the multi-part systems underneath it is Cognitive Hive AI, or CHAI, which organizes different AI components into decision structures with explicit interfaces and authority rules. Its own summary of the goal is a good one: “The point is not to assemble more agents” (Talbot West).
Seen from security, total organizational intelligence is also a total data map, and that cuts both ways. The orchestration layer that lets insight flow across the company is the layer an attacker would most like to reach, so it needs its own identity, its own permissions and its own logs from the first stepping stone, not the last.
A skeleton, then meat on the bones
Stephen said they’re already running a preview of that future, on their own customers. Talbot West builds an internal intelligence source on each client, fed by meeting transcripts and technical documentation, that the team can query in natural language. The design was Jacob’s idea, he said, and it’s a clean picture of how they combine the two kinds of AI:
“What are the natural life cycles of most companies? So whether that’s everything from quote to cash or like what is the business process involved, let’s map that whole thing out and let’s stand that up like a skeleton, and then we can use neurosymbolic AI to create a knowledge graph, to create a framework that defines the attributes that we want to know about this company. And then we can use the latest LLM technology that understands natural language to go through that piece by piece and kind of start putting meat on those bones.”
Stephen Karafiath, at 22:42
Most companies, he said, probably won’t reach total organizational intelligence “for a couple of years,” but this is “eating our own dog food” today. It’s also a useful pattern for anyone building on customer data: the structure decides what the model is allowed to fill in, and every attribute it fills has a place, a source and an owner. If those transcripts include other companies’ confidential information, as consulting transcripts do, the structure is also where you decide who can query what.
Why AI projects stall
Sasha asked the question that sits behind a lot of AI budgets: how do you keep adoption going after the excitement wears off, when the old technology stack quietly wins again? Jacob answered in two parts. The first is scoping, and the MIT report:
“So this whole MIT report where only ninety, you know, ninety-five percent of AI projects fail to see any meaningful ROI. I mean, we think that that’s largely driven by improper scoping and lack of understanding the dependencies and all of the things required for a technical implementation to succeed.”
Jacob Andra, at 24:05
He means the report from MIT’s NANDA initiative, The GenAI Divide: State of AI in Business 2025, which found that about 95% of organizations were getting no measurable return on generative AI (MIT NANDA). His reading is that most of those projects were trying to “shallowly fit a solution in without understanding the landscape.” The second part is people: “Are you properly training the people, doing your change management and all of that?” Talbot West’s own postmortems point the same way. In its guide to why AI initiatives fail, the decision that sank a project is usually made early, by the wrong person, and never revisited (Talbot West).
Jacob also said Talbot West sees “a much higher success rate than that with our clients.” That’s their claim, and the record doesn’t let me check it either way. What I can say is that the two causes he named are the ones we can see from the security side too. A pilot with no owner for its data and no training for the people downstream fails the same way whether the risk is a bad answer or a leak.
Failures that don’t scale away
The second half of Jacob’s answer is the one the episode title points at. There’s “such a frenzy right now over large language models and a fundamental misunderstanding of what their actual capabilities are and aren’t,” he said, and he’s cataloguing the ways they fail:
“And these ways are persistent across releases, leading me to believe that it’s fundamentally structural to the way that these neural networks function. It’s not something that they’re going to throw more compute, more scaling laws at and outgrow it.”
Jacob Andra, at 25:00
He said he planned to publish a paper mapping the categories of error, logical and otherwise. I haven’t found a paper under that description on Talbot West’s site yet. The nearest public version is the conversation he and Stephen recorded for their own podcast in October 2025, whose description lists formatting errors that swing between extremes, logical inconsistencies across large bodies of text, fabricated citations, and habits no prompt seems to remove, including the em dash, which is why you won’t find one on this page.
The practical consequence is a scoping question, asked before any tool is chosen. “Is this a task for a large language model? Is this a task for some type of machine learning solution? Is this an ensemble job where it’s complex enough, you decompose it into subtasks and you bring in multiple different capabilities and kind of orchestrate them together?” And Jacob’s patience with the alternative is short: when people “conflate large language model with AI” and claim it can do everything, “I just roll my eyes, you know, like that’s not serving clients, that’s not steering them properly.”
Talbot West’s company page put the same view in a better metaphor in July 2026.
Large language models are not all of "AI" any more than electric scooters are representative of "transportation." They're getting outsize attention because we love to anthropomorphize and a machine that sounds like us plays on that tendency.
Talbot West · July 20, 2026
A dot-com moment
Stephen picked up the hype thread with the view of someone who has been through one bubble already. He credits Jacob with “a middle of the road approach. Not overly pessimistic, not overly optimistic,” and then he made a historical call:
“And to answer your initial question, my own personal view is we’re probably in the equivalent of the dot-com boom and bust of the late 90s. I think especially around LLMs, I’m seeing those same patterns coming up.”
Stephen Karafiath, at 26:32
For scale, the Nasdaq Composite rose about 600% from 1995 to its peak on March 10, 2000, and had fallen 78% from that peak by October 2002 (Wikipedia). The internet survived the bust, and so did most of what it made possible. That’s the shape of Stephen’s argument too: a correction in the investment, not in the technology.
He admitted he’d been a believer. “I even drank into the Kool-Aid,” he said, once the Transformer paper from Google came out in 2017 and translation got good almost overnight. That paper, “Attention Is All You Need,” by eight researchers at Google, introduced the architecture behind today’s large language models (Wikipedia). “They weren’t just passing the Turing test, they were obliterating the Turing test,” he said, and for a while, through GPT-2, GPT-3 and 3.5, it looked like scaling might simply continue like Moore’s Law. Then the balloon deflated, at least for him: “The idea that LLMs specifically are going to be this artificial general intelligence, I think that balloon has been deflated for me at least. Jacob’s helped me see that as well.”
The more useful half of his answer is what people are missing while they stare at chatbots:
“You know, the stuff that I did historically with Oracle back when we did fraud detection for major credit cards, or when we’re doing predictive sales analytics. You know, we weren’t using any neural networks for any of that, but it was about like mapping with logic, you know, a lot of different interdependencies and making predictions based on that and like Bayesian inference algorithms. All of that stuff is ripe to be combined with the natural language of LLMs to be bigger than the sum of its parts.”
Stephen Karafiath, at 28:04
He wouldn’t be surprised by “a bust in the investment into just scaling LLMs and data centers,” and he’d welcome a pause. “Jacob and I could use a breath of fresh air if the pace of AI development just stagnated for let’s say five years, awesome.” His reason is practical: when the landscape shifts every month, it’s harder for companies to plan or start implementing. That’s true for security programs too. A control written for last quarter’s model integration is often already describing the wrong thing.
The flywheel, in theory
I pushed back with the strongest version of the other side: AI as a flywheel that keeps increasing its own velocity. Stephen said he subscribes to it in theory, a machine smart enough to improve itself, given unlimited resources, could run away, and “we can start talking singularity, basilisk, all of that stuff.” Then he described the practical thing in front of us:
“Then I look at the practical thing in front of us, which is the LLM, which is like an autocomplete engine of like, what’s the next word that might sound like an answer?”
Stephen Karafiath, at 30:07
He’s impressed by “how close [these] things that aren’t actually answers come to being answers,” and Sasha finished his sentence for him: “When it’s wrong.” Stephen agreed and kept going. “Like its hallucinations are convincing.” When he and Jacob push the models with long context and many things to hold at once, “the cracks around the edges and the little hallucinations turn into chasms.” If there’s another discovery on the scale of the Transformer paper, he said, everything is back on the table. Until then he’s “less and less convinced and/or worried that LLMs are going to be the thing to get us there.”
Wrangling a drunk junior developer
The best evidence Stephen offered was his own work. He writes code with AI tools every day, and his description is the one I’ll be quoting for a while:
“And it plays out like even in my own coding, like using Claude Code or using Cursor, like these things are impressive. I can get 10 hours worth of coding done in one hour. I still have to play senior [developer]. It’s like wrangling a drunk junior developer, but, like, it can do work really quickly and if it’s not the wrong work and if it didn’t hallucinate, then it’s amazing.”
Stephen Karafiath, at 31:07
The fast part and the senior developer part belong together. A coding agent is software that does what its prompts, tools and permissions let it do, so the useful question is what it can touch while it works. The more semi-complicated the task, Stephen said, the more often “it falls on its face,” and he wasn’t worried yet about it writing better than human code and improving itself. “But let’s get on again in a year and maybe I’ll have updated my prediction.”
For a security team, “drunk junior developer” translates into a short list. Give the agent its own identity, read access by default, write access to a branch rather than to production, and a review step a person owns. Jacob wrote the operations version of the same rule on LinkedIn in August 2026, about AI tools that write back to legacy systems:
When modernizing infrastructure, be careful about granting AI tools write permissions to legacy systems. When new software writes back to core databases without proper validation, small definition conflicts can stall production lines or corrupt inventory states.
Jacob Andra · August 13, 2026
The guide behind that post sorts integrations into three tiers by blast radius: read-only advisory tools with zero write access, human-in-the-loop execution where the system prepares a change for an operator to approve, and automated write-back to core systems, which it rates as severe risk because errors compound before a person can step in (Talbot West). Between the AI layer and the systems of record it puts a validation and rollback boundary, with one rule: “Capability layers must never bypass the validation gate to reach systems of record.”
That’s close to the ladder we use with clients, read, propose, write behind approval, then autonomous write inside a declared reversible scope, which we laid out in are you actually going to give the agent write access? and in the blast radius playbook. The value of seeing it from an operations firm is the failure example: an agent that writes a routing change with a stale part number doesn’t look like a security incident, and it can still stop a production line.
A looseness that is both its strength and its undoing
Jacob gave the cleanest description of the problem I’ve heard on this show:
“There seems to be something inherent to the neural network archetype that is very kind of loose in its logical structure. And that’s the best way I can describe it: a looseness that is both its strength and its undoing.”
Jacob Andra, at 31:50
The looseness is what lets the model generalize and make connections that “rigid and brittle machine learning or AI technologies wouldn’t be able to do.” It’s also why “it’s not very anchored or tethered to reality.” The path forward he described is the one Stephen had mentioned earlier, neurosymbolic AI: “certain rigid ontological structures that anchor the neural network to a sense of reality,” while the network keeps its flexibility for the parts it does well. Talbot West’s longer write-up is careful not to oversell it. Its opening line reads, “Neurosymbolic AI is one approach to constraining generative AI and rendering it more reliable,” and it says the approach is valid for some use cases and not others, a design decision rather than a company-wide choice (Talbot West). Our own explainer of the architecture is neuro-symbolic AI: why enterprises need more than large language models.
The security reading is direct. Anywhere a decision has to be right and explainable, an access decision above all, the answer should come from something that can show its work, a policy engine or a rule, with the language model translating at the edges.
The easiest win
I asked where a company new to AI should start. Jacob gave two answers, and he was honest about the first: “One is hire Talbot West to do our APEX process and we’ll tell you your easiest win.” The second is free:
“But short of that, you know, just start playing with ChatGPT. Know where to use it, know where not to use it.”
Jacob Andra, at 33:34
ChatGPT, Claude, Gemini, “this batch of, you know, chatbots,” used in a narrow scope with custom GPTs for certain knowledge tasks, “not putting sensitive data in them, et cetera, et cetera,” and “you can be up and running in one day, provided your people know how to do it.” Some clients ask Talbot West to teach them exactly that. The limit is expectations: “help me make ChatGPT do this very complicated data analysis thing and integrate with all these other things. Like, yeah, it’s not going to do that.”
The one clause in there that carries the security program is “not putting sensitive data in them.” It’s also the clause most often skipped, which is why the first job of any AI policy is to say in plain words which data can go where, and the second is to find out what people already did before the policy existed. That second job is what shadow AI detection is for.
Governance, champions and one set of tools
Then I asked the question I hear from late-stage startups every month. Five hundred people, no formal program, everyone using something different. Some people on Augment Code, some on Cursor, some on Windsurf. What should leadership do?
“Yeah, so usually that’s a comprehensive governance framework combined with an internal initiative to find the internal champions, find the tools that are working, consolidate on one set of tools, get everyone on board that one set of tools, work with the internal champions to kind of evangelize to everyone else, put a training program in place, et cetera.”
Jacob Andra, at 35:04
I’d sign every word of that. The order is the part people get wrong: find the champions and the tools that already work before writing the rules, so the rules describe how good work gets done instead of banning it. Consolidating on one set of tools is also the cheapest security decision a company can make about AI, because one tool means one contract, one set of data terms, one admin console and one place to look. For an organization that wants the governance to stand up to a buyer’s review, ISO 42001 is the management system standard built for it, and our AI governance maturity model shows where most companies sit before they start.
Efficiency against security
Stephen then drew the trade-off he and Jacob talk about with clients, for our benefit, since “you guys are security guys”:
“And the most secure thing that companies could do with AI is just not use it. But also, you’re going to trade off, um, [a] ton of wasted efficiency to not use it at all.”
Stephen Karafiath, at 35:39
His policy line is a cost test. As long as there’s no personally identifiable information or customer information involved, and what you’re sharing isn’t “something that’s going to break the bank if it gets out anyway,” you can move into big efficiencies quickly. He’s skeptical of the checkbox that says a provider won’t train on your data, and called trusting it “fanciful.” That’s his view, and the safer habit it points to is the same either way: decide what can leave the building by what it would cost you if it got out, then read the contract for the tools you approve.
Jacob drew the line that keeps the trade-off honest. Stephen was talking about “public commercial large language models,” he said, “because obviously there are a ton of AI solutions that everything you’re saying doesn’t apply because it can be completely private, on-prem or secure cloud, right?” Stephen: “Absolutely.” The rules for a model you rent by the prompt aren’t the rules for one running inside your own boundary, and a good policy says which is which.
Sasha and I have made the same argument from the security side, that a leader who says yes with guardrails ends up knowing more about where the data goes than one who says no and gets shadow AI. Episode 95 with Bedrock Data is the long version, and our post on shadow AI governance is the short one.
Where 10x is real
With that distinction made, Stephen named the quickest wins with large language models, and his test is the most practical idea in the episode:
“And especially if it’s something where the amount of work to produce it is large, but the amount of work to review it is low and you still have your expert humans reviewing it, like you can get 10x efficiency gains on these things.”
Stephen Karafiath, at 36:49
He listed the candidates: text in a templatable, relatively consistent format that “could save hundreds of hours worth of work in half an hour,” from creative writing and marketing content “all the way up to like RFP proposals, understanding them, responding to them, getting that stuff double checked,” and contracts and law review in the legal field. That’s why so many software products already sit on those tasks.
Security questionnaires are the example we live with. Answering a 300-question review from scratch takes days. Checking a draft built from your own policies, your SOC 2 report and your last answers takes hours, if the person checking knows the program. That last condition is Stephen’s, and it’s the one that keeps a fast answer from becoming a wrong promise in a contract. The calculator below runs his test on your own numbers.
When on-prem buys flexibility
I asked how regulated markets, healthcare with HIPAA and HITRUST, defense with CMMC Level 2, get the benefits of AI when the path is unclear. Jacob’s answer was pragmatic: partnerships, and “many solutions emerging to abstract, you know, abstraction layers that take out all of that or mask it,” chosen against what each regulation actually requires, and, when the right solution isn’t on the market yet for a given sensitivity, “steering them away from that or guiding them toward an on-prem solution.”
Stephen added a twist I didn’t expect from someone who spent years moving Oracle customers to the cloud:
“And in some ways, to answer your question, with, like, stuff like CMMC Level 2 environments, with, like, say, a secure on-prem enclave, sometimes you actually buy yourself more flexibility than trying to find either SaaS provided or stuff that can be hosted in the cloud that meets that regulation.”
Stephen Karafiath, at 39:03
If the regulation is mainly about a secure enclave, an on-prem deployment of something like a risk management platform with few connections to other systems may be acceptable where the software-as-a-service version isn’t. “I’ve given up on trying to figure out the reason behind a lot of these onerous federal regulations,” he said, “and just try to advise clients on what they’re allowed to do that will be most useful to them.” I said I was torn, since we just moved everything into the cloud, and he was emphatic about the general case:
“Let me be clear. There’s almost nothing that I would recommend that an enterprise client who’s not subject to secure federal regulations would run in their own on-prem environment. The security of your hosted private cloud [is] going to be far better than anything you’re doing on-prem.”
Stephen Karafiath, at 40:11
“But when the federal government and regulations are involved, sometimes things get backward.” I’ve seen that once or twice myself, as I said on the tape. The useful version for a defense contractor is the one Jacob described for the aerospace client: decide the enclave question first, on purpose, because the tool choices, the vendor list and the evidence plan all follow from it. Our CMMC Level 2 and FedRAMP readiness work starts there, and a healthcare company has the same fork with AI and HIPAA.
Today at the Naval Postgraduate School, I discussed some important implications of AI on national security with Professor Rodrigo Nieto Gomez. Topics included deepfakes, cyberresilience, and the need for standards for interoperable, modular AI capabilities. #ai #cybersecurity
Jacob Andra (@jacobandra) · February 18, 2026
The hardest days
I asked about the most challenging day as an entrepreneur. Jacob passed it to Stephen (“I love every day, so it’s a hard one for me to answer”), and Stephen’s answer was a programmer’s: the worst moments come when he thinks he’s onto a revolutionary architecture, the bugs turn into a rat’s nest, and then the problem underneath turns out to be philosophical rather than technical.
“There is a fundamental philosophical problem with doing this whole thing unstructured in a natural language and it’s not just, you know, oh, we just need a better LLM or a better machine learning algorithm. It’s like this is fundamentally not going to work.”
Stephen Karafiath, at 41:44
He said Jacob reassures him that they’re doing research, and research sometimes doesn’t work out. That’s an unusually honest thing to say about AI products, and it’s the same conclusion as the rest of the episode from a different direction: some problems need structure, and no amount of fluent text supplies it. Some of the architectures he loves most are in BizForesight, the M&A advisory platform Talbot West is building as a side venture out of its consulting hub. Jacob introduced it on his own podcast in September 2025.
Jacob’s hardest day is one that hasn’t happened: “the worst day, is if a client was totally unhappy, if for any reason a client didn’t have like an amazing outcome.” He knows the weight he puts on it, and said so. “But I do feel like this strong commitment to that.” Sasha’s reply was the right one: “Well, I mean, it sounds like you’re a true partner.” It also matches the one rule on Talbot West’s about page, which starts, “Do right by the client.”
Earlier, Stephen had described the business model behind that commitment. They call it escape velocity, enough contracts at enough margin to get past “the gravity that would pull us back down,” so the consulting hub funds side ventures that could be home runs. Jacob’s version was shorter. Asked when they sleep, he said, “It’s all about the right team and delegation.”
Advice to a younger self
My last question was what they’d tell their younger selves. Stephen went first:
“I think if I could go back to like all of the younger versions of myself, it would be mainly a message of, like, compassion and acceptance of: it might be hard now, but like it’s all gonna be okay. And a lot of the things that you’re catastrophizing or worried about are gonna work out.”
Stephen Karafiath, at 42:53
Jacob agreed and added his own: “put myself first a lot sooner. Don’t put up with other people’s BS and let their agendas run my life and actually make that decision way sooner. Do what’s right for me.” For founders who listen to this show, that’s the co-founder story in one line. They were friends first, and each of them got to the company on his own terms.
What happened after the tape
The episode aired on April 21, 2026, and Jacob shared it that evening, along with YSecurity’s post about it.
Thanks, Jon McLachlan and Sasha Sinkevich, for having Steve and I on the podcast. It was great to talk about embedding AI capabilities into real company systems and workflows with you guys and Stephen Karafiath.
Jacob Andra · April 21, 2026
The video version followed on our YouTube channel on June 17, 2026. In the months after, Talbot West published much of what the tape describes. In June it argued that the model layer is commoditizing and the advantage moves to the harness, the custom structure around a model: “The harness is where durable advantage lives” (Talbot West). In August came the guide to why AI initiatives fail, with APEX’s five dimensions, the guide to modernization dependencies, and the blast radius taxonomy for AI that writes to systems of record. Jacob has been teaching the harness idea too.
Thanks to @Neumontu for having me out to give a talk on advanced harness engineering for AI systems. Harness engineering will be one of the defining disciplines of the coming years, and @TalbotWest4093 is leading the way.
Jacob Andra (@jacobandra) · May 18, 2026
He also kept doing what he describes on the tape, warning people off the hype. In March, Talbot West posted about an appearance on a go-to-market podcast on spotting AI snake oil.
Talbot West CEO @jacobandra joined Dr. Amy Cook on the go to market podcast to discuss AI snake oil and what business executives should be asking to evaluate vendors and truly get value from AI technologies. Links to the show in the comments below.
Talbot West (@TalbotWest4093) · March 31, 2026
The regulated world moved too. On July 13, 2026, the Defense Department suspended Phase II of CMMC, the step that would have required third-party Level 2 assessments from November 10, 2026, and began a review of the program. The department’s CMMC page is clear about what didn’t change: “All Phase I self-assessment requirements remain firmly in place” (DoD CIO). DefenseScoop’s report quoted the department’s chief information officer, Kirsten Davies, on the companies that had already done the work: “Every dollar spent on security is a wise dollar spent” (DefenseScoop). For the aerospace client on the tape, and for anyone like it, the enclave decision is still the first fork, and the self-assessments still count.
Talbot West itself has grown into something bigger than the “AI-first digital transformation” Jacob described. Its site now talks about enterprise coherence and modernization capital, and in October 2026 it published the first field report of its American Industrial Capability Project, Beyond the Obvious in Defense Production (Talbot West). The method underneath is the one from the tape. Map the system, find the dependencies, then choose the move. Jacob put it in one line on LinkedIn in September: “Reality rewards one thing only: what works” (LinkedIn).
Did you catch it? A six-question quiz
Interactive · six questions
Did you catch it?
Six questions on what Jacob, Stephen, Sasha and I actually said, with the timestamp for each answer. The answers are under each question for anyone reading without JavaScript.
-
How much does Jacob say you need to move the needle in a large organization to see a massive impact?
- Ten percent
- One or two degrees
- Half of the workforce
b. "On a large organization, if you can even move the needle one or two degrees, you're talking massive impact on bottom line and profitability and all of that." Transcript, 8:42.
-
What does APEX stand for?
- AI Prioritization and Execution
- Automated Process Execution
- Applied Performance Excellence
a. "It's called APEX, that stands for AI Prioritization and Execution." Transcript, 18:28.
-
What era does Stephen compare the current moment for large language models to?
- The space race
- The arrival of the smartphone
- The dot-com boom and bust of the late 90s
c. "And to answer your initial question, my own personal view is we're probably in the equivalent of the dot-com boom and bust of the late 90s." Transcript, 26:16.
-
What does Stephen say coding with AI tools is like?
- Wrangling a drunk junior developer
- Pair programming with a senior architect
- Autocomplete with a better memory
a. "It's like wrangling a drunk junior developer, but, like, it can do work really quickly and if it's not the wrong work and if it didn't hallucinate, then it's amazing." Transcript, 30:25.
-
Where does Stephen say you can get 10x efficiency gains from large language models?
- Anywhere you can remove the human reviewer
- Where the work to produce something is large, the work to review it is low, and experts still review it
- Only in software development
b. "And especially if it's something where the amount of work to produce it is large, but the amount of work to review it is low and you still have your expert humans reviewing it, like you can get 10x efficiency gains on these things." Transcript, 36:27.
-
What does Stephen recommend for an enterprise that isn't subject to federal security rules?
- Move AI workloads back on-prem
- Split everything evenly between cloud and on-prem
- Almost nothing on-prem, because a hosted private cloud will be far more secure
c. "There's almost nothing that I would recommend that an enterprise client who's not subject to secure federal regulations would run in their own on-prem environment." Transcript, 40:11.
Is your AI use case ready to ship?
Jacob and Stephen kept returning to the same order of operations: map the dependencies, pick the right kind of AI, decide what it may change, and only then ship. This scorecard asks eight questions about one use case you’re planning or already running.
Scorecard · eight questions
Is your AI use case ready to ship?
Answer for one use case. If two answers fit, pick the lower one.
-
Before you chose the tool, did you map what the use case depends on?
- We chose the tool first
- We listed the systems it touches
- We mapped the data, processes and people it depends on, and fixed the data at the root
"Because again, back to this idea of dependencies, you've got to map it." See One or two degrees.
-
How did you decide a large language model is the right tool for the task?
- We assumed it was
- The task is mostly language, so it seemed right
- We asked whether it's a job for a language model, classical machine learning or an ensemble, and where the answer has to be right
Jacob's scoping questions come before any tool. See Failures that don't scale away.
-
What can the AI change on its own?
- It writes to production systems with no check
- A person approves each change
- Read-only or proposals by default, with a validation and rollback step before anything writes
Talbot West sorts integrations by blast radius, and we use a ladder that earns write access. See Wrangling a drunk junior developer.
-
Which data can go into public commercial models?
- Whatever people paste
- There's a policy, but nobody checks
- A written rule on which data can go where, and the approved tools' terms reviewed
Stephen's cost test: nothing with personal or customer information, and nothing that would break the bank if it got out. See Efficiency against security.
-
Who checks the output before it counts?
- Nobody, it ships
- Whoever is around
- An expert who can check it faster than they could have written it
The 10x gains come with "your expert humans reviewing it." See Where 10x is real.
-
How many AI tools does the team use for the same job?
- Everyone picks their own
- A short approved list
- One set of tools, with champions who show the rest and a training program
Jacob's answer for a late-stage startup. See Governance, champions and one set of tools.
-
Why this use case first?
- It was the loudest idea in the room
- It had the highest projected return
- It scored well on urgency, value, feasibility, complexity and alignment, and it makes the next project cheaper
The highest-return project may not be the best first one. See APEX: three rounds to one first move.
-
If you're in a regulated market, did you settle the hosting question first?
- Not yet
- We're working it out tool by tool
- Yes, the enclave or cloud decision came first and the tool list followed, or we aren't regulated
One early choice gates the rest. See Forking pathways.
The tool arrived before the map, which is how most AI pilots start. The next step costs a few hours: draw what this use case depends on, decide what it may change, and write down which data can go into it. Most of the risk shows up on the drawing.
You know what you're building and why. The open items are usually in the boundary: what the AI can write, who checks its output, and how many tools the team spreads the same data across. Close those and the pilot can become a system.
Mapped, scoped, bounded and reviewed. Use it as the evidence for the next project: the map you drew, the review step and the data rules are what your buyers and auditors will ask to see, and they make the second use case cheaper than the first.
Where is 10x real for you?
Stephen’s test is that the gain is real where producing the work is expensive, checking it is cheap, and an expert still checks it. This calculator runs the test on one kind of document your team writes over and over, like security questionnaires, proposals or contracts.
Calculator
Where is 10x real for you?
How many documents a month, how long one takes to write by hand, how long an expert takes to review an AI draft, how long the fixes take, and what an hour of that expert's time costs.
20 documents a month at 6 hours each is about 1,440 hours a year by hand. With an AI draft that an expert reviews in 30 minutes and fixes in 20, it's about 200 hours, roughly 7.2 times faster, which gives back 1,240 hours, or $148,800 at $120 an hour.
Every default is a placeholder, not a number from the episode. A year here is 12 months of the same volume, and the estimate leaves out the time to set up the drafting, the templates and the source documents, which is real work the first time. If review and fixes take longer than writing it by hand, the gain is zero, and that's Stephen's test telling you this isn't the task for it.
From plaster and Oracle to the 2030 thesis
Everything on this page sits on one timeline, from Stephen’s first developer team to Talbot West’s first field report. Filter it by thread.
Timeline · 2000 to 2026
From plaster and Oracle to the 2030 thesis
Filter by thread. Dates are from the public record except where marked as part of this show.
- The dot-com peak
The Nasdaq Composite tops out, then falls 78% by October 2002 (Wikipedia).
- Global Exchange Services
Stephen leads a team of senior software developers (LinkedIn).
- TerraFirma Paint & Plaster
Jacob owns and runs it for 11 years (LinkedIn).
- Oracle
Stephen joins as a senior solution architect, the start of almost 17 years there (LinkedIn).
- Marketing director
Jacob runs marketing at Advanced Composites, then U.S. Translation Company and One Click Retail (LinkedIn).
- The Transformer paper
Attention Is All You Need, by eight researchers at Google (Wikipedia).
- SOMA
Jacob is chief marketing officer, based in Helsinki (LinkedIn).
- Developer Innovations Team
Stephen becomes its director at Oracle (LinkedIn).
- CustomerFaucet.com
Jacob becomes chief executive of the copywriting and site architecture firm (LinkedIn).
- ChatGPT
OpenAI releases it (Wikipedia).
- Morfius
Stephen leaves Oracle to be vice president of AI solution engineering (LinkedIn).
- Talbot West
Jacob starts it in Salt Lake City (LinkedIn).
- Stephen joins
As senior vice president of AI solutions architecture (LinkedIn).
- Cognitive Hive AI
Talbot West first publishes CHAI, its architecture for multi-part AI systems (Talbot West).
- 47G
Both are on its AI advisory board by November 2024 (LinkedIn).
- Windsurf
Cognition signs a deal to acquire it (Cognition).
- The GenAI Divide
MIT NANDA finds about 95% of organizations getting no measurable return on generative AI (MIT NANDA).
- The Applied AI Podcast
Jacob's show launches, with Stephen on the first full episode (YouTube).
- BizForesight
The M&A advisory side venture, introduced on the podcast (YouTube).
- Neurosymbolic AI
Jacob's guide to using constraints where the consequences demand them (Talbot West).
- The shortcomings of LLMs
Jacob and Stephen on their own show, on failure modes and neurosymbolic AI (YouTube).
- We record episode 93
"Like if it ain't broke, don't fix it."
- CMMC Phase 1
Contract enforcement begins, with self-assessments at Levels 1 and 2 (DefenseScoop).
- Lucidity Sciences
Jacob becomes a digital transformation advisor to the company (LinkedIn).
- Neumont
Jacob joins its industry advisory board (LinkedIn).
- Naval Postgraduate School
Jacob on AI and national security (X).
- Episode 89
Graham Neray on agents that delete your database (episode 89).
- AI beyond the hype
Jacob moderates a Silicon Valley panel (X).
- Episode 93 airs
This conversation.
- Episode 95
Bedrock Data on security leaders who enable AI instead of blocking it (episode 95).
- The harness
Talbot West argues the advantage moves from the model to the harness around it (Talbot West).
- Episode 93 on YouTube
The video version (YouTube).
- CMMC Phase II paused
Phase I self-assessments stay in place (DoD CIO).
- Why AI initiatives fail
Talbot West publishes APEX's five dimensions and three rounds (Talbot West).
- Blast radius
Three tiers of integration risk for AI that writes to systems of record (Talbot West).
- Episode 102
Chris Kirschke on why a veteran CISO won't give an AI agent write access (episode 102).
- Beyond the Obvious in Defense Production
The first field report of the American Industrial Capability Project (Talbot West).
What to do this week
Five things, in the order Jacob and Stephen would do them.
- Map one use case before you pick its tool. On one page: the data it needs, the processes before and after it, the people who review it, and the number that’s supposed to change. Fix the data problems you find at the root.
- Ask the scoping question. Is this a job for a language model, for classical machine learning, or for an ensemble of parts? Mark the steps where the answer has to be right, and keep those away from a model that only has to sound right.
- Put every AI integration on a blast radius tier. Read-only, proposes for a person to approve, or writes on its own. Move anything in the third tier behind a validation and rollback step, and give each agent its own identity.
- Write down which data can go where. Stephen’s cost test is a fine first draft: nothing with personal or customer information goes into public commercial models, and nothing that would break the bank if it got out.
- Pick the first project for what it unlocks. Score your candidates, then ask which one makes the next three cheaper. That’s your first stepping stone.