An AI that helps engineers write code is only as good as what it knows about their code. Evan Driscoll put the cost of that plainly, a few minutes into the security part of this episode.
“The notion of a large language model that has the context of your codebase means that we need, in some ways, unfettered access to your codebase.”
Evan Driscoll, at 11:24
Evan was the vice president of engineering at Augment, and Dion Almaer was its vice president of product. Augment builds AI for software engineers working in large codebases, and it came out of stealth on April 24, 2024 with a $227 million Series B at a $977 million post-money valuation, after a $25 million Series A led by Sutter Hill Ventures (Augment Code). We recorded a few days later. Dion’s own post that day described what the team set out to build, from an expert understanding of large codebases to IP protection “designed from the first line of code” (Dion Almaer).
A disclosure before we start. YSecurity worked as Augment’s security team through its first SOC 2 audits. When we recorded, Augment had its SOC 2 Type 1 report and was inside the three-month window for its first Type 2 audit, and Evan thanks us by name at the end of the tape. The case study tells our side of it. So the claims below are labeled as theirs, with the public record beside them.
Jon McLachlan and Sasha Sinkevich co-hosted this one, and it aired on May 8, 2024, with 64 words of show notes. This page is the rest of it: a section for each part of the conversation with the sources beside it, eleven figures, sixteen clips from the tape, the full transcript, and four tools: a quiz, a scorecard for how well your security would hold up against a red team with your source code, a calculator for how many pull requests ship without a security reviewer, and a timeline. Augment came back to the show twice, with Scott Dietzen, then its chief executive, on episode 74 and the founding engineer Dirk Meister on episode 77. This is where the story starts.
Thirty-two years in
Sasha opened with Evan’s résumé. He had started his career about 32 years earlier, held leadership roles at many companies, and spent about a decade as vice president of engineering at Pure Storage before Augment.
“So you’re really dating me. So thank you for that. People on the podcast can’t see the gray hair.”
Evan Driscoll, at 1:06
Dion teased him with a question about which era he most enjoyed. Evan’s answer was the one he kept returning to for the rest of the episode.
“Always the most exciting thing is the thing currently in front of me, the challenge around the corner. How do you reinvent yourself for the next big problem?”
Evan Driscoll, at 1:21
His LinkedIn shows how many times he has done it. He started as a QA engineer at GO Corporation in 1992, the pen computing pioneer, then managed QA from 1994 to 1997 at General Magic, the early smartphone company co-founded by Bill Atkinson, Andy Hertzfeld and Marc Porat. He ran quality at Extricity and engineering at Peregrine Systems, and then spent seven and a half years as vice president of engineering at Forterra Systems, which shared a close history with the virtual world There. After a stint at IMVU he joined Pure Storage in November 2010 as a director of engineering, became its vice president of engineering in March 2015, and stayed to July 2021. He joined Augment in July 2022 (LinkedIn).
Not the smartest guy in the room
Pure Storage was itself a reinvention.
“So at Pure Storage, I didn’t know a lick about storage when I initially joined, but I saw a really amazing team of solid engineers in that interesting market.”
Evan Driscoll, at 1:31
It was also where he met Igor Ostrovsky, one of Augment’s co-founders. When Igor became an engineer in residence at Sutter Hill and got funding, he called Evan, who says he didn’t know anything about AI at the very beginning. What he could see was where it was going.
“And yet, you could already see the potential for how this was going to impact the world and how it was going to impact, specifically, our craft, software engineering.”
Evan Driscoll, at 2:05
He explained his move into leadership with the same modesty.
“And I think it’s predominantly because I’m not the smartest guy in the room, right? So typically the engineers are, but I do have a decent common sense about how to get really highly effective teams.”
Evan Driscoll, at 2:28
He calls his model leader-servant, the idea better known as servant leadership: the leader’s job is to build the environment where engineers can do amazing work. Later in the episode Dion said Evan was being humble about it.
A research team and a SaaS service
What made Augment different from Evan’s earlier jobs was that the engineering team was only half of it.
“And Augment has just taken us to the next level, with like, okay, not only are we building an engineering team, but we’re also building a research capability, which is a whole new exciting challenge.”
Evan Driscoll, at 2:53
And the research had to ship as a service that companies would trust with their code.
“And you have to build a product with AI that has to be a solid SaaS service with all the security and privacy guarantees that is still the new surface area for this entire technology.”
Evan Driscoll, at 3:03
That sentence is the episode in miniature. A model is a research problem. A software as a service product that holds other companies’ source code is a security problem, and Augment had to solve both at once. He summed up the job in one line: “Augment, I’ve not had a comfortable day yet.”
Toil at Google X
Sasha turned to Dion, who has spent his career moving between startups and big companies, always building developer platforms. Two jobs led him to Augment. The first was at Google, with a team in X, the company’s moonshot group, that built models focused on code.
“They’re all about, could we help with some toil? Where, like, our internal software engineers, highly skilled, of course, at Google, highly paid, and they’re spending a bunch of time on stuff that’s grunt work.”
Dion Almaer, at 4:20
The models nudged engineers along so they could stay current in Google’s famous monorepo, which changes constantly. That was before large language models, and Dion says it wasn’t yet at the quality the field can reach now. His launch-day post tells the same story, a research team inside X that helped highly skilled engineers keep up with the monorepo in the pre-transformer days (Dion Almaer).
The cool demo in a week or two
The second job was at Shopify, where he was vice president of engineering for developer experience from 2021 to 2023 (LinkedIn). His team built developer tools on large language models, and learned the lesson most AI teams learn the hard way.
“And that’s where I got to see that you could do the cool demo in like a week or two, but it wasn’t just about the foundational models, right?”
Dion Almaer, at 4:54
The questions that mattered were about everything around the model, and about code itself. “What’s special about code?” he asked, and answered that there’s so much about code that’s different from putting out text. The team also felt restricted, because it was calling a model it couldn’t change and there were things it wanted to tweak. In his post he lists what the Shopify project taught him: a cool demo is easy, something great at scale is hard, one model isn’t the answer for every use case, quality data matters, and evaluations are vital as you iterate (Dion Almaer).
They looked at each other and grinned
So Dion went to Sutter Hill, where two friends worked: Luke Wroblewski, a UX director he had worked with at Google, and Sam Pullara, who he described as Sutter Hill’s chief technology officer. He told them he wanted to start another developer tools startup with the new capabilities of large language models.
“they both looked at each other and grinned. Like, you need to talk to some people that are in this building, in fact.”
Dion Almaer, at 5:53
That led him to the two co-founders: Igor, and Guy Gur-Ari, who did research at Google on language models and reasoning. “There’s a Minerva paper that talks all about that,” Dion said. Minerva was a Google model for quantitative reasoning, described in a June 2022 paper whose authors include Gur-Ari (arXiv). Dion’s post fills in the rest. Gur-Ari led Augment’s research, Ostrovsky had been chief architect at Pure Storage and worked at Microsoft before a stint as an entrepreneur in residence at Sutter Hill, and Scott Dietzen had joined as chief executive (Dion Almaer). Dion joined in August 2023 (LinkedIn).
Product and engineering, one team
The last thing that convinced Dion was Evan. He had seen product and engineering turn into siloed worlds pointing at each other.
“Where product is like, hey, this is the thing. I just want you to build it. I don’t want any of your input.”
Dion Almaer, at 6:28
After time with Evan, he knew they could be one team. It comes back at the end of the episode, when both of them describe a company where engineers think through the product side too.
Augment is in the name
Sasha asked for the overview, and Dion started with the company’s name.
“Yeah, so Augment in the name itself tells a lot about the philosophy, right? Like we’re here to augment the humans, not to replace the humans, right?”
Dion Almaer, at 7:22
He was quoted the same way in the launch release that week: “The best software comes from augmenting developers, not replacing them” (Augment Code). The ambition was bigger than a few percentage points of productivity. As an engineer who loves coding, he wanted to change what the job feels like at companies where a lot of it isn’t fun, and he insisted the unit is the team.
“It’s a team sport. It’s not a solo thing, right?”
Dion Almaer, at 8:11
He had seen the opposite in one large enterprise outside tech, where it was painful to watch how slow everything was for the developers trying to build things.
Pairing with your principal engineers
Jon asked the market question. There were a few players already, so what did Augment do better than everyone else? Dion said it started with Igor going deep, as an entrepreneur in residence, on what it would take to understand a really large codebase, one whose dependencies reach into all kinds of other repos.
“But we want something where it feels like you’re pairing with the principal engineers, the most senior engineers at your company that understand the domain, understand the best practices, and that just delivers a very different experience.”
Dion Almaer, at 9:38
The contrast he drew was with tools that feel like a smart engineer who knows your language or framework. Augment wanted the feeling of the people who know your company’s code.
The speed of thought
The second part was speed. Dion compared it to search before Google, when there were tons of search engines and then one came along with a simple interface, fast results and better quality all at once.
“And by having this vertical AI team and building the entire stack kind of soup to nuts, we can deliver something that’s at the kind of speed of thought as you’re writing code.”
Dion Almaer, at 10:10
His launch post made the same point. Slow responses knock developers out of their flow, so the team built its own inference stack, with custom GPU kernels, to keep suggestions fast (Dion Almaer).
Not a CAP theorem
The third part is where security enters.
“And so having these different pieces where it’s not a trade-off of like, oh, I can only do CAP theorem, which one can I do? Everything was baked in right from the beginning, secure by default, high speed, high quality.”
Dion Almaer, at 10:34
The reference is to the CAP theorem, which says a distributed data store has to give up consistency or availability when the network partitions. Dion’s claim was that an AI product doesn’t have to make a choice like that between being good, being fast and being secure, as long as the company owns the whole stack.
Unfettered access
Sasha asked Evan to drill into data security. His first point was cultural. For many products, he said, data security is a bit of an afterthought, something you do after you get market traction. Augment couldn’t do that, because of the access the product needs, the line at the top of this page.
“And we have to do that in such a way that companies feel comfortable giving us that access. And that’s an incredible responsibility that we have.”
Evan Driscoll, at 11:35
The standard he set was the one engineers would set for themselves.
“In terms of what would other engineers, if we were selling this product to ourselves, what would our expectations for security and privacy be?”
Evan Driscoll, at 11:56
Augment’s customers are engineering teams, so the people judging its security are the people best equipped to find the holes. Evan’s answer was to hire for it from the start.
Layers from day zero
Then the architecture, in one breath.
“But you have to start with an architecture that has layers of security, tightly controlled access, operating on principles of least privilege. We’re explicit about data categorization, what can and can’t be logged, and we minimize data collection wherever possible.”
Evan Driscoll, at 12:22
Each item is a known practice: the principle of least privilege, data classification, rules about what may be logged, and data minimization. The list kept going, with encryption on the wire, and then three specific pieces.
“we build things like mTLS, OpenID Connect, and a non-extractable API.”
Evan Driscoll, at 12:40
mTLS is mutual authentication, where both sides of a connection prove who they are, and OpenID Connect is the standard identity layer for signing users in. The third item is Augment’s own, and Sasha asked about it a minute later. Evan was honest about the cost of starting this way.
“And that’s again something that if you start with those types of principles, it takes you a longer time to build the end solution. But you end up with a really solid foundation, right?”
Evan Driscoll, at 12:50
All make-believe until a red team tries
The layers only count if someone outside checks them.
“So, that third-party attestation is super important. So, we’ve done SOC 2.”
Evan Driscoll, at 13:13
The pen test went further than most. A pen testing firm got the source code and the cloud.
“We partnered with an extremely vigorous third-party pen tester, a pen testing firm that we gave access to our source code, access to our cloud, and they were unable to break us, right?”
Evan Driscoll, at 13:18
A penetration test run with full knowledge of the system, its code and its cloud, is the most demanding kind, because the testers don’t spend their time guessing. Without that kind of test, Evan said, it’s all make-believe until a well-informed red team gets its turn. Augment’s own post that September said the same thing in a sentence: “We hire independent experts to test our security systems regularly” (Augment Code). If you want to know what a test like that costs and covers, our page on penetration testing and our post on what enterprise buyers expect from an AI startup’s pen tests lay it out.
An API with no download
Sasha asked what a non-extractable API is. Evan said it’s pretty much how it sounds.
“So in this case, it’s an API where effectively our system will upload information, say, about your codebase, but there’s no corresponding API to actually download said information, right?”
Evan Driscoll, at 14:28
The reasoning is simple once you hear it.
“As a client, you already know your codebase, you don’t need to download it.”
Evan Driscoll, at 14:42
Augment uses the uploaded code at inference time, to answer the developer who uploaded it, and never offers it back. That design shrinks what an attacker can reach.
“The nice thing about having this sort of type of API is that it really reduces the attack surface area of your public API.”
Evan Driscoll, at 14:53
The attack surface of an API is every call an attacker could try. A stolen credential that can’t download code can’t leak it through that API. The hard part, Evan added, is that it has to happen as you work: “So the state of your code exactly when you’re working on it is what we need to reflect,” at very low latency. Augment wrote that design up in January 2025. Its index updates within a few seconds of a change, and before the backend uses a file, the editor must “prove to the backend it knows a file’s content” by sending a cryptographic hash (Augment Code). Dirk Meister walks through that proof of possession on episode 77.
Imposter syndrome
Sasha said trust is a big part of doing business in 2024, and that Augment was being proactive about it by being transparent about its architecture and testing continuously. Then Jon told the pen test story from the other side.
“One of the pen testers came away with a little bit of imposter syndrome”
Jon McLachlan, at 15:50
“Yeah, that was one of the biggest compliments I’ve ever had in my professional career. Granted, it wasn’t me, it was the engineers around me.”
Evan Driscoll, at 15:56
What the pen tester said, as Evan quoted it, was that they couldn’t find a critical. For a company at day zero, Jon said, that kind of commitment to security seemed really novel. Impostor syndrome usually means doubting your own competence. Here it was the testers’ reaction to a target that held up.
Moving fast without breaking things
Jon pushed on the tension every startup feels. There is a natural, healthy pull toward moving fast, breaking things and chasing the white rabbit to see how deep the rabbit hole goes, and Augment seemed to be taking a more mature approach. Why?
“Yeah, you mentioned trust earlier. And like the minute you lose trust, all bets are off.”
Dion Almaer, at 17:13
The team’s test was the golden rule, applied to source code: how would we want a third party to treat our gold? Dion drew the line between where a startup experiments and where it doesn’t.
“There’s lots of areas that you can experiment on, right? We’re experimenting on the UI all the time.”
Dion Almaer, at 17:35
The difference is when the work happens. “I’ve done the retrofits, it’s really hard,” he said. Building it into the architecture from the start is what lets the team move.
“I think there’s a bit of a myth on just, oh, if you have to do something securely, then everything has to move 100 times slower.”
Dion Almaer, at 18:02
Every new hire
Evan added what keeps it true as a company grows.
“And every new hire that you add to the team understands the importance of this because it’s a core value, you can still go fast.”
Evan Driscoll, at 18:24
It isn’t always easy, he said. There are lots of debates about how to handle personal data. Then the line that explains the whole posture.
“If we don’t get this right, we don’t deserve to play in this market.”
Evan Driscoll, at 18:41
Our post on the .env file as your first security program is about the same habit at a smaller company.
We don’t train on customer data
Sasha asked the question every buyer of an AI coding tool asks first: does Augment train models on the customer data? Evan’s answer was no, without qualification, and then he went further than the question.
“And we’re slightly horrified at the idea of even doing fine-tuning on customer data for the purpose of knowing their codebase, right?”
Evan Driscoll, at 19:30
Fine-tuning on one customer’s code is the obvious way to make a model know that customer’s codebase, and it’s the one Evan called the bad architectural direction. Whatever a model learns from training can surface in its output for someone else, and the risk he named was leaking a customer’s IP.
“We don’t train on customer data. That’s just not an option.”
Evan Driscoll, at 19:42
The company put the same promise in writing when it announced its SOC 2 Type II report that September: customer intellectual property will “never be used to train AI models, whether by us or any third party” (Augment Code). Scott Dietzen later turned it into four questions any buyer can ask a vendor, a story told on episode 74.
Permissive licenses
Jon asked where the training data comes from, then.
“So we’re using permissive open source data. We’re very cognizant of the licensing.”
Evan Driscoll, at 19:47
Augment starts from foundation models and fine-tunes them to add capabilities, and Evan said the team prefers open source models with a clear record of the data sets they were trained on. A permissive software license lets anyone reuse the code with few conditions, which is why it matters for training data. Jon explained why he thinks this will matter more than people expect.
“I think permissive licenses are going to be one of those big pieces of AI that are going to sneak around the corner and maybe surprise folks later.”
Jon McLachlan, at 20:11
The open question is who owns what the models produce.
“And it’s a little bit of a gray space. And when things are undecided or a little bit gray, it does present risk.”
Jon McLachlan, at 20:32
Your conventions, your libraries
Evan added a second reason the copying risk is low, and it comes from how the product works.
“When we see great code completions, they’re often lifting reasoning and logic from your codebase, following your conventions, using your libraries. And so the risk of copying code from, say, an open source project is even less.”
Evan Driscoll, at 21:03
The coding expertise comes from the foundation models, he said, but the suggestions lean heavily on the customer’s own code. Jon tied it back to Dion’s pitch: the context is the product, so the paradigms and libraries a team already uses become part of what it gets back.
Indemnification
Jon asked whether anything in the contracts gives customers more peace of mind on IP. The answer came back in one word, indemnification, and Dion explained it.
“So everywhere we can, we don’t want to be just talking the talk, right? We want to contractually put it out there to show how important it is to us and to show that we know how important it is to you.”
Dion Almaer, at 22:13
An indemnity is a promise to cover the other side’s losses if a claim arrives. For an AI coding tool, the claim a customer worries about is someone coming after them over code the tool suggested, and Dion said Augment wanted to stand by its suggestions in the contract. Our post on the security exhibit your startup signed covers the other promises that end up in those contracts.
Do not use any of this
Dion said the contract mattered because of the bind he kept finding on the customer side. The people responsible for internal development wanted their teams more productive and more satisfied, and they were also nervous about the IP, security and legal sides. The first wave of AI coding tools made it worse.
“You know, sometimes, like, when some of these tools first started to come out, right from the top, they’d get told, do not use any of this. Do not allow anyone on the team to use this.”
Dion Almaer, at 23:10
Four to six months later, the engineers had been using the tools on side projects and felt like an arm was tied behind their backs at work. Then the message from the top reversed.
“Then suddenly from the top, they were saying to this, you know, whoever was responsible, okay, find a way for us to enable this, but keeping the security and privacy and all of the other IP concerns together.”
Dion Almaer, at 23:35
The numbers backed him up. Two weeks before Augment came out of stealth, Gartner said 75 percent of enterprise software engineers would use AI code assistants by 2028, up from less than 10 percent in early 2023 (Gartner). When a ban meets that kind of demand, the tools get used anyway, on side projects or quietly at work. Our post on shadow AI is about governing that without blocking it.
Don’t mess up the money tree
Jon had a blunter way to put the risk aversion.
“Yeah, in a very crude sense, one way to think about a business is a giant money tree, especially one that’s successful. Right. And rule number one, when you have a giant money tree, is don’t mess up the money tree.”
Jon McLachlan, at 23:56
Then he asked the question he said Dion probably hears all the time, whether Augment ever sends data to a third-party AI service.
“Never. Absolutely. None of this would make sense if we were just shipping it off to some other third party.”
Dion Almaer, at 24:24
The closed system with the full vertical stack, Dion said, is what lets Augment stand behind the promise. Its engineers later wrote that they made the same choice for search, self-hosting the embedding search on Google Cloud and “avoiding third-party APIs that could expose embeddings” (Augment Code).
On-prem, now and then
Owning the stack also means running it in Augment’s cloud. Jon asked whether customers ask to run the whole thing on premises.
“Now and then, but not as much as I actually expected.”
Dion Almaer, at 24:54
Dion said more and more research points to running something yourself not actually being more secure, and that you’re often safer partnering with a cloud provider. Evan added the practical problem.
“Not only is there security and privacy challenges, but the scaling challenge of how do potential customers get access to the caliber of GPUs needed to run these models and run these models efficiently.”
Evan Driscoll, at 25:38
And whoever deploys it in their own environment, or in a virtual private cloud, has to maintain it too. Sasha listed what that maintenance means: internal teams trained on the product, security networking teams to keep the DMZ around it, and a deployment that lags behind the latest version.
“And it just becomes operationally very expensive.”
Sasha Sinkevich, at 26:39
Let the folks who wrote the software run the software
Jon gave the security case against on-prem, and it was about freshness.
“Once something goes on-prem, you can’t upgrade the thing. It’s stuck in the old models. The space moves so quickly.”
Jon McLachlan, at 26:54
Maybe it makes procurement go a little faster, he said, and he would rather do the work to get the service approved.
“I would rather just do what you need to do to get the SaaS authorized for use. Let the folks who wrote the software run the software.”
Jon McLachlan, at 27:07
Getting a SaaS authorized means passing the buyer’s security review, which is most of the work we do for startups. Our list of the AI security questions your first enterprise buyer will ask is the short version of that review.
One reviewer, ten pull requests
Jon asked Dion whether AI has a role in helping secure code. Dion said yes, and that the principal engineers he wants the product to feel like include the ones who know security. Then he told a story from a customer.
“And I was talking to a customer fairly recently who had one person on the team that was just really deep on the security side.”
Dion Almaer, at 28:06
That person felt a little sheepish, because they were a reviewer on roughly one out of ten pull requests, and the other nine got a different level of review. Dion’s answer was a helper that doesn’t get tired.
“So what if you could have help for that person to be able to, you know, help them guide an AI that’s always running 24-7, always available, not having to handle the time limitations that a human does.”
Dion Almaer, at 28:34
The human keeps the novel problems and the deep calls, he said, and the AI handles the nits. It works better the more a team writes its practices down, in code, in docs and in its systems, so the AI has them to apply. The calculator below puts numbers on the nine.
More code doesn’t mean better software
Dion was clear about what Augment shouldn’t optimize for.
“More code doesn’t mean better software. And so we want to make sure that we’re really helping with code quality.”
Dion Almaer, at 29:20
He wanted the product to be able to delete bad code, not just write more of it. The fear at first, he said, was that AI would create code and that it would be bad. His bet was the opposite.
“If we can do a good job with this, we can do the exact opposite and help secure the world more than humans could before.”
Dion Almaer, at 29:44
Our post on vibe coding security is about the version of that fear that came true for teams without the guardrails, and what to put in place.
Augment on Augment
Jon had to ask whether Augment uses Augment.
“Do we use Augment to work on Augment? Is it augmenting itself?”
Jon McLachlan, at 29:52
The company had just shipped a new website as it came out of stealth, and Dion said it was fun to watch the developers build it.
“They were just like using Augment to generate a bunch of the components and pieces. It’s very meta.”
Dion Almaer, at 30:08
The habit has a name in software, eating your own dog food, and Evan comes back to it a few minutes later when he describes the best day. Jon put it as a principle.
“We should be using IDEs to make the IDE better. We should be using Augment to make the next version of Augment.”
Jon McLachlan, at 30:28
Raise the floor as well as the ceiling
Jon asked what success looks like. Dion started with people.
“So I like to see happy developers.”
Dion Almaer, at 30:42
He wants a world where people enjoy their work more than ever and the toil goes to computers, which are totally happy with it. Then the business result.
“Like every company’s become a software company. To be honest, there’s variability in the quality of software that’s created from these companies. And so we want to raise the floor as well as the ceiling.”
Dion Almaer, at 31:09
Evan’s version was about which problems people get to spend their days on.
“Things that will engage people and where there’s innovative business challenges as opposed to figuring out like, oh, I need to update this makefile and I forgot about it. Right. It’s just like, who likes writing makefiles?”
Evan Driscoll, at 31:31
Anyone who has fought a makefile knows the feeling. He called it the very start of the roller coaster, and asked how the industry uses machines for what they’re good at.
“How do we remove toil from software engineering and ultimately enable companies to go faster?”
Evan Driscoll, at 31:57
Mental cycles
Jon said he loved that adoption sat at the center of the definition of success, because it’s the piece that gets left by the wayside when people play with new technology. Sasha added creativity, since people get more time to think about bigger problems, and pointed at chores like remembering to update a README or a library. Evan named the measure he would want if he could have it.
“Yeah, I do think that the interesting metric, if we could measure it, is like the amount of cycles being spent on actual interesting problems”
Evan Driscoll, at 32:57
The opposite of interesting problems, in his example, is the time spent working out what an error message means and what might be the problem.
“Where in many cases an AI could potentially accelerate that entire process of debugging.”
Evan Driscoll, at 33:19
Like electricity
Jon asked about the best day at Augment. Evan joked that it was when Igor called him to join, then picked a day from the fall of 2023, shortly after Dion joined. The company dogfoods heavily, and early on the product wasn’t good enough for the team to want to use it much.
“I think it was October, we started seeing the first jaw-dropping code completion. What? How did the model get this right?”
Evan Driscoll, at 34:04
It had taken more than a year of research to get there.
“And for a startup guy, that’s just like, that’s like electricity. It’s just like it starts clicking and we really knew we had something.”
Evan Driscoll, at 34:19
So freaking fast
Dion’s best day came right after. First the dogfooding moments, then the day outside developers saw the same thing in a pre-alpha that put the product in people’s hands as early as possible.
“Like for me, I’d had a couple of friends who were amazing developers and hearing from them when they were just like, oh my God, this is so freaking fast.”
Dion Almaer, at 35:06
Sasha said it’s exciting to see innovation create a positive impact on people’s lives, and Dion said the human part is the point. He has worked on systems big enough to lose sight of it.
“I remember at one point I worked on Chrome at Google and you’re looking at billions of users and it felt like staring at the Matrix.”
Dion Almaer, at 35:39
His LinkedIn lists Chrome DevTools and Lighthouse among the products he owned in his second stint at Google, from 2015 to 2021 (LinkedIn). Going back to see actual customers and how the product helps their day, he said, is what makes it enjoyable.
Where do we begin?
Jon asked about a challenging moment. Dion had one he keeps coming back to, prioritizing, and as the first product person he was glad to have engineers who wanted to step up.
“So, in our culture, we don’t have an army of PMs that are off there defining all of the things, and the engineers are twiddling their thumbs, waiting for us.”
Dion Almaer, at 36:19
Evan agreed that it’s hard, and said the engineers’ enthusiasm is part of why.
“Yeah, it’s actually not an easy problem to say, where do we begin? Where do we focus? How do we start?”
Evan Driscoll, at 36:43
The team uses the product itself, so there are plenty of ideas and opinions about what to do next, and people like to prototype. The challenge is to pick an investment and take it as far as it can go.
A new form of software development
Then Evan named the challenge that sets AI products apart. A lot of the roadmap is bound by research, so a great feature idea may or may not work once the team tries to get the models to behave that way.
“And so that’s where this is just in many ways, it’s an entirely new form of software development where you’re building these algorithms through training.”
Evan Driscoll, at 37:53
The risk is that it takes a few tries to get the thing to work.
“And it’s just utterly fascinating, but there is risk here of, yeah, it might take you a couple of different tries to get the thing to work, which is completely foreign from a regular software engineering perspective.”
Evan Driscoll, at 38:00
This is the same point Dion made about Shopify at the start, from the other side. A demo takes a week. Knowing whether the model will do the thing takes research.
Two pillars
Dion’s version of the challenge was the one he has learned from every platform he has built. Developers never use an API exactly the way its designers expected, so you build a platform that lets you see what they do and make it better.
“I feel like there’s two main pillars: there’s the developer experience and there’s the AI.”
Dion Almaer, at 38:54
The fun days are when the two work together, he said, and the team can build different models for different scenarios that enable a totally different developer experience.
Built on trust
Sasha closed the business part of the conversation with who it’s for.
“Any company, any organization can take advantage of this new technology that is built on trust and core security principles.”
Sasha Sinkevich, at 39:32
Dion pointed listeners to augmentcode.com, where the company was hiring and had a waitlist at the time.
The shout-out
Before the goodbyes, Evan said he had something to make explicit.
“So I do have to give a shout out to YSecurity, just to make it very explicit.”
Evan Driscoll, at 40:26
“They partnered up with us to help us achieve SOC 2 in blistering speed, I might add, SOC 2 Type 1.”
Evan Driscoll, at 40:38
Sasha’s answer gave the credit back.
“In all fairness, we stand on a strong foundation that has been built by the leadership and engineering teams”
Sasha Sinkevich, at 41:15
The case study has our account of the work. We worked as Augment’s security team, reviewed 18 core security policies, and had its 40-person team fully SOC 2 ready in two months, then ran the three-month audit window that ended with zero deviations. In the three months after its SOC 2 Type 2, its security-qualified leads grew fifteenfold (YSecurity). Evan’s line in it is “YSecurity helped us raise our security to the next level.” Our posts on SOC 2 Type 1 versus Type 2 and how long a SOC 2 audit takes walk through the same path for other startups.
What happened after the tape
The record since the recording mostly confirms what Evan and Dion said.
The audit. On July 10, 2024, Augment received its SOC 2 Type II attestation after a three-month audit that ran from April 1 to July 1, the window it was inside when we recorded (Augment Code).
The launch. On October 24, 2024, Augment Code launched for teams (Augment Code). Scott Dietzen told the story of the next year on episode 74.
The non-extractable API, in public. In January 2025, Dirk Meister and Markus Rabe wrote up the real-time index behind the context Evan described, with an index that updates within seconds of a change and a proof of possession before any file is used (Augment Code). That April, customer managed keys went live for enterprise customers, a key the customer holds and can revoke (Augment Code). Dirk explains both on episode 77.
A second certification. On May 29, 2025, Augment announced ISO/IEC 42001 certification for the way it manages AI, the standard’s first certification for an AI coding assistant by its own account, and I wrote that post (Augment Code). Coalfire, the certification body, announced it that August (Coalfire).
The people. Dion moved to Tessl as field CTO in November 2024 (Dion Almaer) and has been a director of engineering at Google Labs since July 2025 (LinkedIn). Evan’s LinkedIn lists him at Augment from July 2022 to January 2026, with an adviser role from July 2025 (LinkedIn). Pure Storage, where Evan spent more than ten years, took the name Everpure in February 2026 (Wikipedia).
What holds up best is the design. An API that never hands code back, a product that doesn’t train on customer code, and a pen test with the source in hand were early choices in April 2024, and each one shows up again in the company’s later posts and certifications.
Did you catch it? A six-question quiz
Six questions from the episode. Each answer links the moment in the transcript.
Quiz · six questions
Did you catch it?
Pick an answer for each question, then check your score.
-
What does a non-extractable API leave out, in Evan's description?
- Encryption of the uploaded code
- Any call that downloads the code back
- Logging of who uploaded what
b. "So in this case, it's an API where effectively our system will upload information, say, about your codebase, but there's no corresponding API to actually download said information, right?" Transcript, 14:28.
-
What did Augment give its outside pen testing firm?
- A staging copy of the product
- A list of its security policies
- Access to its source code and its cloud
c. "We partnered with an extremely vigorous third-party pen tester, a pen testing firm that we gave access to our source code, access to our cloud, and they were unable to break us, right?" Transcript, 13:18.
-
What did Evan say Augment uses for training data?
- Permissive open source data
- Each customer's own code
- Anything public on the internet
a. "So we're using permissive open source data. We're very cognizant of the licensing." Transcript, 19:47.
-
In Dion's customer story, how many pull requests did the security expert review?
- All of them
- Roughly one in ten
- Only the ones on the release branch
b. "And this person was saying, like, they felt a little bit sheepish at times because they will be as a reviewer on like roughly one out of ten PRs." Transcript, 28:14.
-
What is rule number one when you have a giant money tree, according to Jon?
- Don't mess up the money tree
- Plant a second one
- Move fast and prune often
a. "And rule number one, when you have a giant money tree, is don't mess up the money tree." Transcript, 24:03.
-
When did Evan say the team saw its first jaw-dropping code completion?
- The day Augment came out of stealth
- During the first SOC 2 audit
- Around October 2023
c. "I think it was October, we started seeing the first jaw-dropping code completion." Transcript, 34:04.
Would your security survive a red team with the source?
Eight questions for a team whose product holds customers’ code or data, drawn from the layers Evan listed. Each one links the part of the episode it comes from.
Scorecard · eight questions
Would your security survive a red team with the source?
Answer for the product that holds your customers' most sensitive data, as it runs today. If two answers fit, pick the lower one.
-
Has anyone outside your team tested you with your source code and cloud access in hand?
- No outside test yet
- A pen test without the source
- Yes, with source and cloud access, in the last year
"So, that third-party attestation is super important." See All make-believe until a red team tries.
-
Is there any API that hands back customer data your product only ever needs to receive?
- We haven't looked
- A few, behind authentication
- No, those paths are upload-only
"As a client, you already know your codebase, you don't need to download it." See An API with no download.
-
Do you train or fine-tune models on customer data?
- Yes, or we're not sure
- Only when a customer opts in
- Never, and the contract says so
"We don't train on customer data. That's just not an option." See We don't train on customer data.
-
Have you written down what may and may not be logged?
- No
- Informally, service by service
- Yes, by data category, and it's enforced
"We're explicit about data categorization, what can and can't be logged, and we minimize data collection wherever possible." See Layers from day zero.
-
How much access does each person and each service have?
- Broad access, for convenience
- Mostly scoped, reviewed now and then
- Least privilege, reviewed on a schedule
"But you have to start with an architecture that has layers of security, tightly controlled access, operating on principles of least privilege." See Layers from day zero.
-
Do your services prove who they are to each other?
- No, the internal network is trusted
- Some do, with shared secrets
- Yes, with mutual TLS or an equivalent
"we build things like mTLS, OpenID Connect, and a non-extractable API." See Layers from day zero.
-
Where is your SOC 2?
- Not started
- A Type 1 report, or inside the Type 2 window
- A Type 2 report in hand
"So, we've done SOC 2." See The shout-out.
-
Does every new hire learn your security commitments in their first week?
- Not formally
- In a policy they sign
- Yes, as a core value, with examples from our own code
"If we don't get this right, we don't deserve to play in this market." See Every new hire.
Your security may be fine, but nobody outside has checked. Start with the two moves that matter most for a product that holds customer data: find every API that hands that data back and close the ones you don't need, and book an outside test that starts with your source code.
The foundations are there and some of them have been checked. Next, close the remaining read-back paths, write down what can be logged for each kind of data, and give your next pen test the source code and the cloud.
You built it in and had it tested. Keep it that way as the team grows: teach the commitments in onboarding, retest after big changes, and keep your report current for the buyers who ask.
How many pull requests ship without a security reviewer?
Dion’s customer had one security expert who saw roughly one pull request in ten. Put in your own numbers to see how many ship without that review in a year.
Calculator
How many pull requests ship without a security reviewer?
Three estimates: how many pull requests your teams merge in a week, the share that gets a security reviewer, and how many weeks a year your teams ship.
That's about 7,500 pull requests a year, 750 of them with a security reviewer and 6,750 without one, about 27 each working day. An AI reviewer that knows your written practices can look at every one of them, and your security expert can spend their time on the ones that need judgment.
One in ten is the share from Dion's story about one customer, not a benchmark. Your repository host can tell you how many pull requests you merge and how many had a security reviewer.
From GO Corporation to Google Labs
Everything on this page sits on one timeline, from Evan’s first job to the company’s later certifications. Filter it by thread.
Timeline · 1992 to 2026
From GO Corporation to Google Labs
Filter by thread. Dates are from the public record except where marked as part of this show.
- GO Corporation
Evan's first job, as a QA engineer at the pen computing pioneer, to 1994 (LinkedIn, Wikipedia).
- General Magic
QA manager at the early smartphone company, to 1997 (LinkedIn, Wikipedia).
- TheServerSide
Dion becomes editor in chief of the enterprise Java community, to 2005 (LinkedIn).
- Forterra Systems
Vice president of engineering at the virtual world company, to January 2010 (LinkedIn, Wikipedia).
- Ajaxian
Dion and Ben Galbraith found a community for Ajax developers, which they run to 2009 (LinkedIn).
- Google
Dion's first stint at Google, as an open web advocate, to October 2008 (LinkedIn).
- Mozilla
Director of developer tools, where he and Ben built an open source web-based code editor at Mozilla Labs (LinkedIn).
- Palm
Director of developer relations for webOS, to November 2010 (LinkedIn).
- Pure Storage
Director of engineering, then vice president of engineering from March 2015, to July 2021 (LinkedIn, on this show).
- Walmart Labs
Vice president of engineering for mobile, after his company Set Direction joined Walmart, to May 2015 (LinkedIn, Wikipedia).
- Google again
Director of engineering, then of product, with Chrome DevTools and Lighthouse among his products, to August 2021 (LinkedIn, on this show).
- Shopify
Vice president of engineering for developer experience, to May 2023 (LinkedIn, on this show).
- Minerva
The Google paper on quantitative reasoning with language models, with Guy Gur-Ari among its authors (arXiv, on this show).
- Evan joins Augment
As vice president of engineering, after Igor Ostrovsky's call (LinkedIn, on this show).
- Dion joins Augment
As vice president of product (LinkedIn, on this show).
- The first jaw-dropping completion
After more than a year of research, as Evan remembers it (on this show).
- The SOC 2 Type 2 window opens
Three months of observation, to July 1 (Augment Code).
- Out of stealth
A $227 million Series B at a $977 million post-money valuation (Augment Code).
- Episode 43
The episode airs (transcript).
- SOC 2 Type II
The attestation arrives, with zero deviations in the audit window (Augment Code, case study).
- Augment Code launches
The product launches for teams (Augment Code).
- Tessl
Dion joins as field CTO (Dion Almaer).
- A real-time index
Updates within seconds, with a proof of possession before a file is used (Augment Code, on this show).
- Customer managed keys
Live for enterprise customers (Augment Code).
- ISO/IEC 42001
Certification for the way the company manages AI (Augment Code).
- Google Labs
Dion becomes a director of engineering at Google Labs (LinkedIn).
- Episode 74
Scott Dietzen on a tech lead for a team of agents (episode 74).
- Episode 77
Dirk Meister on why security starts at line one (episode 77).
- Evan's Augment years
His LinkedIn lists him at Augment to January 2026, with an adviser role from July 2025 (LinkedIn).
- Everpure
Pure Storage takes a new name (Wikipedia).
What to do this week
Five things, in the order Evan’s layers suggest.
- Find your read-back paths. List every API that returns customer data, and close the ones your product only ever needs to receive. The scorecard above is a quick first pass.
- Give your next pen test the source. Scope it with source code and cloud access so the testers spend their time on findings, not guesses. Our penetration testing page shows what that looks like.
- Put the training promise in the contract. If your product uses AI, write down that you don’t train on customer data, and ask your own AI vendors the same questions Scott Dietzen lists on episode 74.
- Count the pull requests nobody security-reviews. The calculator gives you the number. Then write your security practices down where an AI reviewer can apply them.
- Write the logging rule. Decide for each kind of data what can and can’t be logged, and teach it to every new hire. If you’d like a second pair of eyes on any of it, book 15 minutes with us.