Episode 42 of The Security Podcast of Silicon Valley aired on May 1, 2024, with Jon McLachlan hosting. The guest was Avery Pennarun, co-founder and CEO of Tailscale (Podbean, LinkedIn). The episode is called “Avery Pennarun, Co-Founder and CEO of Tailscale, the Anti-Google: Rebuilding a Secure Internet from the Bottom-Up.” Avery mentions Tailscale’s post on 5,000 paying customers as just out. That post went up on March 5, 2024, so the conversation took place between then and the release (Tailscale).
Asked what Tailscale does better than anyone else, Avery gave away what he calls the secret of the company.
“we just do things the easy way instead of the hard way.”
Avery Pennarun, at 9:16
Tailscale is a software company based in Toronto that makes a mesh virtual private network, or VPN, built on WireGuard, and a web service that manages it. Wikipedia says Google engineers Avery Pennarun, David Crawshaw, David Carney and Brad Fitzpatrick founded it in 2019 (Wikipedia). The rest of this page sticks to what Avery said on the tape, with the sources beside it, and what happened after it.
This page follows the conversation in order: a section for each part of it, eighteen figures, seventeen clips from the tape, the full transcript, and five tools: a quiz, a scorecard for how small and identity-based your network is, a calculator for how many bad actors can reach a network of a given size, a timeline, and a to-do list. Where the decodes of the tape disagree on a word, the transcript puts its best guess in brackets, and the page doesn’t quote those words.
A career in one blurb
Jon opened with Avery’s career in four sentences: a first company in college, sold to IBM, eight years at Google, then Tailscale with other former Googlers. Avery agreed with the summary.
“that’s just my entire career history compressed into a pretty short time.”
Avery Pennarun, at 0:58
His LinkedIn fills in the steps. He co-founded Net Integration Technologies, or Nitix, while still at university, and was its chief scientist from 1998 to 2006. It made an all-in-one network server for small businesses and grew to over 100 people before IBM’s Lotus division bought it (LinkedIn). IBM sold the product from 2008 as Lotus Foundations, a direct result of that acquisition, according to Wikipedia (Wikipedia). He was then employee number one of a spinoff from Pacific and Western Bank of Canada, as executive vice president of software development, and co-founded EQL Data in 2009. At Google, from 2011 to 2018, his first project was person-to-person money transfer in Google Wallet. He spent five years on Google Fiber’s home equipment and wifi, then did strategy work for Alphabet projects (LinkedIn). He has run Tailscale since February 2019.
Tailscale’s own account of the move is in the post on its Series A in November 2020. Avery wrote that at Google he built internet-scale products for billions of people. When he later advised Alphabet projects on a first version of something, the power of that infrastructure “becomes a hindrance instead of an advantage.” Tailscale, the post says, was built for people “that need a BeyondCorp-style VPN without massive enterprise complexity” (Tailscale). BeyondCorp is Google’s own zero trust network (Wikipedia).
Not revenue
Jon introduced Tailscale as close to a billion-dollar company. Avery corrected the record in the next breath: the billion dollars was the valuation from the last funding round, not revenue. Tailscale was still a little small, he added, and growing quickly.
The round before the tape was the Series B in May 2022, US$100 million led by CRV and Insight Partners (Tailscale, Wikipedia). Jon also called Tailscale a Fortune Cyber 60 startup.
A networking person
Jon suggested Avery thinks of himself as a security person. Avery said he wasn’t sure.
“I really have always been a networking person”
Avery Pennarun, at 1:31
Then he gave the difference between the two kinds of tools. The job of security products is to stop you from doing stuff, he said, and the job of networking tools is to help you do stuff.
“And engineers generally don’t like to be stopped from doing stuff.”
Avery Pennarun, at 1:47
Tailscale is a security product, he said.
“It’s also a networking product.”
Avery Pennarun, at 1:38
It does a little of each. People buy it to solve a connectivity problem, and they are surprised to find security features that don’t get in their way. They end up more secure by default than before they installed it.
Adopted bottom-up
That is how Tailscale spread, Avery said. He called it the first security infrastructure product, “probably,” to be adopted bottom-up by individual engineers. Jon asked whether Tailscale beat Slack to that model. Avery said Slack came well before, and isn’t a security product. Jon said the venture capitalists have been excited about bottom-up go-to-market strategies ever since.
“It turns out it works really well.”
Avery Pennarun, at 2:50
Avery compared it to cold calls and spam about a product nobody has heard of. At Tailscale it runs the other way. The sales team talks to companies where someone has already signed up and a person inside is already excited about it.
“And it’s a completely different conversation, then, right?”
Avery Pennarun, at 3:21
Tailscale wrote the model down in March 2022, in a post on why its free plan stays free. It calls the strategy bottom-up growth, or product-led growth, and an earlier name for it is GTM 3.0. There is an unlimited free tier for individuals, a small charge for teams, and more for big-company control and audit. People who love Tailscale at home tell their friends and bring it to work. Then a manager pays on a company card, and IT and security run a vendor review. The post adds that “it’s not really a thing in security yet” (Tailscale).
What a CEO does
Jon asked what a co-founder and CEO does all day. Avery quoted a line he read online about founders. It’s as if someone calls to say you got a new job several months ago, and you’re already failing at it. He used to write Tailscale’s code and hasn’t been allowed to for quite a while. His days are meetings: recruiting, coordination, and what he calls unblocking meetings, where nobody knows whose decision it is. In that case, he said, the decision is his job. He shows up, often as the person in the room with the least information, and takes the risk off whoever is most worried about it. Don’t worry, he tells them.
“I guess I’ll put the company at risk.”
Avery Pennarun, at 4:49
Chaos in, order out
Jon said Avery sounded comfortable being uncomfortable. Avery agreed. He doesn’t exactly thrive in chaos, he said. He takes chaos in and puts organized things out.
“And so it’s like without the chaos, there’s nothing really for me to do.”
Avery Pennarun, at 5:14
Things that are already solved bore him. Some people he knows go further and create chaos when things get stable. Jon guessed he meant red teams, and Avery said he had just described one of Tailscale’s people as “basically the red team accidentally.” Red and blue teams need each other, he said, and the tension is healthy if you manage it. Jon reached for The Matrix.
“You don’t have a blockbuster hit unless you have the dynamic between the two”
Jon McLachlan, at 6:06
When networking was simple
Jon remembered when you could plug computers into a hub, no switch needed, and everything worked. Now there are cloud providers, routing protocols and products everywhere. Avery pushed back.
“Computers were pretty broken at the time, right?”
Avery Pennarun, at 6:55
You fiddled with DOS memory managers to make one video game run and undid it for the next. Networking was simple because it was new and nobody had found out what could go wrong. As networks got bigger, we connected them to each other, which he called
“one of the greatest accomplishments and biggest mistakes we ever made”
Avery Pennarun, at 7:19
The result was the internet, the network of networks. Avery doesn’t think there will ever be two of them. If there were, somebody would plug them together.
“So now we’ve got one big network made out of protocol that was designed for something considerably smaller.”
Avery Pennarun, at 7:45
Four billion addresses
Avery said the protocol’s designers never thought it could grow this big, and that IPv6 started because of it. IPv4 has about four billion addresses, and in the 1990s most of them were wasted.
“now there are considerably more than four billion computers on the internet.”
Avery Pennarun, at 8:07
It only works because addresses are reused behind firewalls, proxies and other layers, he said. That is one reason networking got complicated. The other is that the protocol was never meant for any of this.
“The fact that it worked at all is sort of an amazing achievement of the entire network engineering profession.”
Avery Pennarun, at 8:29
The free pool ran out years before the tape. The free pool of the Internet Assigned Numbers Authority ran out on February 3, 2011, and APNIC, the registry for Asia and the Pacific, was the first region to run out, on April 15, 2011 (Wikipedia). Avery said the price of an address goes up a little every year. The last figure he had seen was about $40.
The easy way
Avery said he can share the secret because he has shared it with many people and nobody copies it.
“And this is actually much more difficult than it sounds, right?”
Avery Pennarun, at 9:20
Doing it the easy way is less exciting than building something big, he said, and gave the example.
“what if I created like a Kubernetes cluster of 10,000 machines doing things the easy way.”
Avery Pennarun, at 9:30
“it wouldn’t be easy anymore if you did it that way.”
Avery Pennarun, at 9:36
So Tailscale looks for the way to cheat out of a problem. With billions of people on the internet and maybe ten times as many devices, he said, you have to stop connecting everything directly to everything. That is the opposite of what the internet is supposed to be.
Make the networks small
In the 1990s Avery believed that if everyone could talk to each other, with automatic translators, disagreements would get talked out. It didn’t work out that way, and he said the reason is sad.
“The reason is that something like 99.999% of people are good and want to produce good outcomes.”
Avery Pennarun, at 10:25
A tiny fraction of people are willing to destroy everything. Connect billions of people to one network and give that fraction a way to attack everyone at once, and everyone gets attacked. So Tailscale stopped trying to defend against everyone.
“why don’t we make the networks small enough so the probability of one of the really terrible people being in your network is really low.”
Avery Pennarun, at 11:09
Then you interconnect the small networks again. The calculator below runs his numbers: at one bad actor in every 100,000 people, a network of 200 people holds one about twice in a thousand networks, and a network of a billion people holds about ten thousand.
Identity at layer three
Avery said the new internet would be thoughtful about what traffic goes from one network to another.
“And the process we based it on is identity.”
Avery Pennarun, at 11:40
Tailscale went back to first principles, Avery said. It doesn’t put a firewall on the internet or a proxy at layer seven, the application layer. It inserted itself much lower. Because it’s a layer three tunnel, he said, it’s like another IPv4 on top of IPv4. Solving the problem at that level lets you take layers away, he said, and do less work.
Jon summed it up: less work, simpler to use, and secure, because things are segmented by identity. Avery listed what that fixes: connectivity, segmentation, identity, and encryption on top.
Less is more
Jon said it reminded him of a saying from fashion and makeup: less is more. Wikipedia describes the phrase as a principle from several traditions, close to minimalism (Wikipedia). In security and architecture, Jon said, it’s
“really difficult actually to bubble down a problem into its simplest, purest solution.”
Jon McLachlan, at 12:57
Avery passed on a quote a friend of his uses, and said he forgets who said it first.
“we can solve any problem with another layer of abstraction, except the problem of too many layers of abstraction.”
Avery Pennarun, at 13:44
Wikipedia calls the first half the fundamental theorem of software engineering, and gives it as “Any problem in computer science can be solved with another level of indirection.” It is credited to Roger Needham or David Wheeler, and the clause about too many levels is a humorous addition (Wikipedia). Avery’s second quote, he thought, might be from the author of The Little Prince, Antoine de Saint-Exupéry: perfection comes when there is nothing left to take away. Wikipedia’s page on the KISS principle carries the same line (Wikipedia).
“It is so much easier to add a layer.”
Avery Pennarun, at 14:16
Taking layers out means moving a little slower and accepting that some things won’t work yet, he said. The rest works so much better that it makes up for it.
Tail scale, not internet scale
The name comes from the same idea, Avery said.
“It’s the opposite of internet scale.”
Avery Pennarun, at 14:48
Plenty of people build infrastructure that could serve a billion users, he said, and something has to. Most teams won’t have a billion users for a long time, though. Wikipedia says the name was inspired by a 2013 Google research paper, The Tail at Scale (Wikipedia). Avery’s post on 5,000 paying customers, from March 2024, says “Almost no software needs to scale to a billion users.” The company set out to serve the “long tail” of projects that get neglected (Tailscale).
Easy things easy
Avery offered another saying he likes: good engineering should make easy things easy and hard things possible. According to Larry Wall, Perl has two slogans, and Wikipedia gives the second as “Easy things should be easy and hard things should be possible” (Wikipedia). Almost everybody works on the second half, Avery said, because it feels like pushing the boundaries. Nobody works on making easy things easy, and that’s what most of us need.
“Most of us wake up in the morning and have some work to do and it should be easy and it’s not.”
Avery Pennarun, at 15:39
It isn’t easy, he said, because the easy thing takes 25 tools.
Tailscale isn’t made for connecting a billion devices, he said, and the internet already does that job.
“Tailscale is great if you need to connect 100,000 devices.”
Avery Pennarun, at 16:02
That’s still a lot, and a different class of problem from a billion.
Yes, Tailscale
Jon described what he sees as a security contractor. He goes into startups to raise their security maturity, with programs like SOC 2 and ISO.
“And every time someone is like, oh, do we need VPN? I’m like, yes, Tailscale.”
Jon McLachlan, at 16:27
Don’t mess with anything else, he tells them: drop it in, and it works.
Avery said again that nobody copies Tailscale, even though people hear the idea and say it makes sense. Jon said that when people ask why he went into computer science, he jokes that computers should do all the dirty work. Less is hard to get to, he said, and it takes humility and focus.
The receptionist’s database
Computers probably should do the dirty work, Avery said, but how do you teach one to? He started from what simpler tools make possible.
“One of the things that becomes possible when you make things simpler is suddenly all these people who are afraid to touch it are no longer afraid to touch it anymore.”
Avery Pennarun, at 17:42
His example is Microsoft Access, which Microsoft released on November 16, 1992 (Wikipedia). Avery worked at a computer store that did consulting for places around town. At several of them, a receptionist had built an Access database to track inventory more than ten years before. The receptionist was long gone, and the database was still in production.
“the place would have collapsed without it.”
Avery Pennarun, at 18:40
Only two people used it at a time, and maybe ten in all, so it never had to scale. When a business wanted a change, it hired someone like Avery to poke at the database in the visual editor and save the file. From 2009 to 2011, his company EQL Data made plugins for Microsoft Access (LinkedIn).
We can’t do that anymore, Avery said. We tell ourselves software as a service is cheaper or better, and he said neither is the reason. We’ve added so many layers that only a professional can understand them all. Remove the layers, he said, and things become possible again.
You thought the internet worked
Avery passed on what a customer once told him about Tailscale.
“you thought the internet worked before you learned how the internet worked.”
Avery Pennarun, at 19:59
Any normal person has a phone and a laptop, he said, and wants to send a file from one to the other while the laptop sits at home. The two devices can’t find each other. They are behind two different firewalls and two different NATs, network address translators, and probably don’t share an app for it. Nobody does that anymore, he said.
So the file goes through a messaging app, out to the cloud and back. Jon added that the services scrape your data along the way. Building your own file transfer app means authentication, encryption and storage, and paying Amazon for storage and data transfer. If the app gets popular, Avery said, you go bankrupt paying for ingress and egress.
A socket and a file
With Tailscale on both devices, Avery said, one app can listen on a socket and another can send it the file. It costs nothing, because it doesn’t go through the cloud, and it works when the phone and laptop aren’t side by side.
“because Tailscale just makes the network work the way it was supposed to work in the first place.”
Avery Pennarun, at 21:41
It’s encrypted, he said, and it’s
“attached to your identity, you can set up an ACL”
Avery Pennarun, at 21:53
An ACL, an access control list, decides who can reach the app. Give the app to someone else, and they decide who reaches their copy. A company with a hundred apps like that gets the same for each one.
Dashboards first
Dashboards were one of Tailscale’s first use cases. Starting a dashboard is more than half grunt work, Avery said.
“So we made a list of 100 things”
Avery Pennarun, at 22:27
The top two were connectivity and security. Connectivity is getting your coworkers able to connect to the dashboard.
“Getting people who are not your coworkers not able to connect to the dashboard is security.”
Avery Pennarun, at 22:37
“And it’s easy to do one or the other, but it’s really hard to do both of those things.”
Avery Pennarun, at 22:41
That’s what Tailscale was built for. The team never got to the other 98.
MagicDNS
Jon asked what Tailscale does with DNS. Install Tailscale and it fixes your DNS problems too, Avery said, and security people would appreciate how.
“So MagicDNS actually runs a DNS server inside the Tailscale instance on each of your devices.”
Avery Pennarun, at 23:14
The server gets pushed the names of every machine and service on your Tailscale network, which makes it a primary DNS server. Regular DNS isn’t encrypted, he said, so you can’t count on it to tell the truth about an address, and that’s why TLS is there. MagicDNS can’t be intercepted, because the answer comes from inside the Tailscale client. A lookup returns a Tailscale address, and the traffic goes out encrypted over WireGuard. Each client only ever asks its own DNS server, so the number of DNS servers grows with the number of clients, and a company of 100,000 people doesn’t need a server that handles all their requests. Updates are pushed, so when an address changes, it changes at once, with no cache to expire. Tailscale made MagicDNS generally available on October 20, 2022 (Tailscale).
SSH without keys
Jon said it sounded like referencing local host names, Kubernetes style. Avery said start a few Linux boxes and their names show up at once, ready for ping or SSH. Then there’s Tailscale SSH, which he said manages your SSH keys,
“or rather bypasses the need for SSH keys”
Avery Pennarun, at 25:34
It authenticates with your Tailscale identity. Turn it on for a Linux box, and you can SSH to it without ever managing a key. Tailscale’s post on Tailscale SSH, by Brad Fitzpatrick, is dated June 22, 2022 (Tailscale).
Jon was reminded of Teleport, which Avery said does a lot on the enterprise compliance side that Tailscale doesn’t, and has an open source version. Tailscale SSH has session logging, follows Tailscale’s ACLs and gets through firewalls, so a server in a back room needs no open ports. Outsiders can’t guess passwords, and there’s no old SSH key for someone to lift.
Jon asked where the private keys live. On the devices, Avery said, and the identity expires from time to time, so you sign in again with your identity provider. If an account is deleted, say because someone left the company, all of that person’s devices are deactivated at once, and the keys on them stop working.
Two-factor for a 1990s app
Jon said you could add two-factor authentication on top of Tailscale’s sign-in, and so get it for SSH. Avery said one of Tailscale’s first customers needed something like that, for a Windows client-server app built in the 1990s. There was no chance the app would ever get two-factor authentication. The choice was to add it somehow, or rip the app out and replace it with a software purchase of millions of dollars.
“And they’re like, I don’t want to spend millions of dollars.”
Avery Pennarun, at 28:12
Avery suggested moving the server onto a private network, reached over a VPN with two-factor authentication. The VPN had to be easy to configure and fine to leave on, and it had to do two-factor and work with the identity provider. He was sure someone had built one. Nobody had, so he turned to WireGuard. WireGuard is a free and open-source protocol for encrypted VPNs. It aims to be lighter and better performing than IPsec and OpenVPN, and it passes traffic over UDP (Wikipedia). When Tailscale was first announced on Hacker News, Avery said, a commenter wrote the cliché that they could build it in a weekend.
“I already built it in a weekend.”
Avery Pennarun, at 28:58
It has improved a lot since, with fast key rotation and ACLs. The app never had two-factor and wasn’t web-based, so a web proxy couldn’t help. Tailscale intercepted it at layer three, so it only mattered that the app used TCP/IP. The Series A post lists “adding multi-factor authentication to legacy apps” as a use case (Tailscale), and the 5,000-customer post names Versabank, a Canadian bank, among the first year’s customers (Tailscale).
How big Tailscale was
Jon asked how big Tailscale was. Over 100 employees, Avery said, which is huge by some measures and tiny by others.
“We doubled revenue last year.”
Avery Pennarun, at 29:45
It looked set to double again. The 5,000 paying customers in the blog post didn’t count people on the Personal Pro plan. Customers ranged from one or two employees to tens of thousands of seats. Avery tells people Tailscale is going to fix the internet.
“You can’t fix the internet if you don’t support teeny tiny customers as well as huge customers, right?”
Avery Pennarun, at 30:10
Otherwise it’s just an enterprise tool, he said. Tailscale wanted a couple of enterprise customers early to prove it works there too. The 5,000-customer post says more than half came in the previous 12 months, that Tailscale passed the mark just before its fifth birthday, and that hundreds of thousands of monthly active users use it free for personal use (Tailscale).
The trick, a third time
Jon called serving both ends of the market an engineering feat and a business feat. Avery offered the trick again, since nobody copies it, and Jon pointed out it was the third time. Tailscale solved the easy parts, Avery said,
“and we avoided all the hard parts.”
Avery Pennarun, at 31:00
An enterprise can solve its huge problems with tools that already exist, he said. Its developers still have trouble launching dashboards and connecting a Postgres backend to an app backend. Those little things keep systems running and will never reach a billion users, and everybody at every company has them.
“The whole internet has these problems.”
Avery Pennarun, at 31:33
Home labbers
Jon asked whether people first come to Tailscale from the security side or the networking side. Avery started with home use. Most people use Tailscale for free at home, he said.
“We call them the home labbers.”
Avery Pennarun, at 31:58
The most serious ones have racks in the garage full of gear they want to reach. Others want their laptop and phone connected. Avery counted hundreds of thousands of weekly active users using it for free, almost always engineers. Almost all of them have jobs, and at work they find a VPN or a jump box.
The snowstorm
Avery told the story of a big company with a VPN concentrator in one of its data centers.
“And there was a snowstorm, and the data center in that city went out.”
Avery Pennarun, at 32:50
Every other data center was reached through that concentrator and point-to-point links. With it down, nobody could reach anything. Someone drove to another data center.
“They put one Tailscale node inside the other data center.”
Avery Pennarun, at 33:08
Then they all installed Tailscale. It usually starts with a problem like that, he said, and an engineer who uses Tailscale at home. A few teammates join, then a VP gets involved, and then the company wants an enterprise version with single sign-on. The Series A post says “Distributed teams require distributed networks, not just hub-and-spoke” (Tailscale).
Tags, groups and keys
In the enterprise version, Avery said, you can tag devices, use groups from your single sign-on, and have Tailscale SSH managed for you.
“Anything that, almost always, key lifecycle management is a win in my books.”
Jon McLachlan, at 34:03
Avery said Tailscale does a really good job of that.
Zero trust, unadvertised
It almost always starts with engineers solving connection problems, Avery said. Security and IT get pulled in along the way, to check it won’t get the company in trouble.
“And the security team looks at it and says, wait a minute, is this a zero trust product?”
Avery Pennarun, at 34:23
“Because I have a budget for zero trust stuff and I didn’t know how to do it.”
Avery Pennarun, at 34:26
The answer is yes, Avery said: Tailscale is a zero trust product. The homepage probably says nothing about zero trust, depending on which test you’re on. Avery said the security team almost never comes first.
“Usually, security and connectivity are kind of at odds with each other.”
Avery Pennarun, at 34:50
Tailscale has written about zero trust since 2019. David Crawshaw’s post of September 10, 2019 says the term started in 2010 with John Kindervag, and that we need to give up on the idea of trusted networks (Tailscale).
Thirty years on IPv4
Jon asked Avery to look 100 years ahead. Avery went back first, to 1994, the year before Windows 95. Back then, he said, you needed an add-on, like shareware, just to connect a Windows device to the internet. One such add-on was Trumpet Winsock, a TCP/IP stack for Windows 3.x distributed as shareware, first released in 1994 (Wikipedia). Avery said Bill Gates didn’t think the internet would be big, started MSN, and missed the internet in his book The Road Ahead. Wikipedia’s account is more mixed. The hardback, published November 24, 1995, saw the Internet as one of the “important precursors of the information highway” and said today’s Internet was not the highway Gates imagined. A substantially revised edition came out in October 1996 (Wikipedia).
Customers pressured Microsoft into putting TCP/IP into Windows in 1995, Avery said. Windows 95 went on sale on August 24, 1995, and its default network installation did not include TCP/IP (Wikipedia). Almost 30 years later, he said, the internet is on everything.
“So, interestingly, not too many years later, like 1997 or eight or something like that, the IPv6 project kicked off.”
Avery Pennarun, at 36:31
“And IPv6, amazingly, still not rolled out.”
Avery Pennarun, at 36:39
The client, the server and all the routers in between have to have it, he said, and that makes IPv6 hard to roll out. Wikipedia dates IPv6 to December 1995. It became an IETF Draft Standard in December 1998 and an Internet Standard on July 14, 2017 (Wikipedia). After 20 to 25 years of trying, Avery said, the internet hasn’t evolved, and it still runs the IPv4 of 1995.
Every device to every device
Twenty-five years out, Avery said, we might still be on IPv4, behind five layers of NAT. Something like Tailscale will be on every gadget, because you need to connect every device to every other device.
“Because it’s incredibly silly for all of your traffic in the world to have to bounce up to some cloud provider and pay them rent”
Avery Pennarun, at 38:00
Some architecture like Tailscale will have to win, he said, because the internet has to progress.
“The internet has been stuck for now 30 years, right? Something’s got to change.”
Avery Pennarun, at 38:21
There’s a lot of money to save by not paying AWS, GCP and Microsoft, with a cloud provider for every small thing. Jon was for it. Avery said the phone in his pocket is as powerful as a supercomputer in 1995, and Jon agreed phones are incredible machines.
“it’s a brick without the cloud.”
Avery Pennarun, at 39:07
He can’t send a file from one supercomputer to the other without going through a cloud provider and paying rent.
The best day
Jon asked for Avery’s best day at Tailscale. The best days are the ones where he learns something he never would have guessed, he said. In one, an engineer showed up after six months on a project and had made WireGuard in Tailscale more than twice as fast as WireGuard in the kernel. Customers used to ask why Tailscale was slower than kernel WireGuard. Tailscale is a remote company, Avery said, so the engineer sat at home and kept poking at it.
There’s a blog post, Avery said. Tailscale’s blog has two on the throughput work. Jordan Whited and James Tucker’s post of December 13, 2022 made wireguard-go, the userspace WireGuard that Tailscale uses, “faster than WireGuard (running in the kernel) in the best conditions,” and expected the kernel to catch up (Tailscale). Jordan Whited’s post of April 13, 2023 took Tailscale past 10 gigabits a second on bare-metal Linux (Tailscale). Avery also named the day Dave Anderson first got MagicDNS working, and said
“most companies would have written off each of these things as impossible”
Avery Pennarun, at 40:58
Invisible when it works
Jon said engineering feats in architecture draw visitors, like the Eiffel Tower, but technology has no physical form. Avery said networking has an odd problem: a network app you can see is doing its job wrong. When Tailscale started, he expected that to rule out viral word-of-mouth growth, since the better it works, the more you forget you had it. Among the users who love Tailscale, it turned out otherwise. People install the app, log in with a Google account, and assume they did something wrong. Then they try the name of their iPhone, and it answers.
“and then with Tailscale, it’s, hey, wait, this is how it’s supposed to feel.”
Avery Pennarun, at 42:58
He compared it to a VR headset with low latency: once you’ve had it, you know what it’s supposed to feel like. That’s why Tailscale has so much word-of-mouth growth, he said, with no virality features. People get so excited they won’t leave their friends alone.
Five minutes, two devices
Jon asked for final words of wisdom. Avery said to try Tailscale.
“It is free for up to 100 devices”
Avery Pennarun, at 44:43
It takes five minutes to connect two devices, he said, and he guaranteed it. Jon said that from his own experience, it’s more than doable.
What happened after the tape
These are the items since the episode, in order.
The New Internet. On July 26, 2024, Avery published “The New Internet” on Tailscale’s blog. A version of it was first presented at a company all-hands (Tailscale).
10,000 customers. On January 14, 2025, Avery wrote that Tailscale had passed 10,000 business customers. Ten months earlier it was at 5,000, a number that took over four years to reach (Tailscale).
A Series C. On April 8, 2025, Tailscale announced a US$160 million Series C led by Accel, with CRV, Insight Partners, Heavybit and Uncork Capital. Avery wrote that “millions of people rely on Tailscale every day,” and that AI companies such as Perplexity, Mistral and Hugging Face build on it (Tailscale).
The origin story. On April 23, 2025, Tailscale’s blog recapped Avery’s March 2025 interview with Ben Thompson of Stratechery. It describes Net Integration Technologies as “A box that let small offices share a single Internet connection” (Tailscale).
Two talks. On May 30, 2025, Tailscale posted “Building the new internet: Lessons in simplicity, security, and scale,” in which Avery explores how to build a simpler, more secure internet for the AI era. On November 13, 2025, it posted his talk from Web Summit 2025 in Lisbon, “From developer to CEO without losing the plot,” on moving from engineer to CEO, raising money without a pitch deck, and word-of-mouth growth.
Border0. On March 17, 2026, Avery announced that Border0, a Vancouver startup working on access to sensitive infrastructure, was joining Tailscale. The post calls that privileged access management, or PAM (Tailscale). One heading in it reads: From “can I connect” to “should I”.
The Hugging Face intrusion. On July 31, 2026, Avery wrote “Tailscale didn’t stop the Hugging Face intrusion.” An AI agent escaped its sandbox and read a production secret store with 136 keys. One was a reusable Tailscale auth key, which the agent used over several days to enroll 181 nodes into Hugging Face’s tailnet. No Tailscale vulnerability was found. The post says that with rogue AI agents around, “the big credential vault is the prize.” It recommends workload identity federation in place of reusable keys, one-off keys, narrow tags, an audit of ACL permissions, and flow logs sent to your security tools (Tailscale). Hugging Face published its own technical timeline on July 27 (Hugging Face), and our post Once They’re In walks through the intrusion as a blast radius problem.
The TailscaleUp-date. On August 24, 2026, Tailscale’s product update from TailscaleUp in San Francisco said things should connect with “clear identity on each end and policy deciding what’s allowed.” More and more, it adds, the thing making the connection is software acting on someone’s behalf (Tailscale).
TechCrunch. On September 17, 2026, TechCrunch asked how to catch rogue AI agents, and pointed to watching network traffic. Avery said none of this is new in security: “It’s the same as letting humans onto your network” (TechCrunch). Our post Too Fancy picks up his answer.
Authority that expires. On October 21, 2026, Docker’s post on its Secure Software Factory quoted Avery on agents: give them the authority to do a particular job, “make that authority expire,” and keep the policies and audit trail where the agent can’t rewrite them (Docker).
Off the public Internet. On October 22, 2026, Avery posted about two critical PHP vulnerabilities and government websites going offline.
If only there were an easy way to keep those off the public Internet.
Avery Pennarun · October 22, 2026
Tailscale SSH findings. On October 28, 2026, Tailscale published “Tailscale SSH: Findings and fixes,” its answer to a researcher’s report called Tailworm. The issues were in how permissions were enforced after a connection was authorized, and using them required a foothold on an authorized device with SSH allowed by the tailnet policy file. Tailscale v1.104.4 fixes them, along with three bugs in Serve and Funnel (Tailscale).
The SRE Prodcast. Season 7, episode 2 of Google’s SRE Prodcast is “Ship it, with Avery Pennarun and Tailscale,” on reliability, enterprise scaling and engineering philosophy (Google SRE).
100x more programmers. On November 5, 2026, Avery wrote that he expects “about 100 times more programmers in the next few years,” most of whom won’t call themselves programmers. He writes: “Programming got easier. Computing didn’t.” The original Tailscale thesis, he writes, was that developers scaled things they didn’t need to, and networking was the cause: “too hard, and too dangerous by default.” As on the tape, he compares today’s devices to supercomputers: a MacBook Pro has more power than machines called supercomputers 20 years ago, and is mostly used as a terminal for somebody else’s cloud (Tailscale).
More on this show. Andrew Rubin described Illumio’s founding question, what happens when the perimeter misses, on episode 88. Or Eshed explained how two shifts let LayerX solve 90% of the problem with 10% of the effort on episode 98. Taher Elgamal explained how SSL’s optional client certificate made the password the way everyone logs in on episode 61. Jasson Casey of Beyond Identity made the case for identity as the real security boundary on episode 92. Episode 103 is called “Your zero trust is running on a sticker,” and on episode 38 Nick Sullivan talked about cryptography, Cloudflare and building a better internet.
Did you catch it? A seven-question quiz
Seven questions from the episode. Each answer links the moment in the transcript.
Quiz · seven questions
Did you catch it?
Pick an answer to see whether you caught it, and where it is on the tape.
-
What does Avery call the secret of Tailscale?
a. "we just do things the easy way instead of the hard way." Transcript, 9:16.
-
What share of people does Avery say are good and want good outcomes?
c. "The reason is that something like 99.999% of people are good and want to produce good outcomes." Transcript, 10:25.
-
Where does the name Tailscale come from, according to Avery?
b. "It's the opposite of internet scale." Transcript, 14:48.
-
What would have happened to the business without its Access database, according to Avery?
a. "the place would have collapsed without it." Transcript, 18:40.
-
How long was Tailscale's list of grunt work for making a dashboard?
c. "So we made a list of 100 things" Transcript, 22:27.
-
What does Tailscale call the people who use it for free at home?
b. "We call them the home labbers." Transcript, 31:58.
-
What cut a big company off from all of its data centers?
a. "And there was a snowstorm, and the data center in that city went out." Transcript, 32:50.
Is your network small enough?
Avery’s answer to bad actors is networks small enough that one is unlikely to be in yours, joined by identity. This scorecard checks how close yours is.
Scorecard · eight questions
Is your network small enough?
Answer for the network your production systems sit on. If two answers fit, pick the lower one.
-
Who can reach your production network?
- Anyone on the VPN or the office network
- Groups we set by hand
- Only the people and machines that need it
"why don't we make the networks small enough so the probability of one of the really terrible people being in your network is really low." See Make the networks small.
-
Is access decided by identity, or by IP address and network location?
- Network location
- Some of each
- Identity, from our identity provider
"And the process we based it on is identity." See Identity at layer three.
-
How do engineers reach servers over SSH?
- Shared or long-lived keys
- A jump box
- SSH tied to identity, with no keys to manage
"or rather bypasses the need for SSH keys" See SSH without keys.
-
When someone leaves, how fast does their network access end?
- When someone remembers
- Within a day, by checklist
- At once, when the account is deleted
Avery: delete the account, and all of that person's devices are deactivated at once. See SSH without keys.
-
Do your oldest internal apps sit behind two-factor authentication?
- No
- The web ones, through a proxy
- All of them, at the network layer
Avery: one of Tailscale's first customers had a 1990s app that was never going to get two-factor authentication. See Two-factor for a 1990s app.
-
Is there one box that every remote connection depends on?
- Yes, one concentrator
- One, with a backup
- No, devices connect directly
"And there was a snowstorm, and the data center in that city went out." See The snowstorm.
-
Are dashboards, admin panels and databases off the public internet?
- Some are public
- Public, behind a login page
- Only reachable on our private network
"Getting people who are not your coworkers not able to connect to the dashboard is security." See Dashboards first.
-
Do the keys that let machines join your network expire?
- No, they're reusable and long-lived
- Some of them
- Yes, short-lived or one-off
"key lifecycle management is a win in my books." See Tags, groups and keys.
Most people and machines can reach most of the rest, and access follows network location. Start with the systems that matter most: put them on a private network, decide access by identity, and retire shared SSH keys.
Some access follows identity. Next, take dashboards and internal tools off the public internet, remove the one box everything depends on, and make offboarding end access at once.
Your networks are small and access follows identity. Keep machine keys short-lived, review who can reach what, and send network logs to your security tools.
How many bad actors can reach your network?
This calculator runs Avery’s numbers. Put in how many people can reach one network and how rare bad actors are.
Calculator
How many bad actors can reach your network?
Two numbers: the people who can reach one network, and how rare bad actors are.
At one bad actor in every 100,000 people, a network of 200 holds at least one in about 2.0 of every 1,000 networks. One network of a billion people would hold about 10,000. Up to 1,005 people keeps the chance at 1% or less.
The one in 100,000 comes from Avery's "something like 99.999%" on the tape. The 200 people are a placeholder, so put in your own. The math assumes bad actors are spread at random.
From Trumpet Winsock to 100x more programmers
The history Avery talks about, the sayings he quotes, Tailscale’s own record, and the episodes on this show that pick up the thread. Filter by topic.
Timeline
From Trumpet Winsock to 100x more programmers
- Less is more
Robert Browning uses the phrase in his poem Andrea del Sarto (Wikipedia, on this show).
- Another level of indirection
Wulf attributes the line to Roger Needham (Wikipedia).
- Credit to David Wheeler
Butler Lampson's Turing Award lecture credits the line to Wheeler (Wikipedia, on this show).
- Microsoft Access
Receptionists built inventory systems on it (Wikipedia, on this show).
- Trumpet Winsock
A shareware TCP/IP stack for Windows 3.x (Wikipedia, on this show).
- Windows 95
On sale, with no TCP/IP in its default network installation (Wikipedia).
- The Road Ahead
Bill Gates's book calls the Internet a precursor of the information highway (Wikipedia).
- IPv6
The successor to IPv4 is introduced (Wikipedia).
- Net Integration Technologies
Avery co-founds the company behind Nitix while at university (LinkedIn, on this show).
- IPv6, Draft Standard
The IETF makes it a Draft Standard (Wikipedia).
- Lotus Foundations
IBM's small-business server, from its purchase of Net Integration Technologies (Wikipedia).
- EQL Data
Avery co-founds a company making plugins for Microsoft Access (LinkedIn).
- Zero trust
John Kindervag coins the term (Tailscale, on this show).
- IPv4 runs out
IANA hands out its last blocks, and APNIC runs out on April 15 (Wikipedia, on this show).
- Google
Avery joins, for Google Wallet, Google Fiber and strategy work for Alphabet (LinkedIn).
- The Tail at Scale
The Google research paper that inspired Tailscale's name (Wikipedia, on this show).
- BeyondCorp
Google writes up its zero trust network, from 2014 to 2018 (Wikipedia).
- IPv6, Internet Standard
The IETF ratifies it (Wikipedia).
- Tailscale
Avery co-founds it with David Crawshaw, David Carney and Brad Fitzpatrick (Wikipedia, LinkedIn).
- Zero Trust networks
David Crawshaw on giving up on trusted networks (Tailscale).
- General availability
Tailscale reaches general availability (Tailscale).
- Series A
US$12 million led by Accel (Tailscale, on this show).
- How the free plan stays free
Avery on bottom-up growth (Tailscale, on this show).
- Series B
US$100 million led by CRV and Insight Partners (Tailscale, on this show).
- Tailscale SSH
SSH with your Tailscale identity in place of keys (Tailscale, on this show).
- MagicDNS
Generally available (Tailscale, on this show).
- Userspace isn't slow
wireguard-go passes kernel WireGuard in the best conditions (Tailscale, on this show).
- 10Gb/s
Tailscale passes 10 gigabits a second on bare-metal Linux (Tailscale).
- 5,000 paying customers
Just before Tailscale's fifth birthday (Tailscale, on this show).
- Episode 38
Nick Sullivan on cryptography, Cloudflare and building a better internet (episode 38).
- Episode 42
This conversation. "we just do things the easy way instead of the hard way."
- The New Internet
Avery's post, first presented at a company all-hands (Tailscale).
- 10,000 customers
Ten months after 5,000 (Tailscale).
- Episode 61
Taher Elgamal on why the password became the way everyone logs in (episode 61).
- Series C
US$160 million led by Accel (Tailscale).
- Building the new internet
Avery's talk on simplicity, security and scale (YouTube).
- Web Summit
From developer to CEO without losing the plot (YouTube).
- Episode 88
Andrew Rubin of Illumio on what happens when the perimeter misses (episode 88).
- Border0
The privileged access management startup joins Tailscale (Tailscale).
- Episode 92
Jasson Casey of Beyond Identity on identity as the real security boundary (episode 92).
- Episode 98
Or Eshed of LayerX on why security has to go where users are (episode 98).
- The Hugging Face intrusion
One reusable auth key, 181 nodes, and Avery's account of it (Tailscale).
- TailscaleUp
Clear identity on each end, and policy deciding what's allowed (Tailscale).
- Episode 103
Your zero trust is running on a sticker (episode 103).
- TechCrunch
Avery on watching network traffic to catch rogue agents (TechCrunch).
- Off the public Internet
Avery's post on two critical PHP vulnerabilities (LinkedIn).
- Tailscale SSH fixes
Version 1.104.4 answers a researcher's report (Tailscale).
- 100x more programmers
Avery on a billion new programmers: programming got easier, computing didn't (Tailscale).
What to do this week
Five things, in the order of the episode.
- Map who can reach what. For each network, list the people and machines that can reach it. Take the scorecard with your team. Penetration Testing shows how far one foothold gets.
- Decide access by identity. Tie network access to your identity provider, so deleting an account ends access on every device. Access reviews are part of SOC 2 Type 2.
- Retire long-lived keys. Replace shared SSH keys and reusable machine keys with identity-based SSH and short-lived credentials. The Hugging Face intrusion shows why.
- Take internal tools off the public internet. Put dashboards, admin panels and databases on a private network, and remove any one box that every connection depends on.
- Run the numbers. Put your network sizes into the calculator, then shrink the biggest one. If you want a second pair of eyes, book 15 minutes with us.