0:03My name is Tom Strix. I am a senior principal network engineer here at Cloudflare, and I'm going to talk to you folks through how we've evolved the Cloudflare network over the last 16 years to go from, you know, the teeny tiny startup in 2010 to, you know, a network that can definitely rival some of the hyperscalers from both scale and capabilities. So real quick, who am I? I'm Tom, I've been in part of the network engineering team for the last nine years. I joined Cloudflare in September of 2017 and since then I've worked on both the automation side of things on the software development side in the way that we've configured and provision our network equipment through the operational side as Stan Paseka alluded to earlier in the day -to -day operations and building out that network and running the network on a day -to -day basis.

0:50So like I said the goal of basically today or right now is talk a bit about hey this is where the network was this is what the network looked like and these are the things that we're currently doing to the network to build it and make it ready for that ai revolution that we've been talking about over the last couple of hours and yesterday as well so this is a picture of our san jose deployment in 2019 what you can see here is is a juniper branded mx480 router stuffed to the gills with line cards and it has all of that delightfully yellow fiber optic cable that that tom had uh earlier that he showed and that was what our san jose data center was right that was single data center single edge router connected to a couple of networks and and that's what we built cloud fair on right the the topology in and of itself was incredibly simple and this is still the topology that we're using for a lot of our locations actually today what you see here is up top oh sorry uh could you go back one sledge so up top here's just the internet right is is to to tom's point is the the mobile providers it's your home isp um using anycast you land in in i'm from belgium so for a lot of the my traffic lands in brussels but some of that goes to amsterdam right and that's the anycast component but in and of itself internally within our data centers this is literally the topology or the network topology that we have are used to have in 2016 right

2:25we have that edge router that mx480 i showed you earlier goes down to a top of rack very simple layer two topology and then goes down to our metals the compute nodes the things that serve the traffic that that you request or um from any of our uh customers and we have the exact same topology in the exact same um topology in in amsterdam and in all of the other locations that we have around the world. And that worked really well. But Cloudflare kept growing, right? Like we kept building a business and we're now at more than 20% of the internet or websites go through Cloudflare. So we needed to start understanding, it's like what are some of the things that we focused on before and what are some of the things that we need to focus on now? So before, it's like up until about 2016, 2017, the main driving factor the way that we approached our architecture was it needs to be as simple as possible we need to make sure that it is as easy to reproduce and as easy to debug and understand as possible if anything goes wrong it needs to be very simple that we instantly know this is what the problem is this is how we mitigate it and this is how we fix it right and that was a core component in the way that we approached our architecture but also at the same time like because it's that simple it also makes it incredibly easy to operate makes it incredibly easy to scale out.

3:44It's very easy to kind of just keep doing gradual improvements and keep building on top of it and be happy with it. But unfortunately, that's not always the best aspect, right? It's like if you iteratively and gradually from, you know, 2010 to 2016 to 2026, gradually keep improving and building up top of that, the exact same architecture, you might just get kind of shacks all over. And those shacks aren't always the nice coordinated component that we want. So this is where we kind of are today, right? We were actively working on kind of reducing and changing some of the things that we do. So what you see here is Seattle in 2020. At the bottom here is a QFX 1008 that has eight line cards.

4:31So we can shove eight line cards in there. Here we only have, I think, about three. And you have our data center interconnect up here and that's basically what provides the interconnection it's a lot larger it's a lot bigger and we see that in our topology as well because this is what that topology kind of tries to what that topology builds out to is instead of having a single router in a single location is that's a single point of failure we now have two routers which means that we can take down these routers at any given point out of our network and either have a maintenance do a software upgrade sometimes that can also be a catastrophic failure hardware you know we're we're a software as a service company but we operate infrastructure and infrastructure has the annoying tendency of breaking usually when you least want it to so these routers can break and and we have the redundancy and the capability to just move that traffic immediately over so that our end users don't notice um what happened but below that you have these individual spines that allow us to disaggregate our individual layers.

5:36It's a bit like the availability zones you would have within AWS or within Oracle. We have these availability zones within our metros, within our metropolitan areas. So within Amsterdam, for example, we have, I think at this point, we're at 13 individual availability zones that have some of our compute in there that we can individually splice and dice around so that we can do maintenances on both our hardware and our software to make sure that we still have available capacity. within the metro so that we can keep that traffic as local as possible for as long as possible. And that's where we kind of are, right? Like this is what the majority of our large locations will look like. So this is, for example, that's what Amsterdam will look like, that's what London will look like, that's what Ashburn will look like, New York, name a big city, that's what basically all of these locations look like.

6:27The very big thing here is, and then you might've noticed that in the previous slide, was Amsterdam was a single dataset. At this point in time, we now have at least two different data centers in which Amsterdam lives, and that's a factor of Cloudflare scaling. Cloudflare is becoming larger and larger. We get more and more customers, so we need to build out more servers to accommodate for these customers. Data center space is limited, right? It is a very top priority at this point in time is that build out of gigawatt data centers to put in more compute for AI. We have the same thing for the amount of requests. that we're doing for the AI side of things that we're doing, we also need to put our servers somewhere. So that means we need to have data center space. And as we were growing, we couldn't put these servers in the same data center space anymore.

7:13So that's where that this topology kind of came from was we need to put our servers in different data centers, but that means that the reliability of the interconnection between these data centers is a lot more suspect than within the data center itself, right? A grandmother... with a spade exists outside of the data center, usually they don't let them in inside the data center. Unless they have a visitor pass, then it's fine. But so the majority of the time when you have a cable cut, it's outside of the data center, not inside of the data center. So that causes some resiliency questions, right? It's like, what are we gonna do? How do we deal with this? And that's the architecture that came out of this that allowed us to build out into all of these distinct data centers within the same metropolitan area. And build that out in a reliable and safe manner so that if there is impact, if there is roadworks or any kind of, or if there's power loss or cooling loss, Europe is having a lot of kind of heat waves.

8:10They're not built to deal with constant heat for weeks on end. So power cooling failures are something that we're seeing more and more happen within the data center space within Europe. If that happens, we still need to have a network. We still need to have the capabilities to. deliver and build out for our customers. And that's the topology or the architecture that we have and that we've built over the last couple of years and where we're at now. But that was the big thing, right? So like I said earlier, the architecture principles here is not just the simplicity anymore. It's still a fairly simple topology. It's still a fairly simple network to work on. Whenever we have new hires in the network engineering team, they get on top of this fairly quickly. This is not a complicated rocket. Science, Science Network, and it allowed us to build out for grace. But what are we currently doing?

9:00Because if you're a customer, you may have had a bunch of kind of maintenance notifications over the last couple of months where it's like, hey, we're taking down Ashburn, we're taking down New York, we're doing things in LA, we're doing things in San Jose. And that's basically where we're going. This is the racks. We're doing home deployments. No, this is the rack in my kitchen. Fortunately, it does not run the Cloudflare stack. So this is not where we're going. This is where we're going. Namely, Frankfurt in 2026, this picture is about a month old. This is our entire networks rack. Instead of having a single rack with some network equipment in it, we now have multiple racks. I think we're at four racks with network equipment that will provide both the interconnectivity to external networks, customers as providers, as ISPs.

9:49origin networks all of that as well as internal connectivity because we keep growing we keep adding more and more data center space which also means that we need to have better interconnectivity between these data centers within the same metropolitan area so this is now what our architecture looks like this is what we're building this is the architecture that provides us the confidence to move into the um ai and agentic era to provide that connectivity to our customers and to ourselves because we're a very big dogfooding proponent. So we buy our own products. The big difference here is we still leverage these spines in distinct data centers that allow us to remove that level of compute without impacting the rest of the metro. But we're adding a lot of peering routers that allow us to scale out on a horizontal fashion.

10:37Just keeps adding more and more peering routers allow us to build out more bandwidth to do. engage with our customers. As Stephanie said earlier during the keynote this morning, we're at more than 500 terabits worth of capacity. This is the topology and this is the architecture that enables that. The edge routers for us are a key component to connecting our individual metropolitan areas together. We've built out over the last seven years a massive world -spanning backbone. And the edge routers are a critical component to that because these are the the boxes that connect all of these things together the peering routers are where you know if you're part of an internet exchange or if you have a pni with cloudflare these are the devices that they will land on they aggregate all of the connectivity and provide that as a singular unified thing to our spines that will then um still do the exact same thing through tours and to the metals where we we have our compute with this architecture with this we have the capability at any given time to disable, for example, this spine and all of our workloads will just neatly distribute to the remainder of our capacity so that our customers don't see that or have any impact at that.

11:55So what was the reason? Why did we move from this, right? Like we already had this architecture change that we started pushing through in about 2018, 2019. Why did we need to change? Surely that architecture was already sufficient. But what we... And what we realized is our rate of growth is a lot faster than we originally anticipated when we were building out that architecture in 2018, 2019. So we needed something that allowed us to scale. We needed even more scale. We need to scale faster and we need to scale even larger. And that was the core component here. The only thing that mattered was, yes, we need to keep the resilience. Resilience is important, but we need to get even better at scaling this out. We need to be able to provide silly amounts of capacity to. both our customers as well as internal, to be able to keep that growth going so that the network can never be the bottleneck to what it is that we deploy and how we grow.

12:48That is it for me. This very brief, very short introduction to kind of the ways that we've changed our network and the way that we've grown our network. Thank you very much, folks, and I hope you have a pleasant rest of the conference.