After changing enterprise infrastructure forever, Kubernetes takes a victory lap

For the most part, people who work in technology tend not to dwell on the past; they like to think of themselves as focused on creating the future. But from time to time there are exceptions, and hundreds of people involved with the Kubernetes container-orchestration project came together Thursday night to celebrate the tenth birthday of an open-source project that changed the world.

Designed as a more accessible version of Google's famous Borg internal infrastructure for managing Linux containers, Kubernetes has become the second-most widely used open-source project in the world, behind only Linux itself. Over the course of an evening that ran about 45 minutes over time (a delicious metaphor for a container scheduler many consider a little bloated) a parade of speakers at Google's Bay View campus paid tribute to the community that built and nurtured the code that underpins so much of modern cloud computing.

According to a recent survey conducted by Kubernetes host Cloud Native Computing Foundation, 84% of companies are either running Kubernetes in production or considering it, said Chen Goldberg, general manager and vice president of engineering for Kubernetes and serverless at Google Cloud.

In the early years of the project, it was hard to imagine such an outcome given how many things were simply broken; Goldberg shared a document written by Google's Brian Grant in 2016 that had 13 pages of issues that needed to be fixed before enterprises would feel comfortable deploying the software in production. By 2017, that backlog of issues had been reduced to a manageable number, and that was also the year that both AWS and Microsoft joined the CNCF, which gave the project more of a true community feel and accelerated its growth.

However, Thursday night was very much a victory lap for Google Cloud's ideas for cloud computing, despite the fact that Google has spent years since the CNCF was created in 2015 downplaying concerns that it played too large a role in the Kubernetes project. Google Fellow Eric Brewer had arguably the most compelling account of the history behind Kubernetes, as you might expect from someone who has been a professor at UC Berkeley for three decades.

Brewer divided up the pre-Kubernetes conventional wisdom around building distributing systems into two categories: virtual machines, which counted VMware and AWS as its primary backers, and container-based processes, which was how Google built its infrastructure in the late 1990s before virtual machines were widely used.

Brewer and Google realized that it would have to do something different to compete with AWS, which served a wide variety of use cases but didn't provide the abstraction layer that a platform designed around processes like Google's early cloud product, Google App Engine, did. As Google was trying to define that new approach, Docker exploded in popularity by making Linux containers much easier to use for tech organizations that didn't have Google's in-house expertise.

"And I kind of feel like, this is our chance," Brewer said. "Can we make a bet that we can do both a versatile platform and a higher-level abstraction platform?"

The answer, of course, was yes, and the result now underpins much of the world's technical infrastructure. But Kubernetes is well into its boring phase, as enterprise tech hurtles into the generative AI era and attention turns to a different level of the stack.

Ant Stanley of Senzo X summed it up well in a post earlier this year: "It’s achieved what it set out to achieve, and all the improvements from here are incremental." And given how much open-source software has changed over the last ten years as commercial incentives trumped community building, it's hard to see another project uniting so much of enterprise tech in the same way that Kubernetes did.

Still, there's no question that a group of hundreds of contributors from all walks of life changed the arc of computing history, and that's always a notable accomplishment.

"I've seen a lot of open source projects, and most people that start those projects don't mind if it doesn't go anywhere; they throw it out there, and if people use it, they use it," said Kelsey Hightower, author of the widely read guide Kubernetes The Hard Way. "But Kubernetes was very different."

(This post originally appeared in the Runtime newsletter on June 8th, sign up here to get more enterprise tech news three times a week.)