Cloud computing's mid-market problem

Welcome to Runtime! Today: why cloud providers are falling short with a key segment of the market, OpenAI's Thanksgiving-week management turmoil made customers consider their options, and the quote of the week.

(Was this email forwarded to you? Sign up here to get Runtime each week.)

Stuck in the middle with you

Last year was probably the hardest year for cloud infrastructure computing since its inception, with growth rates down across the board thanks to economic worries and generative AI threatening to upend everything. Attitudes are a little more optimistic going into 2024, but the days in which cloud infrastructure services were the shiny new thing that can solve all your problems are over.

Over the holiday break, former Google Cloud legend Kelsey Hightower hosted an interesting online discussion with David Heinemeier Hansson, the creator of Ruby on Rails and CTO of 37Signals known mostly these days for shitposting. Last year 37Signals decided to break up with AWS and manage its own infrastructure, citing the complexity of both managing cloud services and paying for them.

• Hansson called cloud providers "merchants of complexity," as reported by The New Stack, which is undeniable at this point.
• AWS has a dizzying array of cloud services that companies of all sizes have trouble stitching together, and charges customers for those services in mysterious ways that created an entire industry dedicated to helping cloud customers understand their bills.
• "When it’s that difficult to understand how you’re spending that much money, someone is getting away with something," he said, and there is a lot of truth in that statement.
• 37Signals spent $3.2 million on cloud services in 2022, but ordered $600,000 worth of Dell servers early last year and now manages its own infrastructure.

37Signals might seem like the vanguard of the "cloud repatriation" movement, which started more than two years ago when a16z encouraged some of its portfolio companies to stop paying a cloud tax.

• There are a lot of companies like 37Signals that understand how their applications run, are content with a manageable growth rate, and don't see seasonal spikes in demand for computing resources.
• $3.2 million a year is almost exactly what you'd expect a medium-sized company to spend on cloud services, according to CloudZero, even if it's a rounding error for AWS.
• Those companies are clearly not being served by the Big Three, and could be tempted by stories such as Hansson's.
• In chatting with System Initiative co-founder and CEO Adam Jacob last month at re:Invent, he said something to the effect of "you know, we used to rack our own servers all the time, and stuff worked." (We'd had a few drinks.)

But cloud computing isn't going anywhere; most startups aren't going to spend money on servers, and most enterprises don't want to pay for the expertise needed to run their own infrastructure at scale.

• Still, there is clearly an opportunity for cloud providers to do more for those in the middle segment who feel like they are being left behind.
• Simplifying the basic experience of running cloud services and paying for them could do far more to lift cloud growth than shoving generative AI into every corner of that experience.
• "When the applications and data sets are simple or they support a narrow set of services, such as providing a single tool on demand, the cloud is usually not the right choice," wrote Deloitte's David Linthicum in his own assessment of 37Signals' infrastructure strategy.
• It doesn't have to be that way.

OpenAI allowed an opening

In the chaos of OpenAI's decision to fire and rehire CEO Sam Altman, it seemed very likely that the company's rivals would start immediately working the phones planting those good old-fashioned enterprise-tech seeds of fear, uncertainty, and doubt among its customers. That strategy appears to have worked.

The Wall Street Journal reported Friday that "executives at companies that use OpenAI’s software say they are increasingly looking to also use others’ technology to protect themselves from the risks of problems at any one." The report cited Walmart, a longtime Microsoft customer, and Aviatrix as two companies that immediately asked their tech teams to make sure they weren't overly dependent on OpenAI's models.

OpenAI definitely had a first-mover advantage in the generative AI boom, but other models are quickly catching up when it comes to performance and price. It's actually a healthy sign that enterprise tech buyers are thinking about diversification when it comes to generative AI tools; even Microsoft is adding choices for its customers despite its hefty investment in OpenAI.

Quote of the week

"You want a lawyer to review every single piece of generated output? That's impractical." — Kate Downing, an intellectual property attorney, sizing up what generative AI customers relying on indemnification policies might be in store for, depending on how the courts rule on OpenAI vs. The New York Times.

The Runtime roundup

Dee Templeton, a longtime Microsoft executive, has been selected as a non-voting observer on OpenAI's board, according to Bloomberg.

Merck settled a dispute with its cybersecurity insurers over whether they owed the pharmaceutical company damages after the 2017 NotPetya malware attack, just before an appeals court was set to consider a ruling that the insurers couldn't get out of paying by considering that cyberattack a "hostile/warlike action."

Virginia — home to a huge cluster of data centers, including AWS's flagship region — could consider a new energy efficiency standard for data-center operators in the state that want tax breaks.

Thanks for reading — see you Tuesday!