Blackwell launches Nvidia's arena-rock era

Welcome to Runtime! Today: why tech came to a standstill Monday afternoon to watch Nvidia unveil its latest AI chip, Microsoft puts another outsider at the helm of its AI strategy, and the latest funding rounds in enterprise tech.

(Was this email forwarded to you? Sign up here to get Runtime each week.)

Let it B

It's been a year to forget for the regular occupants of San Jose's SAP Center, with the Sharks all but guaranteed to finish the season as the worst team in professional hockey. Nvidia CEO Jensen Huang's performance in that arena Monday showcased a company on the opposite end of that spectrum.

Nvidia unveiled its Blackwell GPU architecture Monday before 11,000 people gathered to watch Huang show off the company's latest creation. Blackwell is the next generation of Nvidia's Hopper architecture, which produced the H100 and H200 GPUs that powered the generative AI boom.

• The most powerful iteration of the Blackwell line, the B200, will be available from cloud providers and server vendors later this year.
• According to Tom's Hardware, it will offer 20 petaflops of computing performance, compared to 4 petaflops provided by the H100.
• Blackwell will enable the training of large-language models with as many as 10 trillion parameters, compared to the 1 trillion parameters believed to be used by OpenAI's GPT-4.
• And Nvidia continued its push into the CPU side of the data center with Grace Blackwell, a "superchip" consisting of two B200s and its Grace CPU.

The Hopper architecture had an extraordinary run, with demand far outpacing supply for most of 2023. While the frenzied part of the rise of generative AI might be settling down as business realities creep in, there will still be enormous demand for Blackwell chips in the short term.

• Nvidia did not disclose pricing, but Huang told CNBC that Blackwell GPUs will cost between $30,000 and $40,000 per chip, roughly in line with the Hopper generation.
• The Blackwell architecture actually consists of two chips squashed together in a single package, however, so it's not clear exactly what he meant by those prices.
• The B100 version of the new Blackwell design is compatible with Nvidia's existing Hopper systems but B200 customers will have to upgrade to new server designs.
• That's because the B200 demands more power than the H200, although Nvidia promised that overall, servers built around Blackwell chips will be far cheaper to run and use less power than servers built around the Hopper generation.

It could take up to a year before the Blackwell GPUs start to make an impact on AI computing, but they could represent a milestone in the advancement of generative AI.

• In between grand, sweeping promises about its potential to change the world, generative AI vendors readily acknowledge that there are still a lot of issues to be ironed out, most notably (but not limited to) the hallucination problem.
• A big part of the overall pitch centers around the promise that these models will continue to get better over time as companies like Nvidia ramp up their silicon capabilities and model builders learn how to take advantage of that increased horsepower.
• When Blackwell arrives, those companies will be under pressure to show significant improvements in the performance and reliability of their models given how many billions of dollars have already been invested in these efforts.
• And as they continue to rely heavily on Nvidia (and TSMC's) ability to deliver sufficient supplies of those chips, the small market for Nvidia alternatives will continue to simmer.

Inflection point

Microsoft has once again turned to an outsider to oversee a key piece of its AI strategy, poaching Google DeepMind co-founder Mustafa Suleyman from Inflection to run a new consumer-facing business unit called Microsoft AI. Inflection co-founder Karén Simonyan and "most of the staff" from the startup will join that new business unit, according to Bloomberg, leaving Inflection — which Microsoft invested in last summer — in a strange place.

Like its unusual arrangement with Sam Altman and OpenAI, the creation of Microsoft's new business unit appears to have been designed to avoid the antitrust scrutiny that would have come along with an outright acquisition of Inflection. The facilitator was likely Reid Hoffman, a member of Microsoft's board of directors and a co-founder of Inflection who will continue to serve on its board, the company said.

So Microsoft has now placed enormous trust in two men who were each fired from previous roles over their difficult management styles to shape the future of its AI development. Startups seeking investment in the future might be skeptical about taking money from Microsoft, given that it just hollowed out one of its portfolio companies less than six months after trying to do the same thing with OpenAI.

Enterprise funding

Together AI raised $106 million in Series C funding from Salesforce Ventures and existing investors to build enterprise-class features for its AI cloud services platform.

Unstructured landed $40 million in Series B funding to help companies make sense of corporate data that is, well, unstructured.

WarpStream raised $20 million in new funding to build out a streaming data platform the company claims is easier to use than Apache Kafka.

Euno launched with $6.25 million in seed funding to help customers manage business logic around their data pipelines.

The Runtime roundup

Cisco completed its $28 billion acquisition of Splunk, which it hopes will make the company more relevant in the fast-growing observability market.

OpenAI is having trouble finding customers for its GPT Store, which it launched last November in hopes of matching GPT developers with enterprise software buyers, according to The Information.

Broadcom floated plans to let smaller VMware cloud services providers continue to manage SaaS versions of VMware products after cutting them off earlier this year, according to The Register.

Databricks acquired Lilac, a startup working on a tool for data scientists dealing with unstructured data, for an undisclosed amount.

Thanks for reading — see you Thursday!