The AI gold rush is alive in Silicon Valley

Chips ahoy

In a hallway outside a hotel ballroom deep in Silicon Valley, AI chip startup SambaNova Systems senior vice president Marshall Choy boasted that moments ago at the company’s booth, he had signed a $10 million deal for the company’s hardware. Choy flashed what he said was the contract on his phone — too quickly to discern the new client — and sauntered off.

SambaNova and dozens of other AI startups descended on Santa Clara, California, for the AI Hardware Summit. The two-day summit, featuring a modest show floor and technical talks, was a decidedly less buttoned-up affair than a concurrent investor event at the Palace Hotel in San Francisco put on by Goldman Sachs.

• Further evidence emerged at the Goldman conference about the choppy market for chips.
• An executive of flash and hard drive storage maker Western Digital said that the hyperscale cloud computing companies were becoming increasingly cautious with their purchases.
• Though China may be a large potential market to companies such as Qualcomm, for WD and other cloud hardware suppliers, economic turmoil there has hurt sales.
• Nvidia’s Colette Kress echoed WD’s commentary around China’s cloud hardware demand, describing it as “muted” during her Q&A with a Goldman analyst Tuesday.

The steady drumbeat of restrictions on various chip-related technology exported to China — most recently the notices Nvidia and AMD received over advanced AI chips — prompted Goldman Sachs’ analysts to ask nearly every chip company present about the potential effects on their business.

• Kress said the company is working with customers to obtain licenses, or obtain alternative products, though it's worth noting the blocked A100 chips offer significant performance gains over the prior generation.
• AMD data center SVP Forrest Norrod said that the notification took the company by surprise, but that because of recent economic trends AMD hasn’t made China a focus of its business.
• The largest chip tool maker in the U.S., Applied Materials, said that most of its China business is related to less advanced manufacturing systems, and the forthcoming export control rules will have a minimal impact on its operations.

By contrast, none of the chipmaking startups had received notification letters from the Commerce Department, of those polled by Protocol at the AI Summit. The focus at the summit felt far removed from geopolitics and was more directed around nerding out over silicon and software.

• One of the most interesting presentations was from Meta’s recently hired VP of infrastructure engineering Alexis Bjorlin, who outlined how the company deploys AI at the gigantic scale necessary.
• Bjorlin, who has a background in optical networking infrastructure, said the company operates over 100 data centers globally. In 2025 beyond, the company plans to build AI training clusters with roughly 4,000 accelerators.
• And RISC-V startup SiFive buried some news in the middle of its presentation Thursday: The company is working with Google to use its X280 cores inside of its tensor processing units, or TPUs, that power Google’s AI.

A MESSAGE FROM CNCF

The Cloud Native Computing Foundation’s flagship conference gathers adopters and technologists from leading open source and cloud native communities in Detroit, Michigan from October 24 – 28, 2022. Register now and join thousands of attendees, including maintainers for CNCF's 140 Graduated, Incubating, and Sandbox projects, either virtually or in-person.

The metaverse has three buckets

Fifty-two percent of executives polled this week at Goldman Sachs’ inaugural Communacopia + Technology conference see gaming as the most promising market for the metaverse in the next three years, while 21% believe that other entertainment areas will thrive, such as concerts and sports.

Judson Althoff, Microsoft’s chief commercial officer, acknowledged the differing opinions on the subject. “Before all of you roll your eyes on this metaverse thing … You pull 100 people into a room, you get 100 different answers about what the metaverse is and will anybody ever get any real value out of it,” Althoff said at the San Francisco conference, according to a transcript of his talk with Goldman Sachs managing director Kash Rangan.

But Althoff said he looks at the metaverse as falling into three buckets: There’s the consumer metaverse in which Althoff’s avatar might buy Rangan’s avatar a beer, and there’ll be a monetization aspect to that realm, he said.

Also, “there's the commercial metaverse where people will have more engaging and experiential collaboration in the metaverse, and I do think that there's an opportunity there to bring people from around the world with different perspectives to collaborate,” he said.

Then there’s what Althoff calls the industrial metaverse.

“Think of it as combining sets of technologies, IoT capabilities where you come in and create a center fabric for any industrial process, any manufacturing environment, any supply chain or logistics scenario,” Althoff said. “You have that center fabric feed, a large-scale, cloud-based analytics solution, a large data store. And then you reason over top of it with machine learning and create what we call digital twins of those environments and simulate outcomes.”

“If you make anything or you move anything, you create a carbon footprint,” Althoff said. “If I can simulate that for you infinitely in the cloud before you make it and before you move it, I can help you create a better product more cost effectively with lower carbon footprint and lower water utilization more sustainably than ever before. And we see real progress happening today with companies like Coca-Cola, Unilever, AB InBev, GM, Grupo Bimbo — across industry — being able to model these processes, save energy, reduce waste, and these solutions pay for themselves.”

— Donna Goodison (email | twitter)

Open-source AI will only get you so far

When Meta passed its popular open-source AI framework PyTorch to the nonprofit Linux Foundation on Monday for safekeeping, it reinforced the value the AI industry places in sharing its foundational open-source building blocks.

Compared to many other types of technologies incorporating proprietary code, AI is a melting pot of shared research and open-source foundational elements that, when cobbled together and customized, help create AI tools and products.

But there’s a bridge to cross between connecting free AI puzzle pieces and assembling a production-grade system that works at commercial scale. And it’s a toll bridge. (Cue the Python gag).

“[Open-source AI] can make a cool demo; you could build so much cool stuff for free because of all the open-source stuff out there,” said Davis Sawyer, co-founder and chief product officer of Canada’s Deeplite, which provides software that compresses AI so it isn’t too large to function in edge devices such as phones or vehicles.

But, he said, “Open-source will get you 80%. Building a polished product that goes into a security camera [for instance] — that's still a significant investment. That's still going to take millions of dollars even though you're building on open source.”

The difference between a just-OK open-source-based AI product and a great one is dependent on the expertise of its builders.

“The secret sauce today is the piecing together,” said Abhishek Gupta, senior responsible AI leader and expert at Boston Consulting Group and a former Microsoft machine learning engineer. “The real difference comes in your ability to experiment and iterate, which comes from the maturity of the underlying software infrastructure, the underlying engineering practices,” he said.

“That's the distinguisher between people who have good AI systems and people who have great AI systems.”

— Kate Kaye (email| twitter)

Around the enterprise

Uber suffered an embarrassing hack in which an attacker had access to its internal Slack and threatened to release sensitive details about the company.

A MESSAGE FROM CNCF

The Cloud Native Computing Foundation’s flagship conference gathers adopters and technologists from leading open source and cloud native communities in Detroit, Michigan from October 24 – 28, 2022. Register now and join thousands of attendees, including maintainers for CNCF's 140 Graduated, Incubating, and Sandbox projects, either virtually or in-person.