Why Meta Did Not Choose Qualcomm AI Chip?

On April 1, it was reported that Qualcomm is the world's largest supplier of smartphone processors and is well-established in terms of computing power and energy efficiency of its chips. 2019 Qualcomm announced that it will enter the fast-growing market of data center AI chips based on its technology and experience in smartphone chips.

Qualcomm had courted Facebook's parent company Meta Platforms to become a benchmark customer for Qualcomm's first data center AI chip, the AI 100, two people familiar with the matter said. After Qualcomm released the chip in the fall of 2020, Meta tested the chip against a range of other options, including the chip the company had been using previously, as well as a dedicated chip Meta developed in-house to handle AI computing.

Qualcomm's chip performed well in the tests, with the best performance per unit of energy consumption, according to people familiar with the matter. For a company like Meta, energy efficiency improvements can bring huge optimization to operating costs as its data centers serve billions of users.

However, people familiar with the matter said that by the spring of 2021, Meta said it refused to use Qualcomm's chips. The specific reason is that Meta questioned that the Qualcomm chip's supporting software is not mature enough to bring out the chip's best performance in specific computing tasks in the future. A person familiar with the matter said that after evaluating various options, Meta decided to continue to use the existing chip.

This event has never been reported by the media before but also shows that the software has become one of the core factors of AI chips to win customers. IDC analyst Shane Rau (Shane Rau) said that AI chip sales are expected to reach $ 13.5 billion this year, and will grow to $ 41.3 billion by 2026. "The market demand for AI chips is virtually unlimited, at least for the next 15 to 20 years," he said.

Qualcomm chips are at the heart of billions of smartphones worldwide and also underpin AI features such as smartphone photo optimization, but the AI 100 is the company's first attempt at competing with Nvidia. In the data center AI chip space, Nvidia currently holds an overwhelming advantage. The company's dominance comes not only from the chips but also from the accompanying software. NVIDIA's software is the current gold standard in the AI industry.

Everyone, not just Qualcomm, is in an arms race with Nvidia CEO Jen-Hsun Huang," said Peter Barrett, general partner at venture capital firm Playground Global. He follows developments in deep learning and notes the speed at which the technology is evolving. His efforts on the software side have helped maintain the company's leadership position." The playground has also invested in companies like MosaicML, which help AI customers match their models to the right hardware.

To be sure, Meta's rejection is likely just a temporary setback for Qualcomm in the AI chip space. As recently as September 2021, the AI 100 chip scored several firsts in the MLPerf benchmark test, a set of industry standards for measuring AI chip performance, following Meta's testing. Industry watchers expect Qualcomm's chips to also perform well in the tests, which will be conducted again this spring. Qualcomm has already announced its first customer for the AI 100: Foxconn Industrial Internet. The company is using the chip in a server used to analyze video from security and traffic cameras.

Meanwhile, Qualcomm is continuing to court other potential customers such as Microsoft. A Microsoft spokesman declined to comment on this aspect of the development. Qualcomm plans to use the AI 100 chip for inferential computing, which uses AI models trained on massive amounts of data to make real-time decisions. In Meta's scenario, this typically means deciding which content to show to users in millisecond time periods based on a recommendation model.

To achieve better performance, the trained model must also be optimized for the hardware running the model. If the optimization is poor, then the model is likely to use only a fraction of the available performance of the hardware, resulting in wasted power. However, optimization of the model can take up a lot of the developer's time. Typically, software that optimizes code written in various languages and automatically matches the underlying hardware is more likely to be favored by developers. NVIDIA's software excels in this regard. Venkat Mattel, CEO of the startup Ceremorphic, which develops AI processing, said that if the chip was provided directly to developers without accompanying optimization software, it would be like giving a user a bike with 100 gears and then expecting him to explore on his own how to ride over unknown terrain and exactly which gears to use.

You can't give developers 100 gears, but rather you have to make the configuration appear to be 3 gears," he said. But right now, most chip companies aren't doing that."

Engineers capable of writing software to accompany the chip are scarce. It's a challenge for big companies like Qualcomm, as well as dozens of other startups targeting the same market. The development of this type of software requires developers with specialized experience in compilers. Compilers translate the code written by developers into the machine language used by the chip.

This type of talent is sought after and very much in short supply," said Shahin Farshichi, a partner at Lux Capital. This has become a major bottleneck." Lux Capital has invested in AI chip startups Mythic and Flex Logix.