Powerful, on-device processors like Snapdragon’s Hexagon will be needed to drive increasingly ubiquitous AI applications, rather than offloading every task to slow and costly data centers
Artificial intelligence (AI) may be the future of computing, but Qualcomm Technologies Inc. is betting that the future of AI will be hybrid, putting its leading Snapdragon Hexagon processor at the heart of the next PC revolution.
As AI supercharges every kind of application and creates others we have yet to dream of, splitting computation between edge devices and the cloud is “inevitable,” Alex Katouzian, senior vice president and general manager of the Mobile, Compute, and Infrastructure Business Unit at Qualcomm, told COMPUTEX 2023 in a keynote presentation.
“It simply won’t be possible to send everything to the cloud, and for certain applications you won’t want to,” he told a packed auditorium at Taipei’s Nangang Exhibition Center 2 on May 30.
Instead, devices themselves will handle most AI processing, then seamlessly offload to the cloud when complexity increases.
Already, the Snapdragon 8cx Gen 3 with Hexagon can create photorealistic images using stable diffusion on an Android smartphone in under 15 seconds for 20 inference steps—all without an internet connection. And larger models are coming very soon.
The shift will be similar to the evolution in traditional computing from mainframes to desktops and smartphones, Qualcomm Senior VP and GM for Compute and Gaming Kedar Kondap said.
Hybrid AI will not only be faster, but also far cheaper, more sustainable, and secure.
When processing in the cloud, it is estimated that the cost per AI query is 10 times higher than for traditional search thanks to the energy, real-estate, operational, hardware, and other costs needed to run a data center, Kondap said.
Yet when running on a laptop or other edge device, an AI process like stable diffusion can cost as little as nothing, while also using far less power and freeing up cloud bandwidth for more complex tasks.
AI run on a personal device will also learn more about the user over time, creating a more personalized experience to solve each user’s pain points and optimize their workflow.
Last but certainly not least, data processed on-device stays on that device, inherently protecting user privacy and sensitive company information.
Qualcomm has already been working with its partners to create some exciting new AI applications enabled by Snapdragon.
“We are truly at the start of a new computing era,” Microsoft VP of Device Partner Sales Ken Sun said.
Complex AI models are behind programs like Windows Studio Effects, adding background blur, focus, noise reduction, and other features in everyday video calls.
Sun also announced that in June, Microsoft is launching Windows Copilot, a personal assistant capable of tasks like summarizing a PDF or giving suggestions from only a user query.
Some of the most exciting applications will democratize advanced photo and video editing, with Skylum CEO Ivan Kutanin showing the keynote how Luminar Neo’s “Supersharp AI” is able to bring a blurry photo into focus in only 8.4 seconds, versus 2 minutes on a CPU.
“This is just the beginning,” Katouzian said. “We’re very excited to see how developers are going to take advantage of these platforms and enhance all of their solutions.”
