5G I B2B I Product Marketing I Business Development I Smart Factory I IIoT I Semiconductor
There has been a lot of noise lately about Generative AI running on device. Why would anyone want to deploy locally? Generative AI models are often based on deep neural networks that require a lot of memory and computational power to run. So naturally, they are deployed in the cloud. Drawbacks of the cloud implementation: 💲 Cost: According to Sajjad Moazeni from University of Washington, there are hundreds of millions of daily queries on ChatGPT. This many queries can cost around 1 GWh each day, which is the equivalent of the daily energy consumption for about 33,000 U.S. households. 🔐 Privacy: some companies do not allow the use of Chat GPT for generating code due to privacy and security issues. An app for generating code could run on the device without exposing confidential information to the cloud. ⏳ Real time capability: use cases like image editing for adding or removing objects, changing colors and applying filters or speech recognition for captioning, subtitling, translating, or analyzing speech would hugely benefit from running the model on device. The catch is that due to the limited resources on device, Gen AI models need to be optimized and reduced in size. This results in some drawbacks on the quality of the output. However, there is consensus that for very narrow applications, the optimized AI models can give very good results. There is ongoing research to reduce the AI computation demands through various methods (quantization, compilation or hardware acceleration). Recently our research team used generative AI to develop a digital fitness coach. You can see a video of the demo here. You will find other examples below with many more to come in the near future. https://lnkd.in/eVmvjqkY Other links: https://lnkd.in/ef4ir_Wd https://lnkd.in/eutmZwib https://lnkd.in/enBjrRyX https://lnkd.in/euPJkKDj #qualcomm #iiot #iot #genai #ai #snapdragon