"I have a GPU. Why do I need an NPU?"
Well known LLMs such as ChatGPT often use GPUs from Nvidia to perform inference in the cloud. And your PC or smartphone will always include a GPU.
Nvidia has been successfully turning its GPUs into devices that are highly effective accelerators for machine learning training and inference. It has done this through the development of both a high-quality software ecosystem (centered on #CUDA) and through the refinement of its hardware, including adding specialized tensor matrix multiplication units to its GPUs. Other GPU designers have either not done this or have been much less effective in doing so.
Plus the GPU on your PC or smartphone is already needed to perform its original function: drawing images on your display. What happens if your device needs to draw some complex graphics and perform machine learning at the same time? Having a separate NPU removes this potential contention.
#GPU #npu #llm #AI NVIDIA
https://lnkd.in/dPckEKfF