businessline’s Post

Digging into customers’ demand for ML and AI workloads 🤖#theCUBE exclusive at Juniper Networks’ Seize the AI Moment event. Join Raj Yavatkar, the CTO at Juniper Networks, in this #AINativeNow Industry Panel, where he hears from Steve Scott, Corporate Fellow at AMD, on the rising demand for compute consolidation, efficiency, space, power, and flexibility to deal with ML and AI workloads. “AI is just driving a huge appetite for performance. Everybody's constrained by budgets and space and power. One of the things that customers are asking us about is the need to consolidate their general purpose compute to make up space and power for AI,” shares Scott. “A lot of our AI interest is actually on the EPYC side with people using EPYC CPUs to consolidate their older servers because they're very power efficient, and they are seeing a 5x to 8x consolidation possible from older servers,” he furthers. “For the AI infrastructure itself, people are looking for choice, flexibility, and competition. They want better price performance for training, but especially for inference. Most of the inference is done on CPUs, but the high-end inference is all being done on GPUs, and we're pretty consistently hearing that people are looking for a second source for their accelerators. So, there is a lot of interest in the AMD Instinct 30x,” Scott concludes. 📺 Tune in now! https://lnkd.in/g7mscxp4 #theCUBE #SeizetheAIMoment

The linked article mentions that Google already uses SiFive cores. The RISC-V supplier has a big portfolio covering little low-performance CPUs, designs with wide vector engines, and speedy CPUs for high-level OSs and applications. As Google refreshes its TPU, it's not surprising it would license cores again, perhaps using additional or different ones from SiFive. SiFive is well capitalized, but it's hard to make money licensing semiconductor IP--especially anything with an open component and especially to a company with the resources to roll its own.

How to improve spectral efficiency and overall performance per watt? In this blog, the approach of accelerated computing and how it applies to RAN and spectral efficiency is presented. Many wireless algorithms and techniques exist that improve wireless performance and hence spectral efficiency; however, they need lots of compute power. This is where Aerial on NVIDIA Accelerated Computing solves these challenges. The outcome is a better TCO for the telco. Thanks to the team on putting this together. Jim Delfeld Yan Huang Rajesh Gadiyar CC Chong Emeka Obiodu, PhD https://lnkd.in/g6fsFPap

Excellent insights on separating compute for Prompt and Token Generation during LLM inference phases for enhanced Hardware utilisation. separating the prompt computation and token-generation phases onto separate machines. This approach is underpinned by the insight that prompt processing and token-generation are distinct in their computational, memory, and power requirements. By separating these two phases, we can enhance hardware utilization during both phases. #gpus #microsoftresearch #promptengineering #tokenization #llminference https://msft.it/6041icTdP

I love networking! Reading about the complexity, cost, and scale of networking these GPUs for AI workloads is insane! This provides some insight ! | 100k H100 Clusters: Power, Network Topology, Ethernet vs InfiniBand, Reliability, Failures, Checkpointing https://ow.ly/xVuX50Ssslf

New week, another partner talking about their experiences with AMD Instinct MI300X! The latest blog from Oracle demonstrates and details some results that are based on experiments done on the upcoming MI300X GPU bare metal offering from OCI, while running on the open-source AMD ROCm software stack. https://lnkd.in/gvQCwVju #AMD #Instinct #MI300X #accelerator #gpu #ai

Partnership Announcement 🤝 Nuklai is partnering with io.net, a decentralized compute network that leverages #GPU clustering services to turn weaker hardware into advanced compute networks. io.net will provide decentralized, efficient, and customizable GPU power to Nuklai. Our mutual efforts will give communities more #data and GPU power from a variety of choices. Giving enterprises, data enthusiasts, and other users another reliable service partner to work with for their computational needs. Nuklai’s infrastructure will help io.net create private data networks to monetize their data. Furthermore, Nuklai’s community of data enthusiasts and curators will enhance io.net’s data with metadata, turning it into fuel for #AI and LLMs. io.net has already onboarded over 24,000 GPUs in its network. They also have an impressive waitlist of 200,000 other GPUs that could provide instant compute power to the Nuklai #network. Full announcement 👇 https://lnkd.in/e9zFcQU4

Groq is setting new benchmarks in AI with their record breaking speed. Their LPU is designed to overcome the two LLM bottlenecks: compute density and memory bandwidth. An LPU has greater compute capacity than a GPU and CPU in regards to LLMs. This reduces the amount of time per word calculated, allowing sequences of text to be generated much faster. You can try it at groq.com

Link flapping is a developing and costly issue in large GPU clusters – with each link flap event resulting in up to 30 minutes of lost training time and costing up to $200,000! Thanks to Massine Merzouk of X for recognizing how important it is that Credo is taking the lead in addressing this issue. "Cluster reliability is of paramount importance when building the biggest supercomputers in the world with 100,000+ GPUs,” said Massine Merzouk, Network Engineer at X, assisting xAI. “Credo’s HiWire AECs offer the stable transport platform we need to build such massive systems." https://bit.ly/3zPhZUV

The LLM frenzy is slowing down. The crazy run to squeeze a few decimals on LLM leaderboards is coming to an end. Billions $ were wasted on GPUs with little purpose and now it's time to count the ducks. We reached a plateau and GPT-4o is the proof. LLMs will not lead to AGI, not with current architectures. More research is needed and much more data to train on, synthetic or not. A few months ago, it was impossible to get any GPUs, but now they are available on demand! No reservations are required! Sooner or later, a supply excess is inevitable. Dozens of chip companies are getting started, and soon, we will have plenty of compute at dirt-cheap prices. This is good news for open-source because cheap and abundantly available compute will inevitably lead to decentralized and open-source AI that's available to everyone.