Robert Ankarlo’s Post

Want to log Cerebras LLM inference calls? We’ve added streaming support so Log10 users can now log LLM calls with Cerebras Systems as well as other OpenAI-compatible providers such as Perplexity and Mistral AI. 🥳 ICYMI Cerebras launched the "world’s fastest AI inference" that lets developers leverage the power of wafer-scale compute for AI inference via a simple API. Read more here: https://bit.ly/4eaKUlm #LLM #LLMOps

This article introduces the methodology and results of performance testing the Llama-2 models deployed on the model serving stack included with Red Hat OpenShift AI. 

How to fine-tune Llama 3.1 with Ray on OpenShift AI This is the first in a series of articles that demonstrate the OpenShift AI tuning capabilities on a variety of AI accelerators. This post focuses on NVIDIA GPUs. https://lnkd.in/eM_E2VCZ

This article introduces the methodology and results of performance testing the Llama-2 models deployed on the model serving stack included with Red Hat OpenShift AI. 

As AI and ML models become increasingly complex, efficient GPU utilization is crucial for performance and cost-effectiveness. This webinar delves into advanced techniques such as time-slicing, Multi-Instance GPU (MIG), Multi-Process Service (MPS), and other best practice strategies available for optimizing GPU usage on Kubernetes. https://lnkd.in/e8Fskqwv

Hot news about Microsoft's acqui-hiring Inflection AI and the changes at Stability AI. But I want to focus on the latest advancements in model architectures - Mamba. It appears to be gaining traction. What exactly is it? Mamba – a new architecture that rivals the famous Transformer-based models. Mamba's innovations address significant challenges in processing long sequences, a problem that has limited traditional models. Mamba leverages state-space models (SSMs), mathematical frameworks that describe a system's dynamics using state variables and observations. Mamba incorporates Structured State Space (S4) models into a large language model (LLM) framework. The integration enables Mamba to achieve linear scaling with sequence length over the quadratic scaling in traditional Transformer-based models. The streamlined architecture includes selective SSM layers for improved efficiency and flexibility. As a result, Mamba efficiently processes extremely long sequences, surpassing earlier models in performance. Additionally, it benefits from hardware-aware optimizations, maximizing the potential of contemporary GPU architectures. Want to learn more about Mamba? Read the original paper: https://lnkd.in/g-6-yb-X I also discuss it here: https://lnkd.in/gRhidu_Q

Semianalysis looks in depth at the new AI hardware solution from Groq in comparison with Nvidia's offerings with detailed pros and cons https://lnkd.in/gNzfs4-5. #groq #artificialintelligence #largelanguagemodel #semianalysis

I've created a video on LLMs on the Edge... The latest generation LLMs are absolutely astonishing — thanks to their multi-modal capabilities you can ask questions in natural language about stuff you can see or hear in the real world ("is there a person without a hard hat standing close to a machine?") and get relatively fast and reliable answers. But these large LLMs have downsides; they're absolutely huge, so you need to run them in the cloud, adding high latency (often seconds per inference), high cost (think about the tokens you'll burn when running inference 24/7), and high power (need a constant network connection). In this video we're distilling knowledge from a large multimodal LLM (GPT-4o) and putting it in a tiny model, which we can run directly on device; for ultra-low latency, and without the need for a network connection, scaling to even microcontrollers with kilobytes of RAM if needed. Training was done fully unsupervised, all labels were set by GPT-4o, including deciding when to throw out data, then trained onto a transfer learning model w/ default settings. One of the models we train has 800K parameters (an NVIDIA TAO model with MobileNet backend), a cool 2,200,000x fewer parameters than GPT-4o :-) with similar accuracy on this very narrow and specific task. https://lnkd.in/ehJ848RG

Gemma models are a family of open AI models that are lightweight, state-of-the-art, and easy to customize for different tasks. They are built on the same technology as Gemini, Google's flagship AI model, but are more accessible and efficient for developers and researchers. Check it out!

I concur Neal Mann & the AHA moment is powered by the underlying LLM & AI, yet not only that layer…the full experience needs to be as delightful, effortless & customer need aligned as always. Brand, marketing, product, CX, data, technology, processes & people need to be aligned around these. #ai #productmanagement #cx