Ravi Shankar’s Post

One more step toward cheaper or faster LLM inference on AWS? Also what about Neuron support on Lambda and ECS Fargate? (cold start is an issue .. I know) https://lnkd.in/gyZgY-53

This post shows how to accelerate pre-training of large language models by scaling up to 128 trn1.32xlarge nodes, using a 2-7B Llama model as an example. It shares best practices for efficient, stable training of LLMs on AWS Trainium with 100+ nodes, and recovering from failures. #aws #awscloud #cloud #amazonec2 #awsneuron #awstrainium #bestpractices #technicalhowto #distributedtraining #neuron

Check out this awesome blogpost from my colleagues Jianying Lang, Fei Chen et.al. on training SOTA LLMs on AWS Trainium, our AWS ML chips.

Accelerate LLM training with #Meta Llama 3 and #AWSTrainium. 🦙⚡ https://go.aws/3RTFged In this post, you'll learn best practices for training LLMs on AWS Trainium, scaling the training on a cluster with over 100 nodes, improving efficiency of recovery from system and hardware failures, improving training stability, & achieving convergence. #AWS

Accelerate LLM training with #Meta Llama 3 and #AWSTrainium. 🦙⚡ https://go.aws/3RTFged In this post, you'll learn best practices for training LLMs on AWS Trainium, scaling the training on a cluster with over 100 nodes, improving efficiency of recovery from system and hardware failures, improving training stability, & achieving convergence. #AWS

Here’s @anbarth AWS Weekly Roundup 👉 LlamaIndex support for Amazon Neptune, force AWS CloudFormation stack deletion, and more (May 27, 2024) https://buff.ly/3WYH1tR #AWS #News

AWS Weekly Roundup: LlamaIndex support for Amazon Neptune, force AWS CloudFormation stack deletion, and more

AWS Weekly Roundup: LlamaIndex support for Amazon Neptune, force AWS CloudFormation stack deletion, and more

Deploy low-latency, high-throughput inference using AWS Graviton or AWS Inferentia on Amazon EKS. Read this guidance page to learn how to use prebuilt AWS solutions to deploy an ML inference workload. This guidance walks through how to pack up thousands of unique PyTorch deep learning models into a scalable architecture using Graviton or Inferentia on Amazon EKS.

AWS Graviton Weekly # 98 https://lnkd.in/eWT6bJVE Highlights of the week 🗞️ Amazon Web Services (AWS) Graviton-based EC2 instances now support hibernation 🗞️ New Amazon CloudWatch dimensions for Amazon EC2 On Demand Capacity Reservations 🗞️ Amazon EC2 Fleet and EC2 Auto Scaling groups now supports aliases for Amazon Machine Images (AMIs) ⛏️ RAG solution on Amazon Bedrock - Part 9: Optimizing ECS and EKS Infrastructure with AWS Graviton, by Vivek V ⛏️ RISE with SAP on AWS delivers consumer goods innovation, by Justin Honaman ⛏️ Amazon EC2 R8g Instances with AWS Graviton4 Processors Generally Available, by Steef-Jan Wiggers ⛏️ AMD vs Intel vs Graviton: Comparison between Modern Processors, by Visak Krishnakumar ⛏️ Using the MAQAO framework to analyze application performance across instances, by Hugo Bolloré, Cédric Valensi, and William Jalby ⛏️ Cloudera Data Engineering 1.22 for Public Cloud: Support for AWS Graviton, Spark 3.5 + Iceberg 1.4, and More! ⛏️ Top 6 Strategies for AWS EMR (Elastic Map Reduce) Cost Optimization, by Sanika Kotgire ⛏️ Databricks Runtime ML on AWS Graviton instances ⛏️ Get a 50% Price-Performance Boost With StarRocks on AWS Graviton3, by Andy Ye ⛏️ Read how Simon Phillips( CTO at SecureAck) migrated its platform to AWS Graviton ⛏️ Exploring AWS Lambda: Performance and Cost Analysis of Different Architectures and Deployment Methods, by Evgeny Lukashov ⛏️ Move workloads on x86-based instances to AWS Graviton | The Keys to AWS Optimization | S10 E11, with Steph Gooch, Rem Baumann, and Hahnara Hyun ⛏️ Harness the power of Karpenter to scale, optimize & upgrade Kubernetes | Architecting on AWS ⛏️ Watch Jeff Underhill and Sunita Nadampalli demonstrate blazing fast performance of llama3.1 on Graviton, using the highly optimized llama.cpp framework ⛏️ 269: Crowdstrike: Does Anyone Know the Graviton of this Situation? by Justin Brodley, Jonathan Baker, Ryan Lucas, and Matthew K. ⛏️ SUSE x Arm at SUSECON 2024, with Robert Sirchia and Andrew Wafaa ⛏️ Applying FinOps to GenAI: Perspectives and Best Practices from Grammarly and Intuit, by Jason Rhoades and Josh Collier #aws #awsgraviton #finops #kubernetes