Anyscale

Exciting to see DeepSeek AI using Ray and Arrow in their smallpond release today! - Smallpond targets high performance data processing - It provides a high-level dataframe API - Targets petabyte-level scaling Data processing is essential for training data prep. Typically includes a lot of filtering (to increase data quality) and annotation (to extract structured information) as well as deduplication (there is lots of redundancy on the internet, and you want to control the data mixture and not leave it to chance). The challenges around training data prep only grow when you include multimodal data, e.g., images, video, audio, and start running a ton of transcription, captioning, not to mention a lot of synthetic data workloads. https://lnkd.in/gGsKMjwJ

uv is one of the most important projects that's been created recently, and the team at Astral is very very good. It's hard to overstate how important getting dependency management right is for iteration velocity. My favorite thing about uv is that when installing a new Python dependency, instead of modifying the overall global Python environment, you can treat the dependency as scoped locally just to the execution of a particular script. It's very clean.

Python dependency management has been a longstanding challenge facing AI teams. The uv package manager, built by the team at Astral, goes a long long way toward putting that problem to rest, at least for code running on a single machine. The challenge is even bigger in the cluster setting, where developers must iterate rapidly on code and dependencies while keeping everything synced across a potentially autoscaling pool of compute. This integration with Ray extends the benefits of uv to distributed Python applications running on a cluster. It's quite simple. If main.py is your Ray driver script, you can manage the dependencies for the entire Ray application (the driver and all of the Ray worker processes) by running: "uv run <args> main.py". https://lnkd.in/g2-FadxB

Awesome turnout for Anyscale's Cody Yu presentation at the vLLM meetup—nearly 300 people joined to hear about the vLLM roadmap and our team's release of new LLM APIs in Ray Data and Ray Serve. The new batch inference APIs seamlessly integrate vLLM, improving both speed and scalability. See the APIs here: Ray Data + LLMs- https://lnkd.in/gJ_Ucc4W Ray Serve for LLMs- https://lnkd.in/gi2TVSAz

We love seeing the work Cambridge Mobile Telematics is doing with Ray. With sensor data from millions of IoT devices, they scale deep learning training workloads using large datasets with Distributed Data Parallelism (DDP). To scale up to even larger models, they rely on Ray for Full Sharded Data Parallel (FSDP) training. All of this runs on their Amazon EKS cluster. Read on to learn about EKSRay 👇 https://lnkd.in/eSby4GQ7

⚡Anyscale Webinar Alert! - THIS WEDNESDAY⚡ Join us tomorrow, Wed Feb 26 for an Introduction to Anyscale and Ray AI Libraries, a webinar designed to help you get started with Ray’s AI libraries for distributed machine learning. Our training instructors will walk you through: 🔹 How Anyscale addresses today’s AI challenges 🔹 An overview of Ray AI Libraries (Data, Train, Tune, Serve) 🔹 A full ML workflow covering all MLOps stages Don't miss this chance to learn directly from experts and see how Ray AI Libraries can supercharge your ML workflows. 🔗 Register now: https://lnkd.in/dbK8PFw5

Join us next Wednesday, Feb 26 for an Introduction to Anyscale & Ray AI Libraries, designed to help you get started with Ray’s AI libraries for distributed ML. 🙌 Our instructors will cover: 🔹 How Anyscale improves AI scalability & efficiency 🔹A comprehensive look at Ray AI Libraries (Data, Train, Tune, Serve) 🔹 A full ML workflow covering all MLOps stages Don't miss this chance to learn from experts! 🔗 Register now: https://lnkd.in/dbK8PFw5 #MachineLearning #AI #RayDistributed

KubeRay v1.3.0 is here with major upgrades for running Ray on Kubernetes. 🚀 Key highlights: • Enhanced observability: The new RayService Conditions API and RayCluster Conditions API have reached Beta, along with the Kubernetes Events API for better tracking and monitoring. • Stronger reliability: Improved failure recovery, better resource management for more stable Ray clusters, and a doubling of end-to-end tests from 23 to 46. •Improved usability: Ray Kubectl plugin (Beta) simplifies job and cluster management, along with a new unified GCS fault tolerance API. KubeRay v1.3.0 improves observability, reliability, and usability making Ray on Kubernetes easier to monitor, recover, and manage. 🙌 Read full blog article here: https://lnkd.in/g-6w8Cnf

Introducing the Ray Kubectl Plugin (Beta)! 🙌 Now, managing Ray clusters on Kubernetes is faster and more efficient. Key enhancements: • Streamlined workflows - Run your Ray scripts from your laptop on a remote K8s cluster with a single command. • Create Ray clusters & worker groups with a single command, no YAML needed. Read more about streamlining your Ray cluster management on Kubernetes here: https://lnkd.in/gwWWsk-P

⚡Anyscale Webinar Alert! ⚡ Join us on Feb 26 for an Introduction to Anyscale and Ray AI Libraries, a webinar designed to help you get started with Ray’s AI libraries for distributed machine learning. Our training instructors will walk you through: 🔹 How Anyscale addresses today’s AI challenges 🔹 An overview of Ray AI Libraries (Data, Train, Tune, Serve) 🔹 A full ML workflow covering all MLOps stages Don't miss this chance to learn directly from experts and see how Ray AI Libraries can supercharge your ML workflows. 🔗 Register now: https://lnkd.in/dbK8PFw5