Nebius AI

🎩 Cloud solutions architect team: lessons learned Levon Sarkisian, our CSA Team Lead, wrote a piece reflecting on the technical challenges his team (and many others at Nebius) have faced this year. Instead of making a flashy announcement, we encourage you to simply check out Levon’s short article — perhaps it will resonate with you as well: https://lnkd.in/dKAXuWVc #CSA #cloud #architects

🔥 This fall, we’re adding H200 SXM GPUs to Nebius — reserve your cluster right away https://lnkd.in/dXVwpEZw Currently, H200 is the most powerful GPU for your AI and HPC. Its key gains come from: * Memory. H200 is the first GPU with 141 GB of HBM3e memory — nearly doubling the memory capacity of H100 SXM. * Performance. The increased memory bandwidth of 4.8 TB/s allows for better utilization of processing power, making it the new preferred GPU for large models. * Data access speed. With the shared filesystem, you can achieve up to 20 GB/s read speeds from one node, which is crucial for training and inference. This is 6 times more than what’s available with the H100. According to an NVIDIA research, H200 shows up to 2x the LLM inference performance over H100: 1.4x in case of Llama2 13B, 1.6x with GPT-3 175B, and 1.9x with Llama2 70B. Prices start from $2.50 GPU/h. Reserve these new powerhouse GPUs today: https://lnkd.in/dXVwpEZw #H200 #GPU #LLMs #clusters

🦾 Insights from the Nebius LLM R&D team When introducing our LLM R&D team, we mentioned that its members have gathered a wealth of insights along the way. Which, on one hand, helps us deeply specialize the platform, but on the other, is very much worth sharing with the community. In fact, the team has been actively sharing their knowledge since the beginning of this year. If you also handle large models and build MLOps to streamline your workloads, be sure to check out these materials. — The MLOps Community Podcact featuring Simon Karasik provides a gentle introduction to the topic of LLM checkpointing, explaining why is it hard and how big are the checkpoints. Simon discussed various tips and tricks for saving and loading multi-terabyte checkpoints, as well as the selection of cloud storage options for checkpointing: https://lnkd.in/eYg_ksVf On the same topic, Simon wrote an article covering some of the podcast's main points and other parts of the workload: https://lnkd.in/eBNQijT2 — In his talk for the LLMOps Space community, Maksim Nekrashevich discussed building the fine-tuning pipeline for an LLM alignment, including incorporating models into the data collection, techniques for instilling desired behaviors and workflow management: https://lnkd.in/evcTf83x — Yury Anapolskiy wrote an article on data preparation for large models. Follow Yury as he explores methods and technologies for maximizing efficiency in data collection and preparation for training LLMs, outlines the pipeline in detail and discusses our own chosen workload: https://lnkd.in/eKMKDsKN — And finally, the second part of our introductory article provides a sneak peek into distributed training and data processing infrastructure used by the team: https://lnkd.in/ev7v5-C2 Other team members have actively given talks off the record at more local events — for instance, Filipp Fisin has spoken several times about resilient training, and Sergey Polezhaev has also covered the alignment pipeline topic. The LLM R&D team will continue to share their experiences — there are still few teams on the market training LLMs from scratch. #LLM #training #finetuning #talks #conferences

🔀 How transformers, RNNs and SSMs are more alike than you think Recent research has exposed deep connections between different architectural options: transformers, recurrent networks (RNNs), state space models (SSMs) and matrix mixers. This is exciting because it allows for the transfer of ideas from one architecture to another. 
In the next installment of our AI research series, we’ll mainly follow papers like “Transformers are RNNs” and Mamba 2, getting elbows deep in algebra to understand how: * Transformers may sometimes be RNNs. * State space models may hide inside the mask in the self-attention mechanism. * Mamba may sometimes be rewritten as masked self-attention. Read the article on our blog: https://lnkd.in/dQsyEnV5 #transformers #RNN #SSM #research #papers

🤝 Support and architects: how they can assist you We take pride in our technical support and solution architect teams. But sometimes, there might be confusion about which team handles what. So here’s a quick memo. Cloud solution architects (CSAs): — Assist clients throughout the entire workload launch cycle in multi-node and other special cases. Onboarding is not required if you need a single GPU-equipped node — you can get it yourself through our user-friendly console. However, if the setup is more complex than that, you might need help from a CSA. — Develop the Nebius Solution Library, which is a set of Terraform and Helm solutions designed to streamline the deployment and management of your apps. The library is hosted on GitHub: https://lnkd.in/dSWKguHj — Ensure that future onboardings run smoothly. CSAs pre-test our infrastructure, InfiniBand network, new clusters, new types of nodes developed in-house by Nebius, and other components. They also communicate with our LLM R&D team to understand in advance the general requirements of AI labs. Read more on our blog: https://lnkd.in/dA6V55Yh — Host webinars (such as this one: https://lnkd.in/dGENKeY2) and assist the documentation team in creating guides so that clients can get answers to most questions in advance. We always let you know about new guides here and in our monthly digests: https://lnkd.in/dxT2nUAz — Run internal demos and PoCs, among many other tasks. Then there are support specialists, available 24/7 to assist you with a variety of issues, including: * restoring access to the console, * providing documentation links, * diagnosing and fixing errors, * addressing FAQs, * assisting in setting up services, * handling data requests, * recommending service recovery for critical incidents, * helping with third-party software configuration and offering recommendations for OS and third-party software issues. You can contact support through the support center in the console or by email at support@nebius.ai. To learn more, visit this page: https://lnkd.in/dimy434J. Hope that makes things clearer for you! #CSA #support #architects #AIcloud

🟢 Nebius at TechCrunch Disrupt: what to expect? This fall, we will proudly participate in TechCrunch Disrupt 2024 in San Francisco, one of the major industry conferences. If you’d like to learn how to streamline your AI development — or discuss a partnership with us, here are the formats in which we’ll be present: — Our Chief Business Officer Roman Chernin will give a talk about his experience building AI cloud platform and ways to enhance every part of your pipeline. — We invite you to visit our vibrant exhibition booth and take part in activities we’re preparing. Stay tuned to find out more. — Engineers who build our platform, as well as other team members, will also be around and ready to engage in conversation. See you on October 28! #Disrupt #conferences #networking #partnerships #AIcloud

🎩 Optimizing inference with math: TheStage AI and its framework The inference market has grown so significantly that inefficiencies between revenue and inference costs have emerged. TheStage AI’s solution, which uses our infrastructure, is designed to close this gap. Based on an award candidate CVPR article and other papers, the product is an automatic neural network analyzer, or ANNA for short. Learn how it works: https://lnkd.in/dFhy2vWr #CVPR #inference #math #research #papers

📣 Talk by Nebius: what it takes to train a GenAI model Several weeks ago, our Product Director Narek T. gave a talk at The AI Summit Series in London. He shared an insider's perspective on the key steps, essential tools and challenges involved in bringing a generative AI model from concept to production. In this talk, he broke down the real story behind data preparation, experiments to prepare foundational model training, a pre-training process, fine-tuning and inference. Watch the talk here: https://lnkd.in/d3iFi9Jk #GenAI #talks #training #conferences

🔥 Choosing storage for deep learning: a comprehensive guide The rapid evolution of deep learning models has brought about unprecedented growth in both their size and complexity. This trend, while pushing the boundaries of what is technologically possible, has also placed immense demands on the underlying infrastructure, particularly in terms of data management and storage. Drawing from Nebius’ and our clients’ extensive experience, today’s guide and research by our own Igor Ofitserov aims to help engineers choose the most fitting storage solutions for deep learning. Head straight to our blog to read it: https://lnkd.in/drM7sKbH #storage #research #guide #deeplearning

🧪 ML experiments: approaches and best practices Machine learning experiments help you discover the most optimum model version for your specific use case. Read this article on our blog to learn about different types of experiments and what you need to watch out for when conducting them: <https://lnkd.in/d-A2PSV8> #ML #experiments #bestpractices