Lightning AI

🚀 Build a corrective RAG (CRAG) agentic workflow, powered by DeepSeek-R1! Search your docs, evaluate context quality & fall back to web search when needed. Plus, a sleek @streamlit UI for seamless interaction. 🔥 Try out the studio here ➡️ https://lnkd.in/eShDW9_t

🚀 Don't miss Luca Antiga, CTO of Lightning AI, in his #GTC25 speaking session: Train and Serve AI Systems Fast With the Lightning AI Open-Source Stack Learn about Lightning AI’s high-performance set of libraries for training, fine-tuning, and deploying AI systems that builds upon and extends the PyTorch ecosystem and discover new features like multi-dimensional parallelism with DTensors and quantization with torchao. ⚡️ 11:20 AM - 11:35 AM PDT ⚡️ San Jose Convention Center Grand Ballroom Theater (L2) https://lnkd.in/ekeKi4gi

📣 Just announced at #GTC25: We are thrilled to announce the launch of new and exclusive cloud partner benefits available to #NVIDIAInception and #NVIDIAConnect members. https://nvda.ws/4kBRYLi ☁️ Enhanced cloud benefit offers ✅ Simplified process Explore these new offers from our partners: Nebius, Lambda, Scaleway, Yotta, Lintasarta, Weights & Biases, Lightning AI, BRIA AI, Beamr, and more.

We are super excited to work with Lightning AI as we further advance 's AI-powered 3D imaging platform! ⚡ With Lightning AI, we're accelerating our deep learning workflows on the cloud, handling large-scale 3D and 4D microscopy data efficiently and deploying cutting-edge 3D foundation models for drug discovery and precision medicine. At Sentinal4D, our mission is to learn how cells respond to treatments in true 3D by leveraging AI to decode drug mechanisms, identify off-target effects, and enhance preclinical decision-making. Grateful to work alongside innovators like Lightning AI to push the boundaries of precision medicine! #AI #3DImaging #DrugDiscovery #PrecisionMedicine #Oncology

🚀 Join Luca Antiga & Thomas Viehmann for their talk, Make My PyTorch Model Fast, and Show Me How You Did It TODAY at #GTC25! Learn how Thunder - PyTorch-to-Python compiler - optimizes PyTorch models with ease without changing the code! ⚡️ 11:00 AM - 11:40 AM PDT ⚡️ San Jose Convention Center 210F (L2)

We’re thrilled to welcome Sentinal4D as a customer! 🎉 A fellow NVIDIA Inception member, they’re using AI-powered 3D imaging to push the boundaries of precision oncology—accelerating drug discovery and improving cancer treatment. We're proud to support their mission! Learn more about Sentinal4D ➡️ https://lnkd.in/ekzZBVbe

If you're still using FastAPI to deploy Hugging Face LLMs/VLMs - switch to LitApi! FastAPI is a great framework for implementing RESTful APIs. However, it wasn’t specifically designed to handle the complex requirements of serving ML models at scale. The team at Lightning AI is behind LitServe and LitApi to fill in that gap. 🔹 𝗟𝗶𝘁𝗔𝗣𝗜 builds on top of FastAPI, adapting for ML workloads, standardizes the core steps of serving a model. 🔹 𝗟𝗶𝘁𝗦𝗲𝗿𝘃𝗲𝗿 handles the infrastructure side of serving models. 🔸 Here's what you must know: 1. 𝗢𝗻𝗲-𝘁𝗶𝗺𝗲 𝗺𝗼𝗱𝗲𝗹 𝘀𝗲𝘁𝘂𝗽 In the 𝙨𝙚𝙩𝙪𝙥() method, we can load any model only once. 2. 𝗖𝘂𝘀𝘁𝗼𝗺𝗶𝘇𝗲 𝗣𝗿𝗲𝗱𝗶𝗰𝘁 In the 𝙥𝙧𝙚𝙙𝙞𝙘𝙩() method, we implement the inference on inputs logic. 3. 𝗖𝘂𝘀𝘁𝗼𝗺𝗶𝘇𝗲 𝗕𝗮𝘁𝗰𝗵𝗶𝗻𝗴 𝗟𝗼𝗴𝗶𝗰 You can specify a MAX_BATCH_SIZE and a BATCH_TIME_WINDOW and it'll automatically handle the dynamic batching of requests as they come in concurrently. You can use ThreadPoolExecutor to parallelize the preprocessing steps in the 𝙗𝙖𝙩𝙘𝙝() method. 4. 𝗖𝘂𝘀𝘁𝗼𝗺𝗶𝘇𝗲 𝗨𝗻𝗯𝗮𝘁𝗰𝗵𝗶𝗻𝗴 𝗟𝗼𝗴𝗶𝗰 After inferencing on a batch, you'll handle the detach () of GPU tensors and post-process the raw logits in the 𝙪𝙣𝙗𝙖𝙩𝙘𝙝() method. 5. 𝗗𝗲𝗰𝗼𝗱𝗲 𝗿𝗲𝗾𝘂𝗲𝘀𝘁 𝗮𝗻𝗱 𝗲𝗻𝗰𝗼𝗱𝗲 𝗿𝗲𝘀𝗽𝗼𝗻𝘀𝗲 In the 𝙙𝙚𝙘𝙤𝙙𝙚_𝙧𝙚𝙦𝙪𝙚𝙨𝙩() - specify how the API should access the input value from the request. In the 𝙚𝙣𝙘𝙤𝙙𝙚_𝙧𝙚𝙨𝙥𝙤𝙣𝙨𝙚() - specify how the API should return responses to the client. Simple as that! To scale this up for a production workload, you'll use LitServe's scale configuration parameters: ``` LitServer( lit_api: LitAPI, accelerator: str = "auto", devices: Union[str, int] = "auto", workers_per_device: int = 1, timeout: Union[float, bool] = 30, max_batch_size: int = 1, batch_timeout: float = 0.0, stream: bool = False, ) ``` 📙 For a full tutorial, see this article: https://lnkd.in/dGUrVX7s --- #machinelearning #deeplearning #artificialintelligence --- 💡 Follow me for more 𝗲𝘅𝗽𝗲𝗿𝘁 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀 on AI/ML Engineering

Join us for AI After Hours, hosted by Lightning AI at #GTC25! Connect with top AI experts, network with peers, and dive into the latest AI advancements in an exclusive, invite-only setting. ⚡️ Thursday, March 20th | 7:00 PM PT⚡️ Spaces are limited—secure your spot here https://lnkd.in/eEdD7jsw

🚀 Excited to be at #GTC2025 this week! Stop by booth #2403 to meet our team and explore how to scale AI from research to production. Don’t miss our live demos and more! See you there! ⚡

Unlock the power of Fully Sharded Data Parallel (FSDP) & Thunder! ⚡ FSDP is already powerful, but tweaking bucketing logic to better align with hardware speeds could unlock even more efficiency. Faster materialization of tensors on high-performance machines means reduced overhead and improved scalability. Watch the full Thunder Session with Luca Antiga and Thomas Viehmann ➡️ https://lnkd.in/eeJRXuYD