For AI model serving, you have two main options today: TorchServe or a DIY FastAPI server. But people are now choosing LitServe - here's why 👇🏼 - TorchServe is a Java engine, so it's hard to work with Python models (like most models today). - Ollama, VLLM are overfit to LLMs, so they are not in this category (LitGPT is our comparison to these). - FastAPI is easy to start, but hard to scale to AI workloads. So, most people choose FastAPI because the others are inflexible and hard to use (what if you need two models or want to bring your own?). LitServe changes that! Built on FastAPI, it offers full flexibility to run multiple models, integrate your own, and more. https://lnkd.in/diYUG7NW
Lightning AI
Software Development
New York, NY 85,758 followers
The AI development platform - From idea to AI, Lightning fast⚡️. Creators of AI Studio, PyTorch Lightning and more.
About us
The AI development platform - From idea to AI, Lightning fast ⚡️. Code together. Prototype. Train on GPUs. Scale. Serve. From your browser - with zero setup. AI Studio is your laptop on the cloud. Zero setup. Always ready. Persistent storage and environments. Code on CPU. Debug on GPU. Scale to multi-node. Run sweeps, jobs and more. Scale models with PyTorch Lightning, Fabric, Lit-GPT, torchmetrics and more.
- Website
-
http://lightning.ai
External link for Lightning AI
- Industry
- Software Development
- Company size
- 11-50 employees
- Headquarters
- New York, NY
- Type
- Privately Held
- Founded
- 2019
- Specialties
- Artificial Intelligence, Machine Learning, Infrastructure, deep learning, data science, and open source
Locations
-
Primary
Get directions
New York, NY, US
Employees at Lightning AI
Updates
-
Bring your own models into LitServe 🤯🤯 https://lnkd.in/diYUG7NW
-
-
The weekend is here! explore LitServe (AI model serving) with these examples: 🎶 Generate music from text (AudioCraft) 🎆 Upscale small images without losing details (Aura SR) 🧐 Annotate images (Phi 3.5) 🧑🏼🎨 Generate images from text (Flux - the new Stable Diffusion) Link to more examples: https://lnkd.in/ddrvfRPN
-
-
LitServe makes it super easy to build compound AI systems! Mix models, data, and databases to create cool multi-modal setups like image captioning or speech-to-text. Here's an example for image captioning with CLIP. https://lnkd.in/diYUG7NW
-
-
🚀 Introducing LitServe! The next-gen serving engine for AI models—lightning fast ⚡, scalable. ✅ 2x faster than FastAPI (at least) ✅ GPU autoscaling ✅ Batching, streaming ✅ LLMs, NLP, vision ✅ PyTorch, SkLearn, Jax... ✅ ... 10+ features Star ⭐️ on Github: https://lnkd.in/diYUG7NW Example: Deploy a Whisper model.
-
-
Get an in-depth introduction to building RAG applications with open source LLMs! In this video, @svpino uses Llama 3.1 and Lightning AI Studio to develop a basic RAG system that can answer questions from a PDF.
I recorded a gentle introduction to building RAG applications using open-source models. Prerequisite: You should be comfortable reading Python. I wanted to record a video to introduce as many people as possible to building RAG applications and using Large Language Models. Here, I tried to stay away from complex jargon and break everything down as simple as possible. Some of the topics I cover here: • Fundamental components of a RAG application • How to use Llama 3.1 running locally • How to use Lightning AI Studio's dev platform and GPUs • How to orchestrate everything using Langchain It's a long video with lots of detailed explanations. Here is a link to the video (what you see attached is just a trailer): https://lnkd.in/eRwxKn63 You'll find a link to all of the code in the video description.
-
Lightning AI reposted this
Helping companies use GenAI || Data Scientist at QuantumBlack || McKinsey || M.S. Columbia University || CDTM || TUM
We're going to have the best AI Tinkerers from NYC together again at our August Meetup! When: August 28th, 6:00 PM to 9:00 PM Where: Awesome rooftop terrace and office, hosted by Lightning AI Don't miss this chance to connect with other AI tinkerers in NYC! Register now: https://lnkd.in/ekvy-Pw6. You can also still submit a request to speak and show what you are building!
-
Comet is a powerful experimentation platform for tracking, comparing, debugging, and visualizing model training runs. Check out this Lightning AI Studio to see how you can get started quickly with the PyTorch Lightning CometLogger: https://lnkd.in/g6ARWZ24
Comet + Lightning Quickstart - a Lightning Studio by abigailm
lightning.ai
-
We can’t wait to connect with everyone at this week’s Ai4 - Artificial Intelligence Conferences! 🗓️ ”Building Scalable AI for Real-World Business” with William Falcon - Tuesday, Aug 13 @ 11:35 AM PDT 🗓️ Lightning AI booth & live demos - Tuesday, Aug 13 to Wednesday, Aug 14 @ Booth 311
Headed to Ai4 - Artificial Intelligence Conferences in Las Vegas next week? Our CEO, William Falcon, will be giving a talk on “Building Scalable AI for Real-World Business” on Tuesday @ 11:35am PDT. And don’t forget to visit our booth to get a live demo of Lightning AI Studios and snag some exclusive Lightning AI swag!
-
-
We loved bringing the developer community together with Georgian at the Toronto Machine Learning Society (TMLS) Summit for our “Building Applications with Open Source LLMs” hackathon! Everyone had a blast working on innovative projects and collaborating live with Lightning AI Studios. Stay tuned for more upcoming hackathons near you.