Baseten

🔥 We have two huge announcements today! 🔥 We're thrilled to announce early access to our new Baseten Hybrid offering 🎉 As well as our launch on the Google Cloud Marketplace! 🚀 Links to the announcement blogs and waitlist for Hybrid in the comments👇 With Baseten Hybrid, you have complete control over your policies and workloads with the flexibility to scale up as needed on our cloud. This solves a common problem: you want to self-host certain workloads to meet compliance requirements, but when push comes to shove, you need to tap into extra compute. Run sensitive workloads securely in your VPC, meet specific data residency requirements, and fully utilize existing investments in providers like @Google Cloud. When you need extra compute, effortlessly spill over to Baseten Cloud—zero engineering effort required. With our launch on the Google Cloud Marketplace, it’s now easier than ever for Google Cloud users to leverage Baseten’s platform directly within their cloud ecosystem. Our growing partnership provides seamless interoperability, secure data management, and the ability to quickly spin up high-performance AI applications in your Google Cloud environments. Get early access to Baseten Hybrid or get started today with Baseten Cloud on the Google Cloud Marketplace! Special thanks to our infra team for making Hybrid a reality! Philip Howes Colin McGrath Ujjwal Sarin Matthew Howard William Lau Zhang Lu Jonathan Rochette And another shoutout to our partner lead Ed Shrager and the Google team for their great partnership! Eddie White Heather McCormick Vince Bryant Matthew Jones Moritz Schoenheit

You can now deploy models with Baseten as part of Vercel's AI SDK! 🎉 Easily run and test OpenAI-compatible LLMs as part of your Vercel workflows with all the best-in-class performance benefits that Baseten offers. Plus: you can access all of our LLM features (including streaming)—in any JavaScript framework—with just a few lines of code. 💪 👉 Learn more: https://lnkd.in/echrcuV3

What a week! 🧠 💚 Last week concluded our latest company-wide offsite in beautiful Cape Cod. 🌳 From over a dozen hackathon projects (that you'll soon see as full-blown features 👀) to pickleball, a talent show, and more, we're so impressed by everyone who makes Baseten such a wonderful place to work! (P.S., we're growing 😉 👇)

We're excited to welcome two new team members! 🎊 Daniel Sarfati is joining our forward deployed engineering team 🧰 And Tyron Jung is the newest Software Engineer on our core product team! 🛠

Looking to scale LLM Inference and save on costs? Baseten's benchmark post breaks down batch handling, goes deep into performance results, and provides tips on when and how to optimize spend. Get the full scoop here: https://hubs.ly/Q02VTTmv0

We're hitting the road to close out the year: 📅 KubeCon, Nov 12-15, Salt Lake City 📅 re:Invent, Dec 2-6, Las Vegas We'll have demos, swag, prizes, and on-site experts ready to help you optimize your production inference. Come say hi at the booth! KubeCon – https://lnkd.in/gvvWv_z8 re:Invent – https://lnkd.in/gAEZGC-M

After the team at Rime Labs trained astonishingly lifelike speech synthesis models with over 200 voices, they needed fast, reliable infra to bring their API to market. With Baseten, they've maintained <300 ms p99 latency and 100% uptime through 2024. Read their story: https://lnkd.in/gnAzvtiB

We benchmarked the new NVIDIA H200 GPUs for LLM inference with Lambda 📈 H200s crush long input sequences 📈 H200s make huge batches more efficient (high throughput) 📉 H100 GPUs are likely more cost-efficient for many inference workloads Want the details, from tech specs to benchmark results? Check the comments for a link.

NVIDIA's new Nemotron 70B is... 🚀 Better than GPT-4o and Sonnet 3.5 on benchmarks 🍓 Able to count 3 Rs in Strawberry ✅ Available now in the Baseten model library Deploy it in 2 clicks (link the comments) or customize hardware/quantization/sequence lengths in 10 lines of YAML with our Engine Builder. 😎