Modal

🌈 Apply to demo at Modal x Mistral AI's SF demo night 🌈 Calling all AI builders! Modal is hosting a demo night in SF with Mistral on Thursday, March 6th. We want to showcase the coolest projects that the community is building on Mistral & Modal, whether they’re personal projects or for work. You'll have an audience of 80+ founders, builders, and investors in the community! If selected, you'll also get $1000 in Modal credits to keep on building 🎖️ Submit your demo proposal here: https://lnkd.in/egv_aHys

Congrats to Mistral AI for launching a new, super-fast version of their AI assistant ⚡ We're excited that their researchers used Modal Sandboxes to speed up the development of Le Chat's code interpreter feature. The code interpreter allows users to execute code directly within the chat interface, such as for scientific analysis or running simulations. Sandboxes are a Modal primitive to safely run LLM-generated code. It comes with both secure isolation and all the usual high-performance, fast-scaling benefits of our platform.

🌎 Region selection for Functions and Sandboxes is now available across all plan tiers! Region selection allows you to define specific regions for your Functions to run in. It's useful for: 🔐 Complying with data residency requirements 💰 Reducing egress costs 🚤 Reducing latency for apps that need to run near an external DB We've gotten many requests to make this feature more broadly available, and we're excited that anyone can use it now! For more information on the feature and how it's priced: https://lnkd.in/enDewYBQ

New blog post by Jonathon Belotti diving into the technical details of our container memory snapshot restore functionality. Container memory restores can reduce cold start latency on user functions by more than 50% 💥

We have been working a lot to make Modal Functions start really fast at Modal. This is one of my favorite ideas: memory snapshots. https://lnkd.in/eRhTfrcN We take a snapshot after your function boots. We then restore your function's memory when your function starts again. This yields 2.5x faster cold boot times. Neat!

Today on the Modal blog we’re sharing how a warmed up container can be saved to disk and later restored for a 2.5x cold start performance boost. Restoring a Stable Diffusion container and getting the model into the GPU takes only 3.56s at p50! How exactly this works has lots of fun OS stuff: CRIU, FUSE, vDSO, CPU fingerprinting. Saving live container processes to disk turns out to be pretty whacky and interesting! A process is a sophisticated OS resource and wasn’t designed to be savable, but these are computer things and so what isn’t possible if you try?

🐋 How to deploy DeepSeek AI models on Modal: https://lnkd.in/eJWYybWF Check out our end-to-end code sample for running DeepSeek-R1. No need to bring your own infra, and you can try this out well within our $30/mo free credits! This example makes use of Unsloth AI's 1.58 bit quantization of DeepSeek-R1, which fits on four L40S GPUs, and runs it with llama.cpp, a C++ inference library. The full version of this 671B parameter model requires a minimum of 8 H200s (>1 TB of VRAM) to run! More inference-time optimizations to come so stay tuned. And if there’s other flavors of DeepSeek models you want to see examples of, let us know ⬇️

We're saying goodbye to our winter interns Qijia (Joy) Liu, Andrew Liu, and Tarushii Goel this week! ☃️ They worked on some awesome features during their internships like sandbox memory snapshots, proxy auth tokens, and automatic workload profiling. 🚀 And looked great doing it! 💚

We built the prototype for Modal Sandboxes in late 2023, when the first wave of "AI agents" started cropping up. All of them needed a runtime to execute untrusted code, and it was clear that the tech we had built for fast scale-up and strong isolation was a perfect fit for this use case. While that's still a big use case, this primitive is a *lot* more than just a code execution environment. A Sandbox is more like `docker run` but with cloud super powers. You can: - Programmatically define environments (or your LLM can), and run them in the cloud on the fly. - Attach any of the other primitives we've built for Functions, like GPUs, Volumes, Cloud Buckets or static IP proxies. - Expose ports to the public internet. - Snapshot filesystem state and explore forking paths in different sandboxes. - Get great observability out of the box in our web dashboard, and via our telemetry integrations. Very excited that they're finally out of beta, and eager to see the crazy things people build with them :)

🚀 We just GA’d a new product: Modal Sandboxes. Sandboxes are a new primitive in Modal to safely run untrusted code. We've seen demand for this functionality grow quickly with the advent of coding agents and AI-generated code. With Sandboxes, you don’t just get secure isolation—you also get all the usual high-performance benefits of Modal. This means super fast cold starts and being able to spawn hundreds of sandboxes a second. It takes just a few lines of Python to run code in a Sandbox. Excited for you to give them a try!