Fireworks AI

🔥 Announcing FireOptimizer/Multi-LoRA 🔥 I didn't expect what I considered to be a small feature launched last year delivered a powerful impact to our customers. I'm excited to announce Multi-LoRA, an important component of FireOptimizer. Personalized experiences are critical to driving greater usage, retention and customer satisfaction for your product. Without Multi-LoRA, deploying hundreds of fine-tuned models on separate GPUs would be prohibitively expensive. With Multi-LoRA, you can now deliver personalized experiences across thousands of users and use cases, without scaling your costs! More specifically, Multi-LoRA has benefits below: -- Fine-tune and serve hundreds of personalized LoRA models at the same cost as a single base model, which is just $0.2/1M tokens for Llama3.1 8B -- 100x cost-efficiency compared to serving 100 fine-tuned models without Multi-LoRA on other platforms with per-GPU pricing -- Convenient deployment on Fireworks Serverless with per-token pricing and competitive inference speeds, or Fireworks On-Demand and Reserved for larger workloads Multi-LoRA is part of FireOptimizer, our adaptation engine designed to customize and enhance AI model performance for your unique use cases and workload. FireOptimizer capabilities include Adaptive Speculative Execution (https://lnkd.in/ejdD-wGG), that enables up to 3x latency improvements, Customizable Quantization (https://lnkd.in/dwpTU233), to precisely balance speed and quality, and LoRA Fine-Tuning (https://lnkd.in/et2UFzDy) to customize and improve model performance. ⚡Cresta uses Multi-LoRA to personalize their Knowledge Assist feature for each individual customer on the Fireworks enterprise platform. "Fireworks' Multi-LoRA capabilities align with Cresta's strategy to deploy custom AI through fine-tuning cutting-edge base models. It helps unleash the potential of AI on private enterprise data." - Tim Shi, Co-Founder and CTO of Cresta ⚡Brainiac Labs helps businesses leverage their proprietary data to fine-tune and deploy models using Multi-LoRA on the Fireworks self-serve platform. “Using Fireworks, clients with limited AI expertise can successfully maintain and improve the solutions I provide. Additionally, students in my course are able to complete real-world fine-tuning projects, dedicating just a few hours per week to the process.” - Scott Kramer, CEO of Brainiac Labs 👉 Read more in our blog post https://lnkd.in/d3_HGRqy

Check out this demo by our DevRel 👩🏻💻 Mikiko B. hosted on Hugging Face Spaces using Flux (powered by Fireworks) to generate festive holiday cards for when the gift store doesn't have quite what you're looking for. #flux #blackforestlabs #fireworks #builtwithfireworks

Check out this demo by our DevRel 👩🏻💻 Mikiko B. hosted on Hugging Face Spaces using Flux (powered by Fireworks) to generate festive holiday cards for when the gift store doesn't have quite what you're looking for. #flux #blackforestlabs #fireworks #builtwithfireworks

Announcing fast, frugal and flexible FLUX on Fireworks in partnership with Black Forest Labs! Commercially usable FLUX.1 [dev] and FLUX.1 [schnell] models are now available on Fireworks https://lnkd.in/dFP7daXs 💨 Speed and 💰 Cost-Efficiency: FLUX runs on Fireworks with FP8 precision, offering 2x faster inference at half the cost of other platforms ($0.0014 for FLUX.1 [schnell] and $0.014 for FLUX.1 [dev]) ⚙️ Multiple Deployment Options: Use FLUX serverless to start instantly and pay per image. Deploy to private on-demand GPUs for consistent speeds in production ✏️ Customization: Leverage FLUX with custom ControlNet and LoRA adapter support to fine-tune your AI applications. Get started with FLUX.1 models for serverless image generation on Fireworks. Commercial usage is enabled through our partnership with Black Forest Labs. https://lnkd.in/d8zd9JH7 https://lnkd.in/d-rbe7ED Scale as you need: Deploy auto-scaling GPUs to handle production traffic. FLUX.1 [dev] and [schnell] both fit on an A100 or H100 GPU. https://lnkd.in/d3iAsDyc https://lnkd.in/dxD48p8P Use LoRA, ControlNet, or create custom server apps to fit FLUX into your broader AI systems. Ready to build your next AI innovation? Start customizing FLUX today: https://lnkd.in/dFP7daXs

Fireworks AI has grown tremendously in the last 6 months, driven by our vision for the future of AI. Here are the key lessons fueling our success: 1. 10x Thinking, Not Incremental: When I started Fireworks, Ali Ghodsi asked me, “What’s your 10x, and why is it worth building a startup around?” At Fireworks, we think big and constantly challenge ourselves — our goal is to make AI accessible to every developer, not in five years, but in five days, and without the need for an army of infrastructure experts or researchers. We have built not just better tools but a compound AI platform (https://lnkd.in/edhF-eu2) that integrates multiple modalities, tools, and knowledge bases, empowering businesses to achieve high quality results with AI. 2. Speed is Everything: At Fireworks, we operate with a relentless sense of urgency, constantly asking ourselves, “Why not today? Why not faster? Why not better?” This mindset drives us to deliver the fastest inference engine and onboard the latest models and hardware with unprecedented speed. We were the first to enable Llama family, Mistral family, Stable Diffusion 3 and many other SOTA models, as well as to integrate Nvidia and AMD’s cutting-edge hardware (https://lnkd.in/dujY9R-y). Our speed isn’t just about being first— it’s about empowering developers to create innovative UX which impacts our day to day life and shifts the industry. 3. Simplicity Scales: Our design philosophy draws inspiration from PyTorch: a simple UX for model developers, with all production complexities hidden in the backend. At Fireworks, we’ve built simple, yet powerful compute engines and tools like FireAttention (https://lnkd.in/eyN_sM6Z), FireFunction (https://lnkd.in/e7HQyCmB), and FireOptimizer (https://lnkd.in/ejdD-wGG) that allow developers to seamlessly build upon battle-tested, large-scale AI systems, backed by the operational expertise of the Fireworks team. This focus on simplicity allows our users to out-innovate convention and significantly shorten time-to-market for developers and enterprises worldwide. I shared more thoughts in this interview, thanks to EO STUDIO. Check it out:  https://lnkd.in/eRpqAusR Join our world-class team to land industrial-wide impact! https://lnkd.in/etYFJ8Wy

🔥FireAttention v3 -- enabling AMD as a viable alternative in the GPU inference serving market 🔥 Engineers at Fireworks AI have successfully ported FireAttention to AMD MI300s, resulting in 80% more throughput and 60% faster latency than NIM on Nvidia H100s. With these improvements, FireAttention V3 enables AMD MI300 to become a viable alternative for GPU inference. To achieve the performance gain we aimed for, we rewrote our attention kernel from scratch. We took advantage of AMD MI300’s higher memory capacity, and accounted for differences in shapes and element swizzling formats. Performance on the MI300 can get better with future firmware updates to power management and software updates for improved matmul performance. For developers interested in hardware diversity, we are happy to share learnings from writing FireAttention v3. Stay tuned on using FireAttention v3 from our on-demand platform shortly. For enterprises, this finally solves your supply chain resilience concerns: FireAttention V3 enables broader hardware optionality especially when you use Bring Your Own Cloud deployment - get in touch with us to explore running FireAttention v3 on your hardware! Blogpost with details: https://lnkd.in/dujY9R-y

🔥We just made Llama 70B 1.5X faster on Fireworks! 🔥 In the last week, developers using Llama 3.1 70B on Fireworks just got a 1.5X speed up. The speed boost is driven by continuous improvements to FireAttention, our proprietary inference engine, and FireOptimizer, our continuous optimization engine. The speed boost is easily visible on benchmarks - the graph below shows GPU based providers. Quantization didn’t change, traffic load didn’t change, we didn’t overprovision for benchmark traffic or give them any special treatment. The numbers just show incredible speeds on production workloads for all developers. FireAttention, our proprietary inference engine, helps developers access ultra-fast and efficient inference for open models. We’ve consistently updated it to retain its place as the fastest production-ready inference engine for open models, drawing on our deep experience as ex-PyTorch industry veterans. Learn more: https://lnkd.in/gEWKr3AP FireOptimizer, our continuous optimization engine, helps developers precisely customize quality and performance for their specific use case and workload characteristics. We leverage a range of techniques to deliver superior performance both for our serverless platform and custom deployments: https://lnkd.in/grnqb9W9 If you’re a developer building real-time user experiences with AI in production, consider following startups and enterprises like Cursor, Uber, and Cresta in building on the Fireworks platform. Get started at fireworks.ai or contact our team: https://lnkd.in/gnKdVucH We’ll keep adding new improvements and capabilities to the Fireworks platform over the coming few weeks. Stay tuned!

We're thrilled at what participants built at the Fireworks AI & E2B hackathon to kick off SF Tech Week! Thank to Edge & Era for having us! Congrats to the winners! 🏆🥇 Mandroid-Accelerator by Mandar Deshpande 🏆🥈 Cherry Limeade by Nehil Jain, Selvam Palanimalai 🏆🥉 APIGuard by Abhigya Wangoo, Noorvir Aulakh, Dexter Horthy

As part of sf tech week, MongoDB Ventures will be hosting a demo night in SF tomorrow 10/8 with the teams at the frontier of AI agents, Fireworks AI and Cognition. Come join us and our founder/CEO Lin Qiao and Cognition's Russell Kaplan to see what's coming. https://lu.ma/m6ql3t8q

now that the GPUs have had a chance to cool down 😮💨, wanted to also share another fun sunday afternoon experiment can we re-create the notebooklm experience with open-source models, and how far can we go? a few hours later - i got ✨ 𝗼𝗽𝗲𝗻 𝗻𝗼𝘁𝗲𝗯𝗼𝗼𝗸𝗹𝗺 ✨ up. 🔗 try it here: https://lnkd.in/gEEV4m2s drop in a pdf (or two), a hyper link & and you can get a podcast in 13(!!) different languages 🌎 🌍 🌏 i'lll add some examples in the comments 🔍 more deets here: https://lnkd.in/gWa6vz2N 📦 code here: https://lnkd.in/g2atiuyG credits to knowsuchagency and mrfakename for their original repos too! ========== 🛠️ this was built with 🛠️ ========== - 🦙 AI at Meta’s Llama 3.1 405B, hosted on Fireworks AI which supports pydantic schemas in their JSON mode - i think this has easily processed 100M tokens so far in the last few days - 🎙️ MyShell.ai’s MeloTTS & Suno’s Bark - these 2 speech-to-text models are lightweight, have strong multilingual support, and were open-sourced with voice samples too - 📚 Jina AI’s Reader API (they have a similar OSS model `reader-lm` too) which is used to parse html to markdown for the llm - 🤗 Gradio on Hugging Face spaces - which provides the GPU for the text-to-speech model models