NinjaTech AI reposted this
Founder/CEO/CPO of NinjaTech AI (MyNinja.ai), Ex-Sr. Director of Product Management, 11yr Googler | Winner of Google Manager Award | 4x Entrepreneur | Investor, advisor & mentor | AI/ML geek | Stanford & Berkeley
www.MyNinja.ai got a massive next-gen hardware & software upgrade: Ninja is now powered by the world's fastest implementation of agentic Llama 3.1 405B as our core engine. Thanks to our close partners at Amazon Web Services (AWS), Ninja is now running on next-gen hardware which allows us to leverage Speculative Decoding (https://lnkd.in/gpmrqa_X) at scale for Llama 3.1 405B. In our estimate, currently this is the fastest-in-the-world inference implementation of agentic 405B at scale in a production environment leveraging the best GenAI components that AWS has to offer. You can see speed comparisons below between Meta.ai's 405B vs. www.MyNinja.ai (3.75x faster)...And ours is doing more work because it's doing intent-analysis and tool-selection in real-time (it's agentic). You can try it yourself here: https://lnkd.in/gmpxPJ4a We're still optimizing and fixing bugs, so it'll get even better in the next 2 weeks (e.g. currently, under load, it can be jerky; that's a VLLM issue and we're working with the open source community to fix those) - our science and engineering teams are on fire these days :D. With these hardware upgrades, our infra costs are going down too (due to more throughput), so we've decided to pass the cost savings to users. Next week, we'll be rolling out our entry tier for unlimited use (Standard Tier = $5/month) and we'll be reducing our prices for Pro (from $15 to $10) & Ultra (from $30 to $15) tiers as well. All of our existing paid users will get the price adjustments (early Christmas gift!). Our core mission is to "Democratize access to world's best AI agents & models to everyone" and that vision is becoming a reality thanks to the genuine support of Matt Garman Jon Jones Swami Sivasubramanian Nafea Bshara Gadi Hutt Kamran Khan Brennan Demro Samantha Lisowski and many others at Amazon Web Services (AWS): you all have our deepest gratitudes. Also, a few more product features are landing tomorrow 😊 (for those who asked: no, we don't sleep much these days 😇 🤖 ).