CapitalG reposted this
Huge congrats to Eric Steinberger and the Magic team as they announce their 100M token context window; $450M+ in additional funding from Nat Friedman (NFDG), Eric Schmidt, Jane Street, Sequoia Capital; and scaled compute cluster from Google Cloud. We at CapitalG are thrilled to continue to support and partner with the team. It's been a wild 2 years since Eric Steinberger first told me that he was building a new model architecture that would allow for cost-efficient long context - which they would use to power the world's first autonomous software engineer. I could not be more amazed by the intellectual horsepower, dedication to their vision, and strong moral values displayed by every single member of the Magic team as they continue to push the boundaries of AI. They're hiring A+ folks ready to tackle hard problems at the frontier of AI capabilities and AI for software engineering. If that sounds like you, please reach out or apply at magic.dev!
LTM-2-Mini is our first model with a 100M token context window. That’s 10 million lines of code, or 750 novels. Full blog: https://lnkd.in/g5-pKWvi Our LTM (Long Term Memory) mechanism needs >1,000x less compute and memory than Llama 3.1 405B’s attention. Llama 3.1 would need 638 H100s *per user* to store a 100M token KV cache. LTM needs a small fraction of one. SSMs, RNNs, and RAG all exploit weaknesses in evals like Needle In a Haystack, so we made a new eval, HashHop: 1) Incompressible 2) Multi-hop 3) No semantic hints 4) No recency bias With context solved, we now focus on unbounded inference-time compute as the next (and potentially last) breakthrough we believe is needed to build reliable AGI. Imagine if you could spend $100 and 10 minutes on one task and reliably get a great pull request for an entire feature. That’s our goal. We are 23 people (+ 8000 H100s) working on a single project: co-designing for long context, inference-time compute, and end-to-end RL to automate coding and research. Ben Chess (fmr. OpenAI supercomputing lead) just joined to help us scale and we’re hiring more engineers and researchers across ML, CUDA, infra, security, and more: https://magic.dev/careers