Magic

LTM-2-Mini is our first model with a 100M token context window. That’s 10 million lines of code, or 750 novels. Full blog: https://lnkd.in/g5-pKWvi Our LTM (Long Term Memory) mechanism needs >1,000x less compute and memory than Llama 3.1 405B’s attention. Llama 3.1 would need 638 H100s *per user* to store a 100M token KV cache. LTM needs a small fraction of one. SSMs, RNNs, and RAG all exploit weaknesses in evals like Needle In a Haystack, so we made a new eval, HashHop: 1) Incompressible 2) Multi-hop 3) No semantic hints 4) No recency bias With context solved, we now focus on unbounded inference-time compute as the next (and potentially last) breakthrough we believe is needed to build reliable AGI. Imagine if you could spend $100 and 10 minutes on one task and reliably get a great pull request for an entire feature. That’s our goal. We are 23 people (+ 8000 H100s) working on a single project: co-designing for long context, inference-time compute, and end-to-end RL to automate coding and research. Ben Chess (fmr. OpenAI supercomputing lead) just joined to help us scale and we’re hiring more engineers and researchers across ML, CUDA, infra, security, and more: https://magic.dev/careers

We're hiring for an Ops Generalist in San Francisco to help keep things organized as we grow our team. Come join us!

I love my team a lot and sometimes it’s stressful but life has never been so fulfilling. If you want to build AGI on a small team of people who care a lot with thousands of GPUs, please apply :) www.magic.dev

We've raised $117M from Nat Friedman and others to build an AI software engineer. Code generation is both a product and a path to AGI, requiring new algorithms, lots of CUDA, frontier-scale training, RL, and a new UI. This round was led by Nat Friedman & Daniel Gross (NFDG), with participation from CapitalG and Elad Gil, and will allow us to further scale up our models. If you want to solve very hard problems to build safe AGI on a small team with thousands of GPUs, come join us! Link in comments