OSS Capital reposted this
Last December 2024 at NeurIPS, Ilya Sutskever mused this slide to make the case that the AI field needs to push further into the domains of alternative architecture exploration of LLMs and AI models in general. Learning from the evolution of hominids can indeed offer a great deal of insight into how we humans can / should design and scale AI. This message was very open ended, but personally super validating and encouraging of approaches and techniques that take big leaps outside the status quo to learn from nature and explore new experimental architectures (we funded Liquid AI at incorporation for eg., nearly 2 years ago now). This evolutionary approach to AI improvement and iteration will continue as far out as we can imagine. The “wrong” approach is to become overly obsessed with the Transformer variation de jour that only serves to incrementally move the field forward.