Gensyn

Introducing Diverse Expert Ensembles Diverse Expert Ensembles are a model architecture, optimised for decentralised networks. They are trained in an embarrassingly parallel fashion, and use expert diversity to enhance model performance. Why it matters We believe AGI will take the form of an open ecosystem of diverse models, rather than a monolithic model from one company. HDEE (our experiments with Diverse Expert Ensembles) show early evidence of this, finding that diverse model ensembles achieve the best perplexity in 20 out of 21 evaluated domains compared to a homogeneous baseline. How it works Building on top of the embarrassingly parallel training method of Branch Train Merge (BTM) from Li et al, 2022, HDEE works by tailoring each expert based on its data domain and/or compute capabilities, varying the model size and training steps. These configurations ensure that each expert is optimally tuned, resulting in a more capable ensemble, at an equivalent compute budget. Looking Ahead Diverse Expert Ensembles are a glimpse into the future of ML, where developers can train models independently, using configurations that suit their specific data, compute, and areas of expertise, and merge them together on an open network like Gensyn. Learn more: https://lnkd.in/ePvhDguC Run the experiments: https://lnkd.in/ezn9p28V Read the paper: https://lnkd.in/es3mmy_c

Introducing SkipPipe SkipPipe is a new communication-efficient, pipeline-parallel training method. It reduces distributed training time by up to 55% and is scalable to theoretically infinite model size. Today, we're open sourcing it to push the frontier of decentralised ML. Read more: https://lnkd.in/e2wZjW3K Paper: https://lnkd.in/e79WPepJ Open source code: https://lnkd.in/eJSBFqMe

Introducing Verde Verde is a high-scale verification system for machine learning over untrusted nodes. It ensures correct execution of ML operations across any device in the world, from data centres to the edge. It operates at runtime and doesn't require any onboarding. Research: https://lnkd.in/gAqPitwt Blog: https://lnkd.in/ggN-zG75 Demo: https://lnkd.in/gC_xNcf6

Two years ago, we laid out our vision for a machine learning compute protocol. One that connects every device in the world into an open network for machine intelligence, with no gatekeepers or artificial boundaries. This week, we’ll be sharing some of our early progress, beginning with RL Swarm, a peer-to-peer system for collaborative reinforcement learning over the internet. Next month, we’ll open our Testnet, allowing anyone to contribute to the frontier of open machine intelligence. Introducing RL Swarm RL Swarm is a fully open source system for collaborative reinforcement learning over the internet. It is a live demo of our research findings, which show that models training with RL learn faster when they train as a collective swarm than they do on their own. Join our swarm now to see this in practice. You can participate with consumer hardware at home or a powerful GPU in the cloud. You can follow along with the swarm’s progress by following the links below. To join the swarm, click here: https://lnkd.in/gjqjzvq6 To learn more about it, and follow its progress, click here: https://lnkd.in/gPAiRr3m To read the underlying research, click here: https://lnkd.in/gikzdk-Q

Catch the Gensyn team at ETHDenver next week. We've got founders Ben Fielding and Harry Grieve and COO Jeffrey Amico speaking on some expert panels, plus we're hosting 2 awesome events. Check out below. Feb 25: 6pm - Join us for an evening of food, drinks, swag, and networking with conversations on decentralised AI, collaborative machine learning, and verifiable compute. https://lu.ma/z0v7px3g Feb 26: 9pm - Get involved in the most powerful neural network in Denver. (ai)RL - live collaborative learning between biological nodes (you); cohosted with Eden Block, W.ai and Pluralis. https://lnkd.in/eAf7Bx_g Feb 25: Verifiable AI and the Future of DeFi. Catch Co-Founder, Ben Fielding, on stage with Jason Morton, Grigore Rosu and Jeff Wilser at Encode Club's Modular DeFI Research Day talking about when and why to verify AI. (https://lnkd.in/e7ZCUbwu) Feb 26: DEPIN x AI: Building the Future of Intelligent Infrastructure. Hear from our COO, Jeffrey Amico, as he joins a panel at DePIN Day to discuss machine learning's new low-level infrastructure. (https://lu.ma/depin-denver) Feb 26: We're excited to be supporting and speaking at Open AGI Summit again this year! Drop by our booth to chat with the team and hear Co-Founder, Ben Fielding, talk all things Compute on a panel at 2pm.(https://lnkd.in/emcQ4Eyf) Feb 27: Decentralized Training Panel catch Co-Founder, Ben Fielding, on stage with Manveer Basra and Vladyslav Larin at Big Brain Holdings and Blockchain at Berkeley's DeAI Summit. (https://lu.ma/bjki4pj0) Feb 27: We're looking forward to supporting and speaking at RenAIssance ETH Denver this year. Catch Co-Founder, Harry Grieve, on stage with Alexander Long, Bidhan R., Nick Wen and Timothy K. talking about Decentralised Training. (https://lu.ma/fkpems9y) Feb 28: How can AI benefit from ZK? Hear from Co-Founder, Harry Grieve, as he chats about cryptographic verification of machine learning programs at House of ZK. (https://lnkd.in/ebUwNjz9) Feb 28: Security and Reliability of AI Models in Cross-Chain Data Interactions. Hear from Co-Founder, Ben Fielding on stage at Scaling DeAI Summit. (https://lnkd.in/eRquRHST)

Catch the Gensyn team at ETHDenver next week. We've got founders Ben Fielding and Harry Grieve and COO Jeffrey Amico speaking on some expert panels, plus we're hosting 2 awesome events. Check out below. Feb 25: 6pm - Join us for an evening of food, drinks, swag, and networking with conversations on decentralised AI, collaborative machine learning, and verifiable compute. https://lu.ma/z0v7px3g Feb 26: 9pm - Get involved in the most powerful neural network in Denver. (ai)RL - live collaborative learning between biological nodes (you); cohosted with Eden Block, W.ai and Pluralis. https://lnkd.in/eAf7Bx_g Feb 25: Verifiable AI and the Future of DeFi. Catch Co-Founder, Ben Fielding, on stage with Jason Morton, Grigore Rosu and Jeff Wilser at Encode Club's Modular DeFI Research Day talking about when and why to verify AI. (https://lnkd.in/e7ZCUbwu) Feb 26: DEPIN x AI: Building the Future of Intelligent Infrastructure. Hear from our COO, Jeffrey Amico, as he joins a panel at DePIN Day to discuss machine learning's new low-level infrastructure. (https://lu.ma/depin-denver) Feb 26: We're excited to be supporting and speaking at Open AGI Summit again this year! Drop by our booth to chat with the team and hear Co-Founder, Ben Fielding, talk all things Compute on a panel at 2pm.(https://lnkd.in/emcQ4Eyf) Feb 27: Decentralized Training Panel catch Co-Founder, Ben Fielding, on stage with Manveer Basra and Vladyslav Larin at Big Brain Holdings and Blockchain at Berkeley's DeAI Summit. (https://lu.ma/bjki4pj0) Feb 27: We're looking forward to supporting and speaking at RenAIssance ETH Denver this year. Catch Co-Founder, Harry Grieve, on stage with Alexander Long, Bidhan R., Nick Wen and Timothy K. talking about Decentralised Training. (https://lu.ma/fkpems9y) Feb 28: How can AI benefit from ZK? Hear from Co-Founder, Harry Grieve, as he chats about cryptographic verification of machine learning programs at House of ZK. (https://lnkd.in/ebUwNjz9) Feb 28: Security and Reliability of AI Models in Cross-Chain Data Interactions. Hear from Co-Founder, Ben Fielding on stage at Scaling DeAI Summit. (https://lnkd.in/eRquRHST)

Join our ICLR meetup on Decentralized and Open Source AI later today. Date and Time: Friday May 10, 7-9 PM Limited spots available. https://lnkd.in/exP4kGvg

"I want there to be a healthy and vibrant ecosystem. I would be hesitant about five mega corporations taking over." -- Andrej Karpathy This echoes the future that we envision. One where currently gatekept developers can try out new architectures, datasets, and even modalities - ultimately creating novel things together in a thriving ecosystem. It begins with making the fundamental resource for AI - compute power - accessible to everyone.

At Gensyn, we are shaping the future of Machine Learning. Every day, our engineering and research teams take on cutting-edge challenges in distributed computing. From verification of decentralized training to serialization across ML frameworks, we push the boundaries in AI. If this is something that you're excited about, then join our team: https://gensyn.ai/jobs

Stanford Institute for Human-Centered Artificial Intelligence (HAI) highlights the soaring training costs for AI models and the resulting unequal access to compute in the AI Index Report 2024. For AI to equitably proliferate throughout our world, it's key to establish the right to build and computational liberty for everyone – a vision that we at Gensyn are actively pursuing.