Struggling to pick the right AI model?

Greetings Cohere Community,

The world of AI might be facing what psychologist Barry Schwartz famously coined “the paradox of choice.” The proliferation of new language models can feel overwhelming, and for many corporate leaders, it can lead to decision paralysis. 

The Fear of Future Models

Although excitement about the transformative potential of AI remains high, with some 85% of leaders reporting they expect to increase AI spending in 2024, the revenue growth gap between early adopters and laggards is growing and expected to increase 2.4x by 2026. The wait-and-see approach taken by some might ultimately prove to be detrimental. 

In conversation with UK Secretary of State for Science, Innovation, and Technology Michelle Donelan, Cohere’s CEO Aidan Gomez advises that “by engaging early and starting to experiment with the technology, [companies] will be in the best possible position in the future.”

How Do You Leapfrog Forward Without a Clear Line of Sight? 

Picking the right AI model can vary from proof-of-concept to scaling. This month, Cohere experts Sudip Roy and Neil Shepherd published How to Choose the Right AI Model for Your Enterprise to help customers navigate the variety of choices and find the best-fit model. [Full disclosure: we think our models are a pretty darn good choice.]

Their advice? Start by reviewing the following options: 

• Open-source or proprietary: Consider not only the upfront costs, but also the time-to-solution, data provenance, and indemnity options to avoid any unwanted surprises like indemnity obligations some open-source providers include. Then review the level of support and engineering know-how you will need, and the frequency of updates made to the models.
• General or tailored: Rightsizing the model to your use case and performance requirements at scale is critical. For example, does your solution need advanced reasoning (and the costs it entails) for every query? Consider how a fine-tuned model with advanced RAG capabilities may outperform a large general model at a fraction of the cost. Look for models that are optimized for performance with methods like quantization, transformer efficiencies, and model compression techniques. 

• Transformational or incremental adoption: Most organizations start with solutions for tactical benefits, like increasing productivity and lowering costs. A growing trend among customers is improving information retrieval systems with simple integration of a Rerank solution. We are also seeing a surge of leaders quickly advancing to strategic benefits like innovation and growth. According to Deloitte’s survey, 31% of leaders expect substantial transformation in 2024.

For more, check out our latest stories or read on for this month’s highlights and upcoming events!

Product 

Semantic search with embeddings is a critical component of RAG workflows. We are thrilled to announce that you can now scale semantic search systems using Cohere Embed and Pinecone’s serverless solution, a top choice for a vector database. 

For Business

As AI safety efforts take center stage in 2024, catch up on the seven top themes around potential harms from AI. Read The Enterprise Guide to AI Safety.

Developers

Learn how to fine-tune a chatbot in the latest LLM University chapter, The Developer’s Guide to Fine-Tuning Cohere Chat. 

Research

Introducing Aya, the biggest, open-source, multilingual model dedicated to research efforts across the world. Nonprofit research lab Cohere For AI launched Aya and accompanying datasets covering 101 languages in collaboration with 3000 researchers. Try the Aya model in the Cohere Playground. 

Company

Cohere joins the largest collection of AI developers, users, researchers, and affected groups in the world as part of the newly established U.S. AI Safety Institute Consortium (AISIC). 

Upcoming events with Cohere 

• March 6 [Online]: Join Cohere For AI and Adil Salim, Senior Researcher at Microsoft Research, who’ll be discussing small language models.
• March 7 [Online]: Celebrate International Women’s Day with an inspirational webinar on the Architects of AI: Women Powering MLOps.  Women from the Cohere technical team will share their unique experiences and insights with AI, offering an uplifting and educational event for all participants.
• March 11 [Austin, Texas]: Cohere will be at SXSW! Our Head of Solutions Architecture Shaun Hillin will be speaking on the panel, Breaking the Confidentiality Paradox: A Secure GenAI Roadmap, from 10:00 a.m. - 11:00 a.m. CET at the Hilton Austin, Salon C.
• March 18 [Online]: Join Cohere For AI in conversation with Cheng Soon Ong, Senior Principal Research Scientist at the Statistical Machine Learning Group, Data61, CSIRO, and Associate Science Director at Data61.

For all upcoming events, explore cohere.com/events.