What are the most popular LLMs? The results may surprise you.
Users use our product, Traceloop, to detect anomalies or hallucinations in production across many different LLMs. As we track which providers are more common to help us prioritize features, we were curious to see the results from the last couple of months.
Looking at the data from 50k of our users, I was surprised to see that while OpenAI was the clear winner at the beginning of 2024, it all changed with the release of Claude 3 by Anthropic in March. By July, it was a clear winner, and by a lot.
Even the release and adoption of GPT-4o and GPT-4o mini around June and July looked like a small bump compared to Anthropic hockey-stick usage increase.
At the end of the day, it feels like it's all about trends: OpenAI was first to the market and was the go-to model provider for every developer out there. Claude 3 brought a lot of hype, leading many developers to switch over. It performs well in some tasks, but not all.
It's tempting to rely on what's the most trendy model now, but you should evaluate the model for your use case with proper metrics and not get carried away by the hype. Who knows - what's trendy today may not be trendy tomorrow.
What is your preferred Model? Do you see a clear difference when using Claude that justifies this huge difference?