Traceloop’s Post

It's always exciting to have the brightest people you used to work with joining the company you co-founded. Oz Ben Simhon is the best - and he's joining us at Traceloop as a founding engineer! In his own words: "I'm passionate about solving problems and getting things done—whatever you throw at me, you’ll probably hear 'Yalla, let's do it!' I love to travel, hike, and explore new places on foot and by food! I could talk more about my hobbies, but with two ninja toddlers keeping me on my toes, who’s got time for that?"

We decided to build an open-source project and here are a few tips that helped us grow and get huge companies like Amazon, Microsoft, IBM and Google to use and promote our product. 1. Organic reach: we published the project everywhere we could, and repeated that. Hacker News seemed to be the best place to get that first momentum. Yes, the website looks like it was taken from the 90s, but you'd be surprised at how many industry leaders are reading through posts there on a daily basis. Also worked well for us are dedicated Reddit communities, and to some extend Twitter/X. 2. Friendly Experience: we made our OSS friendly for first-time contributors from day 1. We opened around 10 issues in the repository with various degrees of complexity, tagged some with "good first issue" so that GitHub search engine can index our repo and opened a community slack workspace so people can ask questions or request guidance easily. 3. Quick Response: we monitored our open-source activity closely. We set up Zappier integrations so we get notified whenever someone opened an issue or a PR on the repo - so we can respond quickly. The first few contributors are looking for a quick feedback and may quickly abandon your project if they don't see maintainer activity. Community: we actively engaged with the community. We arranged webinars, answered questions, and were quick to fix bugs that were opened. This helped gain the trust that we need to make this succeed. Building and maintaining an open-source requires constant dedication of time and effort. But once you do it right, it’s a great way to get exposure to what you’re building. What is your story? How do you grow your open-source projects?

What are the most popular LLMs? The results may surprise you. Users use our product, Traceloop, to detect anomalies or hallucinations in production across many different LLMs. As we track which providers are more common to help us prioritize features, we were curious to see the results from the last couple of months. Looking at the data from 50k of our users, I was surprised to see that while OpenAI was the clear winner at the beginning of 2024, it all changed with the release of Claude 3 by Anthropic in March. By July, it was a clear winner, and by a lot. Even the release and adoption of GPT-4o and GPT-4o mini around June and July looked like a small bump compared to Anthropic hockey-stick usage increase. At the end of the day, it feels like it's all about trends: OpenAI was first to the market and was the go-to model provider for every developer out there. Claude 3 brought a lot of hype, leading many developers to switch over. It performs well in some tasks, but not all. It's tempting to rely on what's the most trendy model now, but you should evaluate the model for your use case with proper metrics and not get carried away by the hype. Who knows - what's trendy today may not be trendy tomorrow. What is your preferred Model? Do you see a clear difference when using Claude that justifies this huge difference?

OpenAI just made LangChain redundant. Here’s why engineers shouldn’t choose technologies based on today’s tech trends. OpenAI released structured output support last week (link in the comments if you missed the announcement). This is another small but significant feature that makes it easier to just use OpenAI directly instead of other frameworks like LangChain. I believe this update will make LangChain redundant in less than a year. I also think this tells a bigger story: engineering's strong preference for what's trendy today, creating legacy code for the future. This has happened many times in the past (do you remember jQuery?), and will probably continue to happen. So what should we do? How do we choose the right technologies to use on our next projects? I look at 3 key aspects when choosing a technology to adopt for a project: 1. I prefer to use things I'm already familiar with. When starting a new project, I focus on getting s**t done, not learning some weird new constructs. 2. I prefer to start simple and complex stuff only when I need to. Can I just call OpenAI directly? Great. No need to use a framework if I don't need it. 3. I look at the technology itself. It should be stable and widely used. Stability is important, I don't want to spend all day upgrading versions. And if it's not widely used, I may get stuck trying to figure out ways around weird bugs with no way to get help. For us at Traceloop, we chose React and Next.js for our FE; Go, Kafka, and Clickhouse powering our backend; and VertexAI for serving models in production. What are your tools of the trade?