With DeepSeek so widely talked about in the past 2 weeks, our team had to try it out against Gemini for Incrowd's model to find the best moments/clips. 👇👇
🔍 𝗪𝗲 𝗣𝘂𝘁 𝗗𝗲𝗲𝗽𝗦𝗲𝗲𝗸 & 𝗚𝗲𝗺𝗶𝗻𝗶 𝘁𝗼 𝘁𝗵𝗲 𝗧𝗲𝘀𝘁—𝗛𝗲𝗿𝗲’𝘀 𝗪𝗵𝗮𝘁 𝗪𝗲 𝗙𝗼𝘂𝗻𝗱: Incrowd is all about finding the best moments from live performances. We’re constantly refining how we extract the most compelling clips using Large Language Models (LLMs), so when we saw all the buzz around DeepSeek AI—touted for its intelligence, scalability, and cost-effectiveness—we had to put it to the test against Gemini. DeepSeek has been making waves and topping benchmark leaderboards. But the real question for us is: Can it handle the nuanced, high-precision searches that Incrowd depends on? To find out, we ran DeepSeek and Gemini through one of our toughest challenges—finding a specific moment buried inside a long video summary. The prompt? A hefty 16,985 tokens, asking the models to pinpoint a scene with a "woman wearing blue." We used OpenRouter to test both models under default settings, measuring response quality, speed, and cost. The highlights of our experiment are: → Both Gemini-1.5-pro and DeepSeek-R1 were highly effective. They both identified the correct moment. → DeepSeek-R1-Distill-Llama-70B and Gemini-1.5-flash both identified the right answer but also included a few other moments that didn’t precisely match. Notably, DeepSeek-R1-Distill-Llama-70B altered the original description to fit the prompt better, while Gemini-1.5-flash provided actual descriptions that weren’t as closely matched. This wouldn’t fly in Incrowd’s product and probably frustrate people with our implementation. → Overall, the distilled versions of DeepSeek tended to include incorrect information, posing potential issues for applications that demand high accuracy and reliability. → While DeepSeek-V3 failed to pinpoint the moment, it did not produce any incorrect responses either. → Cost-wise, DeepSeek-R1 was 47% cheaper than Gemini-1.5-pro, at $0.016 compared to $0.023. → Despite being the most affordable option for using the DeepSeek-R1 and V3 models, DeepSeek’s API is plagued by frequent downtimes and overloads, as evidenced by reports on DeepSeek’s status page. As a result, users may turn to more expensive alternatives, as shown in the table below. → Gemini offers a generous free tier for all API users, and currently, Azure and Chute provide free inference for DeepSeek-R1 temporarily. This test highlighted each model's strengths and weaknesses in a real-world application for Incrowd. DeepSeek and Gemini both have merits, but DeepSeek-R1 and Gemini-1.5-pro proved the most effective for tasks demanding precision and reliability. These takeaways will shape how we fine-tune our video analysis tools, keeping Incrowd at the forefront of AI-powered content discovery. And the best part? You won’t have to wait long to see it in action. Our next release will let you search through hours of footage and instantly find the moments that matter. Stay tuned! 🚀