Twelve Labs

#IBC2024 was a blast! It’s always inspiring to be at the heart of where the media and entertainment industry meets innovation. 🎤 Our team had the privilege of speaking on multiple panels, sharing insights, and exploring new frontiers in AI and video technology. It was amazing to see how the media and entertainment field is adopting multimodal AI and video understanding for enhanced workflows and user engagement! A huge shout-out to our incredible partners: EMAM, Inc., Vidispine - an Arvato Systems brand, Amazon Web Services (AWS), Monks, Nomad Media, Blackbird plc, Snowflake, Databricks, and NVIDIA for their support and collaboration. 📍 Big thanks to everyone who stopped by our booth and our vibrant spot in the AWS/Nvidia Innovation Village. Your enthusiasm and interactions were truly phenomenal!  🇳🇱 And of course, Amsterdam! Roaming around the conference floors was almost as enchanting as navigating its iconic canals. 🤝  Thank you to everyone who stopped by, engaged with us, and shared this memorable experience. Here’s to more technology, partnerships, and adventures. Until next time!

We are excited to share a new tutorial on integrating Twelve Labs' Embed API with Pinecone's hosted vector database for RAG-based Q&A on videos! 🤝 This tutorial demonstrates: ⬇ • Video embedding and semantic search with Twelve Labs + Pinecone  • Conversational AI on video content using Pegasus and LLaVA-NeXT-Video • Techniques for processing and sampling video data • Integrating multiple solutions for advanced video understanding Perfect for developers and AI engineers looking to build cutting-edge video chat applications. Check it out and let us know what you think! https://lnkd.in/gSGdzmKr

Thanks to IntelliVid Research for the engaging conversation and the opportunity to dive deep into how Twelve Labs is shaping the future of video content management! #VideoAI #TechTalk

In the 62nd session of #MultimodalWeekly, we have three exciting presentations on temporal action localization, hallucination benchmark for Vision-Language models, and learning objectives for Vision Transformers. ✅ Benedetta Liberatori from Università di Trento will discuss T3AL, which stands for Test-Time Adaptation for Temporal Action Localization. In a nutshell, T3AL adapts a pre-trained Vision and Language Model (VLM) at inference time on a sample basis. ✅ Tianrui Guan and Fuxiao Liu from the University of Maryland will discuss three separate works: (1) HallusionBench -- manual hallucination benchmark creation, (2) AUTOHALLUSION -- automatic hallucination benchmark curation, and (3) Eagle -- hallucination alleviation and mitigation. ✅ Manjin Kim from Pohang University of Science and Technology will discuss his work StructSA, which stands for structural self-attention. StructSA is a new attention mechanism that leverages rich correlation patterns naturally emerging in key-query interactions of attention. Register for the webinar here: https://lnkd.in/gJGtscSH 🕶

~ New Webinar ~ The recording of #MultimodalWeekly 57 is up! Watch here: https://lnkd.in/gwfVPq8P 📺 They discussed: 1️⃣ Automatic Data Construction for Composed Video Retrieval (Lucas Ventura) 2️⃣ The Rapid Decline of the AI Data Commons + Multimodal Implications (Shayne Longpre) 3️⃣ Prompting LLMs to Transform Video Annotations at Scale (Nina Shvetsova Anna Kukleva) Enjoy!

Our new tutorial with Roe AI showcases how to create a semantic video search solution that leverages our Embed API and their platform to manage unstructured data. 💡 ▶ Read the tutorial: https://lnkd.in/e62Px3hM ▶ Watch the demo: https://lnkd.in/gbXD6j2W ▶ Review the notebook: https://lnkd.in/gM3F2D-2 The combination of Twelve Labs' advanced video understanding and Roe AI's flexible data handling enables the creation of more intelligent, context-aware applications. This allows developers to process and understand video content in ways that were previously challenging or impractical. 👥

Twelve Labs is excited to share our latest technical blog post, which expands on our recent TWLV-I technical report: https://lnkd.in/grbrxh7X This post goes beyond just showcasing TWLV-I's capabilities - it introduces a comprehensive evaluation framework for video understanding models. ⚖ Our goal is to establish a new standard in assessing both appearance and motion capabilities in video AI. We believe this holistic approach is crucial for advancing the field and developing more robust models. ♎ Key highlights from the blog post: 1️⃣ Detailed evaluation methodologies including linear/attentive probing, K-NN classification, temporal/spatial action localization, and embedding visualizations. 2️⃣ Rigorous comparison of TWLV-I against state-of-the-art models across multiple benchmarks. 3️⃣ Insights from t-SNE and LDA visualizations revealing strengths and challenges in appearance vs. motion understanding. 4️⃣ Discussion of future research directions in scaling, image embedding, and multimodal tasks. We present TWLV-I not just as a high-performing model, but as a case study for our proposed evaluation framework. Our aim is to guide the video AI community towards more comprehensive assessment and development of video understanding models. ✔ Read the full technical blog post here: https://lnkd.in/g2VJjPui ✔ Adopt our evaluation methodologies: https://lnkd.in/gcANeufq Massive thanks to the core contributors (Hyeongmin Lee Jin-Young Kim Kyungjune Baek Jihwan Kim Aiden L.) and the rest of the Twelve Labs ML Research and ML Data teams.

~ New Webinar ~ The recording of #MultimodalWeekly 56 is up! Watch here: https://lnkd.in/dQMzQCik 📺 They discussed: 1️⃣ The Time Interval Machine (TIM) - which addresses the interplay between the two modalities in long videos by explicitly modeling the temporal extents of audio and visual events. Jacob Chalk Jaesung Huh 2️⃣ Movie-Identity Captioner (MICap) - which is a new single-stage approach that can seamlessly switch between id-aware caption generation or fill-in-the-blanks when given a caption with blanks. Haran Raajesh Naveen Reddy D 3️⃣ From Recaps to Story Summarization - which tackles multimodal story summarization by leveraging TV episode recaps — short video sequences interweaving key story moments from previous episodes to bring viewers up to speed. Aditya Singh Dhruv Srivastava Enjoy!

Imagine stepping into an interview room where your every gesture, word, and expression is not just observed, but understood. Welcome to the future of job interviews, where AI becomes your personal coach and ally. 🙅♀️ In today's competitive job market, acing that crucial interview can mean the difference between landing your dream job and watching it slip away. But what if you had a secret weapon—a tool that could analyze your performance, highlight your strengths, and gently suggest areas for improvement? 💼 That's exactly what the AI Interview Analyzer offers (built by Hrishikesh Yadav), using powerful models from Twelve Labs to provide insights once exclusive to seasoned HR professionals. 😉 ☑ Read the complete tutorial here: https://lnkd.in/gndz9C23 ☑ A detailed demonstration video of the application is provided here: https://lnkd.in/gMZrSbYU ☑ You can explore the demo of the application here: https://lnkd.in/gHJyNrBY ☑ You can also experiment with it using this Replit template: https://lnkd.in/gweyVS7X ☑ Find the repository for the notebooks and this application on GitHub: https://lnkd.in/gz2H4R8s

Exciting news! We’ve completed SOC 2 Type 2 certification! What does this mean? This milestone reinforces our commitment to protect our customers data. Here’s what users can look forward to: 🎥 Enhanced Video Content Protection: Rigorous security standards to prevent unauthorized access. ⏱ Reliable Video Processing: Ensures tasks are completed without interruption. 🔐 Strict Confidentiality Measures: Robust encryption and access controls to protect user data. Twelve Labs is dedicated to maintaining the highest standards in security, availability, and confidentiality—today and into the future. Join us as we continue to lead the way in secure, innovative AI video understanding. Your trust, our technology. https://lnkd.in/gG6m8rav #TwelveLabs #MultiModal #soc2