Retrocausal’s Post

Excited to share our latest work on video LLMs for temporal reasoning in long videos. In particular, we introduce (i) a time-aware clip encoder for extracting time-aware fine-grained cues from short-term clips, and (ii) a BiLSTM module for capturing long-range temporal dependencies across multiple clips. Check out the below link/video for more details.

🔍 How can we push the boundaries of video LLMs for temporal reasoning? Our latest work introduces: ✔️ A time-aware clip encoder for capturing precise short-term temporal cues. ✔️ A BiLSTM module for understanding long-term dependencies in lengthy videos. This milestone wouldn’t have been possible without the brilliant work of my fellow authors, Umer Ahmed and Hamza Khan, and without the invaluable mentorship of Quoc-Huy Tran and Zeeshan Zia—thank you all for your dedication and guidance! 🙌 Check out our results in the link/video below and share your feedback! #GenerativeAI #LLMS #DeepLearning #MultimodalAI #Innovation

🚀 Excited to share my latest Medium article where I dive into the intersection of Generative AI and video content! 🎥 We have all been in the crazy situations where earlier we had utilized the rich information platform of YouTube to learn from a video and moved on with the further objectives without thinking too much of it and now, we are in an interview or exam situation where we need to recall something that we learnt from that video right before the game begins. How difficult it becomes to then go through the whole video scrolling the bar trying to search for the exact point at which the video mentions about the information that we’re looking for! In this piece, I explore how to search YouTube videos with a RAG pipeline and ensemble retrieval. 🔍 If you're interested in GenAI, RAG, machine learning, or simply love exploring innovative tech solutions, this one's for you! Check it out and let me know your thoughts! 💬 #GenerativeAI #hybridsearch #VideoSearch #faiss #rag

Introducing Meta Video Seal: a state-of-the art comprehensive framework for neural video watermarking. Try the demo ➡️ https://go.fb.me/bcadbk Model & code ➡️ https://go.fb.me/7ad398 Details ➡️ https://go.fb.me/n8wff0 Video Seal adds a watermark into videos that is imperceptible to the naked eye and is resilient against common video editing efforts like blurring or cropping, in addition to commonly used compression techniques used when sharing content online. With this release we’re making the Video Seal model available under a permissive license, alongside a research paper, training code and inference code.

Here's how Video Seal could be a valuable tool for engineers: 1. Protecting Designs and Intellectual Property: Watermarking CAD files and simulations: Engineers often create valuable designs and simulations using CAD software or other specialized tools. Video Seal can be used to embed watermarks in videos of these designs, making it difficult for others to steal or claim ownership of their work. This is particularly important in competitive industries or when sharing designs with clients or collaborators. Securing training and instructional videos: Engineers often create training videos or tutorials for internal use or to share knowledge with others. Watermarking these videos with Video Seal can help prevent unauthorized distribution or modification. 2. Ensuring Data Integrity and Authenticity: Verifying data from autonomous systems: Engineers working with autonomous systems (e.g., drones, robots) rely heavily on video data for navigation, object recognition, and decision-making. Video Seal can help verify the authenticity and integrity of this data, ensuring that it hasn't been tampered with or corrupted. This is crucial for safety and reliability in applications like self-driving cars and industrial automation. Authenticating video evidence in investigations: In fields like civil engineering or failure analysis, video footage can be crucial evidence. Video Seal can help verify that video evidence hasn't been manipulated, ensuring accurate investigations and reliable conclusions. 3. Building Trust and Transparency in AI-Generated Content: Verifying AI-generated designs and simulations: As AI plays a larger role in engineering design, it's important to be able to verify the outputs of AI models. Video Seal can be used to watermark AI-generated videos of designs or simulations, providing a way to track their origin and ensure they haven't been altered. Increasing confidence in AI-driven decisions: In applications where AI is used to make critical decisions (e.g., structural analysis, medical diagnosis), Video Seal can help build trust by providing a way to verify the integrity of the AI's input data and output visualizations. 4. Other Potential Benefits: Tracking the provenance of video data: Video Seal can help track the origin and history of video data, which can be useful for managing versions, identifying sources of errors, and ensuring compliance with regulations. Detecting deepfakes and misinformation: As deepfake technology becomes more sophisticated, Video Seal could help engineers identify manipulated videos that might be used to spread misinformation or cause harm. By understanding these potential benefits, engineers can leverage Video Seal to protect their work, ensure data integrity, and promote responsible use of AI in their respective fields.

I am excited to see where this leads us in the future regarding traceability, observability, and liability of digital content, especially with the proliferation of generative AI. This is just the first step in building a robust data supply chain and improving traceability. I am also curious about the potential effects on AI training if future datasets include these invisible watermarks. Will they impact the training data and model behaviour if we don't cleanse them first? Alternatively, will we be able to trace the origins of datasets through these watermarks, allowing for better management of ownership and copyright issues?

👆 #AI #Video #watermarking plays a crucial role in distinguishing fake videos from authentic ones and in protecting #intellectual #property rights. ☑️ #Video #Seal #Meta #Neural #Fake #Innonvation #AI #Media #Production

A state-of-the-art comprehensive framework for neural video watermarking: Neural Watermarking Overview: Neural video watermarking uses deep learning techniques to embed watermarks (e.g., logos or text) into video content in a way that is robust, imperceptible, and difficult to remove or alter. End-to-End Framework: The framework is typically end-to-end, meaning it incorporates both watermark embedding and extraction processes within a unified model, improving efficiency and robustness. Invisible Watermarking: One of the primary goals is to make the watermark imperceptible to the viewer, ensuring that it doesn’t degrade the visual quality of the video, even under compression or various transformations. Deep Neural Networks (DNNs): State-of-the-art frameworks often use convolutional neural networks (CNNs) or generative models (GANs) for embedding and extracting the watermark, leveraging their ability to capture complex patterns in video data. Robustness to Attacks: Neural watermarking frameworks are designed to maintain watermark robustness, ensuring the watermark survives common attacks like compression, cropping, frame dropping, and video transformations. Training with Adversarial Loss: To enhance watermark resilience, frameworks often use adversarial loss functions, training the watermarking model with both the watermarking task and a counteracting model to strengthen its robustness. Time and Spatial Consistency: Watermarks in video should maintain consistency over time (across frames) and spatially (in terms of locations within the video), ensuring they remain detectable and intact throughout the entire video. Embedding in Feature Space: Advanced models may embed the watermark not directly in pixel space but in the feature space of the video, improving invisibility and increasing robustness to common video processing operations. Extraction Without Reference: A key innovation in neural video watermarking is blind extraction, meaning the watermark can be extracted without needing the original video or watermark image, relying solely on the trained model. Applications and Use Cases: This framework can be used in various real-world applications, including content protection, intellectual property verification, video tracking, and watermark-based digital rights management (DRM). In summary, a state-of-the-art comprehensive framework for neural video watermarking combines advanced neural network techniques to embed imperceptible, robust watermarks into video content, providing efficient, secure methods for content protection and ownership verification.

Retrieval Augmented Generation (RAG): A Comprehensive Visual Walkthrough 🧠📖🔗🤖 via #TowardsAI → https://bit.ly/4cptRtM

🔬 Breakthrough in diffusion model alignment! ByteDance's SeedEdit introduces a novel approach to image editing by reframing it as a controlled balance between reconstruction and re-generation tasks. ⚡️ Key innovation: Instead of relying on scarce paired editing data or unstable inversion techniques, SeedEdit employs a progressive alignment framework that transforms a text-to-image diffusion model into a robust image editor through iterative refinement. 🧠 Technical highlights: - Causal self-attention architecture for image conditioning - Diverse paired data generation through multiple re-generation techniques - Iterative alignment process maximizing both CLIP directional score and image similarity 📊 The results are impressive: Significantly higher GPT evaluation scores (78.54 vs 54.17) and CLIP directional scores (0.1766 vs 0.1473) compared to previous SOTA, while maintaining superior image consistency. 🔗 Check it out: Paper: https://lnkd.in/gm44nuaw Discussion: https://lnkd.in/g9TcQagj Demo: https://lnkd.in/g5Qc4f74 #MachineLearning #DiffusionModels #ComputerVision #DeepLearning #AIResearch #ByteDance