Ben McGuire’s Post

Multi Modal Knowledge Graph #Embeddings - Key Trends As the industry moves towards building complex #LLM products, a few trends are becoming quite important. These trends are very often driven by the need to build higher ROI #GenAI industry products. Specifically, the trends include (but not limited to):   a.     Context strengthening through better information retrieval & information modeling – specifically, for this post, the use of Knowledge Graphs, semantic web & approaches like the Graph RAG have shown promise.   b.     Use of real-world inputs, such as multi modal data – a recent design example, I’ve been involved with used drone sent images, sensor data & various forms of metered data, to feed into multiple knowledge graph RAGs. c.     Of course, structuring approaches such as advanced RAG, DSPy, agentic flows & others, along with building compound AI systems, through chained inference steps or reasoned function calling. As Graph RAG like approaches arise, it is important to understand the unique needs of Multi Modal Knowledge Graphs. Some base “thought” issues that arise, include: 1.     What are the models of using multi modal data in knowledge graphs? 2.     How do you populate knowledge graphs with multi-modal data & relationships? 3.     What is the nature of the data (especially embeddings) that are maintained in these multi-modal knowledge graphs? It is the last question, on which a fair amount of new research is being done and is critical for defining successful hybrid search approaches when Multimodal LLM-based Graph-RAGs are built. The intent of the blog is to point to key shifting trends in generating Knowledge Graph embeddings for multi modal data, using the backdrop of 2 recent seminal papers. For more details on this line of thinking, see https://lnkd.in/gfPmzBgv Note:: for some early efforts at building multi modal KG RAG pipelines, using LlamaIndex and Neo4j – you can check out - https://lnkd.in/g9xnCSTu

I know firsthand how frustrating it can be to spend hours (if not days) debugging and fine-tuning machine learning models, only to resort to parsing the dataset and find the issue was bad training data. That's why I'm excited to share a tool I’ve developed to help ML engineers easily identify and exclude bad data from image classification datasets, using embedding space and dimensionality reduction to make this process faster and more precise. Check out my latest Medium article where I dive into how this tool works and how it can streamline your data cleaning process. If your team is dealing with similar challenges, or you’re looking for custom AI solutions, I'd love to connect. At gud_data, we specialize in tackling tough machine learning problems and building scalable solutions that deliver results. Let’s chat about how we can help your business. #machinelearning #AI #datacleaning #imageclassification #datascience #AItools #dimensionalityreduction

Built this interesting sigmoid approximation using the Taylor Series expansion logic I described in my previous post. Used a piece wise definition with different coefficients to achieve this fit. This translates to hardware quite well with a small error margin. #ai #hardware #vlsi

Save time by automating the extraction, classification, and detection of information from data such as imagery, video, point clouds, and text. https://ow.ly/3tTN30sAa5u #GeoAI

Exploring the strengths of ORB, SIFT, and FREAK for image alignment. Dive into my latest Medium post for insights into these algorithms and how they can improve your workflows. #ImageProcessing #ComputerVision #DocumentProcessing #MachineLearning #DataScience #AI #AlgorithmAnalysis #TechInnovation

I-JEPA: The first AI model based on Yann LeCun’s vision for more human-like AI Celebrating 10 years of FAIR: A decade of advancing the state-of-the-art through open research Turing Award presented to Yann LeCun, Geoffrey Hinton, and Yoshua Bengio Today, we’re publicly releasing the Video Joint Embedding Predictive Architecture (V-JEPA) model, a crucial step in advancing machine intelligence with a more grounded understanding of the world. This early example of a physical world model excels at detecting and understanding highly detailed interactions between objects. In the spirit of responsible open science, we’re releasing this model under a Creative Commons NonCommercial license for researchers to further explore." https://lnkd.in/d8tqWx7d.

https://lnkd.in/eRadUFsp Diffusion Models: Midjourney, Dall-E Reverse Time to Generate Images from Prompts

Meta AI recently open-sourced an impressive new video understanding model called V-JEPA (Video Joint Embedding Predictive Architecture). This model represents a major step toward advanced AI that can learn visual concepts more like humans do. V-JEPA was trained using a self-supervised approach on unlabeled videos, allowing it to predict missing parts of a scene based on context. It accomplishes this through an efficient "frozen training" methodology. The model is great at spotting detailed interactions between objects in videos. For example, it can tell if someone pretended or actually picked up a pen. https://lnkd.in/gfsje7i5 #ArtificialIntelligence #MachineLearning #ComputerVision