unstructured.io’s Post

Enterprise Search is one of the most effective productivity use cases of generative AI. However building and deploying an enterprise search system is extremely challenging! That's why I'm so excited about DanswerAI + unstructured.io integration which makes it easy to deploy a production grade enterprise search system within your organization on your own infrastructure.

When I found out about unstructured.io and the release of Pinecone Serverless, I knew I had to give them a go. And I have to say I'm quite impressed! If you're thinking about the same thing, check out a brief analysis on building a RAG model using these two tools: https://lnkd.in/eHBcAXk2

Docker published my blog post on containerizing llamafile! There have been many improvements since I worked with llamafile a few months ago, and it's a good time to revisit. Check out the post! https://lnkd.in/gWE-DKVD

If you do not know where to start learning AI, ML, and DL, do not worry; check my GitHub link. I frequently post all courses at no cost. GitHub Link: https://lnkd.in/gcXHcADh Python  Numpy Pandas  Matplotlib  Seaborn  SQL Power BI or Tableau  Statistics and Probability.  Machine Learning - All Alograthims Deep Learning GenAI (All Gen AI concepts)  GCP & AWS AI cloud services and many more topics..!   Happy Learning! Feel free to contact me if you have any questions.   #AI #ML #DL #GenAI

Looking to learn more about building AI applications with Postgres? Join us on May 22nd for a session with John McBride from OpenSauced and learn how StarSearch uses pgvector and GPT-4 to generate powerful insights from GitHub data. Can't make it live? Register and we'll send you the recording via email afterward. Timescale #AI #RAG #PostgreSQL #pgvector

Poor code and repository documentation often leads to hindered collaboration and wasted time. With the recent demonstration from OpenAI on how the new GPT-4o model could read and understand a full code so fast and interpret it accurately, I thought LLMs could be leveraged to improve future and existing code on source control management systems, leading to improved code quality and saved time. I tested this by automating the process of creating a repository on GitHub, linking it to a local repo and crafting a description and readme file that represent what the project is doing by analyzing its code. The outputted description and readme are more accurate than I expected. More on this project: https://lnkd.in/ed68M8Ve

I've had people ask me about embeddings. Quite simply, an embedding is a meaningful numerical representation of text. You can compare two embeddings to determine their similarity. Embeddings are used with large language models (LLMs) and retrieval augmented generation (RAG) as part of a generative AI workflow. In this example, I'm using Amazon Bedrock to create an embedding of text data (film descriptions from IMDB), which is then stored in OpenSearch Serverless as a vector database. Using the built-in features of a vector database, I'm using OpenSearch to find text that is similar to what I'm searching for. In this case, I'm looking for movies that are similar to a description I've provided. To get started, clone the repo, run the CloudFormation template to set up the infrastructure, then run the demo.ipynb file to step through the example.

Want to see the 𝗰𝗼𝗱𝗲 on 𝗵𝗼𝘄 to 𝗯𝘂𝗶𝗹𝗱 and 𝗱𝗲𝗽𝗹𝗼𝘆 a 𝗿𝗲𝗮𝗹-𝘁𝗶𝗺𝗲 𝗲𝗺𝗯𝗲𝗱𝗱𝗶𝗻𝗴 𝗽𝗶𝗽𝗲𝗹𝗶𝗻𝗲 to do 𝗥𝗔𝗚 with 𝘂𝗽-𝘁𝗼-𝗱𝗮𝘁𝗲 𝗱𝗮𝘁𝗮 for your 𝗟𝗟𝗠𝘀? Then, I am excited to tell you that I 𝘀𝘁𝗮𝗿𝘁𝗲𝗱 𝗿𝗲𝗹𝗲𝗮𝘀𝗶𝗻𝗴 an 𝟴-𝗹𝗲𝘀𝘀𝗼𝗻 𝗠𝗲𝗱𝗶𝘂𝗺 𝘀𝗲𝗿𝗶𝗲𝘀 on our 𝗛𝗮𝗻𝗱𝘀-𝗼𝗻 𝗟𝗟𝗠𝘀 course. 𝘐𝘯 𝘵𝘩𝘦𝘴𝘦 𝘸𝘳𝘪𝘵𝘵𝘦𝘯 𝘭𝘦𝘴𝘴𝘰𝘯𝘴: → I go in-depth into the theoretical aspects → explain all the system design choices → present the code & deployment process step-by-step . That being said... We just 𝗿𝗲𝗹𝗲𝗮𝘀𝗲𝗱 the 𝟮𝗻𝗱 and 𝟯𝗿𝗱 𝗹𝗲𝘀𝘀𝗼𝗻𝘀 on how to build a real-time feature pipeline that syncs the financial news data source and vector DB as follows: 1. listens to financial news 24/7 using a WebSocket 2. cleans, chunks, and embeds all the financial news using Bytewax, unstructured and sentence-transformers 3. loads everything to a Qdrant vector DB 4. deploy the feature pipeline to AWS as a Docker container 5. build a CI/CD pipeline using GitHub Actions → Now, when doing RAG, the vector DB is constantly updated with the latest financial news to retrieve the proper results. . ↳ Check out Lesson 2 for theoretical aspects and system design choices: 🔗 𝘞𝘩𝘺 𝘺𝘰𝘶 𝘮𝘶𝘴𝘵 𝘤𝘩𝘰𝘰𝘴𝘦 𝘴𝘵𝘳𝘦𝘢𝘮𝘪𝘯𝘨 𝘰𝘷𝘦𝘳 𝘣𝘢𝘵𝘤𝘩 𝘱𝘪𝘱𝘦𝘭𝘪𝘯𝘦𝘴 𝘸𝘩𝘦𝘯 𝘥𝘰𝘪𝘯𝘨 𝘙𝘈𝘎 𝘪𝘯 𝘓𝘓𝘔 𝘢𝘱𝘱𝘭𝘪𝘤𝘢𝘵𝘪𝘰𝘯𝘴: https://lnkd.in/dmyNqyV5 . ↳ Check out Lesson 3 to go through the code step-by-step: 🔗 𝘛𝘩𝘪𝘴 𝘪𝘴 𝘩𝘰𝘸 𝘺𝘰𝘶 𝘤𝘢𝘯 𝘣𝘶𝘪𝘭𝘥 & 𝘥𝘦𝘱𝘭𝘰𝘺 𝘢 𝘴𝘵𝘳𝘦𝘢𝘮𝘪𝘯𝘨 𝘱𝘪𝘱𝘦𝘭𝘪𝘯𝘦 𝘵𝘰 𝘱𝘰𝘱𝘶𝘭𝘢𝘵𝘦 𝘢 𝘷𝘦𝘤𝘵𝘰𝘳 𝘋𝘉 𝘧𝘰𝘳 𝘳𝘦𝘢𝘭-𝘵𝘪𝘮𝘦 𝘙𝘈𝘎: https://lnkd.in/dEgD3E4t #machinelearning #mlops #datascience . 💡 Follow me for daily lessons about ML engineering and MLOps.

Today's links look at a new Docker service for cloud-based container builds, how to build an app from scratch with AI-assisted tooling, and comparing RAG with fine-tuning.

A great article about ML model versioning! Thanks Gerben Oostra ! https://lnkd.in/dk9eHrgN In my previous work for production grade ML systems, we trended to a similar approach independently but it's great to see it expressed so concisely! Must read if you're working with dynamic ML system.