paradime.io’s Post

⚙️ 𝗖𝗿𝗲𝗮𝘁𝗲 𝗦𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲𝗱 𝗱𝗮𝘁𝗮𝘀𝗲𝘁 𝘂𝘀𝗶𝗻𝗴 𝘁𝗵𝗲 𝗜𝗻𝘀𝘁𝗿𝘂𝗰𝘁𝗼𝗿 ⚙️ Build a dataset for Named Entity Recognition (NER) using schema-based extraction. This helps in performing semantic searches based on the entities. Pydantic, the backbone of Instructor, enables high customization and utilizes return datatype hints for seamless schema validation. It seamlessly integrates with LanceDB and directly inserts data into tables. ✅ Code Implementation - https://lnkd.in/gmjepTcg 🌟 Checkout for related examples and tutorials https://lnkd.in/gWYgJD8z #instructor #pydantic #semanticsearch #vectordb

New features ✨ are here, including our new Anthropic integration! ➡️ Work with #Claude in prompt and data workflows ➡️ Organize prompt outputs with custom delimiters ➡️ Easily upload data in CSV format ➡️ Experiment with the limited release of batch actions to annotate data at scale. We're excited to see what you build in June! #JuneReleases #DataEngineering #PromptEngineering #LLMs #Anthropic

RAG is one of the biggest applications of #LLMs and #GenerativeAI. In a very crude analogy, think of #RAG as an equivalent to SQL in structured data setups. In a structured setup, you had tables and you would write SQL queries to pull out the data / information. RAG does the same things on documents. Given huge datasets, RAG techniques allow you to generate answers from these documents / data. I believe RAG techniques are going to be incredibly useful for all developers and AI professionals in future. To experience RAG, try out NotebookLM from Google. One of the best products to experience power of RAG systems. If you want to start learning RAG, here is a free course for the same on Analytics Vidhya - https://lnkd.in/gXakBrnC The course uses LlamaIndex to quickly build your first RAG system - go ahead and play!

Did you ever wonder if you could select only certain people from your life and filter the rest of them with AI, well you can't but you can use AI to select data with SQL. Yes !! With LlamaIndex libraries we can convert text to sql queries and create a RAG pipeline with a relational database . In this video we will use Llama3 model with Groq to do Text to SQL in just 15 mins. Jerry Liu Thank you for this amazing feature. Colab notebook - https://lnkd.in/ewEtszWb #llamaindex #llama3 #rag #groq #sql #texttosql #relationaldatabase #rds https://lnkd.in/e4yZS7sa

For some parts of our AI stack, we have also been converting over to using Knowledge Graphs generated by LLMs (and other sources) rather than using LLMs in line. It has taken a lot of work so far, but has shown far lower latency, lower cost and more predictable engineering progress. Compared to techniques like RAG, It takes a number of other NLP tools to make this approach work for the mid/long tail of cases we need to cover in a real production grade system. Beyond a demo, we needed to use natural language and human-expert-readable script output, log output, alert snippets, metrics, etc. We’re gluing all these NLP tools together. OK for now given the awesome UX latency improvements we get from this approach. This reminds me of the early transition from Web to mobile+web where there were fast and great prototypes but production-grade apps had many fits and starts to deal with screen size diversity, OS diversity, caching / network speed diversity, etc.

Tired of crazy data extraction costs? JsonLLM saves you 50 bucks! Ever felt ripped off by GPT-4's data extraction prices? There's a new sheriff in town: JsonLLM! JsonLLM: Get the data you need, WAY cheaper! This awesome tool lets you easily build programs (APIs) that suck the info you want right out of PDFs and documents. Just tell it what format you need the data in, and boom! Structured data, delivered fast. Best part? It costs 50 times LESS than GPT-4! That's right, you can finally ditch the crazy bills and get the same results for way cheaper. Here's why JsonLLM rocks: ✅Super Easy: No coding skills needed, anyone can use it! ✅Lightning Fast: Extracts data from your documents in a flash. ⚡️ ✅Saves you Money: Stop wasting cash on expensive tools! JsonLLM is also: ✅Accurate: Gets you the precise data you need, every time. ✅Flexible: Works with PDFs, text documents, and more! Ready to ditch the data extraction robbery?  Try this:- https://lnkd.in/gZfuWjVk P.S. Support us on here:- https://lnkd.in/gg79_PTF #JsonLLM #DataExtractionMadeEasy #SaveMoney #BoostEfficiency

SheetChat is a tool I have created to make data science easier. It is like a conversation, and you get information from data. #datascience #statistics #openai #chatGPT #dataanalysis https://lnkd.in/dqaNyX3Y

🚀 Exciting Updates to TidyDensity! 🚀 I'm thrilled to announce some fantastic new features and improvements in the latest update of the TidyDensity package! 📈 What's New? - Negative Binomial Distribution: Calculate AIC with `util_negative_binomial_aic()`. - Zero-Truncated Distributions: Parameter estimation, AIC calculation, and summary tables for Negative Binomial, Poisson, Geometric, and Binomial distributions. - F Distribution: New functions for parameter estimation and AIC calculation. - Pareto, Paralogistic, and Inverse Distributions: Enhanced support with new parameter estimation and summary table functions. - Generalized Distributions: Expanded capabilities for Gamma and Pareto distributions. Minor Improvements - Optimized Parameter Estimation: `util_negative_binomial_param_estimate()` now uses `optim()` for better accuracy. - Improved Data Handling: `quantile_normalize()` now includes column names for clearer data presentation. These updates significantly enhance the analytical capabilities of TidyDensity, providing more robust tools for distribution analysis. Whether you're dealing with standard or specialized distributions, these new features will streamline your workflow and improve your results. Don't miss out on these powerful new tools—update your TidyDensity package today and take your data analysis to the next level! 📰 News: https://lnkd.in/ea7mX_Xg Happy coding! 💻 #Rstats #DataScience #TidyDensity #Analytics #DataAnalysis #RProgramming #Update #Coding #Statistics #StatisticalDistributions Anna Anisin Hadi Heidari Mehdi Gorji nia Khalili Margarita S. David Langer David Kun Jake Waddle Jake Riley Joachim Schork Darko Medin Olga Palkovskaya Veerle van Leemput 🔥 Matt Dancho 🔥

Discover the power of Seaborn for your data visualizations! 🤠 With beautiful default styles🤩, built-in statistical functions, seamless Pandas integration, concise syntax, and strong community support, Seaborn makes creating stunning plots a breeze. Say goodbye to the struggles of matplotlib 🫸and hello to effortless visualization with Seaborn👋.✌️ #Seaborn, @datacamp

Curious about converting any text data into a Knowledge Graph? 👉 Get the answer here: https://lnkd.in/gmMqki6R Throughout the week, I've been exploring the most effective methods to achieve this. Starting with Leann Chen’s insightful tutorial on leveraging spacy-llm to extract entities and relationships, and also inspired by Milena Trajanoska’s innovative approach using schema.org (https://lnkd.in/gNQj-zrT). In the end, I discovered that utilizing LLAMA3 & Groq for converting text into a knowledge graph is the most efficient method. Best part? It's entirely free, and I'm genuinely impressed with the results. Feel free to experiment with my code for your own data and use cases. Last but not least, after you’ve watched my tutorial, you might be wondering if there's a solution that wraps it all up neatly. Enter LLMGraphTransformer, a remarkable creation by Tomaz Bratanic (https://lnkd.in/g-Kzngsb). While incredibly useful, it's currently limited to OpenAI and Mistral models. If you've had experience constructing a knowledge graph from unstructured text, I'd love to hear about it! Share your insights in the comments below.