Patronus AI’s Post

Patronus AI has a new open-weights, SOTA eval model for hallucinations, probably the most common kind of eval for RAG applications! If you've found LLM-as-a-judge inadequate for your RAG eval, may be worth trying this. Lynx beats prompted frontier models like 4o and sonnet 3.5. It's super encouraging to see: (1) Companies like Patronus pushing open source forward not just on the fancy demos and capabilities side, but also on core pieces of infra like eval. (2) Results like this that demonstrate finetuned open models being practically useful and valuable over large frontier models in the right hands.

The first step towards removing hallucinations from language models is detecting them! Patronus AI just released a new state of art model for detecting a class of hallucinations that outperforms GPT4!

Evaluating hallucinations in LLM outputs has long been a challenging yet crucial task especially for regulated industries like healthcare and finance. 🏥💰 PatronusAI introduces a new state-of-the-art (SOTA) eval model for hallucinations - open source, weights, data! Lynx outperforms leading models like 4o and Sonnet 3.5. Congrats to the team.

Today, we are excited to release Lynx v1.1, a smaller, state of the art RAG hallucination detection model 🚀 Even though companies use RAG to reduce hallucinations, LLMs can still produce unsupported or contradictory information. Since we released Lynx v1.0 a few weeks ago, thousands of developers have used it in all kinds of real world applications. Lynx v1.1 is the best performing RAG hallucination detection model of its size, enabling real-time hallucination detection in AI applications ✨ - Beats Claude-3.5-Sonnet on HaluBench by 3% - Outperforms GPT-4o on medical questions and answers by 6.8%  - 1.4% higher accuracy than Lynx v1.0 on HaluBench - Outperforms all open source models on LLM-as-judge tasks - Open source, open weights and open data Use Lynx 1.1 with any of our Day 1 integration partners like NVIDIA, MongoDB, and Nomic AI 🚀 Check out the Hugging Face Spaces demo: https://lnkd.in/gcjVmeNG Download Lynx v1.1 on Hugging Face:  https://lnkd.in/gvMbRddM Download Lynx v1.1 (Quantized) on Hugging Face: https://lnkd.in/g3xAFJZh Read the arXiv paper: https://lnkd.in/eznVjrWA Read the blog: https://lnkd.in/eYaP5Zpe

A great day with exciting insights and many good conversations lies behind us: This week, the #AWSAutomotiveSymposium took place in Munich. We presented our scenario generation solution, which uses the generative AI technologies of AWS AI. 😊🚗 The new scenario generation feature is based on a large language model (LLM) available on Amazon Bedrock, and a retrieval augmented generation (RAG) architecture that uses scenario data provided by dSPACE. This enables users to easily create a large number of complex and realistic scenarios by simply entering a textual scenario description. We also showed our customers how to scale their testing efforts with SIMPHERA and seamlessly test their HPC applications with the combination of VEOS and Graviton. 💻🔧 #scenarioGeneration #dSPACE #SIMPHERA #VEOS

Improve your #generativeAI model performance with NVIDIA NeMo Curator. Now easily installable via PyPI ➡️ https://nvda.ws/3wTv6Tz ⚡ This GPU-accelerated data curation library helps you prepare large-scale, high-quality datasets for pretraining and customization. ✨Features include data download and text extraction, language identification, quality filtering, deduplication, and more.

It delves into the process of constructing a retrieval-augmented generation (RAG) pipeline using NVIDIA AI Langchain and AI endpoints . When building your own RAG pipeline, it’s important to correctly split the vector store documents into chunks by optimizing the chunk size for your specific content and selecting a LLM with suitable context length. For some cases, complex chains of multiple LLMs may be required.

This is big... More and more Enterprises are looking to train foundational models, but are limited by GPU resources from the hyperscalers. With our acquisition of MosaicML and our NVIDIA partnership, Databricks has GPUs readily available TODAY. Allen Institute for AI (AI2) was able to train OLMo in under 2 months with these new resources. All their data, in one secure environment. https://lnkd.in/gMN3Cywa

NeuroMesh wants to build the open, accessible middle layer for AI developers to build atop of and access GPU compute Watch our full conversation with NeuroMesh Foundation CTO, Yujian Hong, to learn more about distributed machine learning 🔗 https://lnkd.in/eKjwrns9

🔍 Just attended an enlightening panel at Nvidia's GTC24, featuring insights from Kaggle Grandmasters and NVIDIA's leading data scientists, including David Austin, Jiwei Liu, Ph.D., Kazuki Onodera, @Chris Deotte, and Laura Leal-Taixé. The panelists discussed the evolving landscapes of challenges involving AI and the frontiers of Large Language Models (LLMs), providing various perspectives on how these technologies are shaping the future of data science and machine learning. A notable point for me was the increasing importance of synthetic data generation in Kaggle competitions. Moreover, the panelists also pointed out the growing usefulness of Retrieval-Augmented Generation (RAG) technology, which enhances LLMs by integrating external knowledge sources. The discussion also painted Kaggle competitions as not only a platform for improving machine learning skills but also as a significant career stepping stone, offering exposure to real-world problems and the latest technologies, such as llama 2 and Mixtral (open-source LLMs). The emphasis on RAG and LLMs highlights their potential in addressing complex problems. This session has inspired me to explore these technologies further and consider participating in Kaggle competitions 🚀 #NVIDIA #GTC24 #AI #MachineLearning #DataScience #Kaggle #LLM #RAGTechnology #DataScience