DSC ADRIA’s Post

𝗥𝗲𝗳𝗹𝗲𝗰𝘁𝗶𝗼𝗻-𝟳𝟬𝗕 𝘂𝗻𝗱𝗲𝗿 𝗶𝗻𝘃𝗲𝘀𝘁𝗶𝗴𝗮𝘁𝗶𝗼𝗻 𝗳𝗿𝗼𝗺 𝘁𝗵𝗲 𝗰𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝘆! ⚙️ I've posted last week about Reflection-70B, a model with a crazy performance for its relatively small size beating the behemoths GPT-4o and Claude-3.5 Sonnet. But this whole thing is now under serious scrutiny. Already end of last week, many people pointed out that this Reflection tuning method was similar to fine-tuning a model to perform chain-of-thought (CoT). To me, this is not that much of an issue : if you can effectively fine-tune a model to perform better on all tasks without needing a hack like adding a CoT suffix to your prompt, it means your model did really become stronger. And I'm a big believer in adding more tokens for reflection anyway: for now LLMs don't use nearly enough. But 𝘀𝗲𝗿𝗶𝗼𝘂𝘀 𝗱𝗼𝘂𝗯𝘁𝘀 𝗼𝗻 𝘁𝗵𝗲 𝗺𝗼𝗱𝗲𝗹 𝗵𝗮𝘃𝗲 𝗲𝗺𝗲𝗿𝗴𝗲𝗱 𝘀𝗶𝗻𝗰𝗲 𝘁𝗵𝗶𝘀 𝗺𝗼𝗿𝗻𝗶𝗻𝗴: 🤔 First, company Artificial Analysis posted their independent evaluation of the model weights posted on HF. It turns out the model's performance is under Llama-3-70B, so far from the author's claims. ➡️ Matt Shumer provided an API access to the model for people to try out. But people have noticed strange behaviours + it's hard to really test a model through an API that can change any time! 👉 Others have shown since that the model weights on HF could be a LoRA fine-tune of Llama-3-70B ⏱️ Matt Shumer and team have said that they're investigating the model weights to re-upload better ones on HF: Let's wait for the results to draw a conclusion! Anyway, this advocates for even more transparency in LLM releases and rigorous evaluation! https://lnkd.in/eKC8fQDm

The #LLM landscape has changed a lot since Galileo launched the first Hallucination Index in November 2023, with larger, more powerful open and closed-sourced models being announced monthly. Since then, two things happened: the term #hallucinate became Dictionary.com’s Word of the Year, and Retrieval-Augmented-Generation (#RAG) has become one of the leading methods for building AI solutions. Galileo new index evaluates how well 22 of the leading models adhere to given context, helping #developers make informed decisions about balancing price and performance. The report highlights 4 trends: 1. #Anthropic outperforms OpenAl: during testing, Anthropic's latest Claude 3.5 Sonnet and Claude 3 Opus consistently scored close to perfect scores, beating out GPT-4o and GPT-3.5, especially in shorter context scenarios. 2. Larger is not always better: in certain cases, #smallermodels outperformed larger models. 3. Performance is not impacted by #context lenght: models perform particularly well with extended context lengths without losing #quality or #accuracy, reflecting how far model training and architecture has come. 4. Open source is closing the gap: while closed-source models still offer the best performance thanks to proprietary training data, open-source models like #Llama continue to improve in #hallucinationperformance without the cost barriers of their close-sourced counterparts. If you want to dive deep into the results of this research you can access the report and executive summary at this link: https://lnkd.in/ggJhF7cx

🚀 Excited to share that I have completed the "Knowledge Graphs for RAG" course organized by Neo4j and DeepLearning.AI! 🌐📊 This knowledge opens up new possibilities in the integration of knowledge graphs with Retrieval-Augmented Generation (RAG), enhancing the way we process and generate information in AI and ML. 💡 A big thanks to the organizers for an incredible learning experience!

I finish the "Building and Evaluating Advanced RAG" course provided by DeepLearning.AI LlamaIndex Jerry Liu TruEra Anupam Datta. I learn new retrieval concept (sentence-window and auto merge) which may help to increase RAG performance. Both of them are trying to help us improve our context quality. so we can provide better context before fit into prompt to LLM within RAG pipeline. TruEra[1] is a new evaluation tool which allow you to measure your performance within each steps of RAG. It looks like very cool. I definitely will explore more within their document!  - Context relevance: Is the retrieved context relevant to the query? - Answer Relevance: if the response relevant to the query? - Groundedness: Is the response supported by the context? [1] https://lnkd.in/gP5DFrfE https://lnkd.in/gJmnK8Jf #llamaindex #deeplearning.ai #truera #LLM #Generativeai #Genai #RAG #evaluation #LLMOps

We are thrilled to announce that we will be presenting our joint work, Sharpened Lazy Incremental Quasi-Newton Method at the 27th International Conference on Artificial Intelligence and Statistics (#AISTATS) in Valencia, Spain. This research is a collaborative effort with Aakash Lahoti, Ketan Rajawat, and Alec Koppel. Our paper introduces SLIQN, which is the first incremental Quasi-Newton algorithm that, a) Features an explicit superlinear rate of convergence b) Incurs minimal per-iteration cost of 𝑂(𝑑^2) c) Enjoys superior performance over other existing Quasi-Newton algorithms. For more details, you can access the full paper at https://lnkd.in/ghs9sMqR, and the code to reproduce our results here at https://lnkd.in/gZyHVFg8. If you're attending AISTATS, please drop by our poster during Poster Session 3, Auditorium 1 Foyer on May 4. We would be delighted to meet and discuss our work with you. #AISTATS #ML #Optimization #iitk #cmu #usc

𝗟𝗟𝗠 𝗛𝗮𝗹𝗹𝘂𝗰𝗶𝗻𝗮𝘁𝗶𝗼𝗻 𝗥𝗮𝗻𝗸𝗶𝗻𝗴𝘀 𝗮𝗿𝗲 𝗵𝗲𝗿𝗲! Galileo, which offers a platform for evaluating AI models, tested 22 models to see whether they hallucinated after retrieving information from documents of various lengths. The findings showed that 𝗺𝗲𝗱𝗶𝘂𝗺-𝗹𝗲𝗻𝗴𝘁𝗵 𝗱𝗼𝗰𝘂𝗺𝗲𝗻𝘁𝘀 (𝟱-𝟮𝟱𝗸 𝘁𝗼𝗸𝗲𝗻𝘀 𝗼𝗿 𝟰~𝟭𝟵𝗸 𝘄𝗼𝗿𝗱𝘀) 𝗮𝗿𝗲 𝘁𝗵𝗲 𝘀𝘄𝗲𝗲𝘁 𝘀𝗽𝗼𝘁 for minimizing false information, as models performed best in this context. The standout performer was 𝗖𝗹𝗮𝘂𝗱𝗲 𝟯.𝟱 𝗦𝗼𝗻𝗻𝗲𝘁, achieving impressive scores across various contexts. This highlights the importance of understanding context length when designing AI applications—𝗺𝗼𝗿𝗲 𝗰𝗼𝗻𝘁𝗲𝘅𝘁 𝗺𝗮𝘆 𝗹𝗲𝗮𝗱 𝘁𝗼 𝗯𝗲𝘁𝘁𝗲𝗿 𝗮𝗰𝗰𝘂𝗿𝗮𝗰𝘆 than we typically assume. As we continue to refine these models, reducing hallucinations could prove critical for applications where accuracy is paramount. Checkout the detailed rankings on:

2025 will be another fast-paced year for GenAI! To start, three new approaches to RAG are being tested: HtmlRAG, Multimodal RAG, and Agentic RAG. - HtmlRAG uses HTML directly, preserving structure and context through cleaning and pruning, enhancing retrieval accuracy for complex documents. - Multimodal RAG combines text and image data using separate or integrated vector databases, enabling retrieval from diverse sources like images and PDFs. - Agentic RAG employs an AI agent to reformulate queries, critique retrievals, and retry iteratively, improving accuracy of complex queries. Read the full analysis below. #generativeai https://buff.ly/4285gs6

🔥 Read our Highly Cited Paper 📚 Deep Convolutional Generative Adversarial Networks-Based Data Augmentation Method for Classifying Class-Imbalanced Defect Patterns in Wafer Bin Map 🔗 https://lnkd.in/gyJJisV4 👨🔬 by Sangwoo Park and Cheolwoo You 🏫 Myongji University #waferdefects #deeplearning #dataimbalance #dataaugmentation

Competencies Replaceable by Artificial Intelligence in the Tuning Project for Latin America

Google's PaliGemma model is making waves with its robust features and versatile applications. Here's a concise breakdown of what you need to know: Introduction to PaliGemma: The model is designed for a range of applications, offering significant capabilities in natural language processing. Foundational Papers: Insights are drawn from the PaLI-3 and SigLIP papers, which lay the groundwork for understanding PaliGemma's architecture and functionality. Hugging Face Blog: A detailed summary on PaliGemma's features and use cases is available, highlighting its potential in various scenarios. Pre-trained Checkpoints: PaliGemma comes with three pre-trained checkpoints, catering to different needs and facilitating ease of use. Model Sizes and Releases: Multiple sizes and releases of PaliGemma ensure flexibility and adaptability for diverse requirements. Hugging Face Spaces Demo: A practical demonstration of PaliGemma is accessible on Hugging Face Spaces, showcasing its real-world applications. ScreenAI Datasets: These datasets provide a robust foundation for training and fine-tuning PaliGemma. Practical Coding: Detailed instructions on using PaliGemma with the Transformers library and fine-tuning the model are provided, making it easier for developers to implement and customize. For those interested in diving deeper, resources such as Colab codes for inference and fine-tuning, as well as GitHub tutorials, are available for further exploration. #PaliGemma #AI #MachineLearning #NLP #TechInnovation #HuggingFace #GoogleAI ---------------------- Learn more here: https://lnkd.in/exTHXbb9