Kili Technology

🔍 Ever wondered how LLMs are evaluated for safety against adversarial attacks? We’re pulling back the curtain on our recent red teaming study. 🧐 In our latest blog, we explore: 🔹 The adversarial prompts that challenge LLMs in both English and French. 🔹 Surprising insights into language disparities in model vulnerability. 🔹 The red teaming techniques used to uncover weaknesses across multiple harm categories. Our research is pushing the boundaries of multilingual AI safety. Dive into the behind-the-scenes details here: https://lnkd.in/dy9gZxSH #AI #LLM #redteaming

🔍 It’s finally here! We just released our comprehensive study on LLM vulnerabilities, testing different models to find the most effective adversarial prompting techniques and to understand better how models behave when confronted with adversarial prompts in other languages. Key findings that caught our attention: - LLM safety measures can weaken during longer conversations - English prompts are significantly more effective at bypassing safety than French ones - Pattern-based attacks proved surprisingly successful, with up to 92.86% effectiveness This is just phase one - we're expanding to include more languages, models, and a broader dataset of prompts. Interested in AI safety? Check out our full report! LLM Benchmark Report: https://lnkd.in/d_BFaQjt #AISafety #RedTeaming #LLM #PromptEngineering

🤔 What’s next after GPT4o? Llama 3.2? And every high-performing frontier AI existing today? Well. We’ll need more data to find out. 👏 Edouard D., Kili Technology’s Co-founder and CTO, will take the stage at the ai-Pulse event by Scaleway and share how better, more specialized data can push AI forward, making it more effective in finance, law, and more. 😍 ✅ This is a must-go if you want to learn how data and expert knowledge are shaping the next big breakthroughs in AI. Where? STATION F of course! When? November 7 😉 Limited places available. Be fast! #AIPulse #Data #AI

How do we measure the effectiveness of red teaming? 🧐👇 😬 Well, it's not easy. But researches have suggested the following concepts: 1. Attack Success Rate (ASR) The percentage of adversarial attempts that successfully elicit undesired behavior from the model. 🚨 This rate measures the percentage of times that an adversarial input—crafted specifically to trick or mislead a model—successfully induces the model to make errors or behave in unintended ways. 2. Diversity Measures how varied the successful attacks are. 🚨 A diverse set of successful attacks = more comprehensive evaluation of the model's vulnerabilities. 3. Transferability The extent to which attacks effective against one model also work on other models. 🚨 High transferability = more vulnerabilities in LLM design or training. 4. Human Reliability For adversarial prompts, this metric assesses how natural or interpretable the prompts are to humans. 🚨 More readable prompts = more sophisticated and potentially dangerous attacks. 5. Specificity Measures how targeted the attack is. 🚨 Some attacks might cause general model failures, while others might elicit very specific undesired behaviors. 6. Robustness Evaluates how consistent the attack's success is across different scenarios. 🙂↕️ Understanding these basic concepts provides a foundation for more advanced red teaming techniques and strategies. 💪 Everything you need to know is in our latest Red Teaming Blog Article 😉 #AI #LLM #RLHF

🧐 Q: ’’For a regex generation system to extract structured data from a pdf document, can we consider these methods to improve LLM-generated regexes?‘’ 😉 A: Yes, there's a very interesting line of research on synthetic feedback and compiler feedback. Essentially, you can add a reward based on whether the regex is syntactically valid whether you have the data, and whether it compiles or not. ✅ Our latest webinar with Paul G., Andrew Jardine, and Daniel Hesslow, was clarifying everything about surpassing frontier LLM performance on your tasks using RLHF. We answered questions live and we’re still actively answering on LinkedIn! ✨ Don’t hesitate to ask your questions on our posts 👇 #webinar #AI #RLHF

Key insights from OpenAI’s red teaming of the o1 model family 💡 🔹 Uncovering Vulnerabilities: Red teaming revealed how the o1 models, like o1-preview and o1-mini, resist adversarial prompts better than earlier versions, especially in tough jailbreak tests. 🔹 Diverse Risks Explored: Experts from various domains evaluated the models, leading to improvements in handling threats like harmful content generation. 🔹 Human & Automated Testing: Both human experts and automated tools tested the models, highlighting increased robustness but revealing a few gaps. 🔹 External Evaluators: Collaboration with external teams improved objectivity and transparency, bringing fresh perspectives to AI safety. 🔹 Continuous Improvement: Red teaming is ongoing, helping the o1 models evolve and strengthen against emerging threats. 🔥 Red teaming remains a key driver in making AI safer and more reliable for real-world applications. 🙂↕️ Read our Red Teaming blog to be guided 👇 #AI #RedTeaming #OpenAI

Just in case Walter White wants an AI-assistant 🙂↕️ Question from our webinar: 🤔 ‘’Is supervised fine-tuning better for domain knowledge like chemistry?’’ 🧪 Our speaker’s answer: 👉 For domain-specific knowledge like chemistry: ✨ Use continued pre-training to embed new factual information ✨ Apply fine-tuning (SFT or RLHF) to optimize how the model applies this knowledge and improve task-specific performance This two-step approach ensures both accurate knowledge acquisition and effective application in domain-specific contexts. 😉 We hope this helps! 😌 We have a webinar recap that summarises everything we talked about. 👇 #AI #LLM #RLHF

ChatGPT was once manipulated to share personal information through repeating the word "poem" 😬 Well, this can happen if your model isn’t trained with the right red-teaming practices. 🤔 🔥 One of the practices is Threat Models (in the context of LLM red teaming is a structured approach to identifying potential security risks and vulnerabilities.) It helps in understanding how an adversary might attempt to exploit the system. 🙂↕️ For LLMs, common threat models include: 🔹 Jailbreaking: Attempts to bypass the model's safety constraints to generate prohibited content. 🔹 Data Extraction: Efforts to make the model reveal sensitive information from its training data. 🔹 Prompt Injection: Manipulating the model's behavior by inserting malicious instructions into the input. Understanding these threat models allows engineers to design more comprehensive testing scenarios and develop more robust defenses. ✅ We drafted a full article to understand and apply the best practices to fine-tune your LLM and avoid dangerous situations. 😉 #article #AI #redteaming

How could you miss our webinar? 😩 ✨ Making your LLM perform better is great. But you need to train it on more specific use cases or tasks through fine-tuning. Why is it important? Addressing limitations: Real-world applications often require more focused and tailored responses. Adapting to human-like interactions: Fine-tuning helps steer models to behave like helpful assistants, engaging with human-like interactions. Improving output quality: Fine-tuning can significantly enhance model outputs' style, format, and relevance for specific applications. In our webinar recap, we discussed the best methods to adapt your language model. You know what to do 😉 #webinar #AI #RLHF

🎬 Webinar Replay: Quality Metrics explained in less than 4min 👀 ✅ Now that you have selected your framework to perform RLHF on your LLM, you need to measure quality metrics to keep track of the fine-tune. 📉📈 A few best practices to adopt: 🔸 QA Score 🔸 Metrics Trackers (Behavioural distributions, Bias, and Data diversity) 🔸 Agreement (Preference ranking) 😉 Everything is explained by Paul G. in this video 👇 🔥 Need to deep-dive the subject a bit more? Our full webinar recap is available here: https://lnkd.in/efbTz67Q #webinar #AI #RLHF