Robust Intelligence enables enterprises to secure their AI transformation with an automated solution to protect against security and safety threats. Our platform includes an engine for detecting and assessing model vulnerabilities, as well as recommending and enforcing the necessary guardrails to mitigate threats to AI applications in production. This enables companies to meet AI safety and security standards with a single integration, automatically working in the background to protect applications from development to production. Robust Intelligence is backed by Sequoia Capital and Tiger Global, and trusted by leading companies including ADP, JPMorgan Chase, Expedia, Deloitte, Cisco, and the U.S. Department of Defense to unblock the enterprise AI mission.
External link for Robust Intelligence
The Robust Intelligence platform automates testing for security and safety vulnerabilities of AI models in development and their protection in production. The platform includes an engine for detecting and assessing model vulnerabilities as well as the necessary guardrails to deploy safely in production. This consists of two complementary components, which can be used independently but are best when paired together: AI Validation detects and assesses model vulnerabilities to various attack techniques and safety concerns through automated testing and provides the recommended guardrails required to deploy safely in production. AI Protection secures applications against attacks and undesired responses in real time with guardrails that are tailored to the specific vulnerabilities identified during model assessment. It’s simple to get started with our API-based service. Just point at a model endpoint to initiate the assessment and generate specific guardrails custom-fit to your model.
555 19th St
San Francisco, California 94107, US
Robust Intelligence reposted this
CEO at Robust Intelligence | ex Professor of Computer Science at Harvard University | ex Google AI
When not busy protecting the world from AI risk, the good people of Robust Intelligence show off their running skills at the JPMorganChase corporate Challenge!
Robust Intelligence reposted this
Red Teamers and Model/Application owners. Check out this AI Application Red Teaming event...finalists get to show their stuff in a red team / blue team exercise alongside CAMLIS!
US Science Envoy, Artificial Intelligence | CEO, Humane Intelligence | Investor | Board Member | Startup founder |TIME 100 AI | ex- Twitter, ex- Accenture
HumaneIntelligence is excited to announce an upcoming nationwide AI red-teaming exercise supported by the National Institute of Standards and Technology (NIST). We are recruiting: 😎 Individuals interested in red teaming models online OR in-person 🤖 Model developers building generative AI office productivity software, including coding assistants, text and image generators, research tools, and more. 🏁 Our goal is to demonstrate capabilities to rigorously test and evaluate the robustness, security, and ethical implications of cutting-edge AI systems through adversarial testing and analysis. This exercise is crucial for helping to ensure the resilience and trustworthiness of AI technologies. The online event is a red-teaming pilot for NIST's newly-announced ARIA GenAI evaluation program (link in comments) This event will demonstrate: 📝 A test of the potential positive and negative uses of AI models, as well as a method of leveraging positive use cases to mitigate negative. 📝 The use of NIST AI 600-1 to explore GAI risks and suggested actions as an approach for establishing GAI safety and security controls. Why Participate? ☑ Contribute to the advancement of secure and ethical AI. ☑ Network with leading experts in AI and cybersecurity, including in U.S. government agencies. ☑ Gain insights into cutting-edge AI vulnerabilities and defenses. Participants in the qualifying red teaming event may be invited to compete in an all-expenses paid event alongside CAMLIS, scheduled for October 24-25, 2024 in Arlington, VA. More details in comments. Sign up here:
The JAM (Jailbreak Against Moderation) method was introduced in a recent research paper as a method to bypass moderation guardrails in LLMs using cipher characters to reduce harm scores. In experiments on four LLMs — GPT-3.5, GPT-4, Gemini, and Llama-3 — JAM outperforms baseline methods, achieving jailbreak success rates about 19.88 times higher and filtered-out rates about six times lower. Learn more about this and other AI security threats in our most recent monthly roundup: https://lnkd.in/g-6BhVqi #AIsecurity #LLMsecurity #cybersecurity #threatintel #LLMjailbreak
⚠️ Within hours of OpenAI's release of Structured Outputs, our AI security researchers identified a simple yet concerning exploit that bypasses the model's safety measures, including its refusal capabilities. We found that by defining a structure with specific constraints, we could force the model to generate content in a way that bypasses its safety checks. We reached out to the OpenAI team to inform them about this exploit and suggested countermeasures. This jailbreak is particularly significant for 3 reasons: 1️⃣ Simplicity: The method is remarkably straightforward, requiring only a carefully defined data structure. 2️⃣ Exploit of Safety Feature: The jailbreak takes advantage of a feature specifically designed to enhance safety, highlighting the complexity of AI security. 3️⃣ Dramatic Increase in Attack Success Rate: Our tests show a 4.25x increase in ASR compared to the baseline, demonstrating the potency of this exploit. This relatively simple jailbreak underscores the importance of third-party red teaming of AI models, as well as the need for model-agnostic guardrails updated with the latest threat intelligence. To learn more about our bleeding-edge AI security research and end-to-end AI security platform, check out our website. For in-depth analysis of our OpenAI Structured Outputs exploit, see our blog: https://lnkd.in/gHDJtNNk #AIsafety #AIrisk #AIsecurity #LLMsecurity #genAI #genereativeAI #redteaming
🎥 Our CTO Hyrum Anderson was interviewed by Sarah Young, Sr. Cloud Security Advocate at Microsoft, for Copilot L33T Sp34k - a new webinar series for security professionals focused on generative AI. Listen as Hyrum and Sarah discuss a variety of topics, including the evolution of adversarial machine learning, security concerns specific to AI, and how you can get ahead of these threats. 💬 Hyrum on the evolution of adversarial machine learning: "What’s changed most is how we think about AI security as not just the model-centric view, but also when you start building applications around AI. What are the other elements of security that you have to think about? Because often times, the security vulnerabilities are in the cracks between system components, and that can’t be more true than in modern AI applications." 🖥️ Watch the full interview here: https://lnkd.in/g2_N98Rx #AIsecurity #LLMsecurity #AIsafety #AIrisk #generativeAI #genAI #machinelearning #redteaming
A “ChatBug” is a common vulnerability in LLMs that arises from the use of chat templates during instruction tuning. While these templates are effective for enhancing LLM performance, they introduce a security weakness that can be easily exploited. In our latest blog, we explore three examples of chat template exploits: the format mismatch attack, the message overflow attack, and the Improved Few-Shot Jailbreak (I-FSJ). Learn more about this and other AI security threats in our most recent monthly roundup: https://lnkd.in/g-6BhVqi #AIsecurity #LLMsecurity #cybersecurity #threatintel #LLMjailbreak
Robust Intelligence reposted this
Sr. Manager, Enhanced Services at F5 | Strategic Leader developing innovative & effective support solutions
As #AI reshapes the digital landscape, security is no longer just about keeping up—it's about staying ahead. That's why F5 has partnered with Robust Intelligence, an AI security pioneer, to pave the way for a secure AI-enabled future. Together, we're empowering organizations to confidently deploy AI applications, ensuring innovation, speed, and performance without compromising security. Discover how we're driving the future of AI security. Read our latest blog: http://ms.spr.ly/6047lU5LF
♠️ As we reflect on a wonderful week in Las Vegas, we'd like to thank everyone at Black Hat and DEF CON who met with us to discuss the state of AI application security. Highlights included our partnership announcement with F5, lightning talk on protecting #GenAI applications, AI security leaders dinner, and sponsorship of AI Village at DEF CON. Thank you to our valued customer and partners, as well as the incredible cybersecurity community, for such a rewarding week! If we missed you in Vegas, we hope you'll reach out to learn how we can protect your AI applications from safety and security threats: https://lnkd.in/g6_CuhZU #AIsecurity #LLMsecurity #AIrisk #guardrails #LLMs #generativeAI #blackhat #BHUSA #DEFCON
As AI applications become more critical to the enterprise and handle greater volumes of sensitive data, bad actors are increasingly motivated to target them. For organizations harnessing this transformative technology, AI security is an imperative. However, AI applications are fundamentally different from traditional applications, which makes existing tools and processes ineffective. In our latest white paper, we unpack these differences and explain how traditional application security concepts like open-source scanning, vulnerability testing, and data loss prevention apply to AI. Check out the preview below, and for the full paper visit https://lnkd.in/gvGiune9 #AIsecurity #AIsafety #LLMsecurity #AIredteaming #LLMguardrails
Robust Intelligence reposted this
Another favorite moment from hacker summer camp is this AIRT reunion. We miss our alum but are so proud of the work they are doing at Dreadnode and Robust Intelligence!!