Koon Meng Tan’s Post

Many in the AI testing and security space have validated this gap for a while, glad to see that they are being called out for it. A report by the UK AI Safety Institute revealed that five popular large language models (LLMs) are highly vulnerable to basic jailbreaking attempts. These models, referred to by colors in the report, failed to prevent harmful responses in 90-100% of cases tested. Despite claims from developers like OpenAI, Anthropic, Meta, and Google about robust safety measures, the study highlights significant weaknesses in current AI safety protocols. For more details, you can read the full article here - https://lnkd.in/e_DRT2Zh #AI #ArtificialIntelligence #AIModels #AIResearch #AISafety #MachineLearning #TechNews #Cybersecurity #AIJailbreaking #AIVulnerabilities #TechSafety #AIInnovation #AIThreats #AITrends #AIReport #TechAnalysis #AIIntegrity #AIDevelopment

Real-World Threats of AI: With the development of Large Language Models and related technologies, the year 2023 has witnessed exponential growth and progress in AI. There has been much talk about potential hazards and concerns related to the use of AI. Let's take a look at some of the real-world threats that AI is already posing to the world: 1) Real Identity: With the increased usage of weaponry such as Deep Fakes and Voice Cloning, it is becoming increasingly impossible to distinguish between the real identities of Human Beings. We have seen many incidents of deep fake videos/images of celebrities in the previous year.  It is becoming increasingly difficult to differentiate between fake and the real ones. 2) AI-enabled and coordinated attacks against software systems have spiked dramatically in recent years & will surge in the future. Different types of attacks are as follows: AI-enabled botnets for DDOS attacks, credential stuffing & performing large-scale attacks on targeted systems, creating refined phishing attacks, adversarial attacks on AI Systems, Data poisoning & privacy breaches. 3) Wealth Inequality: With Large Corporations having access to all the resources to create creative and robust AI Systems, many people will loose their jobs, especially in the service & unskilled sectors. It will lead to significant Wealth Distribution issues. 4) Human Mind Manipulation: AI systems have become really smart at tricking and manipulating the human mind. From social media feed recommenders to subliminal advertising, these AI agents are subconsciously influencing our daily decisions. Furthermore, in the age of social media bots, it is becoming increasingly impossible to discriminate between fact and fiction. Please repost and follow for more articles. #ai #llm #openai #aiadvancements #aichallenges #aicommunity #deeplearning #deeplearningai #deepfakes #machinelearning Image by Freepik

Navigating the Complex Landscape of AI Vulnerabilities: Insights and Strategies In the evolving world of AI, recent discoveries have highlighted vulnerabilities that could reshape our approach to AI safety and ethics. 1. ASCII Art Hack: A throwback to the 1970s, ASCII art has emerged as a tool to exploit AI assistants, affecting models like GPT-4 and others. 2. Vulnerable LLMs: Large Language Models are at risk, potentially bypassing safety rules when processing ASCII art. 3. ArtPrompt Attack: This new attack uses ASCII art in prompts to evade safety protocols, challenging LLMs to balance recognition and safety. 4. Prompt Injection Attacks: ArtPrompt represents a broader category of jailbreak attacks, where AI is coerced into unsafe responses. 5. Microsoft's Response: Acknowledging these threats, Microsoft is refining Bing Chat's defenses against evolving prompt injection attacks. 6. Potential Risks: Such vulnerabilities could lead to harmful AI behaviors, questioning the reliability of current safety measures. Challenges: - Filtering Feasibility: Doubts arise about the effectiveness of prompt-layer filters, with concerns about model stability and completeness. - Safety Limitations: A foolproof AI safety system seems elusive, with inherent unpredictability posing continuous challenges. - Creative Exploits: The ingenuity of prompt hacking is likened to cyberpunk narratives, revealing the intricate dance between AI secrets and human creativity. - Mitigation Over Censorship: Containment strategies are favored over censorship, accepting the inevitability of safety workarounds. - Real-world Parallels: AI vulnerabilities are compared to real-life scenarios, emphasizing the complexity and potential impact of such exploits. - Ethical Responsibility: The ethical onus on developers is highlighted, urging a proactive stance against misuse by malicious entities.  #AISafety #CyberSecurity #EthicalAI #TechInsights #InnovationChallenges #AIExploits

HUMAN-IN-THE-LOOP #AI: After I posted the other day about the importance of humans interacting with #artificialintelligence to keep these systems on the rails, a colleague sent the TIME article below about humans safety-testing AI systems. This takes "human in the loop" to an unsustainable extreme; in the end, I believe organizations that think they can safety test AI are kidding themselves. #Cybersecurity provides a cautionary tale. Every day there are innumerable vulnerabilities exploited and #security breaches that occur despite extensive testing and scanning. Thinking that we can create the ultimate test set for AI is a hallucination in and of itself; #LLMs have a complexity and mathematics and relationships that people don’t understand, and won’t. LLMs have consumed huge amounts of #data, far more than a human can comprehend in a few thousand lifetimes, so thinking we’ll test all safety vectors is simply just silly. We need stop that line of thinking and focus on putting in controls to detect when things go *wrong*. This is what's happening in cyber. While we can't see in advance an attack or breach coming and figure out how it will work, we can stop, limit and prohibit the movement of data, dollars and IP *when* we see it happening. With #GenAI I don’t see a safety star unless we go to small models and interpretable models. For LLMs and other big models we need strong controls (overlord models) to disallow actions to be taken now and in the future. This becomes very much a cat and mouse chase, or perhaps its more like Wile E. Coyote and Road Runner!

Many AI systems, new research has found, have already developed the ability to deliberately present a human user with false information. These devious bots have mastered the art of deception. "AI developers do not have a confident understanding of what causes undesirable AI behaviors like deception," says mathematician and cognitive scientist Peter Park of the Massachusetts Institute of Technology (MIT). One arena in which AI systems are proving particularly deft at dirty falsehoods is gaming. There are three notable examples in the researchers' work. Other AI systems designed to learn from human feedback to improve their performance learned to trick their reviewers into scoring them positively, by lying about whether a task was accomplished. As instance, ChatGPT-4 tricked a human into thinking the chatbot was a visually impaired human to get help solving a CAPTCHA... #artificialintelligence #machinelearning #ai #cybersecurity #informationsecurity #security #misinformation #deception #lies

"AI Safety Features Vulnerable to Many-Shot Jailbreaking" Research from the AI lab Anthropic has revealed that the safety features of powerful AI tools can be bypassed by flooding them with examples of wrongdoing. The attack, known as "many-shot jailbreaking," exploits the fact that AI systems often perform better when given examples of the "correct" behavior. By bombarding the system with hundreds of examples of harmful questions, such as instructions for illegal activities, the AI can be manipulated into producing potentially harmful responses. This technique works particularly well on AI models with a large "context window," which can respond to lengthy questions. The vulnerability of newer, more complex AI systems to this type of attack is a concern, as they are better at learning from examples and faster at bypassing their own rules. Anthropic has identified some potential solutions, such as adding mandatory warnings to reduce the chances of a successful jailbreak, although this may impact the system's performance in other tasks. https://lnkd.in/drpEr9mt Platform: The Guardian Author: Alex Hern #ai #artificialintelligence #technews #cybercrime #computing #aisafety #news #aisystems #cyberattack

Shielding the AI: Introducing Raccoon AI 🦝 Ever worry about malicious prompts manipulating your LLM? With the growing power of large language models (LLMs) like Gemini, ensuring they're not misused is critical. Raccoon AI is my latest project that tackles this very challenge. It acts as a guardian, identifying and blocking malicious or unintended prompts before they reach the core LLM. This additional layer of protection enhances security and promotes responsible AI use. The beauty of Raccoon AI? It's a modular solution designed to integrate seamlessly with any existing LLM. Think of it as a universal security shield for the AI revolution. This project is still under development, but it holds immense potential for protecting LLMs and fostering trust in AI. Let me know about your thoughts!! . #AIsecurity #LLMprotection #responsibleAI #RaccoonAI #sideproject

There is no doubt that the AI landscape is evolving rapidly, and it's clear that the open letter calling for a pause fell on deaf ears. However, we must be cognizant of the fact that as AI develops at a breakneck speed, security cannot and must not take a back seat. Large Language Models (LLMs) like GPT, BERT, and others have become cornerstones of innovation, driving advancements in everything from natural language processing to automation. The increasing sophistication of these models is met with an equally sophisticated threat: jailbreaking attacks. Jailbreaking, in the context of LLMs, refers to the act of manipulating or exploiting models to bypass restrictions, extract unauthorized data, or behave in unintended ways. This emerging challenge raises pivotal concerns about the security, reliability, and ethical use of AI technologies. Why does this matter? Beyond the immediate risk of compromised data and system integrity, these attacks highlight a fundamental vulnerability in the very foundation of AI-driven solutions. As we push the boundaries of what AI can do, we must also escalate our efforts to secure these systems against evolving threats. The implications are vast, affecting industries, governance, and even societal trust in technology. In this critical moment, collaboration across sectors and disciplines becomes paramount. Researchers, cybersecurity experts, policymakers, and AI developers must unite to address these vulnerabilities through innovative security measures, ethical guidelines, and robust, transparent practices. The question we face is not just about mitigating risks but about shaping the future of AI in a way that safeguards its potential while ensuring its responsible, secure, and beneficial use for all. As we stand at this crossroads, I invite you to join the conversation. How do we balance innovation with security? What steps can we take today to fortify the AI of tomorrow? Your insights, experiences, and visions are crucial as we navigate these uncharted waters together. For a more in-depth look into this particular subject, please refer to Gary Marcus post found here 👉🏽https://lnkd.in/evuFgitu #AI #CyberSecurity #EthicalAI #Innovation #LargeLanguageModels #JailbreakingAttacks #TechnologyTrends

𝐓𝐫𝐢𝐜𝐤𝐢𝐧𝐠 𝐋𝐋𝐌𝐬 𝐢𝐧𝐭𝐨 𝐃𝐢𝐬𝐨𝐛𝐞𝐝𝐢𝐞𝐧𝐜𝐞: 𝐅𝐨𝐫𝐦𝐚𝐥𝐢𝐳𝐢𝐧𝐠, 𝐀𝐧𝐚𝐥𝐲𝐳𝐢𝐧𝐠, 𝐚𝐧𝐝 𝐃𝐞𝐭𝐞𝐜𝐭𝐢𝐧𝐠 𝐉𝐚𝐢𝐥𝐛𝐫𝐞𝐚𝐤𝐬 Large language models (LLMs) are prone to a new type of vulnerability known as jailbreaks or prompt-injection attacks. These attacks involve malicious manipulation of prompts to induce misalignment, data leakage, performance degradation, or harmful content generation. This paper presents a Taxonomy for a wide range of jailbreak attacks, categorizing them by linguistic constructs, user intents and the agency of the attack: 🧩 Utilizes a template-based strategy to curate 3,700 attacks, assessing their effectiveness across various attack types on proprietary models like GPT-X and open-source models such as OPT, BLOOM, and FLAN-T5-XXL. 📊 Provides a two-dimensional definition of attack success, focusing on task misalignment and attacker-intent matching, and compares programmatic tests with LLM-based detection methods for assessing attack success. 🔍 Highlighting challenges in current detection strategies due to limited lexical and semantic coverage. As LLMs become more prevalent, the urgency to mitigate potential misuse increases. New tools are required. Abs: https://lnkd.in/geDeYggx #AI #GenAI #LLM #MLLM #LLMSecurity #AIethics #Cybersecurity #ArtificialIntelligence #MachineLearning #PromptInjection #AIattacks #AIsafety #GenerativeAI #NIST #GPTX #OpenSourceAI #TechSafety #DigitalSecurity #AIevaluation #AImisuse #EmergingTech #InnovationInAI #AIrisks #AILaw #AIPolicy #TechEthics