Armilla AI’s Post

New #blog on the #MI portal: Artificial Inundation: AI is the Future and We're Living in It. Take a deep dive into the #executiveorder and #OMB memo shaping #AI policies and learn more about the emphasis on security, privacy, and governance. https://bit.ly/47W2eXL Can’t get enough AI info? Join us on December 13th for an exclusive webinar, as we dive into exactly how federal offices are utilizing the latest artificial intelligence (AI) technology to craft solutions, and most importantly, how you too can penetrate the market as additional opportunities arise. https://bit.ly/47qSFjp

Exciting developments in the world of AI! Following a comprehensive executive order focusing on AI risk management, the Biden-Harris administration has unveiled key actions to shape the future of artificial intelligence. In October, President Biden issued an executive order, emphasizing new standards for AI safety and security, privacy protection, equity and civil rights advancement, and more. The AI landscape is evolving rapidly, and the government is actively fostering partnerships with industry and academia. I'm pleased to see a focus on cybersecurity, AI training, and data integrity. Security leaders are weighing in on the importance of government and industry collaboration in AI innovation and security. Transparency, integrity, and responsible AI use are key themes. Read the full insights from experts in the field. 👇 #ArtificialIntelligence #AI #Cybersecurity #Innovation #GovernmentAI #TechLeadership

'The purpose of this publication is to provide organisations with guidance on how to use AI systems securely. The paper summarises some important threats related to AI systems and prompts organisations to consider steps they can take to engage with AI while managing risk. It provides mitigations to assist both organisations that use self-hosted and third-party hosted AI systems.' https://lnkd.in/gyD9BWuj

🚨 Breaking News in AI Security 🚨 A Wake-Up Call: Understanding AI System Vulnerabilities In a groundbreaking publication, the National Institute of Standards and Technology (NIST) has shed light on the various types of cyberattacks targeting AI systems. The report, titled "Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations", marks a pivotal step in understanding and mitigating the vulnerabilities of AI and ML systems. 🔍 Key Insights: - AI Vulnerabilities: The report uncovers how AI systems can be misled or 'poisoned' by unreliable data, leading to significant malfunctions. - Types of Attacks: It categorizes attacks into four major types: evasion, poisoning, privacy, and abuse attacks, each with unique characteristics and implications. - No Foolproof Defense: Currently, there's no absolute method to shield AI from these threats. The report emphasizes the necessity for AI developers and users to be vigilant against overconfident claims of infallibility. 📈 Impact on AI Development: - Raising Awareness: This publication serves as a crucial reminder for AI developers and users about the potential risks and encourages the development of more robust defenses. - Guiding Framework: It offers a comprehensive overview of attack techniques and methodologies, aiding developers in anticipating and preparing for potential threats. - Continuing Challenges: The NIST's work highlights ongoing theoretical problems in securing AI algorithms, underscoring the need for continual research and development in this area. 🤔 Thought-Provoking: As we integrate AI more deeply into our lives, from autonomous vehicles to healthcare diagnostics, understanding and mitigating these vulnerabilities becomes paramount. How can we, as a community, collaborate to enhance the security and reliability of AI systems? Let's discuss it! 🔗 https://lnkd.in/dM-BS6fw #AI #MachineLearning #Cybersecurity #NIST #ArtificialIntelligence #AdversarialMachineLearning #AIsecurity

In our last post we talked about the rapid deployment of Ai solutions can create security debt. Following that same topic and with the rapid deployment of AI systems, the National Institute of Standards and Technology (NIST) has issued a warning about the significant security and privacy challenges. The integration of AI into various online services, including generative systems like OpenAI's ChatGPT and Google's Bard, has escalated these risks. Key concerns identified by NIST include: 1. Adversarial Attacks: Manipulation of training data, model vulnerabilities, and adversarial output generation can severely disrupt AI performance. 2. Data and Privacy Breaches: Threats like data model poisoning, prompt injection attacks, and privacy attacks can expose sensitive information and undermine data integrity. 3. Lack of Robust Defenses: The current security measures are insufficient to counteract these evolving threats, with significant gaps in securing AI algorithms. If we put our threat modelling hat, we classify some of the threat highlighted by NIST in the following category: ·       Data Integrity and Trustworthiness: given that AI systems often source data from multiple source, the risk of compromised data integrity is high, especially In environment where adversaries can easily tamper with the data. ·       Lack of Comprehensive Monitoring: Due to the massive amount of data in play and the nature of the data used in training, it is almost entirely impossible to monitor and continuously vet these data which create a significant security gap. ·       Dependency on Data Security: The effectiveness of AI systems is heavily dependent on the security and integrity of the training data, making them vulnerable to attacks that corrupt this foundational aspect. As quoted by the author “Despite the significant progress AI and machine learning have made, these technologies are vulnerable to attacks that can cause spectacular failures with dire consequences,” he said. “There are theoretical problems with securing AI algorithms that simply haven’t been solved yet. If anyone says differently, they are selling snake oil.” To make AI systems more secure, security practices need to be integrated throughout their development lifecycle, from design to deployment. This includes implementing security controls and regular assessments. Furthermore, by training AI models with simulated adversarial attacks, they can learn to recognize and resist manipulation attempts in the real world, minimizing potential risks.   🔗 For a more in-depth look at these issues, visit NIST's latest findings. https://lnkd.in/eANUjXiJ #AI #AISecurity #Cybersecurity #NIST #PrivacyRisks #TechCommunity

"Adversaries can deliberately confuse or even "poison" artificial intelligence (AI) systems to make them malfunction—and there's no foolproof defense that their developers can employ. Computer scientists from the National Institute of Standards and Technology (NIST) and their collaborators identify these and other vulnerabilities of AI and machine learning (ML) in a new publication. Their work, titled Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, is part of NIST's broader effort to support the development of trustworthy AI, and it can help put NIST's AI Risk Management Framework into practice. The publication, a collaboration among government, academia, and industry, is intended to help AI developers and users get a handle on the types of attacks they might expect along with approaches to mitigate them—with the understanding that there is no silver bullet." #security #cybersecurity

Yesterday the ACSC (ASD) released guidelines on securely engaging with AI. From their web page: "The purpose of this publication is to provide organisations with guidance on how to use AI systems securely. The paper summarises some important threats related to AI systems and prompts organisations to consider steps they can take to engage with AI while managing risk. It provides mitigations to assist both organisations that use self-hosted and third-party hosted AI systems." The full guidance can be found here: https://lnkd.in/gkjSa73f

“The purpose of this publication is to provide organisations with guidance on how to use AI systems securely. The paper summarises some important threats related to AI systems and prompts organisations to consider steps they can take to engage with AI while managing risk. It provides mitigations to assist both organisations that use self-hosted and third-party hosted AI systems.” https://lnkd.in/erkvd9ji

To help support the development of trustworthy AI, the National Institute of Standards and Technology (NIST) has published a new report on the different types of cyberattacks that can manipulate AI systems’ behavior, entitled “Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations.”   Adversarial machine learning (AML) works by extracting data about the functions and characteristics of a ML system and manipulating the inputs to create a specific outcome.   This may be done through multiple types of attacks, the major four being:   • 𝗘𝘃𝗮𝘀𝗶𝗼𝗻: After deploying an AI system, an attack is attempted by altering inputs in order to change the response of the system. • 𝗣𝗼𝗶𝘀𝗼𝗻𝗶𝗻𝗴: During the training phase, corrupted data is introduced. • 𝗣𝗿𝗶𝘃𝗮𝗰𝘆: During deployment, attacks aim to discover sensitive information about the AI system, or about the data it was trained on, so it can be misused. • 𝗔𝗯𝘂𝘀𝗲: Incorrect information is inserted into a source that the AI system absorbs.   NIST’s guidance shares approaches to help mitigate these attacks. However, “there is no silver bullet.”   Learn more about the new report: https://lnkd.in/eANUjXiJ   #NIST #MachineLearning #cybersecurity #AI

Indirect Prompt Injections by Federal Office for Information Security (BSI) What are Intrinsic Vulnerabilities in Application-Integrated AI Language Model ? A few examples when employees are using an LLM to summarize or analyze text from external sources * Attackers could manipulate the result in a targeted manner • Use of a chatbot accessing modified web pages * Results of queries could be manipulated in a targeted manner * The chatbot could exhibit undesirable behaviour and, for example, make legally questionable or undesirable statements * The chatbot could motivate users to access a (malicious) link * The chatbot could attempt to obtain sensitive information from users (e.g., credit card information) * Attackers could extract sensitive information from the chat history if, for example, the possibility to call web pages or display external images exists Some measures recommanded ? . To reduce the impact of a potential attack, actions may be restricted to be reversible or executed in a segregated environment (“sandbox”). You can continue with The BSI publication “Large Language Models: Opportunities and Risks for Industry and Authorities” I also recommand to read the excellent pdf from OWASP® Foundation "LLM AI Cybersecurity & Governance Checklist From the OWASP Top 10 for LLM Applications Team" You can bookmark the Checklist with Adversarial Risk, Threat Modeling, AI Asset Inventory, AI Security and Privacy Training, Establish Business Cases, Model Cards and Risk Cards, RAG: Large Language Model Optimization and AI Red Teaming. Other measures recommanded: - Scrutinize how competitors are investing in artificial intelligence. - Although there are risks in AI adoption, there are also business benefits that may impact future market positions. - Investigate the impact of current controls, such as password resets, which use voice recognition which may no longer provide the appropriate defensive security from new GenAI enhanced attacks. - Update the Incident Response Plan and playbooks for GenAI enhanced attacks and AIML specific incidents. #PurpleHackademy #cybersecurity #LLM #riskmanagement #AI #whitepaper