Saima Fancy’s Post

I met with Saima Fancy and Sahil Agarwal recently to tackle the new vulnerabilities enabled because of the rise of large language models. Some key areas in this article: • Countries with the most robust cybersecurity infrastructures include Finland, Norway, and Denmark. • US and Canada maintain strong defences in cybersecurity but only rank 9th and 10th overall when it comes to safety. • Countries with the highest risk: Venezuela, Honduras, Bolivia are much more vulnerable today to cyber espionage, and attacks on critical infrastructure and national safety, and the ensuing economic impacts. • Saima says organizations could legally opt to acquire data and train their models in a structured way, but instead "There's a race to release new technologies, which can inadvertently cause harm. It's often just a case of 'let's release it and see what happens." • Sahil says, "Since the introduction of ChatGPT, there has been a significant increase in awareness among the public, legislators, and companies about the potential risks of AI. This heightened awareness has surpassed much of what we've seen over the past decade, highlighting both the potential and the dangers of AI technologies.” • LLMs have enabled attackers to conduct more effective social engineering attacks by generating personalized messages or responses based on more extensive knowledge of the target’s online activity, their preferences and behaviors. • The speed at which LLMS can also automate certain stages of cyber-attacks by generating targeted queries, crafting exploit payloads, or bypassing security measures, is making it increasingly difficult to identify these before they’re implemented. Read more below: Note: Linkedin is not properly posting this URL so have added the image and title just for context https://lnkd.in/g4w76EBu Will LLM Adoption Demand More Stringent Data Security Measures?

While at RSA earlier this month, I had the opportunity to meet and speak with Hessie Jones from Forbes about how startups and enterprises are building and adopting LLMs, focusing on the critical intersection of innovation and safety. Check out the full discussion for insights on why innovation and safety in AI development are inseparable. Follow Enkrypt AI's journey as we bring safety, security, and trust to Generative AI. #AI #ResponsibleAI #Innovation #TechEthics

This article in SecurityWeek looks at the many ways AI's journey is evolving significantly, with #GenAI like ChatGPT leading the charge. As we head towards mid-2024, the landscape is set for even more transformative shifts. Sacha Labourey of Cloudbees provides a nugget about the rise of 'pilot' #AI solutions, capable of executing decisions autonomously, marking a new era in technology's role, "2024 will see the emergence of true ‘pilot’ solutions that will not just propose changes, but perform them, relying on the overall security and stability test harness of the underlying product to make sure they are the proper changes.” This progression in AI is not solely about innovation. AI will affect areas we haven’t even considered. For example, AI will probably expedite changes to patent laws. In late December 2023, the UK’s Supreme Court agreed with a similar ruling in the US: products produced by AI cannot be patented to an AI inventor. Since the owner or developer of the AI cannot claim to be the inventor of the product, the result is clearly untenable. Read the article to learn of more ways AI is impacting us all. https://lnkd.in/g5B6_d84

The rapid adoption of AI technologies has brought about unprecedented efficiency, but it's crucial to remain vigilant about potential security risks. Prompt injection, a technique where malicious actors manipulate AI models through deceptive prompts, poses a serious threat. The following article is a good read to bring you up to speed.

The NSA's research director, Gilbert Herrera, recently shared insights on the impact of large language models (LLMs) like ChatGPT on the intelligence community. Here are the key takeaways: 1. The conversational abilities of ChatGPT mark a significant advance, even though the NSA has been researching AI for over two decades. 2. Due to privacy laws and budget limitations, the NSA faces hurdles in developing its own LLMs compared to tech giants. However, this doesn't significantly hinder their core mission of foreign intelligence. 3. The NSA could leverage commercial LLMs for reverse engineering cyber defenses. Techniques like retrieval augmented generation (RAG) could allow LLMs to analyze NSA data while adhering to privacy regulations. 4. The proliferation of AI introduces new security risks, such as enhanced phishing attacks. To address challenges like model theft and data leakage, the NSA has established an AI Security Center. The way forward is evident - the intelligence community must identify methods to harness the capabilities of commercial LLMs while safeguarding privacy and security. This will necessitate cutting-edge technical solutions, strong cybersecurity measures, and continued AI security research. Action items: - Explore technical approaches like RAG to enable privacy-preserving use of LLMs on sensitive data - Invest in robust cybersecurity practices to protect AI models and training data - Foster public-private partnerships to share knowledge and develop AI security best practices - Support ongoing research into AI security threats and countermeasures Knight, W. (2024, March 21). The NSA Wants a ChatGPT For Spooks. Fast Forward. Wired. Artificial Intelligence Security Center #aisc https://www.nsa.gov/AISC/ #ai #generativeai #llm #rag #NSA #aisecurity #publicprivatepartnership #emergingtechnologies

In the ever-evolving landscape of technology, some exciting developments have caught our eye in recent AI news: Exciting news for iOS users as Apple confirms at their WWDC keynote that access to GPT-4o will be free of charge. Despite not paying for the integration, the tech giant assures that privacy concerns are unwavering with OpenAI's inability to save user data. A pioneering move that signals new frontiers for device-integrated AI. https://lnkd.in/dCrpGfDp IRONSCALES ups the ante in email security with the general availability of their GPT-powered spear phishing simulation. It's a breakthrough for cybersecurity, providing sustainable protection against sophisticated email-based attacks. With AI blending into security protocols like never before, this marks a significant step toward smarter, responsive systems that can learn and adapt in real-time. https://lnkd.in/dKcjtEis On the partnership front, OpenAI is expanding its horizons by teaming up with Oracle to increase computing capacity for running ChatGPT. This collaboration signifies a substantial move towards leveraging high-performance computing to further AI's capabilities and ensuring that innovative technologies like ChatGPT continue to grow and integrate seamlessly into our digital lives. https://lnkd.in/dMuPXJTP Each of these articles points to a future where AI becomes more ingrained in our everyday experiences, from the natural integration in mobile operating systems, enhancing cybersecurity defenses, to forming partnerships that solidify the infrastructure it operates on. With these advances, we're not just looking at what AI can do for us today, but how it's paving the way for a smarter tomorrow.

#ai #promptinjection "Prompt injection attacks are widely considered the most dangerous of the techniques targeting AI systems. Prompt injection is a method used to trick an AI tool, such as ChatGPT or Bard, into bypassing its normal restrictions. Attackers do this by using prompts -- text or inputs fed to a large language model (LLM) -- that do one of the following: -Override the controls that define how and by what rules the AI interacts with the user. -Fool the system into thinking it does not need to follow those rules anymore." -techtarget

🚀 Elevating Awareness: The Dual-Edged Sword of LLMs in Cybersecurity As we continue to embrace the transformative power of Large Language Models (LLMs) across various industries, it's imperative to spotlight the nuanced landscape of cybersecurity risks and rewards they introduce. A recent discussion sheds light on how these advanced AI models, from generating compelling content to enhancing customer experiences, are becoming integral to organizational growth and innovation. Yet, their rise also opens new avenues for cybercriminals. Explore the security dimensions of LLMs. 🎯 LLMs: A Spectrum of Use Cases LLMs like PaLM, GPT, and Megatron-Turing NLG have showcased their versatility by powering a wide array of applications, from spam detection to sentiment analysis. Their capability to understand and generate human-like text is revolutionizing how we interact with digital platforms, making AI tools more intuitive and accessible. ⚙️Navigating the Cybersecurity Landscape However, this technological leap is not without its vulnerabilities. The article highlights several key security concerns associated with LLMs, including sensitive data exposure and the malicious use of AI for bypassing security protocols. It's a stark reminder that with great power comes great responsibility. ☎Mitigating Risks in the Age of AI Adopting LLMs necessitates a balanced approach to harness their potential while mitigating inherent risks. This involves: 🔹 Implementing robust input validation measures. 🔹 Setting API rate limits to prevent misuse. 🔹 Employing proactive risk management strategies with advanced threat detection systems. 🤝 A Collaborative Path Forward As the landscape of AI continues to evolve, fostering a culture of security-first in the development and deployment of LLMs is crucial. It requires a concerted effort from developers, legal teams, and cybersecurity experts to ensure that the benefits of LLMs can be realized without compromising on data privacy and security. 🛠 Embracing the Future with Preparedness The journey with LLMs is just beginning, and their potential to drive progress is immense. By staying informed and proactive in addressing the security dimensions of LLMs, we can safeguard our digital future and continue to innovate with confidence. Let's engage in a dialogue about how we can collectively navigate the cybersecurity challenges posed by LLMs, ensuring a safer and more secure digital landscape for all. 👉 https://lnkd.in/dR-bwamM #Cybersecurity #AI #LLMs #DataPrivacy #Innovation #llmsecurity #genaisecurity

Here Come the AI Worms Security researchers created an AI worm in a test environment that can automatically spread between generative AI agents—potentially stealing data and sending spam emails along the way. Nassi, along with fellow researchers Stav Cohen and Ron Bitton, created the worm, dubbed Morris II, as a nod to the original Morris computer worm that caused chaos across the internet in 1988. In a research paper and website shared exclusively with WIRED, the researchers show how the AI worm can attack a generative AI email assistant to steal data from emails and send spam messages—breaking some security protections in ChatGPT and Gemini in the process. https://lnkd.in/eKy3mwKk Now, in a demonstration of the risks of connected, autonomous AI ecosystems, a group of researchers have created one of what they claim are the first generative AI worms—which can spread from one system to another, potentially stealing data or deploying malware in the process. “It basically means that now you have the ability to conduct or to perform a new kind of cyberattack that hasn't been seen before,” says Ben Nassi, a Cornell Tech researcher behind the research.

Have you considered how use of AI could impact your company’s security? Throw some ideas in the comments below before reading on 🗨️👇 One in four employees are using AI tech in ways that contravene their company policies, ranging from using AI for unapproved tasks to handling sensitive data in ways that could lead to breaches.  Unlike other services employees often use, AI can encourage employees to share sensitive information in the race to be more productive, but this exposes them to new threats. For example, in March 2023 when a bug in ChatGPT’s source code led to the exposure of sensitive data of ChatGPT Plus subscribers during a nine-hour window (names, email addresses, payment details, and the last four digits of credit card numbers). Our latest blog post explores the security risks of using AI in your organization: https://lnkd.in/eTvKGerN