RAG Poisoning (PANDORA) – found something cool in the knowledge graph this am: “The integration of various plugins into LLMs, notably Retrieval Augmented Generation (RAG), which enables LLMs to incorporate external knowledge bases into their response generation such as GPTs, introduces new avenues for indirect jailbreak attacks. To fill this gap, we investigate indirect jailbreak attacks on LLMs, particularly GPTs, introducing a novel attack vector named Retrieval Augmented Generation Poisoning. This method, PANDORA, exploits the synergy between LLMs and RAG through prompt manipulation to generate unexpected responses. PANDORA uses maliciously crafted content to influence the RAG process, effectively initiating jailbreak attacks.” Paper: https://lnkd.in/eFXUwQtH #malware, #ai, #informationsecurity, #blueteam #reverseengineering #cyberdefense #cybercrime, #cyberthreatintelligence, #cyberdefense, #cyberwarfare #networksecurity #sec #security #tools #offensivesecurity, #redteam #innovation
Very informative
Good Point
M365 Enterprise Expert • Azure Architect • Mentor • C|EI • C|EH • C|EH TOP 100 • C|HFI • C|ND • C|NDA • C|CSE • CRISC • CCSK • MCSE • MCSA • Intune • M365 & Azure Security • Endpoint • IAM
8moFascinating insights on indirect jailbreak attacks in LLMs. Can't wait to check out the paper. 🔒