Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Bases

This research paper explores how to protect private information in AI systems, especially those that use Retrieval-Augmented Generation (RAG). RAG systems help large language models (LLMs) access and use external knowledge bases to provide better answers. However, hackers can trick these systems into revealing private information from these knowledge bases. The authors developed an automated attack strategy called “Pirates of the RAG” that uses a smaller LLM and cleverly designed questions to extract hidden information. This attack is adaptive, meaning it learns from its attempts and gets better at stealing data over time. The researchers tested their attack on three different virtual agents, each representing a real-world application of RAG, and found that “Pirates of the RAG” outperformed other attack methods in terms of how much information it could steal and how quickly it could do so. The paper highlights the need for stronger security measures to protect private information in RAG systems and emphasizes that simply relying on “Guardian” LLMs, designed to prevent unsafe outputs, is not enough.

https://arxiv.org/pdf/2412.18295