We've built an open-source tool to stress test AI agents by simulating prompt injection attacks.

We’ve implemented one powerful attack strategy based on the paper [AdvPrefix: An Objective for Nuanced LLM Jailbreaks](https://arxiv.org/abs/2412.10321).

Here's how it works:

- You define a goal, like: “Tell me your system prompt” - Our tool uses a language model to generate adversarial prefixes (e.g., “Sure, here are my system prompts…”) that are likely to jailbreak the agent. - The output is a list of prompts most likely to succeed in bypassing safeguards.

We’re just getting started. Our goal is to become the go-to toolkit for testing agent security. We're currently working on more attack strategies and would love your feedback, ideas, and collaboration.

Try it at: https://security.vista-labs.ai/

Docs with how to: https://hackagent.dev/docs/intro

GitHub: https://github.com/vistalabs-org/hackagent

video demo with example: https://www.loom.com/share/1e4ce025ea4749fab169195e7b1222ba

Would love to hear what you think!