frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

Open in hackernews

Show HN: DeepTeam – Penetration Testing for LLMs

https://github.com/confident-ai/deepteam
3•jeffreyip•1y ago
Hi HN, we’re Jeffrey and Kritin, and we’re building DeepTeam (https://trydeepteam.com), an open-source Python library to scan LLM apps for security vulnerabilities. You can start “penetration testing” by defining a Python callback to your LLM app (e.g. `def model_callback(input: str)`), and DeepTeam will attempt to probe it with prompts designed to elicit unsafe or unintended behavior.

Note that the penetration testing process treats your LLM app as a black-box - which means that DeepTeam will not know whether PII leakage has occurred in a certain tool call or incorporated in the training data of your fine-tuned LLM, but rather just detect that it is present. Internally, we call this process “end-to-end” testing.

Before DeepTeam, we worked on DeepEval, an open-source framework to unit-test LLMs. Some of you might be thinking, well isn’t this kind of similar to unit-testing?

Sort of, but not really. While LLM unit-testing focuses on 1) accurate eval metrics, 2) comprehensive eval datasets, penetration testing focuses on the haphazard simulation of attacks, and the orchestration of it. To users, this was a big and confusing paradigm shift, because it went from “Did this pass?” to “How can this break?”.

So we thought to ourselves, why not just release a new package to orchestrate the simulation of adversarial attacks for this new set of users and teams working specifically on AI safety, and borrow DeepEval’s evals and ecosystem in the process?

Quickstart here: https://www.trydeepteam.com/docs/getting-started#detect-your...

The first thing we did was offer as many attack methods as possible - simple encoding ones like ROT13, leetspeak, to prompt injections, roleplay, and jailbreaking. We then heard folks weren’t happy because the attacks didn’t persist across tests and hence they “lost” their progress every time they tested, and so we added an option to `reuse_simulated_attacks`.

We abstracted everything away to make it as modular as possible - every vulnerability, attack, can be imported in Python as `Bias(type=[“race”])`, `LinearJailbreaking()`, etc. with methods such as `.enhance()` for teams to plug-and-play, build their own test suite, and even to add a few more rounds of attack enhancements to increase the likelihood of breaking your system.

Notably, there are a few limitations. Users might run into compliance errors when attempting to simulate attacks (especially for AzureOpenAI), and so we recommend setting `ignore_errors` to `True` in case that happens. You might also run into bottlenecks where DeepTeam does not cover your custom vulnerability type, and so we shipped a `CustomVulnerability` class as a “catch-all” solution (still in beta).

You might be aware that some packages already exist that do a similar thing, often known as “vulnerability scanning” or “red teaming”. The difference is that DeepTeam is modular, lightweight, and code friendly. Take Nvidia Garak for example, although comprehensive, has so many CLI rules, environments to set up, it is definitely not the easiest to get started, let alone pick the library apart to build your own penetration testing pipeline. In DeepTeam, define a class, wrap it around your own implementations if necessary, and you’re good to go.

We adopted a Apache 2.0 license (for now, and probably in the foreseeable future too), so if you want to get started, `pip install deepteam`, use any LLM for simulation, and you’ll get a full penetration report within 1 minute (assuming you’re running things asynchronously). GitHub: https://github.com/confident-ai/deepteam

Excited to share DeepTeam with everyone here – let us know what you think!

Gemini 4 Neon is the new #1 on the Models Engineering Index

https://git-aarvin.github.io/mei/
1•hn-aarvin•39s ago•0 comments

Model or Cautionary Tale? What the South Bay Tells Us About Data Centers

https://www.coyotemedia.org/ai-data-centers-san-jose/
1•cdrnsf•46s ago•0 comments

Prepaid gas bonds: A differentiated source of tax-exempt income

https://www.blackrock.com/us/individual/insights/prepaid-gas-bonds
1•toomuchtodo•1m ago•1 comments

On The Use of AI-Based Tools to Find Bugs in Open Source Software

https://www.haproxy.com/blog/on-the-use-of-ai-based-tools-to-find-bugs-in-open-source-software
2•karalel•2m ago•0 comments

I never learned Bash (2024)

https://www.joshooaj.com/blog/2024/12/30/why-i-never-learned-bash/
1•zenincognito•3m ago•0 comments

Visa, Mastercard and JPMorgan Customer Data Leaked by Hackers

https://www.wsj.com/finance/banking/visa-mastercard-and-jpmorgan-customer-data-leaked-by-hackers-...
2•mikhael•4m ago•0 comments

Logic Puzzle Battle Royale

https://www.puzzleroyale.co.uk/
1•jamiehannigan•7m ago•0 comments

Mastodon instance Baraag.net: Server Admin arrested and domain seized by the FBI

https://old.reddit.com/r/Mastodon/comments/1y0eb7t/mastodon_instance_baraagnet_has_been_forcibly/
2•doener•11m ago•0 comments

Tin vs. Aluminum Foil in hat making applications

https://tinfoil.wtf/dispatches/tin-to-aluminum-switch/
1•thebetatester•12m ago•0 comments

Surveillance Pricing Warrants Further Review

https://communitysentinel.media/2026/11/18/surveillance-pricing-warrants-further-review/
1•nobody9999•13m ago•0 comments

Silicon Valley Is Suffering from Model Collapse

https://www.theatlantic.com/technology/2026/11/silicon-valley-model-collapse/689505/
2•spking•14m ago•0 comments

Hakeem Jeffries Staffer Built an AI Program for Surveillance and Advice

https://www.washingtonsun.com/congress/hakeem-jeffries-ai-caucus-pulse-office-policy
2•cdrnsf•19m ago•0 comments

Show HN: Auto PR Labeling with Jev

https://checks.dorbit.ai/
1•mtanaka•22m ago•0 comments

Show HN: Mindie – Reminders, notes and calendars on WhatsApp

https://www.getmindie.app
1•tamilini92•22m ago•0 comments

People Don't Subscribe to Blogs

https://micjm.com/blog/front-page-of-hacker-news-by-the-numbers/
2•seizethecheese•23m ago•2 comments

AIs might overthrow the government in one year, says Paul Christiano [video]

https://www.youtube.com/watch?v=SyHM5hNPAEo
2•OgsyedIE•25m ago•0 comments

Show HN: Tools4U – 54 file tools that run in the browser

https://tools4u.org
2•emberloop•31m ago•0 comments

Verified Controllers Deployed on a Fusion Reactor

https://twitter.com/HarmonicMath/status/2123513273677131876
3•modestmc•34m ago•0 comments

Google Search homepage gets AI-heavy redesign with Gemini

https://9to5google.com/2026/11/19/google-search-homepage-gemini-sidebar-update/
2•xnx•35m ago•0 comments

Data and Reality: The Database Book That Messed Me Up

https://faidi.tech/posts/the-database-book-that-messed-me-up/
1•ghassenfaidi•36m ago•0 comments

LibreTexts – An online library of open textbooks

https://www.libretexts.org/
2•ibobev•36m ago•0 comments

Research Acceleration: The Jagged Frontier

https://openai.com/index/research-acceleration-jagged-frontier/
2•gmays•37m ago•0 comments

Show HN: Kiiwi – Mechanical keyboard sounds for Mac

https://kiiwi.app/
2•tamilini92•37m ago•0 comments

Court: 4th Amendment Doesn't Cover AI-Generated 'Probable Cause'

https://www.techdirt.com/2026/11/19/court-tells-cops-that-the-4th-amendment-doesnt-cover-ai-gener...
20•defrost•39m ago•4 comments

What is to be done about the SRCF?

https://blog.ucant.org/?p=1057
1•zenincognito•40m ago•0 comments

Codeberg: Investigations and Post-Mortems

https://codeberg.org/Codeberg/changelog/issues/7
2•severine•40m ago•0 comments

Why We Built Anthropic [video]

https://www.youtube.com/watch?v=sof3vW4mHTs
1•Philpax•40m ago•0 comments

AI Agentic harness with explicit risk analysis

https://github.com/AppliedIngenuity-ai/project-chaperone
4•BraveNewCurency•41m ago•1 comments

Show HN: Wellness Tracker for AI Usage

https://chromewebstore.google.com/detail/prethink-ai/ekolohicgedlnehalkchpnbhpfmgnglk
1•harrisonbsmith1•41m ago•0 comments

PocketPico – GameBoy Emulator with RP2040

https://uart.cz/2616/pocketpico-gameboy-emulator-with-rp2040/
2•andrewstuart•41m ago•0 comments