frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

My minute-by-minute response to the LiteLLM malware attack

https://futuresearch.ai/blog/litellm-attack-transcript/
88•Fibonar•1h ago
Related: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised - https://news.ycombinator.com/item?id=47501426 (483 comments)

Comments

Fibonar•1h ago
Callum here, I was the developer that first discovered and reported the litellm vulnerability on Tuesday. I’m sharing the transcript of what it was like figuring out what was going on in real time, unedited with only minor redactions.

I didn’t need to recount my thought process after the fact. It’s the very same ones I wrote down to help Claude figure out what was happening.

I’m an ML engineer by trade, so having Claude walk me through exactly who to contact and a step by step guide of time-critical actions felt like a game-changer for non-security researchers.

I'm curious whether the security community thinks more non-specialists finding and reporting vulnerabilities like this is a net positive or a headache?

Bullhorn9268•1h ago
Not a security researcher, but this is IMHO obviously positive that the other side of the arms race is also getting stronger, and I would argue it's stronger than on the bad guys' side, due to the best being somewhat responsible and adding guardrails.

I like the presentation <3.

rgambee•1h ago
I've heard stories lately of open source projects being inundated with vulnerability reports and PRs. But in this case, it seems like AI assistance was clearly a boon for root-causing and reporting this so quickly.
dot_treo•1h ago
Looks like we discovered it at essentially the same time, and in essentially the same way. If the pth file didn't trigger a fork-bomb like behavior, this might have stayed undiscoverd for quite a bit longer.

Good thinking on asking Claude to walk you through on who to contact. I had no idea how to contact anyone related to PyPI, so I started by shooting an email to the maintainers and posting it on Hacker News.

While I'm not part of the security community, I think everyone who finds something like this, should be able to report it. There is no point in gatekeeping the reporting of serious security vulnerabilities.

notatallshaw•1h ago
> I had no idea how to contact anyone related to PyPI

https://pypi.org/security/:

> If you've identified a security issue with a project hosted on PyPI Login to your PyPI account, then visit the project's page on PyPI. At the bottom of the sidebar, click Report project as malware.

0cf8612b2e1e•1h ago
The existing account to report is an unfortunate obstacle. Presumably not a huge deal if you were auditing code for vulnerabilities, but still an annoyance.
notatallshaw•43m ago
The threat actor was sophisticated enough to spam GitHub issues with dozens of different accounts. I imagine they could completely overwhelm PyPI with unauthenticated reports.
Fibonar•55m ago
The best part was that I didn't even mean to ask Claude who to contact! I was still in disbelief that I was one of the first people affected, so I asked for existing reports on the assumption that if it was real I definitely wasn't the first.

The fork-bomb part still seems really weird to me. A pretty sophisticated payload, caught by missing a single `-S` flag in the subprocess call.

gbrindisi•1h ago
thanks for raising the alarm and sharing this, very insightful

(also beautifully presented!)

lq9AJ8yrfs•13m ago
As a sometimes peripheral and sometimes primary program manager for vulnerability disclosure, for companies you nearly can't avoid, $0.02 follows.

It's a signal vs noise thing. Most of the grief is caused by bottom feeders shoveling anything they can squint at and call a vulnerability and asking for money. Maybe once a month someone would run a free tool and blindly send snippets of the output promising the rest in exchange for payment. Or emailing the CFO and the General Counsel after being politely reminded to come back with high quality information, and then ignored until they do.

Your report on the other hand was high quality. I read all the reports that came my way, and good ones were fast tracked for fixes. I'd fix or mitigate them immediately if I had a way to do so without stopping business, and I'd go to the CISO, CTO, and the corresponding engineering manager if it mattered enough for immediate response.

cedws•1h ago
GitHub, npm, PyPi, and other package registries should consider exposing a firehose to allow people to do realtime security analysis of events. There are definitely scanners that would have caught this attack immediately, they just need a way to be informed of updates.
Fibonar•1h ago
So I've been thinking about this a lot since it happened. I've already added dependency cooldowns https://nesbitt.io/2026/03/04/package-managers-need-to-cool-... to every part of our monorepo. The obvious next thought is "am I just dumping the responsibility onto the next person along"? But as you point out it just needs to give automated scanners enough time to pick up on obvious signs like the .pth file in this case.
cedws•58m ago
It is in a sense dumping responsibility, but there’s a legion of security companies out there scanning for attacks all the time now to prove their products. They’re kind of doing a public service and you’re giving them a chance to catch attacks first. This is why I think dep cooldowns are great.
simonw•1h ago
PyPI does exactly that, and it's been very effective. Security partners can scan packages and use the invite-only API to report them: https://blog.pypi.org/posts/2024-03-06-malware-reporting-evo...
staticassertion•1h ago
PyPI is pretty best-in-class here and I think that they should be seen as the example for others to pursue.

The client side tooling needs work, but that's a major effort in and of itself.

cedws•1h ago
Thanks, TIL.
charcircuit•19m ago
It is not effective if it just takes a simple base64 encode to bypass. If Claude is trivially able to find that it is malicious then Pypi is being negligent.
simonw•5m ago
The package in question was live for 46 minutes. It generally takes longer than that for security partners to scan and flag packages.

PyPI doesn't block package uploads awaiting security scanning - that would be a bad idea for a number of reasons, most notably (in my opinion) that it would be making promises that PyPI couldn't keep and lull people into a false sense of security.

ting0•38m ago
I feel like they should be legally responsible for providing scanning infrastructure for this sort of thing. The potential economic damage can be catastrophic. I don't think this is the end of the litellm story either, given that 47k+ people were infected.
dmitrygr•1h ago
Consider this your call to write native software. There is yet to be a supply chain attack on libc
ddp26•1h ago
Sure, but this is a pretty onerous restriction.

Do you think supply chain attacks will just get worse? I'm thinking that defensive measures will get better rapidly (especially after this hack)

dmitrygr•39m ago
> Do you think supply chain attacks will just get worse? I'm thinking that defensive measures will get better rapidly (especially after this hack)

I think the attacks will get worse and more frequent -- ML tools enable doing it easily among people who were previously not competent enough to pull it off but now can. There is no stomach for the proper defensive measures among the community for either python or javascript. Why am i so sure? This is not the first, second, third, or fourth time this has happened. Nothing changed.

applfanboysbgon•29m ago
Not only do the tools enable incompetent attackers, they also enable a new class of incompetent library developers to create and publish packages, and a new class of incompetent application developers to install packages without even knowing what packages are being used in the code they aren't reading, and a new class of incompetent users who are allowing OpenClaw to run completely arbitrary code on their machines with no oversight. We are seeing only the tip of the iceberg of the security breaches that are to come.
dmitrygr•21m ago
100%
ting0•36m ago
They will certainly get worse. LLMs make it so much easier.
woodruffw•1h ago
This is presumably because libc just doesn't change very often (not meaning code changes, but release cadence). But the average native software stack does have lots of things that change relatively often[1]. So "native" vs. not is probably not a salient factor.

[1]: https://en.wikipedia.org/wiki/XZ_Utils_backdoor

everforward•42m ago
I think that article proves the opposite.

> While xz is commonly present in most Linux distributions, at the time of discovery the backdoored version had not yet been widely deployed to production systems, but was present in development versions of major distributions.

Ie if you weren’t running dev distros in prod, you probably weren’t exposed.

Honestly a lot of packaging is coming back around to “maybe we shouldn’t immediately use newly released stuff” by delaying their use of new versions. It starts to look an awful lot like apt/yum/dnf/etc.

I would wager in the near future we’ll have another revelation that having 10,000 dependencies is a bad thing because of supply chain attacks.

consp•29m ago
This is the security equivalent of having a better lock than your neighbour. Won't save you in the end but you won't be first. Then again, yours could also be broken and you don't get to tick of that audit checkbox.
woodruffw•17m ago
Per below, xz is also an example of us getting lucky.

> I would wager in the near future we’ll have another revelation that having 10,000 dependencies is a bad thing because of supply chain attacks.

Yes, but this also has nothing to do with native vs. non-native.

dmitrygr•41m ago
your link disproves your claim. no naive app depended on xz version >= latest. Most sane distros take time to up-rev. That is why the xz backdoor was, in fact, in NO stable distro

And not changing often is a feature, yes.

woodruffw•18m ago
I don't think it does; I think the industry opinion on xz is that we got lucky in terms of early detection, and that we shouldn't depend on luck.

(I don't know what a "sane" distro is; empirically lots of distros are bleeding-edge, so we need to think about these things regardless of value judgements.)

dmitrygr•16m ago
Sane: debian-stable
hrmtst93837•31m ago
Native code still have plenty of attack surface. If you do everything through pip/npm you might as well publish your root password, but pretending a clean C build from source makes you safe is just cosplay for people who confuse compiler output with trust. If anything people are way too quick to trust a tarball that builds on the first try.
dmitrygr•28m ago
100% with you. Anything that builds from the first try is 100% malicious. No real software builds without 5-30 tweaks of the makefile. And anything on npm/pip is malicious with a fixed chance that you have no control over, as seen in this attack.

But the data remains: no supply chain attacks on libc yet, so even if it COULD happen, this HAS and that merely COULD.

mr_mitm•17m ago
Native software? You mean software without dependencies? Because I don't see how you solve the supply chain risk as long as you use dependencies. Sure, minimizing the number of dependencies and using mostly stable dependencies also minimizes the risk, but you'll pay for it with glacial development velocity.
dmitrygr•16m ago
Slower development velocity but no third-party-induced hacks surely has a market. :)
simonw•1h ago
First time I've seen my https://github.com/simonw/claude-code-transcripts tool used to construct data that's embedded in a blog post, that's a neat way to use it. I usually share them as HTML pages in Gists instead, e.g. whttps://gisthost.github.io/?effbdc564939b88fe5c6299387e217da...
Fibonar•1h ago
I’m a big proponent of it within our company! CC tried to style it to blend in with our blog but it was kind of a disaster. Definitely had a new appreciation for the out-of-the-box experience. I also tried to include the individual sub-pages of Claude investigating but it really trawled my whole machine looking for malware. Don’t know if you’ve thought of any systematic ways of redacting the endless pages of detailed logs?
moralestapia•1h ago
*salutes*

Thank you for your service, this brings so much context into view, it's great.

S0y•43m ago
> Where did the litellm files come from? Do you know which env? Are there reports of this online?

> The litellm_init.pth IS in the official package manifest — the RECORD file lists it with a sha256 hash. This means it was shipped as part of the litellm==1.82.8 wheel on PyPI, not injected locally.

> The infection chain:

> Cursor → futuresearch-mcp-legacy (v0.6.0) → litellm (v1.82.8) → litellm_init.pth

This is the scariest part for me.

RALaBarge•13m ago
Maybe the people who use emacs for everything are the only safe ones?
Bullhorn9268•42m ago
The fact pypi reacted so quickly and quarantined the package in like 30 minutes after the report is pretty great!
Shank•21m ago
Probably one of the best things about AI/LLMs is the democratization of reverse engineering and analysis of payloads like this. It’s a very esoteric skill to learn by hand and not very immediately rewarding out of intellectual curiosity most times. You can definitely get pointed in the right direction easily, now, though!
cdcarter•4m ago
If it weren't for the 11k process fork bomb, I wonder how much longer it would have taken for folks to notice and cut this off.
intothemild•3m ago
Thats the thing, i noticed it almost instantly when trying to install a package that depended on it, as soon as it started, it hard locked my laptop, didn't get to infect it.. but if they had slowed down that fork bomb.. it would have done more damage.
__mharrison__•3m ago
Interesting world we live in.

I just finished teaching an advanced data science course for one of my clients. I found my self constantly twitching everytime I said "when I write code..." I'm barely writing code at all these days. But I created $100k worth of code just yesterday recreating a poorly maintained (and poor ux) library. Tested and uploaded to pypi in 90 minutes.

A lot of the conversation in my course was directed to leveraged AI (and discussions of existential dread of AI replacement).

This article is a wonderful example of an expert leveraging AI to do normal work 100x faster.

Moving from GitHub to Codeberg, for lazy people

https://unterwaditzer.net/2025/codeberg.html
345•jslakro•4h ago•160 comments

My minute-by-minute response to the LiteLLM malware attack

https://futuresearch.ai/blog/litellm-attack-transcript/
91•Fibonar•1h ago•46 comments

OpenTelemetry profiles enters public alpha

https://opentelemetry.io/blog/2026/profiles-alpha/
47•tanelpoder•1h ago•5 comments

Show HN: Claude skill that evaluates B2B vendors by talking to their AI agents

https://github.com/salespeak-ai/buyer-eval-skill
32•ogotlieb•1h ago•0 comments

Personal Encyclopedias

https://whoami.wiki/blog/personal-encyclopedias
672•jrmyphlmn•22h ago•134 comments

Interoperability Can Save the Open Web (2023)

https://spectrum.ieee.org/doctorow-interoperability
136•janandonly•2h ago•38 comments

From zero to a RAG system: successes and failures

https://en.andros.dev/blog/aa31d744/from-zero-to-a-rag-system-successes-and-failures/
215•andros•2d ago•69 comments

My home network observes bedtime with OpenBSD and pf

https://ratfactor.com/openbsd/pf-gateway-bedtime
61•ibobev•3d ago•14 comments

Colibri – chat platform built on the AT Protocol for communities big and small

https://colibri.social/
5•todotask2•16m ago•2 comments

End of "Chat Control": EU parliament stops mass surveillance

https://www.patrick-breyer.de/en/end-of-chat-control-eu-parliament-stops-mass-surveillance-in-vot...
336•amarcheschi•5h ago•196 comments

Running Tesla Model 3's computer on my desk using parts from crashed cars

https://bugs.xdavidhu.me/tesla/2026/03/23/running-tesla-model-3s-computer-on-my-desk-using-parts-...
788•driesdep•20h ago•270 comments

Light on Glass: Why do you start making a game engine?

https://analogdreamdev.substack.com/p/light-on-glass
18•atan2•3d ago•3 comments

Swift 6.3

https://www.swift.org/blog/swift-6.3-released/
265•ingve•10h ago•159 comments

Obsolete Sounds

https://citiesandmemory.com/obsolete-sounds/
159•benbreen•13h ago•33 comments

Building a Blog with Elixir and Phoenix

https://jola.dev/posts/building-a-blog-with-elixir-and-phoenix
8•shintoist•29m ago•0 comments

HyperAgents: Self-referential self-improving agents

https://github.com/facebookresearch/hyperagents
4•andyg_blog•2d ago•0 comments

Stripe Projects: Provision and manage services from the CLI

https://projects.dev/
11•piinbinary•1h ago•4 comments

Shell Tricks That Make Life Easier (and Save Your Sanity)

https://blog.hofstede.it/shell-tricks-that-actually-make-life-easier-and-save-your-sanity/
388•zdw•17h ago•185 comments

SpaceStarCarz KoolWheelz Paper Models

https://davesdesigns.ca/dcc/html/spacestarcarz_.html
21•exvi•2d ago•4 comments

Intel Announces Arc Pro B70 and Arc Pro B65 GPUs

https://www.techpowerup.com/347703/intel-announces-arc-pro-b70-and-arc-pro-b65-gpus-maxes-out-xe2...
83•throwaway270925•3h ago•35 comments

Niche Museums

https://www.niche-museums.com/
83•bookofjoe•2d ago•39 comments

Optimizing a lock-free ring buffer

https://david.alvarezrosa.com/posts/optimizing-a-lock-free-ring-buffer/
54•dalvrosa•2d ago•53 comments

AI users whose lives were wrecked by delusion

https://www.theguardian.com/lifeandstyle/2026/mar/26/ai-chatbot-users-lives-wrecked-by-delusion
104•tim333•4h ago•92 comments

ARC-AGI-3

https://arcprize.org/arc-agi/3
474•lairv•23h ago•305 comments

Ashby (YC W19) Is Hiring Engineers Who Make Product Decisions

https://www.ashbyhq.com/careers?ashby_jid=c3c7125d-7883-4dff-a2bf-f5a55de4a364&utm_source=hn
1•abhikp•10h ago

French e, è, é, ê, ë – what's the difference?

https://jakubmarian.com/french-e-e-e-e-e-whats-the-difference/
78•kerblang•2h ago•80 comments

Optimization lessons from a Minecraft structure locator

https://purplesyringa.moe/blog/optimization-lessons-from-a-minecraft-structure-locator/
48•ftk_•5d ago•6 comments

Earthquake scientists reveal how overplowing weakens soil at experimental farm

https://www.washington.edu/news/2026/03/19/earthquake-scientists-reveal-how-overplowing-weakens-s...
206•Brajeshwar•1d ago•110 comments

LibreOffice and the art of overreacting

https://blog.documentfoundation.org/blog/2026/03/25/libreoffice-and-the-art-of-overreacting/
178•bundie•7h ago•111 comments

My DIY FPGA board can run Quake II

https://blog.mikhe.ch/quake2-on-fpga/part4.html
210•sznio•3d ago•62 comments