If one user could have found this using AI. Then I would imagine anyone else could have found it.
Right, including Sony. AI finding security flaws is very good in general, but one downside is that it will become easier to make "secure" devices that are hostile toward their owners.
People are still fixated on using AI to produce code (to reduce salary costs/dev time) rather than using it to audit bugs in one own's code, which they are better at.
The latter has "always" been obvious to me.
It's just boring. And the people that crow about "their" accomplishments are tiresome.
> Slop kiddies found the only hypervisor bug left, which I had also found a while ago, and decided to report to Sony,” the modder said on social media. “I asked them to at least wait for GTA 6 to come out so that people would have the opportunity to legally purchase the game and also enjoy linux. They agreed to wait, but not a day passed and they decided to waste it instead.”
So what has really happened is the LLMs have lowered the participation floor for this space enough the dynamics are changing. The new comers would rather have a few dollars vs. the founders who would rather have a project. It’s likely the projects days were numbered either way because Sony could also just take a frontier LLM and examine any released installer for the project and reverse engineer and bug.
How old was this bug? How much energy has been devoted to RE of this proprietary console? Not enough, apparently.
remember hacktoberfest 2020? that's just all public-facing source 365 days a year now, except instead of "updated README.md" it's some vaguely-plausible fix... then you read the PR body and someone couldn't even be bothered to, or, just as likely couldn't explain it themselves. and that sort of sinking dread sets in.
Oh man, if you read the threads about that it's such a time capsule of a different era:
e.g. from this thread: https://news.ycombinator.com/item?id=31628342
> I am honestly surprised how little SPAM there is on GitHub in general. Please don’t take that as a challenge!
I entirely sympathize with these maintainers too. I've had 2 instances where an LLM has surfaced a bug and I just couldn't get myself to open a PR and dump more work onto these maintainers, even after manually writing one up (neither were critical bugs, it's fine).
Seeing popular projects (like hermes) having 5k issues and 5k pull requests is madness.
For example, I recently came across Hucre, https://github.com/productdevbook/hucre, which is a JS spreadsheet library apparently developed entirely in the past few months and which has already gained 2.2k GH stars.
Is it any good? Who knows, but the alternatives for spreadsheet handling in JS aren't great, either being limited, semi-commercial, or unmaintained. And creating a new one without LLM-assistance is a huge, painful, mostly thankless undertaking. It's not surprising that a zero-dependency option that promises to do everything this can do would be popular.
Everyone has had the experience of butting their head against functionality limitations of open-source libraries. Fully vibecoded libraries will be able to add functionality at a rate that human-authored ones can't. Code quality might suffer, but if a library has a feature you need, and you've got a deadline to hit, are you going to avoid using it? Is the average developer? Or the average LLM?
I think we're about to hit an interesting period of human-only projects competing with vibecoded ones, and it'll be an acid test as to whether the pro and anti claims about AI-coding bear out in terms of code reliably.
People who hate their job of course like to come out of the woodwork and say no one should enjoy it, but isn't it nice to have a society where at least some people can enjoy their work? I used to enjoy teaching but I'd never get a teaching job now because of how AI is being used in that sector.
Technophiles will say that we should embrace the future because its inevitable but how many good people quitting does it take before they realize that a vibe-coded future of fun isn't all there is to life?
That’s an interesting question. I thought about it for a minute and was surprised that I came to the conclusion that the answer isn’t obviously “yes”.
An analogous question is “isn't it nice to have a society where at least some people are rich?”, and I think that question should make it clearer to many people why that is so.
There’s a crucial difference between the two questions. “Rich” is a relative term: If everyone had the same amount of money, no one would be rich, you need someone to be worse off (poor) to have someone better off (rich). But someone enjoying their work has no bearing on someone else’s enjoyment of theirs.
It's an issue that the dev attempted to collect the bug bounty, AI or no
Not to mention how the code can then be sourced by an AI model in an instant without any credit.
That would indeed be incredibly demoralizing, even crushing. Quite a dick move.
Now that said, I imagine it was only a matter of time anyway until Sony found/fixed or someone else did the same thing. It's just a different world now.
As much as I'd love a non locked down PS5, anyone buying one knows what they're getting. If you care about user empowerment, PC (Valve hardware, etc) is the place to put your money and time. Sony is actively hostile toward you and wants you locked down, and trying to fight it with exploits is destined to be a nasty cat and mouse game. Go with a vendor that is more aligned to your values.
Ever since consoles became moving targets there's been deliberate gatekeeping - specifically, drip-feeding of bugs - to maximize the chance someone can actually use them to break DRM and install Linux. This relies on the fact that most people do not want to have to become FreeBSD kernel experts in order to install non-PlayStation software on their PlayStation. But if everyone is vibe-hacking their PS5s then none of this logic applies. Any bug Claude can find is one Sony also knows about and will get patched, possibly before you even release an exploit for it.
How this ultimately plays out depends on if it's even possible to write software without bugs. Maybe this reaches a new equilibrium where people are paying Claude to vibe-code jailbreaks - as I'd initially hoped. But it's equally as likely that this winds up reinforcing DRM rather than weakening it, for a few reasons:
1. Anthropic's AI safetyism culture encourages the prohibition of vibe-coded jailbreaks. The fact that the model runs on a server and people are spying on your chatlogs means Anthropic has actual knowledge of who is actually using their service to find PS5 hypervisor bugs. Letting Claude break DRM is legally risky; DMCA 1201 implies the only lawful way to break DRM is for you to find your own bugs. So it's probably not going to be long until everyone vibe-coding jailbreaks will get banned.
2. Sony will not be getting banned from these services, they will get trusted access as they're big enough for Anthropic to sue if it gets misused.
3. The attack surface of the thing you have to actually compromise to get code execution on any locked-down system is really small. The Xbox 360 had a 15+ year gap[0] of no softmodding because they'd isolated all the memory protection into a hypervisor. Apple learned the same lesson and iPhone jailbreaking went from incredibly commonplace to "if you know how to do it someone at Zerodium will hand you a million dollars to write spyware with it".
[0] AFAIK, the only two actual softmoddable bugs on Xbox 360 were the King Kong hack right at launch, which got patched in like a week, and that BadUpdate thing last year.
I might be showing a lot of unc energy here, but if you can't afford a raspberry pi or something to play with linux, or a used pc, gaming and having a playstation 5 should be de prioritized for a bit while you get your life together.
On the other hand, it’s been clear for a while that software as we knew will end up at a close to 0 marginal value.
Hopefully "people" doesn't mean scrapers
AI skynet is currently winning the war.
LLMs have taken over so many projects already. If I were young, would I want to contribute to projects maintained these days via AI slop? I would not find that interesting at all.
To quote Dr Ian Malcolm:
"If I may... Um, I'll tell you the problem with the scientific power that you're using here, it didn't require any discipline to attain it. You read what others had done and you took the next step. You didn't earn the knowledge for yourselves, so you don't take any responsibility for it. You stood on the shoulders of geniuses to accomplish something as fast as you could, and before you even knew what you had, you patented it, and packaged it, and slapped it on a plastic lunchbox, and now [bangs on the table]"
This guy has used on working with people that understand the scene, it seems that they were in contact, they agreed on something and then the next day they did something different.
This is really important in the context of what happened and probably the cherry on top of the current "hacking" cake, with the bar being lowered so much that anybody can jump in with some minor skill and luck and get something just for themselves.
Why not set up a triage bot with only read permissions on the project and restricted network access in order to triage issues?
Perhaps this way one could turn the influx of spam into a source of useful information.
At first, my brain parsed "spam" as in ye olde email spam. It took a moment for it to sink in that you meant code spam, instead.
But now that the two concepts are linked in my little pea brain: I kind of want to see how a system like SpamAssassin would work when applied to pull requests like it has been applied to email.
It can use reputation, real-time blacklists, triggers for form, and et cetera, with weighted scores for every aspect that are ultimately factored into one final score.
If final score is passing, the PR is presented for a human to review like a PR was (say) 5 years ago. If the final score fails, it goes into the circular file where it will probably die.
This is definitely the case for me. Before AI tools became mainstream, it was already a difficult proposition to find other smart people who you could get along with and talk to, do something interesting together. The Internet made all the difference in my life because I was able to get outside my geographic region to do big things through open source and hacker communities. Now, that very important filter mechanism no longer works. It's Eternal September all over again. In a way, it's very much a domination of "ends" over "means" in the wider community that is being forced upon those who long focused on "means". For a lot of intelligent people, understanding something is /valuable on its own/, but for the wider world there is no value in simply knowing things, but what you do with that knowledge (or now that lack of knowledge). I even experienced this recently at DEFCON 34 where I saw other participants in some of the CTFs with me using AI tools and not really understanding what the tools were doing or what was happening, but just kind of bruteforcing/tokenmaxxing their way through. This isn't to say that those AI tools are fundamentally a bad thing to use in building open source software, security research, or even as a tool in a CTF, but that the "understanding" step needs to still be present or it destroys the fun in everything.
I'm certainly not having as much fun with computers these days, even as I've invested a lot of effort myself in local LLMs and trying to understand the tools and understand how to apply them reasonably, I've found I prefer much more analog entertainments. Thankfully there's always photography and lockpicking to entertain me at the moment and provide a pathway to meet other interesting fellows.
Same thing did happen to many work places. People at all levels proxy questions through LLMs and don't even bother to read/trim/edit the response.
Funny, how suddenly a tight, 1-2 sentence response on point is a sign of skill.
Wasn't it always?
And no just because some company gave you the title after jobs hoping multiple 1-2 year stints where you never had to deal with the consequences of your actions and decisions doesn't mean you sudenly unlock new knowledge.
He got fired quite quickly after his boss realized it was all bullshit.
Communicating well has always been a sign of skill. Talking to most devs was like pulling teeth even before LLMs.
It is insane just how miserable the experience of FOSS hardware hacking in public is.
The expectation does not even the slightest match reality. You'd think "ah yes many eyes meaning all bugs get shallow", but instead you get worst of XDA-Developers and toxic wastelands where no sane person even answers anymore.
But this predates LLMs and was as miserable as it is now before as well.
In fact, I might even argue that LLMs made this _better_, because you can now avoid the "opening up in hopes of finding 1 helpful person below 99 annoying ones" through simple GPU compute.
Previously, you had to hope that through this self-inflicted horror, you'd find an expert that can augment you. Now that expert rests on huggingface.
It's still.. not perfect, of course. But it might be less miserable - provided that you adapt to the current state of things and stop with the "trawling for volunteers".
__
Of course, this does not help you against "skids let claude find bugs that you'd need and burn it for no reason" as per the twitter thread, but I wouldn't rule out that they would not have done that, if less information was public.
And.. that too was an issue before LLMs.
Information wants to be free, but so does your cat, and your cat gets run over by some crackhead in a 30 year old rusty pickup if you just put it outside without thinking.
So don't let it go out unattended.
That said, this "pep talk" is more directed at myself than at poor Andy. This must all suck big time.
the idea of gatekeeping is often seen as elitist or exclusionary (fairly enough in many cases) but i think this can be an exaggerated description of a culture that necessarily requires a standard of... let's say, desire to engage with material on a sufficiently deep level. many things can be for anybody, but they are not for everybody.
in the past i've seen the phenomenon of "opening up" subcultures or previously-niche intellectual/art fields described as democratization but i think that ignores a big part of what causes this, which can more accurately be described as commodification. access to difficult things was formerly measured by one's ability to expend effort in order to get there (ignoring, for now, the structural conditions that set one up for success in that effort). when these things are commodified, these efforts and identities become products to purchase, the right to which is nearly never limited; in fact, it's almost seen as evil to imply that they should not be available to any individual at any time that individual may desire access.
i know we all have to start somewhere, but there has always been some historical filtration that leads people who are not truly engaged with a subject on a deep level to disengage at some point. that's changing at a novel rate and scale across so many subcultures. it seems like a lot of us are just surprised at how many people even want access to what were once our sanctuaries and we're not capable of grappling with what it takes to adapt to these changes.
i don't think this will pass or that things will return to "the way they were" but there will always be pockets that resemble previous subcultural forms. we just have to re-learn how to identify them with a keener eye and how to better avoid or more-appropriately interact with those who have a more superficial interest in them.
And once in a blue moon something good comes out of it back to the real project.
arnaudsm•45m ago
https://x.com/theflow0/status/2099987019954831744
seki285•44m ago
arnaudsm•35m ago
Vibecoding has been the norm for months, it's the embargo violation today that triggered his resignation.
lokar•33m ago
lovich•29m ago
You know, it’s petty, but I think one of the things I hate the most about AI is that it surpassed front end JavaScript frameworks in terms of changing what the standard is every few years.
What’s the point of learning anything if it becomes obsolete faster than the seasons change?
Daishiman•23m ago
sublinear•21m ago
It was the downstream effect of the browser wars settling down and W3C making big moves on getting vendors to finally implement standards that were sorely needed. It had nothing to do with web dev culture.
arnaudsm•21m ago
analognoise•3m ago
As such, to them, it’s a BENEFIT if the systems change at a rate no mortal would ever agree to. Why learn anything, after all?
sva_•33m ago
Sounds like he is salty that somebody found an exploit using LLM that he found manually (which probably took a significant amount of time.) I can partly understand it, but I mean that's also just the game that somebody else might find an exploit, LLMs just make it easier. LLMs finding bugs is not a bad thing, just sucks for enjoyers of open source software in this particular case.
You can probably find more though.
deletedie•29m ago
Qiu_Zhanxuan•24m ago
Den_VR•20m ago
tetromino_•24m ago
Nope. He is salty because someone made public the secret exploit that Linux-on-PS5 apparently relies upon to bypass Sony's hypervisor protection. Now Sony will fix the exploit, and there will be no more Linux on the PS5 until another exploit is found.
kotaKat•24m ago
But I guess Sony being a gatekeeper wasn't a gatekeeper enough for certain regulatory agencies.
softwaredoug•7m ago
I don’t know if he’s mad AI scooped a bug? Or that LLM slop kiddies don’t pay attention to that something is already known?