Curl: We still have not seen a single valid security report done with AI help

https://www.linkedin.com/posts/danielstenberg_hackerone-curl-activity-7324820893862363136-glb1

187•indigodaddy•2h ago

Comments

jacksnipe•1h ago

Something that really frustrates me about interacting with (some) people who use AI a lot is that they will often tell me things that start “I asked ChatGPT and it said…” stop it!!! If the chatbot taught you something and you understood it, explain it to me. If you didn’t understand or didn’t trust it, then keep it to yourself!

x3n0ph3n3•1h ago

Thanks for this. It's a great response I intend to use going forward.

esafak•1h ago

I had to deal with someone who tried to check in hallucinated code with the defense "I checked it with chatGPT!"

If you're just parroting what you read, what is it that you do here?!

giantg2•1h ago

Manage people?

tough•1h ago

then what the fuck are they doing commiting code? leave that to the coders

giantg2•55m ago

That sounds good, but not might be how it works in Chapter Lead models.

hashmush•1h ago

As much as I'm also annoyed by that phrase, is it really any different from:

- I had to Google it...

- According to a StackOverflow answer...

- Person X told me about this nice trick...

- etc.

Stating your sources should surely not be a bad thing, no?

nraynaud•1h ago

the first 2 bullet points give you an array of answers/comments helping you cross check (also I'm a freak, and even on SO, I generally click on the posted documentation links).

spiffyk•1h ago

Well, it is not, but the three "sources" you mention are not worth much either, much like ChatGPT.

gruez•1h ago

>but the three "sources" you mention are not worth much either, much like ChatGPT.

I don't think I've ever seen anyone lambasted for citing stackoverflow as a source. At best, they chastised for not reading the comments, but nowhere as much pushback as for LLMs.

comex•1h ago

From what I’ve seen, Stack Overflow answers are much more reliable than LLMs.

Also, using Stack Overflow correctly requires more critical thinking. You have to determine whether any given question-and-answer is actually relevant to your problem, rather than just pasting in your code and seeing what the LLM says. Requiring more work is not inherently a good thing, but it does mean that if you’re citing Stack Overflow, you probably have a somewhat better understanding of whatever you’re citing it for than if you cited an LLM.

spiffyk•1h ago

I have personally always been kind of against using StackOverflow as a sole source for things. It is very often a good pointer, but it's always a good idea to cross-check with primary sources. Otherwise you get all sorts of interesting surprises, like that Razer Synapse + Docker for Windows debacle. Not to mention that you are technically not allowed to just copy-paste stuff from SO.

mynameisvlad•1h ago

I mean, if all they did is regurgitate a SO post wholesale without checking the correctness or applicability, and the answer was in fact not correct or applicable, they would probably get equally lambasted.

If anything, SO having verified answers helps its credibility slightly compared to a LLM which are all known to regularly hallucinate (see: literally this post).

bloppe•1h ago

SO at least has reputation scores and people vote on answers. An answer with 5000 upvotes, written by someone with high karma, is probably legit.

dpoloncsak•50m ago

...isn't that exactly why someone states that?

"Hey, I didn't study this, I found it on Google. Take it with a grain of caution, as it came from the internet" has been shortened to "I googled it and...", which is now evolving to "Hey, I asked chatGPT, and...."

hx8•1h ago

It depends on if they are just repeating things without understanding, or if they have understanding. My issue is that people that say "I asked gpt" is that they often do not have any understanding themselves.

Copy and pasting from ChatGPT has the same consequences as copying and pasting from StackOverflow, which is to say you're now on the hook supporting code in production that you don't understand.

tough•1h ago

We cannot blame the tools for how they are used by those yielding them.

I can use ChatGPT to teach me and understand a topic or i can use it to give me an answer and not double check and just copy paste.

Just shows off how much you care about the topic at hand, no?

multjoy•1h ago

How do you know that ChatGPT is teaching you about the topic? It doesn't know what is right or what is wrong.

tough•53m ago

It can consult any sources about any topic, ChatGPT is as good at teaching as the pupil's capabilities to ask the right questions, if you ask me

multjoy•33m ago

It may well consult any source about the topic, or it may simply make something up.

If you don't know anything about the subject area, how do you know if you are asking the right questions?

theamk•56m ago

If you used ChatGPT to teach you the topic, you'd write your own words.

Starting the answer with "I asked ChatGPT and it said..." almost 100% means the poster did not double-check.

(This is the same with other systems: If you say, "According to Google...", then you are admitting you don't know much about this topic. This can occasionally be useful, but most of the time it's just annoying...)

misnome•53m ago

We can absolutely blame the people selling and marketing those tools.

tough•49m ago

Yeah, marketing always seemed to me like a misnomer or doublespeak for legal lies.

All marketing departments are trying to manipulate you to buy their thing, it should be illegal.

But just testing out this new stuff and seeing what's useful for you (or not) is usually the way

jacksnipe•45m ago

I see nobody here blaming tools and not people!

layer8•25m ago

This subthread was about blaming people, not the tool.

stonemetal12•57m ago

In general those point to the person's understanding being shallow. So far when someone says "GPT said..." it is a new low in understanding, and there is no more to the article they googled or second stackOverflow answer with a different take on it, it is the end of the conversation.

mentalpiracy•24m ago

It is not about stating a source, the bad thing is treating chatGPT as an authoritative source like it is a subject matter expert.

rhizome•19m ago

All three of those should be followed by "...and I checked it to see if it was a sufficient solution to X..." or words to that effect.

yoyohello13•1h ago

Seriously. Being able to look up stuff using AI is not unique. I can do that too.

This is kind of the same with any AI gen art. Like I can go generate a bunch of cool images with AI too, why should I give a shit about your random Midjourney output.

h4ck_th3_pl4n3t•1h ago

How can you be so harsh on all the new kids with Senior Prompt Engineer in their job titles?

They have to prove to someone that they're worth their money. /s

alwa•14m ago

I mean… I have a fancy phone camera in my pocket too, but there are photographers who, with the same model of fancy phone camera, do things that awe and move me.

It took a solid hundred years to legitimate photography as an artistic medium, right? To the extent that the controversy still isn’t entirely dead?

Any cool images I ask AI for are going to involve a lot less patience and refinement than some of these things the kids are using AI to turn out…

For that matter, I’ve watched friends try to ask for factual information from LLMs and found myself screaming inwardly at how vague and counterproductive their style of questioning was. They can’t figure out why I get results I find useful while they get back a wall of hedging and waffling.

evandrofisico•55m ago

It is supremely annoying when i ask in a group if someone has experience with a tool or system and some idiot copies my question into some LLM and paste the answer. I can use the LLM just like anyone, if i'm asking for EXPERIENCE it is because I want the opinion of a human who actually had to deal with stuff like corner cases.

jsheard•52m ago

If it's not worth writing, it's not worth reading.

pixl97•51m ago

I mean, there is a lot of hand written crap to, so even that isn't a good rule.

mcny•43m ago

It is a necessary but not sufficient condition, perhaps?

colecut•36m ago

That rule does not imply the inverse

pixl97•12m ago

I mean we have automated systems that 'write' things like tornado warnings. Would you rather we have someone hand write that out?

It seems the initial rule seems rather worthless.

layer8•28m ago

That sounds like https://en.wikipedia.org/wiki/Denying_the_antecedent.

leptons•5m ago

>I mean, there is a lot of hand written crap to

You know how I know the difference between something an AI wrote and something a human wrote? The AI knows the difference between "to" and "too".

I guess you proved your point.

ModernMech•50m ago

It's the 2025 version of lmgtfy.

layer8•43m ago

Nah, that’s different. Lmgtfy has nothing to do with experience, other than experience in googling. Lmgtfy applies to stuff that can expediently be googled.

ModernMech•40m ago

In my experience, usually what people had done was take your question on a forum, go to lmgtfy, paste the exact words in and then link back to it. As if to say "See how easy that was? Why are you asking us when you could have just done that?"

Yes is true there could have been a skill issue. But it could also be true that the person just wanted input from people rather than Google. So that's why I drew the connection.

layer8•37m ago

I largely agree with your description, and I think that’s different from the above case of explicitly asking for experience and then someone posing the question to an LLM. Also, when googling, you typically (used to) get information written down by people, from a much larger pool and better curated via page ranking, than whoever you are asking. So it’s not like you were getting better quality by not googling, typically.

ModernMech•26m ago

That's why I said it's the 2025 version of that, given the new technology. I'm not saying it's the same thing. I guess I'm not being clear, sorry.

layer8•11m ago

It’s not clear to me in what way it is a version of that, other than the response being different from what the asker wanted. The point of lmgtfy is to show that the asker could have legitimately and reasonably easily have found the answer by himself. You can argue that it is sometimes done on cases where googling actually wouldn’t provide the desired information, but that is far from the common case. This present version is substantially different from that. It is invariably true that an LLM response won’t give you the awareness and judgement of someone with experience in a certain topic.

jacksnipe•36m ago

That’s exactly how I feel

soulofmischief•9m ago

The whole point of paying a domain expert is so that you don't have to google shit all day.

cogman10•50m ago

I recently had this happen from a senior engineer. What's really frustrating is I TOLD them the issues and how to fix it. Instead of listening to what I told them, they plugged it into GPT and responded with "Oh, interesting this is what GPT says" (Which, spoiler, was similar but lacking from what I'd said).

Meaning, instead of listening to a real-life expert in the company telling them how to handle the problem they ignored my advice and instead dumped the garbage from GPT.

I really fear that a number of engineers are going to us GPT to avoid thinking. They view it as a shortcut to problem solve and it isn't.

delusional•49m ago

Those people weren't engineers to start with.

layer8•46m ago

Software engineers rarely are.

I’m saying this tongue in cheek, but there’s some truth to it.

colechristensen•31m ago

If I had a dollar for every time I told someone how to fix something and they did something else...

Let's just say not listening to someone and then complaining that doing something else didn't work isn't exactly new.

colechristensen•22m ago

>They view it as a shortcut to problem solve and it isn't

Oh but it is, used wisely.

One: it's a replacement for googling a problem and much faster. Instead of spending half an hour or half a day digging through bug reports, forum posts, and stack overflow for the solution to a problem. LLMs are a lot faster, occasionally correct, and very often at least rather close.

Two: it's a replacement for learning how to do something I don't want to learn how to do. Case Study: I have to create a decent-enough looking static error page for a website. I could do an awful job with my existing knowledge, I could spend half a day relearning and tweaking CSS, elements, etc. etc. or I could ask an LLM to do it and then tweak the results. Five minutes for "good enough" and it really is.

LLMs are not a replacement for real understanding, for digging into a codebase to really get to the core of a problem, or for becoming an expert in something, but in many cases I do not want to, and moreover it is a poor use of my time. Plenty of things are not my core competence or anywhere near the goals I'm trying to achieve. I just need a quick solution for a topic I'm not interested in.

candiddevmike•48m ago

This happens to me all the time at work. People have turned into frontends for LLM, even when it's their job to know the answer to these types of questions. We're talking technical leads.

Seems like if all you do is forward questions to LLMs, maybe you CAN be replaced by a LLM.

mrkurt•43m ago

Wow that's a wildly cynical interpretation of what someone is saying. Maybe it's right, but I think it's equally likely that people are saying that to give you the right context.

If they're saying it to you, why wouldn't you assume they understand and trust what they came up with?

Do you need people to start with "I understand and believe and trust what I'm about to show you ..."?

jacksnipe•33m ago

I do not need people to lead on that. That’s precisely why leading on “I asked ChatGPT and it said…” makes me trust something less — the speaker is actively assigning responsibility for what’s to come to some other agent, because for one reason or another, they won’t take it on themselves.

laweijfmvo•29m ago

the problem is that when you ask a ChatBot something, it always gives you an answer...

JohnFen•9m ago

I agree wholeheartedly.

"I asked X and it said..." is an appeal to authority and suspect on its face whether or not X is an LLM. But when it's an LLM, then it's even worse. Presumably, the reason for the appeal is because the person using it considers the LLM to be an authoritative or meaningful source. That makes me question the competence of the person saying it.

unsnap_biceps•1h ago

For those of you who don't want to click into linked in, https://hackerone.com/reports/3125832 is the latest example of a invalid curl report

parliament32•1h ago

Didn't even have to click through to the report in question to know it would be all hallucinations -- both the original patchfile and the segfault ("ngtcp2_http3_handle_priority_frame".. "There is no function named like this in current ngtcp2 or nghttp3.") I guess these guys don't bother to verify, they just blast out AI slop and hope one of them hits?

indigodaddy•1h ago

Reminds me of when some LLM (might have been Deepseek) told me I could add wasm_mode=True in my FastHTML python code which would allow me to compile it to WebAssembly, when of course there is no such feature in FastHTML. This was even when I had provided it full llms-ctx.txt

alabastervlog•1h ago

I had Google's in-search "AI" invent a command line switch that would have been very helpful... if it existed. Complete with usage caveats and warnings!

This was like two weeks ago. These things suck.

spiffyk•1h ago

> I guess these guys don't bother to verify, they just blast out AI slop and hope one of them hits?

Yes. Unfortunately, some companies seem to pay out the bug bounty without even verifying that the report is actually valid. This can be seen on the "reporter"'s profile: https://hackerone.com/evilginx

pixl97•41m ago

>"ngtcp2_http3_handle_priority_frame"

I wonder if you could use AI to classify the probability factor that something is AI bullshit and deprioritize it?

soraminazuki•12m ago

Considering that even the reporter responded to requests for clarification with yet another AI slop, they likely lack the technical background.

hx8•1h ago

It's probably a net positive that ChatGPT isn't going around detecting zero day vulnerabilities. We should really be saving those for the state actors to find.

vessenes•1h ago

Reading the straw that broke the camel's back commit illustrates the problem really well: https://hackerone.com/reports/3125832 . This shit must be infuriating to dig through.

I wonder if reputation systems might work here - you could give anyone who id's with an AML/KYC provider some reputation, enough for two or three reports, let people earn reputation digging through zero rep submissions and give someone like 10,000 reputation for each accurate vulnerability found, and 100s for any accurate promoted vulnerabilities. This would let people interact anonymously if they want to edit, quickly if they found something important and are willing to AML/KYC, and privilege quality people.

Either way, AI is definitely changing economics of this stuff, in this case enshittifying first.

emushack•58m ago

Reputation systems for this kind of thing sounds like rubbing some anti-itch cream on bullet wound. I feel like the problem seems to me to be behavior, not a technology issue.

Personally I can't imagine how miserable it would be for my hard-earned expertise to be relegated to sifting through SLOP where maybe 1 in hundreds or even thousands of inquiries is worth any time at all. But it also doesn't seem prudent to just ignore them.

I don't think better ML/AI technology or better information systems will make a significant difference on this issue. It's fundamentally about trust in people.

delusional•45m ago

I consider myself a left leaning soyboy, but this could be the outcome of too "nice" of a discourse. I won't advocate for toxicity, but I am considering if we bolster the self-image of idiots when we refuse to call them idiots. Because you're right, this is fundamentally a people problem, specifically we need people to filter this themselves.

I don't know where the limit would go.

orthecreedence•15m ago

Shame is a useful social tool. It can be overused or underused, but it's still a tool and people like this should be made to publicly answer for their obnoxious and destructive behavior.

squigz•27m ago

I guess I'm confused by your position here.

> I feel like the problem seems to me to be behavior, not a technology issue.

Yes, it's a behavior issue, but that doesn't mean it can't be solved or at least minimized by technology, particularly as a technology is what's exacerbating the issue?

> It's fundamentally about trust in people.

Who is lacking trust in who here?

bflesch•56m ago

there is a reputation system already. according to hackerone reputation system, it is a credible reporter. it's really bad

uludag•1h ago

I can imagine that most LLMs, if you ask it to find a security vulnerability in a given piece of code, will make something up completely out of the air. I've (mistakenly) sent valid code with an unrelated error and to this day I get nonsense "fixes" for these errors.

This alignment problem between responding with what the user wants (e.g. a security report, flattering responses) and going against the user seems a major problem limiting the effectiveness of such systems.

rdtsc•48m ago

> evilginx updated the severity from none to high

Well the reporter in the report that stated it that they are open for employment https://hackerone.com/reports/3125832 Anyone want to hire them? They can play with ChatGPT all day and spam random projects with the AI slop.

gorbachev•8m ago

Growth hack: hire this person to find vulnerabilities in competitors' products.

bogwog•43m ago

If I wanted to slip a vulnerability into a major open source project with a lot of eyes on it, using AI to DDOS their vulnerability reports so they're less likely to find a real report from someone who caught me seems like an obvious (and easy) step.

Looking at one of the bogus reports, it doesn't even seem like a real person. Why do this if you're not trying to gain recognition?

jsheard•16m ago

> Why do this if you're not trying to gain recognition?

They're doing it for money, a handful of their reports did result in payouts. Those reports aren't public though so there's no way to know if they actually found real bugs, or the reviewer rubber stamped them without doing their due diligence.

zulban•17m ago

Shame they need to put up with that spam. However, every big open source project has by now had good contributions with "AI help". Many millions of developers are using AI a little as a tool, like Google.

molticrystal•11m ago

There is or at various times was, nitter for twitter, Invidious for youtube, Imginn for instagram, and even many variations of ones for hackernews like hckrnews.com & ones that are lighter, work better in terminals, etc.

Anything for linkedin, a light interface that doesn't required logging in?

I pretty much stopped going to linkedin years ago because they started aggressively directing a person to login. I was shocked this post works without login. I don't know if that is how it has always been, or if that is a recent change, or what. It would be nice to have alternative interfaces.

In case some people are getting gated here is their post:

===

Daniel Stenberg curl CEO. Code Emitting Organism

That's it. I've had it. I'm putting my foot down on this craziness.

1. Every reporter submitting security reports on #Hackerone for #curl now needs to answer this question:

"Did you use an AI to find the problem or generate this submission?"

(and if they do select it, they can expect a stream of proof of actual intelligence follow-up questions)

2. We now ban every reporter INSTANTLY who submits reports we deem AI slop. A threshold has been reached. We are effectively being DDoSed. If we could, we would charge them for this waste of our time.

We still have not seen a single valid security report done with AI help.

---

This is the latest one that really pushed me over the limit: https://hackerone.com/reports/3125832

===

perching_aix•3m ago

> Anything for linkedin, a light interface that doesn't required logging in?

I just opened the site with JS off on mobile. No issues.

ianbutler•8m ago

Counterpoint we have a CVE attributable to ours and I suspect the difference is my co-founder was an offensive kernel researcher so our system is tuned for this in a way your average...ambulance chaser is unable to do.

https://blog.bismuth.sh/blog/bismuth-found-the-atop-bug

https://www.cve.org/CVERecord?id=CVE-2025-31160

The amount of bad reports curl in particular has gotten is staggering and it's all from people who have no background just latching onto a tool that won't elevate them.

danielvf•6m ago

I handle reports for a one million dollar bug bounty program.

AI spam is bad. We've also never had a valid report from an by an LLM (that we could tell).

People using them will take any being told why a bug report is not valid, questions, or asks for clarification and run them back through the same confused LLM. The second pass through generates even deeper nonsense.

It's making even responding with anything but "closed as spam" not worth the time.

I believe that one day there will be great code examining security tools. But people believe in their hearts that that day is today, and that they are riding the backs of fire breathing hack dragons. It's the people that concern me. They cannot tell the difference between truth and garbage.

Node v24.0.0

Family uses AI to create video for deadly victim's own impact statement

Will Supercapacitors Come to AI's Rescue?

The Agent Maturity Model – How far we are from human-like Agents and a roadmap

Deepfakes Now Outsmarting Detection by Mimicking Heartbeats

To Speed Up AI, Just Outsource Memory

Spain to reduce the standard 40-hour work week for 12.5 million employees

Single, or not? Japanese dating app launches relationship verification system

Show HN: An agent to run your newsletter

Ukraine or the Ukraine: Why do some country names have 'the'? (2012)

Controllers Briefly Lost Contact with Planes at Newark Last Week

TypeTalk: Kerning Principles (2022)

Trump administration cuts off all future federal funding to Harvard

AI-Enhanced Social Engineering Will Reshape the Cyber Threat Landscape

Google's Waymo ramps up U.S. robotaxi production

How I cut GTA Online loading times by 70% (2021)

Stad Ship Tunnel

Performance and Energy Usage of Compiled Python

Physicists Develop Novel Quantum Theory of Gravity

Vim for Visual Studio Code

Ask HN: Is vibe coding overhyped or am I missing something?

Common Lisp disassembly through SBCL on RISC-V architecture

Meta Pixels Case Dismissed by Second Circuit–Solomon vs. Triller

Instagram Urged 'Groomers' to Connect with Minors

Gene Switch Reboots Sight and Sound: A Breakthrough in Sensory Regeneration

Helping the AI Industry Secure Unreleased Models Is a National Security Priority

Show HN: Korey – a product management agent for software teams

A Cool HTTP Feature for Simple Real-Time Updates

Tnote – command line note taking app

Show HN: Debate Uncle Bob – Is SQL Dead? (Voice RPG)

Node v24.0.0

Family uses AI to create video for deadly victim's own impact statement

Will Supercapacitors Come to AI's Rescue?

The Agent Maturity Model – How far we are from human-like Agents and a roadmap

Deepfakes Now Outsmarting Detection by Mimicking Heartbeats

To Speed Up AI, Just Outsource Memory

Spain to reduce the standard 40-hour work week for 12.5 million employees

Single, or not? Japanese dating app launches relationship verification system

Show HN: An agent to run your newsletter

Ukraine or the Ukraine: Why do some country names have 'the'? (2012)

Controllers Briefly Lost Contact with Planes at Newark Last Week

TypeTalk: Kerning Principles (2022)

Trump administration cuts off all future federal funding to Harvard

AI-Enhanced Social Engineering Will Reshape the Cyber Threat Landscape

Google's Waymo ramps up U.S. robotaxi production

How I cut GTA Online loading times by 70% (2021)

Stad Ship Tunnel

Performance and Energy Usage of Compiled Python

Physicists Develop Novel Quantum Theory of Gravity

Vim for Visual Studio Code

Ask HN: Is vibe coding overhyped or am I missing something?

Common Lisp disassembly through SBCL on RISC-V architecture

Meta Pixels Case Dismissed by Second Circuit–Solomon vs. Triller

Instagram Urged 'Groomers' to Connect with Minors

Gene Switch Reboots Sight and Sound: A Breakthrough in Sensory Regeneration

Helping the AI Industry Secure Unreleased Models Is a National Security Priority

Show HN: Korey – a product management agent for software teams

A Cool HTTP Feature for Simple Real-Time Updates

Tnote – command line note taking app

Show HN: Debate Uncle Bob – Is SQL Dead? (Voice RPG)

Curl: We still have not seen a single valid security report done with AI help

Comments