Oops, sorry - orig was double pasted and it won't let me edit :/

Yes link not working. Need deep dive to know more on Mythos.

According to our software (which is, of course, imperfect), your account has repeatedly been posting AI-generated and/or AI-edited comments. If so, can you please stop? It's not allowed here, and will eventually get your account banned.

(See https://news.ycombinator.com/newsguidelines.html#generated and https://news.ycombinator.com/item?id=47340079.)

As long as it shifts the zero-sum game in the favor of the defender, it is improving things.

I disagree. Assuming code complexity is roughly fixed, more sophisticated code analysis will result in a smaller surface area for bugs. Bugs will still be found, but there will be less bugs to be found and less opportunities to exploit.

We know about physical-layer attacks that break some of the abstractions that software relies on, allowing an attacker to use physical access or physical proximity to violate security guarantees that are enforced by software alone. (I worked on some of these a while ago!)

For a purely remote attacker (although maybe we have to get clear on what distance counts as "physical proximity" because we need to clarify what phenomena spy satellites, for example, can observe), it seems pretty straightforward to me that there is such a thing as actually secure software.

You can make a very strong model of what the software computes and then prove that it never does some undesired thing. It's not common to do this at all, and even formal verification work may not use very strong models or models that capture some important part of the behavior, but it is possible to mathematically reason about what software does and doesn't or can and can't do.

To summarize some of the problems that I partly just mentioned (in no particular order)

(1) We may not have the will, the skill, or the economic demand to make software secure in a very strong sense.

(2) Attackers may subvert our infrastructure or organizations so that we don't actually apply the processes or controls, or run the software, that we expect.

(3) Physical proximity (for active or passive attacks) might sometimes include distances that are actually attainable for attackers. Maybe there are passive or active attacks involving lasers that can be mounted from multiple kilometers away, as an example. In that case most software users might not be able to be sufficiently isolated from the attackers to be protected against those attacks.

(4) Software or hardware other than the specific software whose security we're talking about might be compromised in its supply chain in a way that people don't have a plan, or resources, to detect or mitigate.

(5) Some systems might be compositionally insecure (their pieces might be secure in some relevant model, but the pieces might interact in a way that isn't secure overall, for example related to timing and concurrency problems).

(6) Our proofs of security for cryptosystems rely on unproven hardness assumptions for various primitives, some of which might turn out to be wrong.

(7) Some security properties, especially related to communications security, might be inherently unattainable even with correct software. For example, there's an argument that Roger Dingledine (Tor lead developer) once told me about that implies that no anonymity system is perfectly secure in the long run against a very powerful active adversary, unless the system is willing to make extreme trade-offs like shutting down completely in response to any attack. So it might be that we can't actually build any useful communications system that can absolutely guarantee perfect traffic analysis resistance, essentially because of inherent architectural trade-offs.

But I don't want to lose sight of the idea that you can actually meaningfully reason about what software does and so there is such a thing as the software being correct or incorrect, relative to some specification or goal for its behavior, and correct software actually does exist (which computes correct outputs for every input).

to say that defense doesn't win in the limit is the same thing as saying there is an attack that can not be defended against.

So to re-phase the question to more clearly have an answer: does there exist an attack which no one will ever be able (for all time) to come up with a defense against? (the very existence of such an attack would end the (open) internet, wholly and completely, if the only winning move is not to play...)

There will be an exhaustion of possibilities in the end. New attacks eventually run out after each surface area is hardened against those attacks.

In the limit, defense wins.

There is only one case (that i see) where this may fail. if there is a 'predicament' with the state of security: ie, if securing against attack A requires you to be insecure against attack B and vise versa (this could be a 'whack-a-mole with many different kinds of attacks' situation). But that would be 'provable'. So if such a case exists, we will know about it. And it may be true that predicaments like this could be exercised if they even can exist, we might still be able to avoid/mitigate them.

So large bets on defense winning in the end.

Because having humans in the loop slows things down, much faster if the attacker can break into the system directly.

We are replacing those with AI agents anyways. It'll be AI agents all the way down!

@deadbabe 100%

Code vulnerabilities can be attacked industrially at scale by smaller groups.

Social engineering requires a lot more organisation from attackers.

Did you just assume every hacker has all the source code in the world?

I don't see that it makes much difference until we know the distribution of issues that Mythos finds and how reliably it discovers them? Vulns from inspection are discovered via a stochastic process of someone looking at the code, knowing about bug classes and paying sufficient attention to notice them. That's still the case.

IMHO the main thing thats interesting about AI assisted bug hunting is that it changes the balance of power from people who had a lot of free time & attention to the state and big business, who have money and frontier model access.

Mythos hacked the site, wrote, and published the article

IDK, 12 em dashes?

... or it riffs with your nick and rhymes with stunt.

If it is "vapor" then how are the various mega corps able to use the preview release? Do you think they are all in on some giant conspiracy?

Maybe, but they’ve got everyone scared shitless. My entire org (30k employees, not just engineers) is in sprint 2 of a remediation effort, where we are systematically fixing every high+ finding across hundreds of workloads with decades of system bloat.

I’ve never seen us so aligned on a goal! Wiz is doing pretty well for itself also…

It's not vapor if people actually have access to it, which they do.

Copy Fail was found using AI and made our last week miserable (thanks RedHat). Anyone with access to a SOTA model can point it at any package in your stack and go fish. You're underreacting.