I actually expected the models to perform better than they did, but I'm a bit of an optimist. It makes sense for the more mature SAST tool to outperform them just based on maturity alone.

This makes sense. Most SAST tools have years of engineering behind them specifically for static analysis, while LLMs are general-purpose models trying to approximate reasoning from patterns.

The interesting question isn’t whether LLMs outperform SAST today, but whether they can complement them — for example, identifying logic-level issues, insecure design patterns, or unusual edge cases that rule-based tools might miss.

It feels like the future is hybrid: deterministic scanners for known classes of vulnerabilities, and LLMs for higher-level semantic and architectural analysis.