frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

LLMs audit code from the same blind spot they wrote it from. Here's the fix

3•brodeurmartin•1h ago
The platform I built is live in beta at FluentLogic.org, serving real families. I’m a high school teacher with a physics and philosophy background (no software engineering experience) who spent 10 months building it — roughly 350,000 lines of production TypeScript, written entirely with AI assistance. I don’t know TS from JS, but I do know assembler and C++. No matter how many times I asked the model to audit the same piece of code, I kept finding the same categories of bugs — until I forced a completely different angle. New class of bugs appeared. Then a plateau. New angle. New class. Plateau again. Before realizing this, I tried the obvious approach: firing hundreds of varied prompts, changing phrasing, and hoping coverage would emerge from volume. I spent several hundred dollars on this shotgun method. It doesn’t work. You’re simply sampling the same semantic neighborhood from slightly different entry points. Shotgun auditing is same-axis repetition with extra noise. The fix is almost embarrassingly simple: add one word to your audit prompts — “orthogonal.” Instead of: “Find bugs in this code” (or any target surface) Try: “Audit this surface from the most orthogonal direction to what you just found.” Then fix the bugs, rotate the axis, and repeat until you hit the P2 floor. The models aren’t broken. When you ask the same model that generated your code to audit it, you’re sending the auditor back into the same semantic compression manifold the generator already exhausted. Same manifold = same blind spot. I call this Generator-Auditor Symmetry (GAS). “Orthogonal” routes the model through a genuinely different neighborhood, producing non-overlapping findings consistently. What I formalized:

Confidence-Coverage Divergence (CCD): Same-axis repetition decreases output entropy (rising false certainty) while bug-class coverage stays flat. P2 Floor: When your false-positive rate crosses ~40% on two consecutive fresh-axis waves with zero new critical bugs, the surface is clean. The FP rate acts as an entropy meter. Rotation > Diversity: Rotating a single model across 3 orthogonal axes outperformed using 3 different models on the same axis.

Scale of the test: Earlier this week I ran a 36-hour marathon audit across 150+ surfaces. Yield: 60+ P0 bugs fixed and ~150 P1 bugs catalogued (e.g., OAuth sentinel bypasses, silent cache-invalidation race conditions). Each was invisible to other probe axes. The web app now feels the snappiest it’s ever been. Same-axis repetition plateaus at ~20% bug-class discovery yield, while orthogonal rotation reaches ~80% — a 4–5× advantage. I took the full 350K-line codebase to systemic P2 floor. The app is perceptibly faster afterward. I wrote a short paper formalizing the method and the supporting topological observations. To verify this wasn’t just a prompting trick, I ran persistent homology (Vietoris-Rips on Gemini semantic embeddings of 58 production bug classes). It revealed 20 significant β₁ interior loops — evidence that the bug classes form geometric structure in semantic space that same-axis probing structurally cannot exhaust. Preprint (Zenodo): https://doi.org/10.5281/zenodo.19223166 This is a single real-world codebase, not a controlled experiment. The survival curves are strong evidence, not final proof. What I’m genuinely curious about:

Has anyone else seen meaningfully better LLM bug detection by rotating audit axes? Does Confidence-Coverage Divergence (CCD) appear in LLM evaluation loops (RLHF, Constitutional AI)? What does the survival curve look like on a codebase you didn’t build yourself?

(19-year Ontario teacher | M.A., B.A. Philosophy · B.Sc. Physics. Built this for real families.)

Comments

chunpaiyang•1h ago
I'm a software engineer building an app as a side project. I quickly realized AI bullshits a lot.

But you know, engineers bullshit each other all the time too. The difference is we have a way to verify it - logical chains. You have to build an argument that holds up before anayone buys in it.

So I though, can I make AI build its own logical chain ? Let it pass its own logic check before telling me the result.

That's how I created my own "think" skill. It's based on Meta's CoT paper: https://arxiv.org/abs/2501.04682

It roughly works like this: 1. FRAME - Challenge the question itself, hidden assumptions.

2. GROUND - Map what you know, what you need, what's missing.

3. ASSOCIATE - Launch multiple independent agents in parallel to generate hypotheses, avoid anchoring bias.

4. VERIFY - Break each hypothesis into atomic claims, verify each independently

5. CHAIN - Build a logical chain from survivors

6. PROVE and LOOP - Walk backwards from conclusion to premises, seearch for evidence, repair if broken

7. DELIVER - Start with "I was wrong if ...."

It helps me a lot. Whenever I need to check if Claude Opus 4.6 is bullshitting me. I just say "/think verify the above reasoning is correct" or "/think verify the above fix is correct and complete."

formrecap•1h ago
The concept of blind spots in same-model auditing is sound, but I'm skeptical that just adding "orthogonal" to a prompt solves it. Which axis was the model using before? Which should it use next? Without knowing that, you're just hoping for variety.

What actually works in my experience is two things:

First, prompting with specific personas. "You are a security auditor looking for multi-tenant isolation failures" unlocks genuinely different reasoning from "review this code." The lens matters more than the word "orthogonal" — it gives the model a concrete perspective to reason from.

Second — and I think this gets overlooked — anchoring AI review in deterministic tooling. Semgrep, ESLint, dependency audits. These tools have been catching bugs reliably for years. A model asked to "review this code" will always find something — they're trained to be helpful, I've never had one say "nope, it's perfect." But pairing that with deterministic tools gives you consistency and catches the things models miss by construction.

It's not really new. It's just working with AI agents the way you'd work with another team member — while knowing their limitations (like regurgitating semantically similar ideas when asked the same question twice).

Caveman Mode Save Token?

https://twitter.com/om_patel5/status/2040279104885314001
1•brightball•2m ago•2 comments

My 11-step GraphRAG pipeline, what worked, and what's still broken

1•pauliusztin•4m ago•0 comments

A diary of an agentic retro-gamer – Part 1

https://keanw.com/2026/03/a-diary-of-an-agentic-retro-gamer-part-1.html
1•throwaway_2494•5m ago•0 comments

Absurd In Production

https://lucumr.pocoo.org/2026/4/4/absurd-in-production/
1•ingve•5m ago•0 comments

Post-quantum blockchain QRL has passed its fourth audit

https://www.theqrl.org/press/halborn-audit-validates-qrls-postquantum-cryptography-library/
1•munrocket•5m ago•0 comments

Trying for 1 month but can't learn pixel art still

3•limondas•6m ago•0 comments

Ask HN: What Are You Working On? (April 2026)

2•subdomain•6m ago•0 comments

AI Is Rewiring India's Film Industry

https://www.reuters.com/technology/ai-is-rewiring-worlds-most-prolific-film-industry-2026-04-04/
2•alephnerd•7m ago•0 comments

The ICEBlock App Has Helped People Avoid Immigration Agents. Is It Legal?

https://www.newyorker.com/news/the-lede/the-rise-and-fall-of-ice-tracking-apps
2•janandonly•9m ago•0 comments

An AI agent called every pub in Ireland to index the cost of a Guinness

https://guinndex.ai
2•sarusso•9m ago•0 comments

Show HN: Hacker News RSS Feed Directory

https://huggingface.co/datasets/open-index/hacker-news-rss
2•tamnd•10m ago•0 comments

Show HN: Content Negotiation in PHP – API Without API (Symfony, Laravel, Temma)

https://old.reddit.com/r/PHP/comments/1sc8ste/content_negotiation_in_php_your_website_is/
2•amaury_bouchard•13m ago•0 comments

DeepSeek's V4 model will run on Huawei chips

https://www.reuters.com/world/china/deepseeks-v4-model-will-run-huawei-chips-information-reports-...
2•wg0•17m ago•0 comments

Activating Two Trap Cards at Once

https://gist.github.com/MostAwesomeDude/560185c24f959f6fec229739cb5a6735
2•mpweiher•17m ago•0 comments

The science behind Japan's perfectly crafted vending machine drinks

https://monocle.com/design/at-your-convenience-2/
1•austinallegro•17m ago•0 comments

Impact of screen size on cognitive training task performance: An HMD study

https://www.sciencedirect.com/science/article/abs/pii/S0167876021001835
2•FrojoS•20m ago•0 comments

Tab Cemetery – the first graveyard for browser tabs

https://chromewebstore.google.com/detail/tab-cemetery/mdnkgcefilpflenkmfoghookblcnleea
2•Patlakh•24m ago•1 comments

YouTube's auto-dubbing rewrites meaning of sentences

https://blog.unstacked.cc/posts/when-translation-becomes-interpretation/
2•unstacked•29m ago•2 comments

Fooling Go's X.509 Certificate Verification

https://danielmangum.com/posts/fooling-go-x509-certificate-verification/
2•hasheddan•31m ago•0 comments

A visual guide to Iran's coastline and strategic islands

https://interactive.aljazeera.com/aje/2026/iran-coastline-islands/
2•giuliomagnifico•32m ago•0 comments

Senators Tell Americans That VPN Use Might Subject Them to Domestic Surveillance

https://www.techdirt.com/2026/04/03/senators-ask-tulsi-gabbard-to-tell-americans-that-vpn-use-mig...
3•redeux•33m ago•0 comments

Components of a Coding Agent

https://magazine.sebastianraschka.com/p/components-of-a-coding-agent
2•MindGods•38m ago•0 comments

Putting Intelligence to Work

https://vivekkaushal.substack.com/p/putting-intelligence-to-work
2•kaushalvivek•39m ago•0 comments

Vitamin D deficiency can lead to autoimmune diseases (2024)

https://medicalxpress.com/news/2024-10-vitamin-d-deficiency-autoimmune-diseases.html
1•OutOfHere•39m ago•2 comments

Using Perfetto in ZJIT

https://railsatscale.com/2026-03-27-using-perfetto-in-zjit/
1•gbourne1•39m ago•0 comments

"The internet could go down if Brundage spent too much time on his exams"

https://www.wsj.com/tech/kimwolf-hack-residential-proxy-networks-a712ab59
2•hank1931•42m ago•0 comments

The New Twist Browser for Devs

https://www.youtube.com/watch?v=fkrsH79HyKw
1•cintronsoftware•43m ago•0 comments

Slap: Functional Concatenative Language with a Borrow Checker?

https://taylor.town/slap-000
1•birdculture•46m ago•0 comments

Show HN: Batty – Run a team of AI coding agents in tmux with test gating

https://github.com/battysh/batty
1•Zedmor•52m ago•0 comments

Awesome

https://archive.openrockets.com/Litha2024-main/
1•techcast•53m ago•0 comments