frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

Ask HN: Are diffs still useful for AI-assisted code changes?

4•nuky•8h ago
I’m wondering whether traditional diffs are becoming less suitable for AI-assisted development..

Lately I’ve been feeling frustrated during reviews when an AI generates a large number of changes. Even if the diff is "small", it can be very hard to understand what actually changed in behavior or structure.

I started experimenting with a different approach: comparing two snapshots of the code (baseline and current) instead of raw line diffs. Each snapshot captures a rough API shape and a behavior signal derived from the AST. The goal isn’t deep semantic analysis, but something fast that can signal whether anything meaningful actually changed.

It’s intentionally shallow and non-judgmental — just signals, not verdicts.

At the same time, I see more and more LLM-based tools helping with PR reviews. Probabilistic changes reviewed by probabilistic tools feels a bit dangerous to me.

Curious how others here think about this: – Do diffs still work well for AI-generated changes? – How do you review large AI-assisted refactors today?

Comments

nuky•8h ago
Just to clarify - this isn’t about replacing diffs or selling a tool

I ran into this problem while reviewing AI-gen refactors and started thinking about whether we’re still reviewing the right things. Mostly curious how others approach this.

DiabloD3•5h ago
You know there are other kinds of diffs, right?

Its common to change git's diff to things like difftastic, so formatting slop doesn't trigger false diff lines.

You're probably better off, FWIW, just avoiding LLMs. LLMs cannot produce working code, and they're the wrong tool for this. They're just predicting tokens around other tokens, they do not ascribe meaning to them, just statistical likelihood.

LLM weights themselves would be far more useful if we used them to indicate statistical likelihood (ie, perplexity) of the code that has been written; ie, strange looking code is likely to be buggy, but nobody has written this tool yet.

nuky•4h ago
Yeah difftastic and similar tools help a lot with formatting noise really.

My question is slightly orthogonal though: even with a cleaner diff, I still find it hard to quickly tell whether public API or behavior changed, or whether logic just moved around.

Not really about LLMs as reviewers — more about whether there are useful deterministic signals above line-level diff.

nuky•4h ago
It was precisely because this was going too far that I thought the consequences of the active adoption of LLM tools could be made visible. I'm not saying LLM is completely bad—after all, and not all tools, even non-LLM ones, are 100% deterministic. At the same time, reckless and uncontrolled use of LLM is increasingly gaining ground not only in coding but even in code analyze/review.
uhfraid•5h ago
> How do you review large AI-assisted refactors today?

just like any other patch, by reading it

nuky•4h ago
fair — that’s what I do as well)
ccoreilly•5h ago
There‘s many approaches being discussed and it will depend on the size of the task. You could just review a plan and assume the output is correct but you need at least behavioural tests to understand what was built fulfilled the requirements. You can split the plan further and further until the changes are small enough to be reviewable. Where I don’t see the benefit is in asking an agent to generate test as it tends to generate many useless unit tests that make reviewing more cumbersome. Writing the tests yourself (or defining them and letting an agent write the code) and not letting implementation agents change the tests is also something worth trying.

The truth is we’re all still experimenting and shovels of all sizes and forms are being built.

nuky•4h ago
That matches my experience too - tests and plans are still the backbone.

What I keep running into is the step before reading tests or code: when a change is large or mechanical, I’m mostly trying to answer "did behavior or API actually change, or is this mostly reshaping?" so I know how deep to go etc.

Agree we’re all still experimenting here.

Ask HN: Share your personal website

414•susam•9h ago•1317 comments

Ask HN: How do you safely give LLMs SSH/DB access?

61•nico•7h ago•85 comments

Ask HN: Weird Archive.today Behavior?

5•rabinovich•4h ago•1 comments

Ask HN: Iran's 120h internet shutdown, phones back. How to stay resilient?

106•us321•1d ago•94 comments

Tell HN: Properly using dishwasher reduced friction with my wife

9•xylo•6h ago•10 comments

The $LANG Programming Language

258•dang•1d ago•66 comments

Ask HN: ADHD – How do you manage the constant stream of thoughts and ideas?

110•chriswright1664•1d ago•133 comments

Distributed SQL engine for ultra-wide tables

2•synsqlbythesea•4h ago•1 comments

Ask HN: How are you doing RAG locally?

29•tmaly•11h ago•6 comments

Ask HN: Quantum Computation, Computers and Programming

31•rramadass•1d ago•26 comments

Ask HN: What are you working on? (January 2026)

256•david927•3d ago•857 comments

Ask HN: Are diffs still useful for AI-assisted code changes?

4•nuky•8h ago•8 comments

Ask HN: Vxlan over WireGuard or WireGuard over Vxlan?

44•mlhpdx•1d ago•81 comments

Tell HN: DigitalOcean's managed services broke each other after update

76•neilfrndes•2d ago•46 comments

Ask HN: Discrepancy between Lichess and Stockfish

21•HNLurker2•1d ago•11 comments

Ask HN: Looking for Windows contributors for meeting-detection engine

7•Ayobamiu•1d ago•1 comments

Anything Down?

3•Artur-Defences•8h ago•2 comments

Ask HN: What makes someone hate their job?

5•agcat•8h ago•12 comments

A Proposal to Modernize Xorg as a Protocol-Only Graphics Layer

3•powerwordtree•8h ago•3 comments

Ask HN: Any evidence AI coding assistants are helping open source projects?

6•UncleOxidant•7h ago•0 comments

Tell HN: Intel could blow up the Console Wars if it had the guts

7•noumenon1111•1d ago•10 comments

Tell HN: I Downgraded from macOS Tahoe to Sequoia

7•inatreecrown2•15h ago•6 comments

Ask HN: Who remembers AWS Spot's auction era before the 2017 pricing change?

3•aleroawani•1d ago•0 comments

Ask HN: 500 citation MSc CS, stuck in a low-trust region. How to move forward?

19•throwawaysafely•1d ago•12 comments

Tell HN: The Google Tenor GIF API has been shut down

23•dfajgljsldkjag•1d ago•17 comments

Ask HN: How to find gaps and oppurtunities in the AI era?

6•SRMohitkr•20h ago•4 comments

Ask HN: Learning Discoverability

2•learnwithmattc•1d ago•0 comments

Ask HN: Are you underutilizing your insurance too?

7•nemath•1d ago•5 comments

Is "AI vibe coding" making prototyping worse inside real companies?

16•arapkuliev•1d ago•5 comments

Ask HN: Personal website featured on HN, list of restaurants in NYC

4•laffOr•14h ago•0 comments