frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

Has anyone figured out how to guard LLMs on kernel level?

2•isaacnsisong•3h ago•1 comments

Tell HN: It's a Lot

5•mettamage•4h ago•2 comments

Ask HN: Is neuromorphic computing going to replace traditional AI?

3•lennart-rth•4h ago•1 comments

Codex Is Down

11•freakynit•4h ago•5 comments

Ask HN: How would you harden AI changes to a 1M-line legacy SaaS before review?

3•thegreatkahuna•6h ago•6 comments

Postgress in Rust

2•poldex•7h ago•1 comments

ChatGPT Is Down

9•anujbans•4h ago•5 comments

Tell HN: Namecheap gave my account to an unverified third party

478•Thrashed•1d ago•174 comments

Companies are optimizing models for specific benchmarks

2•mzubairtahir•8h ago•0 comments

Ask HN: What's Up with Archive.today?

5•nobody9999•15h ago•8 comments

GPL Infection of LLMs?

4•sl_convertible•17h ago•2 comments

Claude used my pipeline to find a counterexample to the Jacobian conjecture

6•JGPTechCo•14h ago•4 comments

Ask HN: Is there a website that tracks excessive writes to SSDs in OS/app betas?

30•amichail•1d ago•28 comments

Stripe closed my account with a 0.18% dispute rate

10•rellyattd•14h ago•8 comments

Ask HN: How do I find startups that are looking for a dev? How to approach them?

5•need_a_work23•20h ago•7 comments

Two Gov Websites Compromised

7•relunsec•21h ago•1 comments

Tell HN: ChatGPT exports do not contain all conversation messages

5•afpx•22h ago•1 comments

Ask HN: How to get in touch with Paddle (MoR)

6•probst•1d ago•5 comments

Ask HN: If OpenAI hacked HuggingFace, why aren't OpenAI prosecuted?

29•sam_lowry_•2d ago•36 comments

Ask HN: What are your current favorite monitoring and observability solutions?

3•Exoristos•17h ago•1 comments

Ask: Why does everything Microsoft create lately feel fragile and half-baked?

12•nothrowaways•1d ago•11 comments

Ask HN: Is anyone giving out tokens for benchmarking LLMs?

3•emosenkis•21h ago•1 comments

Ask HN: Can we get a day that is free from AI news?

8•MichaelMoser123•22h ago•5 comments

Ask HN: How are you getting decent front end interface results out of LLMs?

2•VladVladikoff•23h ago•1 comments

Ask HN: How much software do you build for yourself these days?

4•lellow•1d ago•10 comments

Ask HN: Has your typing gotten worse with AI?

2•aizk•1d ago•7 comments

Single-click and remote OpenCode session right in the browser

3•jashandeep31•1d ago•0 comments

Ask HN: What changed in your life when you started meditating (and how)?

32•dondraper36•2d ago•27 comments

A VSCode extension for AI to understand your project

2•bailingyuan•1d ago•2 comments

Ask HN: Any Good Pi.dev Setups?

3•galsapir•1d ago•1 comments
Open in hackernews

Ask HN: How would you harden AI changes to a 1M-line legacy SaaS before review?

3•thegreatkahuna•6h ago
I’m not a software engineer, but I’ve been running an experiment to see whether agentic development could produce a useful prototype on top of an existing SaaS codebase.

The codebase is 1M+ lines, 15 years old, hosted on Azure, and primarily written in C# and React.

The prototype needs to be available for customer testing in September. No developers were available to work on it full-time, although I could occasionally get help with specific technical issues. An engineer will evaluate the implementation in August and decide how much confidence we can have in the AI-generated code so that we can decide how to “convert” the code to production-grade.

My question is what I can do before August, primarily using AI tools and manual testing, to make the code as robust and reviewable as possible. I want to increase the likelihood that the AI-generated code would be so good that the path to production-grade would be closer to “copy-paste” than building everything again from scratch.

Environment The prototype is being developed in a separate branch, deployed to a separate internal environment, and connected to its own database and schema.

Planning process We interviewed customers in June and turned the resuls into an MVP spec.

The process was approximately: 1. Write a PRD. 2. Use an LLM to convert the PRD into an architecture document, which was reviewed by an architect. 3. Create product designs consisting of screen images and md files containing interaction details ans edge cases with Claude Design. 4. Use an agent to break the work into epics using the PRD and architecture document as guardrails.

The epics were the most granular planning artifacts that received review by me and the architect.

Development process The development flow was intended to run with little intervention: 1. A planner agent converted epics into md story files and Jira stories 2. A coding agent implemented the stories including tests and opened PRs 3. A reviewer agent reviewed the PRs, requested changes, and merged them into the prototype branch

The coding agent polled PRs for review comments and could escalate issues back to the planner.

Agents also had a “stop and ask” list for decisions they were not allowed to make autonomously. I (and a few times an engineer) were involved by resolving those escalations and by manually testing the accumulated changes end to end each day.

Most implementation and initial review were done with Claude-based agents. For riskier PRs, I also used Codex as a reviewer. The second-model review found substantially more relevant issues in the Claude-generated code, but token quotas limited the use.

I had separate refactoring and harden runs for the code as well.

Results The planning and setting up the environment and agentic flow took about two weeks and then the agents built the whole MVP in about two weeks. Size-wise it was 13k lines of functional code + the same amount for tests.

What I would like advice on Assuming that I cannot get substantial developer involvement before the review, how can I increase the likelihood that the code is as close to production-grade as possible?

Here are some of the questions I have been thinking about: 1. What checks or development loops would give the largest improvement in confidence before an engineer reviews the code? 2. How would you use independent agents or models to reduce the risk that the coder and reviewer make the same incorrect assumptions? 3. Should tests be generated by a separate agent from the one that wrote the implementation? 4. What documentation or evidence would make the eventual engineering review faster and more reliable? 5. If you had only a few weeks to improve this prototype before handing it to an engineer, what would you prioritize?

Comments

krembo•2h ago
At this point what stops you from asking AI this question? But honestly, extensive QA. There are probably more quirks than you could imagine
thegreatkahuna•1h ago
I have asked the AI, but was hoping for a second opinion, ideally by someone who has experience from doing something a bit similar. Your point about QA is interesting. Bob Martin, doesn't review AI code any more, just does extensive QA (https://x.com/unclebobmartin/status/2080257779395154409). Perhaps the key move would be for me to now focus on QA and try to nudge the engineering review to also emphasize testing over reading code?
scaredreally•1h ago
Once you no longer "own" the code, does it matter? If you provide a very detail set of specifications and success criteria then you can turn it into a black box. The code is no longer the asset, just the business that uses it. You can use AI to extract the requirements from the existing code and then modify as needed however, I would recommend doing this in two phases: first the conversion, then the enhancements.
thegreatkahuna•53m ago
I suppose you are right at least what comes to the functionality. Security is the one thing where I am sure that my company will require some peaking under the hood.
dnemmers•1h ago
In the past, problems like these were solved with software engineers. Those trained through schooling and experience to be able to read through and understand obscure code conventions and programming quirks.

Their kind has almost gone extinct in this universe...

thegreatkahuna•1h ago
I hear you. This must have been a long time ago, like last year or something. Those were the times.