I used GPT-5 as my personal lawyer the other day. I uploaded contracts and agreements, and the number of things it got wrong was mind blowing. It made me look like a complete amateur with the emails it drafted. GPT-5 is extremely dumb and incompetent when it comes to understanding real-world problems and offering solutions. It wasn't even at the level of a junior lawyer, it was ten times worse. I was honestly shocked by the results.
When it comes to programming, I have to keep replaying with "Nope", "No", "Again", "Wrong", "It doesn't work" and a couple of times "Do better" before it finally produces something complex that actually works.
With coding at least I know when it's wrong. The real problem is when I don't.
ohr•5mo ago
But if you let it run the tests, it’ll just do that on its own (or delete the tests, but usually it won’t), which is very useful and seems to match the article’s sentiment.
pyman•5mo ago
Test-driven development is back.
Since LLMs rely on patterns learned from large amounts of text, the results are relative. If the training data contained more Rust repos, that could explain why it feels stronger in Rust.
The way AI companies talk about "intelligence" now is shifting. They admit LLMs can't truly reason with the current architecture, so intelligence is being framed as the ability to solve problems using patterns learned from text, not reasoning on their own. That's a big downgrade from the original idea of AI reaching human-level thinking and developing AGI.
Also, my understanding is that since Microsoft invests in Copilot, it doesn't want ChatGPT to get better at coding. Instead, it wants it to get better at being a lawyer.
Capricorn2481•5mo ago
The creator of zig did not seem to care for the results
Of course. The author had to be bullshitting. It tried it many times already and it is impossible to let any frontier model program in Zig. The language is evolving too fast and can have breaking changes everyday and the knowledge cutoff for most AI models is only somewhere around the end of last year. Btw, the knowledge cutoff always being so far out-of-date is one of the things I absolutely hate using AI for.
pyman•5mo ago
When it comes to programming, I have to keep replaying with "Nope", "No", "Again", "Wrong", "It doesn't work" and a couple of times "Do better" before it finally produces something complex that actually works.
With coding at least I know when it's wrong. The real problem is when I don't.
ohr•5mo ago
pyman•5mo ago
Since LLMs rely on patterns learned from large amounts of text, the results are relative. If the training data contained more Rust repos, that could explain why it feels stronger in Rust.
The way AI companies talk about "intelligence" now is shifting. They admit LLMs can't truly reason with the current architecture, so intelligence is being framed as the ability to solve problems using patterns learned from text, not reasoning on their own. That's a big downgrade from the original idea of AI reaching human-level thinking and developing AGI.
Also, my understanding is that since Microsoft invests in Copilot, it doesn't want ChatGPT to get better at coding. Instead, it wants it to get better at being a lawyer.