Ask HN: Have people started to see cracks in the ChatGPT answers with time
3•sandeepkd•3mo ago
Context: I have a co-worker who is used to execute the suggested scripts from Chat-gpt with high confidence. Lately one of the scripts to use Github API was not working and he asked me to debug. Turns out, Github had changed/updated the attribute names in the output and they no longer matched with the Chat-Gpt provided script. Hard part, this individual is really good engineer, however he/she was low on confidence about how to debug the Chat-gpt provided script
Comments
speedgoose•3mo ago
Skill issue.
First, switch to Codex. Then, consider using the context7 mcp server to let agents automatically fetch the latest documentation.
Debugging is not so different than debugging code written by someone else. It’s a skill you can practice.
By the way, agents are also good at debugging.
techblueberry•3mo ago
This is a great advertisement:
Agents: harder, slower and more expensive than just doing it yourself.
speedgoose•3mo ago
It depends on the task.
pavelai•3mo ago
It's a very overgeneralized statement. Agents could do a lot of work, which most of people wouldn't want to do. and agents can do this on a pretty good level for now. Probably we would enhance their ability to do this in near future. People should stop write code and start to do real life things.
techblueberry•3mo ago
What you are failing to understand, is that this is literally the worst these models will ever be. Uber wasn’t profitable for like the first 20 years of their existence, and now look at them. People said the same things about the internet.
raw_anon_1111•3mo ago
And thousands of other companies weren’t profitable and disappeared into obscurity and failed, using Uber as proof of anything is the ultimate in survivorship bias.
pavelai•3mo ago
There a lot of examples of things been critisized in the beginning and strived, there are a lot of things which have gone. It only means that there could be only reasonable critics. Can you say if your criticism is reasonable? I saw a lot of things being thrashed in the beginning. And these things people use everyday. You can choose from thousands of examples
First plane didn't have cockpit, and windows, could fly only hundreds of meters, and looked more like a clothes dryer than a plane. Someone could call it inefficient (and really it wasn't efficient). It was easy to call it dangerous. But as we can see, planes evolved since then. When the first radio was invented there was no radio station to listen. When the steam train was invented it was crazy expensive and there was no government which has enough money to build all the railroads we have today
All of these technologies were modified by talented people, some of them were inventors, others later adopters. But they changed how this technologies work, and what goods they brought to us
AI is too young technology to tell if it fail or not. You can critisize it or you can change the direction of the progress. It's up to you
raw_anon_1111•3mo ago
And if you can point to thousands, then I can literally 100x for all the things that failed. You only have to go back 25 years in tech to see all of the failures - or YC companies that IPO’d more recently that collectively have a -49% return while the broader market has seen record gains
It ends up for instance that Steve Jobs was right about Dropbox - it is a feature not a product.
pavelai•2mo ago
Ok. We are talking about different things. There are fundamental rules which make technologies to strive or disappear. But there are other fundamental rules for products, features, etc. So you talk about products and I talk about technology.
YCombinator is about products, not technologies, so their example is not suitable here. Agents is a technology. They have fundamental advantage over premade software. They are already here and they are doing valuable work. They will stay for the near future. You can argue against reasons to use particular agents or particular products built around them, or particular features, but not against the whole technology.
The technology is verified, we need to find the way to make it safer and more sustainable. This would require to solve many technical tasks: build safeguards, create eco-friendly energy production, solve explainability, build physical infrastructure, develop new technical standards, etc.
pavelai•3mo ago
The only thing I'm failing to understand is why you are using my arguments against my point of view, while you are who's critisizing agents. Can you elaborate your statements then?
almosthere•3mo ago
Don't use chatgpt for that kind of work, use agents. It will read the error message and adapt, perhaps even go as far as open the release notes for the tools you're using and fix the problems it encounters.
bjourne•3mo ago
IMnsHO, nope! People trust the bot way more than their own experience.
speedgoose•3mo ago
First, switch to Codex. Then, consider using the context7 mcp server to let agents automatically fetch the latest documentation.
Debugging is not so different than debugging code written by someone else. It’s a skill you can practice.
By the way, agents are also good at debugging.
techblueberry•3mo ago
Agents: harder, slower and more expensive than just doing it yourself.
speedgoose•3mo ago
pavelai•3mo ago
techblueberry•3mo ago
raw_anon_1111•3mo ago
pavelai•3mo ago
First plane didn't have cockpit, and windows, could fly only hundreds of meters, and looked more like a clothes dryer than a plane. Someone could call it inefficient (and really it wasn't efficient). It was easy to call it dangerous. But as we can see, planes evolved since then. When the first radio was invented there was no radio station to listen. When the steam train was invented it was crazy expensive and there was no government which has enough money to build all the railroads we have today
All of these technologies were modified by talented people, some of them were inventors, others later adopters. But they changed how this technologies work, and what goods they brought to us
AI is too young technology to tell if it fail or not. You can critisize it or you can change the direction of the progress. It's up to you
raw_anon_1111•3mo ago
https://medium.com/@Arakunrin/the-post-ipo-performance-of-y-...
It ends up for instance that Steve Jobs was right about Dropbox - it is a feature not a product.
pavelai•2mo ago
YCombinator is about products, not technologies, so their example is not suitable here. Agents is a technology. They have fundamental advantage over premade software. They are already here and they are doing valuable work. They will stay for the near future. You can argue against reasons to use particular agents or particular products built around them, or particular features, but not against the whole technology.
The technology is verified, we need to find the way to make it safer and more sustainable. This would require to solve many technical tasks: build safeguards, create eco-friendly energy production, solve explainability, build physical infrastructure, develop new technical standards, etc.
pavelai•3mo ago