Yep agreed

It's a mixed bag. It depends on your problem domain, the problem you are trying to solve (within that domain), the context you provide the LLM, the output it generates (you are using libraries to coerce the output into predictable (JSON) structures, right?). What's more, based on what you are trying to do, the LLM you are using might have sufficient training data, but not necessarily so (resulting in possible hallucinations/confabulations). So, there's that. Also, LLMs are not deterministic, they can (and will) generate a different response every time you call them (even if the context you provided is the same). So, yeah... sometimes these things really deliver and other times, it's just... meh!

I found the same thing, so I don't bother with ai-gen code AT ALL. I found that the time wasted fixing up the slop was not worth it - it's more efficient to code it yourself, as shown by studies (eg referred to here: https://www.linkedin.com/pulse/vibe-coding-myth-when-feeling...).

No, your vibe-coding is not more productive, unless your only metrics for productivity are commit counts, PR counts, deployment counts. I can commit, PR and deploy crap all day long and "score well" - and this is what people are clinging to with their ai-gen defenses. I'm really sorry to inform you that your experienced "speed-up" is just a trick of the brain (remembering from an article written, iirc, by Gurwinder, but I'm having trouble finding it now) - you're actually going slower, and your brain is tricking you into thinking it's faster because whilst the ai was "coding", you didn't have to, so it feels like more of a win than it actually is, considering the results.

Every single time I try to use it for research or learning it ends up spitting subtly invalid code. Results range from imaginary APIs that don't exist to straight up invalid syntax, not to mention outdated info, contradictory reasoning and flat out wrong explanations.

Maybe spending $200/mo or whatever to access the top-of-the-line models will mitigate some of that, but I'd rather come up with the solution and gain the understanding myself, then spend the money on something worthwile.