I've heard of companies that are shoehorning it into everything, I feel this is many companies just playing the game to get better valuations.
Google-Fu is being replaced with Prompting-Fu
not being allowed or choosing not to spend time learning the limits, benefits and drawback of different LLM models is basically handicapping yourself.
I'd like to see an actual LLM+search system that dreams up hypothesis and then tries to falsify them and confirm them with actual search. That seems like it could be a great search system.
But that's not what we have today afaik. What we have are systems that pump out "you can't lick a badger twice" type misinformation on a massive scale, all unique.
Is this similar to companies forcing TDD or extreme programming or pair programming on their employees? Some manager hoping to get more productivity by imposing a tool or technique?
bingo -let‘s see how that works out…
Everybody focuses on programming, but the real value in is project management imho.
- update docs
- update user stories on whatever project tracking tool you use
- check for inconsistencies between requirements and current flows
Those are all things that should be trivial-ish for an AI, and where the real value-speed up is.
We are allowed to use AI for coding, with the understanding that we are responsible for the code generated by the AI, not legally obviously, but functionally.
I recently did for the first time. Spent 15 minutes writing a long prompt to implement a ticket. A repeated pattern of code, 5 classes + config per topic that deeply interact with each other and it did the job perfectly.
It convinced me that the current code monkey jobs, which are >90%, >95%? of software engineering jobs, will disappear within 10 years.
We‘ll only need senior/staff/architect level code reviewers and prompt engineers.
When the last generation that manually wrote code dies out, all people will do is prompting.
Just like assembler became a niche, just like C became a niche, high level languages will become a niche.
If you still don‘t believe, you haven‘t tried the advanced tools that can modify a whole project, are too incompetent to properly prompt or indeed work in one of the rare, arcane frontier- state-of-the-art niches where AI can‘t help.
And what will you do when all the seniors retire and there's no juniors to take their place because they were replaced by AI?
Just like nowadays assembler is only a side note, C is only taught in specialized classes (OS, graphics) and most things are taught in high level languages.
The same way most of us review our compiler generated code today (ie not at all). If it works it works, if doesn't we fix the higher level input and try again. I won't be surprised if in a few more generation the AI will skip the human readable code step and generate ASTs directly.
How can I visit this fantasy world of yours where LLMs are as reliable and deterministic as compilers and any mistakes can be blamed solely on the user?
Wait 20 years.
It's really easy to make unsubstantiated claims about what will happen decades from now, knowing your claims will be long forgotten when that time finally comes around.
We for example have a whole generation of programmers who have no idea what the difference between a stack and a heap is and know nothing about how memory is allocated. They just assume that creating arbitrarily complex objects and data structures always works and memory never runs out. And they have successful careers, earning good money, delivering useful software. I see no reason why this won't continue.
I think there's a cohort thing going on here, because Google has been spam rekt for long enough that entire classes of students have graduated, entered the workforce, and been promoted all since search and SO went to hell, so the gizmo that has the working code and you just need to describe it seems totally novel.
But we've been through this before: one day there's this box and you type a question and bam, reasonable code. Joing a FAANG is kind of like that too: you do the mega grep for parse_sockaddr_trie and there's this beautifully documented thing with like, here's the paper where it shows its O(ln).
But you call the thing and it seems to work and you send the diff and the senior person is like, that doesn't do IPv6 and that's rolling to us next quarter, you should add IPv6. And the thing was exploiting the octets and so its hard.
The thing is, a sigmoid looks exactly like an exponential when you're standing on it. But it never is. Even a nuclear bomb is exponential very briefly (and ChatGPT is not a nuclear bomb, not if it was 100x more capable).
Think about defense, or trading, or anything competitive like that: now you need the LLM because the other guy has it too. But he's not watching YouTube all day, he's still chain-smoking and taking adderall except he has an LLM now too.
So yeah, in the world where any of 9 TypeScript frameworks would all make roughly the same dark mode website and any of them get the acquisition done because really the founder knows a guy? That job is really risky right now.
But low effort shit is always risky unless you're the founder who knows a guy.
the problem with that is that if there are no juniors left...
The client I work at, through them, has made some tools available but no-one is using them for anything.
Our company is positioned right at the edge of the wave for this though so it's understandable.
What are best practices? What tools are genuinely helpful, such as automatic reviews in a build street, or sentiment analysis in commit messages?
There are also many workshops about how to build with AI etc, so it's slowly becoming part of everyone's work
I'm kinda worried about how the massive usage of AI coding tool will affect the understanding of large codebases and complex systems, but to be totally honest I'm really impressed by Claude Code and how it can write Terraform/Helm/Ruby based on the company's idioms (and I'm talking about a repository with 250k+ lines of HCL!).
I'm certainly seeing the benefits. A lot of tasks are faster with AI. Even some quite fiddly bits of log-diving and finding subtle bugs can be done by AI, which would have taken me considerably longer.
I'm still finding that overall architecture needs to be done by me, though. Once you make the task big enough, AI goes off the rails and makes some really odd stuff.
Lately it's taken over code reviews, for myself and when I review other people's code. Extremely helpful. It's made software development fun again.
Quality far exceeds that of human reviews. It's becoming my favorite use case for AI.
The last place I worked at had bots reviewing bot PRs and passing them. It went about as well as you’d expect.
* The codebase consists of many modules with their own repo each
* The test back end has a lot of gotchas, borked settings or broken ongoing work by the back-end team
* The code is incredibly layered
I’m spending up to 10% of my time writing actual code. The rest is overhead like multi-repo PR’s, debugging, talking to people etcetera.
Once I found the issue, the code is the easy part and explaining it all to the LLM is more work.
Assistive coding tools need to get a lot better
I've already communicated that I don't want to see nor hear the "but AI generated it this way" opinions. But other than that, I can see the potential, and I'm using it as well. Just not for generation of production code, rather to test assumptions, maybe initial implementations, to make things faster, but in the end I'm always reimplementing it anyway.
Also, to be completely honest, AI does better code reviews than most of my coworkers.
Also I can't imagine how being handed a bunch of autogenerated terraform and ansible code would help me. Maybe 10% of my time is spent actually writing the code, the rest is running it (ansible is slow), troubleshooting incidents, discussing how to solve them & how to implement new stuff, etc.
If someone works in a devops position where AI is more of a threat, I'd like to hear more about it.
It’s also quite good at getting to a solution by using validate/plan loops and figuring out syntax/state issues.
The biggest challenge is the lack of test sophistication in most terraform setups.
But llms generally are _amazing_ for my ops work. Feeding a codebase into one and then logs I’ve seen Claude code identify exact production issues with no other prompting. I use llms to help write incident reports, translate queries in the various time series db we use, etc.
I’d encourage you to try an llm for your tasks, for me for ops it’s been a huge boon.
The development and project teams I primarily work with are all encouraged to identify suitable use cases for GenAI. Most development teams have already started trials with AI assisted coding but reported a relatively low adoption rate of 5–10%.
I found the Microsoft Copilot to be reasonably good when given a complete context with extremely limited scope such as being provided a WSDL for a SOAP service and asked to write functions that make calls and then writing unit tests for the whole thing. This had a right way and a wrong way of doing things and it did it almost perfectly.
However, if you give it any problem that requires imagination with n+1 ways of being done it flounders and produces mostly garbage.
Compared to the Microsoft Copilot I found the Github Copilot to feel lobotomised! It failed on the aforementioned WSDL task and where Microsoft's could be asked "what inconsistencies can you see in this WSDL" and catch all of them, Github's was unable to answer beyond pointing out a spelling mistake I had already made it aware of.
I have personally tinkered with Claude, and its quite impressive.
My colleagues have had similar experiences, with some uninstalling the AI tooling out of frustration at how "useless" it is. Others, like myself, have begun using it for the grunt work; mostly as "inteligent boilerplate generator."
I'd never thought about LLM failure modes like this, but it makes sense.
Cross too many possibility streams, instead of prompting / contexting down to one, and it vacillates between them and outputs garbage.
Permitted for development with the explicit caveat that code is always the responsibility of the people connected to the pull request.
To be honest. I think it's pretty cool tech (I mostly use copilot with either Claude Sonnet 3.7 or 4, or otherwise GTP 4.1). Agent mode is cool. I use it every day and it has helped me work faster, do better work by it preemptively catering for things that might have otherwise taken many iterations of releases to discover, so yeah, I think AI is pretty good for software developers overall. It's a great tool to have. Is it going to do my work and leave me redundant? Not any time soon. I think the company I work for will fail in their enforced AI efforts, spend a gazillion dollars and will go quietly back to outsourcing overseas when the dust settles. I feel sad for the junior devs though as they are basically vibe coding their way through Jira tickets atm. I am a graybeard, 30+ years in the industry.
Has your company tried running the models locally, or is that maybe just presumed to be not worth the effort?
Putting company code into a private github repo would be a firing offense where I work.
I suppose us engineers familiar with the product had just a bit more context than the investors issuing a blanket statement to their portfolios to use AI.
Claude Code i will admit i find occasionally useful, but the flood of overly verbose and lackingly meaningful "AI Summaries" I'm being forced to waste time reading is really grating on me. Copilot PR summaries turning a 20-line PR into a fifty-line unhelpful essay is driving me insane.
The division I worked in was demanding that developers use AI at least once a week and they were tracking people's usage. The boss nagged about it every day.
I had no problem meeting the requirement but did find it's contributions to be very hit or miss as to usefulness.
Not using AI is akin to not using an IDE to program. Sure you can code in vim or notepad, but surely there's better options?
We've also been forced to use it for a few projects that it ...didn't work very well for. (Though to be fair other departments have found some valid and helpful uses for it.)
It feels like way more time is being spent trying to make AI work than will ever be saved by using it—at least for me. The time spent finding + giving context, prompting, generating, sifting through garbage outputs, then finally editing whatever output is given (which is required 100% of the time) is usually greater than the time required to just do the thing.
Quiza12•7mo ago