In my personal experience, LLMs help with:
- answering questions
- generating simple code/scaffolding in a vacuum
At the same time I don't have much success using LLMs to generate code in a simple CRUD application (around 20K LOC).
What I am looking for, is a video showing w/o time lapses/breaks, how an experienced prompt engineer uses an LLM to add a non trivial feature to a code base with at least 20K LOC.
What I am looking for:
- It must be used to add a feature on a bigger code base (>= 20 LOC)
- The added feature cannot be a leaf feature (means it must integrate with the rest of the system at multiple points)
- The prompting has to be less effort/faster than to type the solution in the programming language
- Any programming language/framework is fair game
- Any LLM is fair game
- The code base can be a bigger open source project (since I assume all LLMs were trained on open source projects, this should make it easier for LLMs to perform)