I follow a specific process with agents, and it works reasonably well: 1) describe my intended feature or change and then say “please recapitulate my intention and write an implementation plan” 2) read the plan carefully and refine it, make any corrections 3) tell the agent to go, but I check in frequently as it’s going. I watch the stream of thought and make course corrections or pull it out of rabbit holes 4) carefully review the result. Checking that tests actually fail, removing unnecessary changes 5) add instructions in AGENTS.md if I’ve found a new behavior that needs redirection
> But GPT also offers a $100 plan with five times the usage limits. And how long would that last me?
Sol usage is basically unlimited on the $100 plan if you're human-in-the-loop, reviewing the code, and not doing some useless Gas Town crime against humanity.
Luna medium effort is very slightly better than its output and completes tasks in 5-10x less time.
If it's used for that then using anything more powerful is just inefficient usage of your tokens.
It’s worth taking the time to setup a style guide for your agent to remove llm-isms like this. Somewhat paradoxically, you can even have the agent help write it. Just include a couple of snippets of “bad” phrasing from previous chats, add some commentary about why it’s bad, and what to replace it with, and put that in your system prompt. I could provide examples but the whole point is that you can tailor it to exactly how you want it. This has the added bonus of being a quick sanity check for model quality. If you start noticing llm-ism appear, then likely the model is not following your instruction’s and the thread should be discarded/you need to switch to a better model
> Okay, let’s move the rule to the top. In the next reply, right in the same chat, I get text with quotation marks again.
Nerves of steel? Not even close.
I think 90% of the article is the author complaining about (predictably) bad small-model behavior. The large models are much better!
My summary of the article: I use a really cheap LLM, and suffer like everyone else using GPT 3.1. This is definitely a throwback to the days not so long ago when LLMs were completely incompetent.
What are some uses for which you found Luna to be unusable garbage?
From my side, I use Harbor to score tasks and Luna certainly performs when the prompt is either well or overspecified.
dognotdog•1h ago