I can't republish anything that happens in a production/proprietary environment.

One of the things that I think is pretty great about being able to share these particular prompts is that you can run this on one of your own repos to see how it turns out.

ACTUALLY!! Hold on. A couple weekends ago I spent some time doing some underlying research with huggingface/transformers and I have it on a branch.

https://github.com/AlexChesser/transformers/tree/personal/vi...

You can look at the results of an architectural research prompt.

Unfortunately I don't have a "good mode" side by side with a "bad mode" at the moment. I can work on that in the future.

The underlying research linked has the experimental design version of this with each piece evaluated in isolation.

oh my please read TFA it has exactly the answers you seek.

I'll admit that it's a little shoehorned in at the end :)

... but you know how editors are with writing the headline for clicks against the wishes of the journalist writing the article. You'll always see Journos sayign stuff like "don't blame me, that's my editor, I don't write the headlines"

I did toy with the idea of going with something like: `Prompt Engineering is a wrapper around Attention.`

But my editor overruled me *FOR THE CLICKS!!!*

Full disclosure: I'm also the editor

Having a lot of context in the main agent is an antipattern. You want the main agent to call subagents to keep high level reasoning context clean. If you're not using subagents you should start a new chat any time you're doing something fairly different from what you were doing before.

I think the author is confusing the attention mechanism with attention in the general sense. He refers to a phenomenon called “attention drift”, but neither Google nor LLM searches return any relevant references to that term. In fact, ChatGPT only cited this same blog post.

Yea they sound like plausible enough arguments.. I often test my vague input against structured ones and for many tasks it didn't seem like hallucinations happened significantly less. Honestly the more structured you have to be, the less "general" your model probably is, and in theory you want your model to be as general as possible. Seems like here, you're simply helping it over-fit.. At least that's what my intuition tells me but I am yet to really check that out either.

I use Grok to write the prompts. Its excellent. I think human created prompts are insufficient in almost all cases.

Write your prompt in some shape and ask grok

Please rewrite this prompt for higher accuracy

-- Your prompt

100% yes! There've been some other writers who've been doing parallel work around that in the last couple weeks.

https://www.dbreunig.com/2025/06/10/let-the-model-write-the-... is an example.

You can see the hands on results in this hugging face branch I was messing around in:

here is where I tell the LLM to generate prompts for me based on research so far

https://github.com/AlexChesser/transformers/blob/personal/vi...

here is the prompts that produced:

https://github.com/AlexChesser/transformers/tree/personal/vi...

and here is the result of those prompts:

https://github.com/AlexChesser/transformers/tree/personal/vi.... (also look at the diagram folders etc..)

This is pretty much DSPy.

The LLM is basically a runtime that needs optimized input bc the output is compute bottlenecked. Input quality scales with domain knowledge, specificity and therefore human time input. You can absolutely navigate an LLMs attention piecemeal around a spec until you build an optimized input.

yes, just prepend your request to llm with "Please give me a well-structured LLM prompt that will solve this problem..."

GPT 5 prompt optimizer: https://platform.openai.com/chat/edit?models=gpt-5&optimize=...

Does Claude code do this by default? It seems for most prompts that I give it, it ends up breaking things into TODO lists and reformulating the prompt. This seems to work well for most tasks.

What’s the problem?

-A Prompt Architect

TBF I always read prompt engineering as the 2nd definition, not the first.

> 1. the branch of science and technology concerned with the design, building, and use of engines, machines, and structures.

2. the action of working artfully to bring something about.

So you're trying to learn what / how to prompt, in order to "bring something about" (the results you're after).

Not sure if this is a part of it, but there are places where "engineer" is a protected title that requires a degree or license. I work with a bunch of folks in Quebec and they have to use the title "software developer" unless they are a member of the Order of Engineers. I find this to be pretty silly considering someone can have a degree in Mechanical Engineering and use the title "Software Engineer" but someone with a degree in Computer Science can't.

> There are all kinds of attention alternatives (e.g. Hyena) and tricks (e.g. Sliding Window, etc.) that make this analogy, as presented, flat out incorrect.

Not to doubt you but could you explain why so?

also LLM providers optimize your request routing to a cheaper model sometimes, I wonder if there is a way to structure prompt that will route to a large (more expensive, and arguably better) model for better results