How soon until this becomes part of the "no one wants to work anymore" argument

Don't worry, once we achieve post-scarcity they will have more tokens than they could ever dream for spending.

I know right? What did the leadership think would happen when they give some of the worlds greatest software engineers (supportably), a easily quantifiable metric to target?

> Who could possibly have predicted that happening?

Charles Goodhart :-)

How dare you question the most effective allocators of capital.

Now come on, there was a recent post where the author argued that infallible management knew this would happen, but was part of the double-secret-probation strategy to get the cogs to finally start using AI.

A past employer thought it was a good idea to put up a leaderboard of who sent the most Slack messages. They celebrated the people at the top for being so active.

Predictably, everyone started talking in Slack like their jobs depended on it. Everyone was responding to everything. Instead of writing out a complete message and pressing enter, they'd send each fragment of the sentence as a new line.

The Slack leaderboard was never shown again. Unfortunately the habit remained because people were afraid they were going to be secretly judged by how much Slack activity they generated.

I expect the same thing is going to happen at companies who had token leaderboards. Once you've instilled that fear in people, they internalize the expectation.

Tokens are less valuable than the eyeball metric of the Dotcom era. At least the eyeballs were real then.

I'd argue most of the AI value is related to how 'Dead' the internet is.

I'd guess through LLM embedded PoC projects.

You can rack up token consumption extremely quickly if you do things like "have an LLM summarize logs every hour". "Do this automation every 5m that does X"

Or something kind of crazy like that (LLM log summarization as described here is not a good idea or concept, mind you).

I'm building something in that vein, but cognizant and aware to avoid running the automation too frequently during dev.

You can fire off a script that is effectively hundreds of coding dev sized prompts in one batch.

I'd be very surprised if these numbers are just typical coding usage with no scripting/pipeline/automation stuff

One thing I find fascinating as a software engineer who talks to non software engineers who use AI tools is how "reading PDFs" is not more of a solved problem. What I mean is that uploading a PDF into a chatbot tool seems to be an extraordinarily obvious use case that non technical (and technical) users would want to do.

IMO claude, chatgpt/codex, etc should be able to optimize the PDF use case to be extremely token efficient as it's a very obvious use case. But when I start to explain to my wife/friends why it burns through so much quota, I find myself thinking "why should they have to understand this aspect of it". to me, that the details of PDF parsing and extracting are relevant to users (instead of solved such that you don't have to pay attention to it) shows how these tools are not nearly as "ready" as they are made out to be. I may be preaching to the choir on this one, but just my 2c

What outcomes though? The ones I’ve seen posted are still nonsensical metrics that a publicly traded firm absolutely doesn’t care about.

It wants to see faster R&D, higher revenues from existing assets, greater operating margins, higher sales to invested capital ratio and so on…

The best way to measure that for a software firm is up-time of services and project completion duration