frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

Ask HN: How to measure how much data one can effectively process or understand?

17•mbuda•9h ago
Is there a scale of how much data one can effectively process, something similar to the "Kardashev scale for data"? What would be a name for such a thing? During Memgraph's Community Call (https://youtu.be/ygr8yvIouZk?t=1307), the point is that AgenticRuntimes + GraphRAG moves you up on the "Kardashev scale for data" because you suddenly can get much more insight from any dataset, and everyone can use it (a large corporation does not control it). I found something similar under https://adamdrake.com/from-enterprise-decentralization-to-tokenization-and-beyond.html#productize, but the definition/example looks very narrow.

Comments

mbuda•9h ago
Here are clickable links: https://youtu.be/ygr8yvIouZk?t=1307, https://adamdrake.com/from-enterprise-decentralization-to-to...
kellkell•6h ago
The Kardashev scale measures energy control, not information processing. If we were to define a “Kardashev scale for data,” it wouldn’t be about raw volume, but about effective abstraction capacity.

Humans don’t process data directly — we process compressed representations. So a meaningful scale would measure:

1- Throughput — how much structured data an agent can analyze per unit time.

2- Compression efficiency — how much insight is extracted per unit of data.

3- Relational depth — how many meaningful relationships can be modeled simultaneously.

Tools like Agentic Runtimes + GraphRAG don’t just increase data volume access — they expand relational modeling capacity and contextual memory. In that sense, they move users up a scale of informational leverage, not just scale of data.

mbuda•6h ago
Yep, amazing points!

Agree with the measures; follow-up question: what's the insight definition? I think exposing some of those measures would help people better understand what the analysis covered, in other words, how much data was actually analyzed. Maybe an additional measure is some kind of breadth (I guess it could be derived from the throughput).

"Informational leverage" reminded me of "retrieval leverage" because yeah, the scale of data didn't change, the ability to extract insights did :D

kellkell•1h ago
Good question.

By “insight” I mean a measurable reduction in uncertainty that improves decision quality or predictive accuracy.

In practical terms, an insight could be defined as:

•A hypothesis generated and testable from the dataset

•A model parameter adjustment that increases predictive performance

•A structural relationship discovered that reduces entropy in the system representation

So compression efficiency would be something like:

(uncertainty reduced) / (data processed)

Breadth is interesting — I’d treat it as dimensional coverage: how many independent variables or graph regions are meaningfully integrated into the model.

“Retrieval leverage” is a great term. It highlights that the dataset size remains constant, but navigability and relational traversal improve — which increases effective cognitive reach.

Some of these broader ideas around informational sovereignty and anomaly-driven cognition have been explored in independent empirical work, though they’re still niche.

Natfan•5h ago
lol comment, ignored.
mikewarot•4h ago
The limiting factor would be the density of information in the source material, followed my the cognitive impedance match of the receiver.

Fir example, a correct grand unified theory isn't useful if you don't know the physics to understand it.

allinonetools_•4h ago
Interesting question. In practice, I’ve found the limit isn’t how much data exists but how much you can turn into action without friction. The clearer and faster the feedback loop, the more data you can effectively “use,” regardless of volume.
rgavuliak•2h ago
I would measure data by time to action. If you're not actioning data it's worthless.

Reddit Ads support is leaking PII and actively crossing user sessions

5•arashvakil•57m ago•1 comments

Ask HN: How to measure how much data one can effectively process or understand?

17•mbuda•9h ago•8 comments

Thank HN: You helped save 33k lives

1135•chaseadam17•3d ago•113 comments

Ask HN: Is it worth learning Vim in 2026?

23•zekejohn•12h ago•17 comments

Google Cloud APIs (gcloud CLI) seems to be down or broken

3•thej•9h ago•0 comments

Ask HN: Anyone else tired of working in tech?

30•boredemployee•1d ago•34 comments

Googling on Brazil about "Gemini said" shows unrevised content from Gemini

3•yrds96•1d ago•1 comments

Ask HN: Are hackathons still worth doing?

8•kwar13•1d ago•6 comments

Ask HN: Can a license make large corporations give back?

2•arboles•1d ago•3 comments

Ask HN: (Your) Request for Startups?

9•dontoni•1d ago•7 comments

Ask HN: How do you overcome imposter syndrome?

11•fdneng•2d ago•16 comments

Ask HN: How do you motivate your humans to stop AI-washing their emails?

27•causal•3d ago•35 comments

Ask HN: In Cursor/agents, do plugins hide MCP tools from the main agent?

4•azebazenestor•1d ago•3 comments

Ask HN: Why are there no talks about Seedance 2.0 on Hacker News?

8•ElectroNomad•2d ago•7 comments

Watching an elderly relative trying to use the modern web

48•ColinWright•3d ago•19 comments

Tell HN: Attackers using Google parental controls to prevent account recovery

19•TazeTSchnitzel•2d ago•4 comments

Top non-ad google result for "polymarket" in Australia is a crypto scam

17•rtrgrd•4d ago•3 comments

Ask HN: Claude web blocked its assets visit via csp?

6•xgstation•2d ago•2 comments

Ask HN: Companies that advertise being a "best place to work", is it a red flag?

12•jrs235•4d ago•16 comments

SEL Deploy – Cryptographically chained deployment timeline

2•chokriabouzid•2d ago•1 comments

Picknar – Lightweight YouTube Thumbnail Extractor (No Login, No API Key)

3•Picknar•3d ago•1 comments

Grand Time: Time-Based Models in Decentralized Trust

3•AGsist•3d ago•0 comments

Ask HN: How do you debug multi-step AI workflows when the output is wrong?

4•terryjiang2020•2d ago•7 comments

Ask HN: Any AI / Agent power users out there? Do you have any tips?

8•uejfiweun•2d ago•9 comments

You've reached the end!