frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

Open in hackernews

Algorithms on billion-scale graph using 10GB RAM: I love DataFusion

https://semyonsinchenko.github.io/ssinchenko/post/datafusion-graphs-cc-2/
21•speckx•1h ago

Comments

slopblast•1h ago
Really cool visualization, amazing how it resembles a neural network.
chrisweekly•26m ago
> "I can compute PageRank on a directed graph with one billion edges (graph500-26 from the Graphalytics dataset) using 5 GB of memory. Alternatively, I can identify all the weakly connected components in a graph with two billion edges (twitter_mpi from the same dataset collection) using 10 GB of memory. Neither NetworkX nor Igraph can do this; most existing graph algorithms require the graph to fit into memory. Previously, I thought you needed Apache Spark and GraphFrames for billion-scale graph analytics. Now, however, I think all you need is a laptop. I have completely changed my old opinion about using Apache DataFusion for graph analytics."

Impressive!

anon7725•4m ago
> most existing graph algorithms require the graph to fit into memory.

You can get pretty far with sparse graphs, which are just arrays, in combination with memory mapping.

esafak•21m ago
Does it support out-of-core or multi-processor processing?
theLiminator•15m ago
DataFusion is really cool, it's kind of like the LLVM of the OLAP world.

Elevators

https://john.fun/elevators
282•Jrh0203•2h ago•103 comments

Big Food vs. the People

https://www.lighthousereports.com/investigation/big-food-vs-the-people/
49•jruohonen•1h ago•11 comments

DeepSeek V4 Flash 0731 Intelligence, Performance and Price Analysis

https://artificialanalysis.ai/models/deepseek-v4-flash
397•theanonymousone•9h ago•217 comments

DeepSeek-V4-Flash Update

https://api-docs.deepseek.com/updates/
544•dnhkng•11h ago•261 comments

Algorithms on billion-scale graph using 10GB RAM: I love DataFusion

https://semyonsinchenko.github.io/ssinchenko/post/datafusion-graphs-cc-2/
22•speckx•1h ago•5 comments

A GTK4 SSH-askpass in Zig

https://xn--gckvb8fzb.com/a-gtk4-ssh-askpass-in-zig/
28•surprisetalk•1h ago•6 comments

Miso (YC S16) is hiring for U.S. expansion

https://www.ycombinator.com/companies/miso/jobs/g2uAlMG-founding-business-lead-u-s-expansion
1•victorology•20m ago

Arch Linux disables AUR package adoption

https://lwn.net/Articles/1086489/
50•database64128•3h ago•19 comments

Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

https://www.jeffgeerling.com/blog/2026/getting-25g-ethernet-mac-thunderbolt/
11•speckx•1h ago•2 comments

The End of an Era

https://hughhowey.com/the-end-of-an-era/
290•harscoat•5h ago•327 comments

The session you cannot take with you

https://earendil.com/posts/session-portability/
674•apitman•13h ago•194 comments

The C ``Clockwise/Spiral Rule''

https://c-faq.com/decl/spiral.anderson.html
23•etrvic•3h ago•8 comments

13 Models and 4 Agents on SWE Tasks: Go, Java, Python, Rust, TS

https://swe-rebench.com
12•ibragim_bad•1h ago•3 comments

The Art of Decision-Making (2019)

https://www.newyorker.com/magazine/2019/01/21/the-art-of-decision-making
13•EndXA•1h ago•0 comments

Google fixed more Chrome bugs in June than over the past two years, thanks to AI

https://blog.google/security/chrome-stronger-with-every-update/
405•Garbage•9h ago•394 comments

Show HN: BitBang – Reach machines behind NAT from a browser, no account

https://github.com/richlegrand/bitbang-cli
14•narragansett•2h ago•2 comments

Anti-fraud tools can't keep pace with scammers exploiting cheap internet calling

https://broadbandbreakfast.com/how-to-fight-back-against-fraudulent-robocalls/
23•dredmorbius•3h ago•12 comments

How JPEG works: Interactively explore JPEG's lossy compression methods

https://cgjennings.ca/articles/jpeg-compression/
7•at1as•4d ago•0 comments

IMAX vs. IMAX 70mm: The difference between these two cinema formats

https://www.engadget.com/2220571/differences-between-imax-70mm/
51•ksec•6d ago•65 comments

New Defcon Badges Pack a Unique Open-Source Chip That Doubles as a Security Key

https://www.wired.com/story/defcon-34-badge-baochip-andrew-bunnie-huang/
41•hn_acker•1h ago•7 comments

Danube's record low levels force shutdown of Hungary's only nuclear plant

https://www.bbc.com/news/articles/cn0nqv05g0do
68•vrganj•9h ago•88 comments

Is AI reasoning right for the wrong reasons?

https://www.quantamagazine.org/is-ai-reasoning-right-for-the-wrong-reasons-20260731/
72•retupmoc01•1h ago•75 comments

Show HN: What should the GUI for AI agents look like?

https://marbleos.com/demo
81•akbabu•12h ago•48 comments

The Maxwell Conjecture Is False (GPT 5.6 Sol)

https://arxiv.org/abs/2607.27197
111•rahen•5h ago•102 comments

Show HN: I built a cross-browser extension that controls fingerprinting surfaces

https://privacything.com/en/
13•tomaszjanusz•2h ago•2 comments

Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

https://www.bloomberg.com/news/articles/2026-07-31/moonshot-s-kimi-built-on-20-000-nvidia-chip-cl...
67•gk1•3h ago•44 comments

Situational Awareness down 67% in July in AI stock rout

https://www.wsj.com/finance/investing/situational-awareness-down-67-in-july-in-ai-stock-rout-cd19...
109•pondsider•3h ago•107 comments

Winding Down Artichoke Ruby

https://hyperbo.la/w/winding-down-artichoke-ruby/
42•ksec•5d ago•5 comments

The most official water costs $120k a gallon

https://signoregalilei.com/2026/07/26/the-most-official-water-costs-120000-a-gallon/
25•surprisetalk•2h ago•3 comments

Show HN: Gander, an Android file viewer that asks for no permissions

https://github.com/mokshablr/gander
175•mokshablr•11h ago•62 comments