frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

Open in hackernews

Load Test GlassFlow for ClickHouse: Real-Time Dedup at Scale

https://www.glassflow.dev/blog/load-test-glass-flow-for-click-house-real-time-deduplication-at-scale
1•super_ar•4h ago

Comments

super_ar•4h ago
Hi HN, A few weeks ago, we shared GlassFlow: Open Source streaming ETL to dedup and join streams from Kafka for ClickHouse (https://news.ycombinator.com/item?id=43953722).

One of the top questions we received was: “How well does it perform at high throughput?”

We ran a load test and would like to share some results with you.

Summary of the test:

- Tested on 20m records

- Kafka produced 55,000 records/sec

- Processing rate of GlassFlow (deduplication): 9,000+ records/sec

- Measured on a MacBook Pro (M3 Max)

- End-to-end latency: <0.12 ms per request

Here is the blog post with full test results and tried with different parameters (rps, # of publishers, etc.): https://www.glassflow.dev/blog/load-test-glass-flow-for-clic...

It was important to us to set up the testing in a way that everybody could reproduce. Here are the docs: https://docs.glassflow.dev/load-test/setup

We would love to get feedback, especially from folks consuming high-throughput in ClickHouse.

Thanks for reading!

Ashish and Armend (founders)

Show HN: ZenQuery – Query your data files (CSV, parquet, etc.) in plain English

2•freakynit•2m ago•0 comments

Huang's Law

https://thechipletter.substack.com/p/huangs-law
1•rbanffy•7m ago•0 comments

Postly – A privacy-first, open social platform for creators

https://postlyapp.com
1•Malik-Whitten•9m ago•1 comments

Show HN: LLM-Bible – A Visual Interface for Exploring the Latest LLM Research

https://llm-bible.github.io
1•sjm213•11m ago•0 comments

Show HN: Cursor AI for Thinking

https://irulan.xyz
1•tobilobaaa•12m ago•2 comments

State Power Sponsored by Coinbase

https://www.citationneeded.news/issue-86/
1•wslh•12m ago•0 comments

Show HN: New Python GUI Framework (better than others?)

https://github.com/mebaadwaheed/winup
1•ebaadesque•13m ago•0 comments

Show HN: pshunt: Terminal app for easily killing processes

https://github.com/jamesma100/pshunt
1•battle-racket•15m ago•1 comments

Can Europe Level Up Its Defense Game?

https://defragzone.substack.com/p/can-europe-level-up-its-defense-game
1•frag•17m ago•0 comments

AI Pressure from the Top: CEOs Urge Workers to Adapt

https://gazeon.site/ai-pressure-from-the-top-ceos-urge-workers-to-adapt/
1•eligrid•18m ago•2 comments

Desktop Pictures taken by Steve Jobs (2020)

https://techreflect.org/2020/05/desktop-pictures-taken-by-steve-jobs/
1•chaosmachine•18m ago•0 comments

Show HN: Everyone's Building AI Startups – But Most Are Missing This One Thing

1•sahil423•25m ago•1 comments

How do people see SEO transforming into GEO

1•yambox•25m ago•0 comments

Can You Choose an AI Model That Harms the Planet Less?

https://www.nytimes.com/2025/06/19/climate/ai-emissions-chatbot-accuracy.html
2•babushkaboi•26m ago•2 comments

I'm 17, Building in Public Instead of Waiting – Lessons from Failing Fast

1•sahil423•27m ago•0 comments

New code in Spotify's app references the long-awaited 'lossless' tier

https://techcrunch.com/2025/06/18/new-code-in-spotifys-app-references-the-long-awaited-lossless-tier/
1•babushkaboi•29m ago•0 comments

China's Manufacturing Dominance: State and Competition

https://www.dwarkesh.com/p/arthur-kroeber
1•consumer451•29m ago•1 comments

Will long context windows solve all your problems?

https://frontierai.substack.com/p/will-long-context-windows-solve-all
2•cgwu•30m ago•0 comments

Juneteenth in Photos

https://texashighways.com/travel-news/the-history-of-juneteenth-in-photos/
14•ohjeez•34m ago•0 comments

Select FOR UPDATE considered harmful in PostgreSQL

https://www.cybertec-postgresql.com/en/select-for-update-considered-harmful-postgresql/
3•samokhvalov•35m ago•0 comments

Apple Created a Custom iPhone Camera for 'F1'

https://www.wired.com/story/apple-created-a-custom-iphone-camera-for-f1/
3•tosh•35m ago•0 comments

Hailuo 02 AI Video Generator Online – Hailuo02.video

https://hailuo02.video/
1•MxcAlex•35m ago•1 comments

macOS Tahoe Beta Drops FireWire Support

https://www.macrumors.com/2025/06/19/macos-tahoe-beta-drops-firewire-support/
2•tosh•35m ago•0 comments

Corporate crypto treasury bubble risks are real

https://www.theblock.co/post/358875/crypto-treasury-company-bubble-risks-real-but-more-nuanced-than-past-boom-bust-cycles-analyst
2•wslh•36m ago•0 comments

Cataphract: Medieval-fantasy roleplaying wargame, in the Black-Sea C. 1300

https://samsorensen.blot.im/cataphracts-design-diary-1
1•vidro3•36m ago•0 comments

Africa's IPv6 Deployment Is Taking Off

https://pulse.internetsociety.org/blog/africas-ipv6-deployment-is-taking-off
1•KomoD•37m ago•0 comments

16B hit in 'one of largest data breaches in history' – what's been exposed

https://www.tomsguide.com/computing/online-security/16-billion-hit-in-one-of-largest-data-breaches-in-history-whats-been-exposed-and-how-to-protect-yourself
2•bookmtn•37m ago•0 comments

When Earth iced over, early life may have sheltered in meltwater ponds

https://phys.org/news/2025-06-earth-iced-early-life-meltwater.html
1•Stratoscope•39m ago•1 comments

In Praise of "Normal" Engineers

https://charity.wtf/2025/06/19/in-praise-of-normal-engineers/
17•zdw•39m ago•3 comments

The Universe's Darkest Mysteries Are Coming into Focus

https://www.nytimes.com/2025/06/19/science/telescope-vera-rubin-dark-matter.html
2•sharkweek•41m ago•0 comments