frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

From Pipeline Sprawl to Table-Based Data Flows: ETL with Pub/Sub for Tables

3•immortan_dag•4h ago
Pub/Sub for Tables is neither a replacement for messaging systems like Kafka, nor a substitute for OLTP or OLAP databases.

It's a declarative approach to data flows, applying publish /subscribe semantics to tables rather than messages or events.

Why tables?

Because nearly all data used in analytics, whether it originates from APIs, logs, streams, or files—eventually becomes tabular. Starting from tables, rather than converting to them later, allows metadata to stay intact throughout the lifecycle. This means semantics can be preserved end-to-end, enabling better trust, alignment, and clarity across the organization.

This is the foundation we are building at Tabsdata.

Tabsdata is an open-core system for managing data flows around immutable, versioned tables. It provides:

* Declarative data publishing, transformation, and subscription

* Full lineage and provenance for every table and update

* Free-form Python transformations in a controlled, reproducible environment

* A Rust-based core for safety and performance

* Python bindings for developers

* Both a browser-based UI and a comprehensive scriptable CLI

Here is a short overview and demo we shared recently:

https://www.youtube.com/watch?v=qCZIRC9khmA

More details on the project:

https://tabsdata.com

We are still early and would deeply value your feedback, pushback, or suggestions. If you have dealt with pipeline sprawl, loss of trust in data lineage, or the burden of orchestration, please share your thoughts.

What resonates? What does not? What would make something like this more useful in your world?

OpenAI #15: More on OpenAI's Paranoid Lawfare Against Advocates of SB 53

https://thezvi.substack.com/p/openai-15-more-on-openais-paranoid
1•paulpauper•2m ago•0 comments

I Love the 2025 Economics Nobel Laureates

https://arnoldkling.substack.com/p/on-the-2025-economics-nobel-laureates
1•paulpauper•2m ago•0 comments

Choosing Friction

https://phirephoenix.com/blog/2025-10-11/friction
1•panic•7m ago•0 comments

Show HN: Link Management Platform for Affiliate Marketers and Content Creators

1•amirawel•9m ago•0 comments

Panther Lake's Reveal at ITT 2025 – By George Cozma

https://chipsandcheese.com/p/panther-lakes-reveal-at-itt-2025
1•rbanffy•10m ago•0 comments

Putin Is Not Winning

https://www.theatlantic.com/international/archive/2025/10/overestimating-putin-russia-ukraine-war...
2•JumpCrisscross•11m ago•0 comments

Create My Token

https://createmytoken.net
1•marvin76•11m ago•0 comments

NumPy-First AI: Persona-Aware Semantic Models Without GPUs

https://github.com/farukalpay/Semantic-Lexicon
1•HenryAI•13m ago•1 comments

Mapping RNA-protein chats could uncover new treatments for cancer, brain disease

https://phys.org/news/2025-10-rna-protein-chats-uncover-treatments.html
2•PaulHoule•13m ago•0 comments

Who Said Neural Networks Aren't Linear?

https://arxiv.org/abs/2510.08570
1•ComplexSystems•14m ago•0 comments

A modern way to go to market with content, trust and authority

1•theomniaiclub•14m ago•0 comments

How to Turn Liquid Glass into a Solid Interface – TidBITS

https://tidbits.com/2025/10/09/how-to-turn-liquid-glass-into-a-solid-interface/
3•tambourine_man•16m ago•0 comments

Your Data Model is Your Destiny

https://notes.mtb.xyz/p/your-data-model-is-your-destiny
2•hunglee2•16m ago•0 comments

Better SRGB to Greyscale Conversion

https://30fps.net/pages/better-srgb-to-greyscale/
2•ibobev•17m ago•0 comments

A Nobel for thinking about long-term growth

https://www.noahpinion.blog/p/a-nobel-for-thinking-about-long-term
1•paulpauper•18m ago•1 comments

CHIP8 and How Emulators Work

https://retrogamecoders.com/chip8-and-how-emulators-work/
1•ibobev•18m ago•0 comments

3D-printed device can pull drinking water out of thin air

https://www.popsci.com/technology/water-from-air-project/
1•geox•18m ago•0 comments

Soon You'll Be Able to Shop Walmart in ChatGPT. Here's Why It Matters

https://www.wsj.com/business/retail/chatgpt-walmart-shopping-3e411e83
3•calstad•19m ago•0 comments

Nvidia's DGX Spark AI mini-PC goes up for sale October 15 – 1 petaFLOP

https://www.tomshardware.com/desktops/mini-pcs/nvidias-dgx-spark-ai-mini-pc-goes-up-for-sale-octo...
2•diymaker•19m ago•1 comments

Classic Demo Effects, Fire

https://www.4rknova.com//blog/2019/03/17/fire-effect
1•ibobev•19m ago•0 comments

America Is Sliding Toward Illiteracy

https://www.theatlantic.com/ideas/archive/2025/10/education-decline-low-expectations/684526/
36•JumpCrisscross•20m ago•5 comments

Show HN: I've built a prototype for a physical e-ink todo list with voice input

https://github.com/ilyaLibin/todoink
1•cassettelabs•21m ago•0 comments

Zawinski's Law Was Never About Email

https://danverbraganza.com/writings/reinterpreting-zawinskis-law
1•nvader•22m ago•0 comments

Rumdl: A Markdown Linter Written in Rust

https://github.com/rvben/rumdl
1•todsacerdoti•23m ago•0 comments

Guide: Setting up Nvidia DGX Spark with ggml

https://github.com/ggml-org/llama.cpp/discussions/16514
1•homarp•24m ago•0 comments

In the ruins of a forgotten California desert utopia

https://kubicki.org/letters/the-dogs-of-llano-del-rio-i/
2•falibout•25m ago•0 comments

Show HN: An open source access logs analytics script to block bot attacks

https://github.com/tempesta-tech/webshield
2•krizhanovsky•27m ago•0 comments

Sikorsky Converts Black Hawk into U-Hawk, a Battle-Ready Autonomous UAS

https://news.lockheedmartin.com/2025-10-13-Sikorsky-Converts-BLACK-HAWK-into-U-Hawk-A-Battle-Read...
1•gnabgib•28m ago•0 comments

TokEstimator – Estimate LLM Inference Performance on Private Hardware

https://tokestimator.com
1•thinkelastic•28m ago•0 comments

Americans are losing millions to scammers at crypto ATMs

https://www.cnn.com/interactive/2025/10/us/crypto-atm-scams-companies-profit-invs-vis/
8•thelastgallon•30m ago•1 comments