frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

Frozen DuckLakes for Multi-User, Serverless Data Access

https://ducklake.select/2025/10/24/frozen-ducklake/
29•g0xA52A2A•5d ago

Comments

gopalv•2h ago
The useful part is that duckdb is so easy to use as a client with an embedded server, because duckdb is a great client (+ a library).

Similar to how git can serve a repo from a simple http server with no git installed on that (git update-server-info).

The frozen part is what iceberg promised in the beginning, away from Hive's mutable metastore.

Point to a manifest file + parquet/orc & all you need to query it is S3 API calls (there is no metadata/table server, the server is the client).

> Creating and publishing a Frozen DuckLake with about 11 billion rows, stored in 4,030 S3-based Parquet files took about 22 minutes on my MacBook

Hard to pin down how much of it is CPU and how much is IO from s3, but doing something like HLL over all the columns + rows is pretty heavy on the CPU.

ryanschneider•1h ago
Even cooler, let's say you need to "update" a subset of your parquet files after they are written. Once you have your parquet files in a ducklake, you can "virtually" update them (the files themselves aren't touched, just new ones created). Something like:

- create your frozen ducklake

- run whatever "normal" mutation query you want to run (DELETE, UPDATE, MERGE INTO)

- use `ducklake_rewrite_data_files` to make new files w/ mutations applied, then optionally run `ducklake_merge_adjacent_files` to compact the files as well (though this might cause all files to change).

- call `ducklake_list_files` to get the new set of active files.

- update your upstream "source of truth" with this new list, optionally deleting any files no longer referenced.

The net result should be that any files "touched" by your updates will have new updated versions alongside them, while any that were unchanged should just be returned in the list files operation as is.

Affinity Studio now free

https://www.affinity.studio/get-affinity
394•dagmx•3h ago•315 comments

The ear does not do a Fourier transform

https://www.dissonances.blog/p/the-ear-does-not-do-a-fourier-transform
131•izhak•2h ago•51 comments

Launch HN: Propolis (YC X25) – Browser agents that QA your web app autonomously

https://app.propolis.tech/#/launch
59•mpapazian•2h ago•14 comments

987654321 / 123456789

https://www.johndcook.com/blog/2025/10/26/987654321/
386•ColinWright•4d ago•71 comments

Show HN: I made a heatmap diff viewer for code reviews

https://0github.com
90•lawrencechen•4h ago•27 comments

Free software scares normal people

https://danieldelaney.net/normal/
231•cryptophreak•4h ago•161 comments

UDP isn't unreliable, it's a convertible (2024)

https://www.proxylity.com/articles/udp-is-a-convertible.html
30•mlhpdx•5d ago•21 comments

Minecraft HDL, an HDL for Redstone

https://github.com/itsfrank/MinecraftHDL
6•sleepingreset•20m ago•0 comments

Rapid Brightening of 3I/Atlas Ahead of Perihelion

https://arxiv.org/abs/2510.25035
13•bikenaga•54m ago•12 comments

ZOZO's Contact Solver for physics-based simulations

https://github.com/st-tech/ppf-contact-solver
46•vintagedave•3h ago•24 comments

I have released a 69.0MB version of Windows 7 x86

https://twitter.com/XenoPanther/status/1983477707968291075
60•rvnx•1h ago•13 comments

TruthWave – A Platform for Corporate Whistleblowers

https://www.truthwave.com
4•mannuch•22m ago•0 comments

Some people can't see mental images

https://www.newyorker.com/magazine/2025/11/03/some-people-cant-see-mental-images-the-consequences...
56•petalmind•1h ago•119 comments

US declines to join more than 70 countries in signing UN cybercrime treaty

https://therecord.media/us-declines-signing-cybercrime-treaty?
239•pcaharrier•4h ago•159 comments

Zig's New Async I/O

https://andrewkelley.me/post/zig-new-async-io-text-version.html
147•todsacerdoti•1d ago•39 comments

Show HN: In a single HTML file, an app to encourage my children to invest

https://roberdam.com/en/dinversiones.html
153•roberdam•8h ago•288 comments

Qt Creator 18 Released

https://www.qt.io/blog/qt-creator-18-released
92•jrepinc•2h ago•11 comments

Springs and bounces in native CSS

https://www.joshwcomeau.com/animation/linear-timing-function/
6•feross•2d ago•0 comments

Israel demanded Google and Amazon use secret 'wink' to sidestep legal orders

https://www.theguardian.com/us-news/2025/oct/29/google-amazon-israel-contract-secret-code
337•skilled•1d ago•121 comments

Learn Multiplatform Z80 Assembly Programming with Vampires

https://www.chibiakumas.com/z80/
11•surprisetalk•4d ago•0 comments

Frozen DuckLakes for Multi-User, Serverless Data Access

https://ducklake.select/2025/10/24/frozen-ducklake/
29•g0xA52A2A•5d ago•2 comments

Replacing EBS and Rethinking Postgres Storage from First Principles

https://www.tigerdata.com/blog/fluid-storage-forkable-ephemeral-durable-infrastructure-age-of-agents
82•mfreed•1d ago•34 comments

NaN, the not-a-number number that isn't NaN

https://piccalil.li/blog/nan-the-not-a-number-number-that-isnt-nan/
72•tobr•1w ago•64 comments

Tell HN: Azure outage

856•tartieret•1d ago•779 comments

Minecraft removing obfuscation in Java Edition

https://www.minecraft.net/en-us/article/removing-obfuscation-in-java-edition
945•SteveHawk27•1d ago•405 comments

You can't turn off Copilot in the web versions of Word, Excel, or PowerPoint

https://support.microsoft.com/en-us/office/turn-off-copilot-in-microsoft-365-apps-bc7e530b-152d-4...
44•artbristol•2h ago•7 comments

Aisuru botnet shifts from DDoS to residential proxies

https://krebsonsecurity.com/2025/10/aisuru-botnet-shifts-from-ddos-to-residential-proxies/
4•feross•1d ago•0 comments

Acronymy (Can we define every word as an acronym?)

https://acronymy.net/
29•cubefox•5d ago•10 comments

Spinning Up an Onion Mirror Is Stupid Easy

https://flower.codes/2025/10/23/onion-mirror.html
161•speckx•1w ago•61 comments

Typst's Math Mode Problem

https://laurmaedje.github.io/posts/math-mode-problem/
98•marcianx•6d ago•44 comments