We have a tool live that uses it and I'm quite happy so I'm both looking for interesting use cases from others but also to be honest I'm reasonably sure I've just identified today that DuckDB is leaking memory quite seriously [2] so I'm curious to hear if other people have noticed this or if it's maybe something that's not as relevant to others since people might be running DuckDB pipelines in ephemeral envs like lambdas etc. where a memory leak might not matter as much.
[1] https://news.ycombinator.com/item?id=46645176
[2] https://github.com/duckdb/duckdb/issues/20569
delaminator•5h ago
DuckDB (like most database systems and many applications using memory allocators like jemalloc or mimalloc) doesn't immediately release memory back to the OS after freeing it internally.
Memory allocator strategy - DuckDB uses an allocator that keeps freed memory in a pool for reuse. Returning memory to the OS is expensive (system calls, page table updates), so allocators hold onto it anticipating future allocations.
yakkomajuri•4h ago
Maintainers have acknowledged problems like this on other issues too.
But I'll nevertheless read more into this!
delaminator•3h ago
yakkomajuri•4h ago
"Running DROP TABLE should free the memory used by the table"
Source: https://duckdb.org/docs/stable/sql/statements/drop