A personal project — building a storage engine for fun, exploring LSM design, and developing with an LLM as a partner.
Comments
emschwartz•1h ago
Seems neat. Could you plug it in to DataFusion to use their SQL layer?
nlz•1h ago
Yes! This is a great idea. Thanks for mentioning it! GlueSQL could be another cool target too (https://github.com/gluesql/gluesql). I think there's a fun exploration in taking a storage engine and seeing if it's compatible with different SQL layers — GlueSQL as a simpler starting point, DataFusion as the more complete option (https://github.com/apache/datafusion). Plugging into a real SQL engine seems like a great strategy for uncovering bugs, fixing correctness issues, and discovering what use cases the storage layer actually needs to support. It's also interesting how certain SQL engines and storage engines tend to align well with each other — the right pairing can unlock a lot. Definitely a direction I want to explore.
emschwartz•1h ago
nlz•1h ago