However, I wonder whether the GPU's are a good fit for this to begin with.
Counterpoint: Xilinx side of the AMD shop has developed Alveo-series accelerators which used to be pretty basic SmartNIC platforms, but have since evolved to include A LOT more programmable logic and compute IP. You may have heard about these in video encoding applications, HFT, Blockchain stuff, what-have-you. A lot of it has to with AI stuff, see Versal[2]. Raw compute figures are often cited as "underwhelming," and it's unfortunate that so many pundits are mistaking the forest for the trees here. I don't think the AI tiles in these devices are really meant for end-to-end LLM inference, even though memory bandwidth in the high-end devices allows it.
The sauce is compute-in-network over fabrics.
Similarly to how PG-Strom would feed the GPU with relational data from disk, or network directly, many AI teams on the datacenter side are now experimenting with data movement, & intermediate computations (think K/V cache management) over 100/200/800+G fabrics. IMHO, compute-in-network is the MapReduce of this decade. Obviously, there's demand for it in the AI space, but a lot of it lends nicely to the more general-purpose applications, like databases. If you're into experimental networking like that, Corundum[3] by Alex Forencich is a great, perhaps the best, open source NIC design for up to 100G line rate. Some of the cards it supports also expose direct-attach NVMe's over MCIO for latency, and typically have as many as two, or four SFP28 ports for bandwidth.
This is a bit naive way to think about it, but it would have to do!
Postgres is not typically considered to "scale well," but oftentimes this is a statement about its tablespaces more than anything; it has foreign data[4] API, which is how you extend Postgres as single point-of-consumption, foregoing some transactional guarantees in the process. This is how pg_analytics[5] brings DuckDB to Postgres, or how Steampipe[6] similarly exposes many Cloud and SaaS applications. Depending on where you stand on this, the so-called alternative SQL engines may seem like moving in the wrong direction. Shrug.
[1] https://heterodb.github.io/pg-strom/
[2] https://xilinx.github.io/AVED/latest/AVED%2BOverview.html
[3] https://github.com/corundum/corundum
[4] https://wiki.postgresql.org/wiki/Foreign_data_wrappers
I think the GPU could be a great fit for OLAP, but when it comes to the nasty OLTP use cases the CPU will absolutely dominate.
Strictly serialized transaction processing facilities demand extremely low latency compute to achieve meaningful throughput. When the behavior of transaction B depends on transaction A being fully resolved, there are no magic tricks you can play anymore.
Consider that talking to L1 is at least 1,000x faster than talking to the GPU. Unless you can get a shitload of work done with each CPU-GPU message (and it is usually the case that you can), this penalty is horrifyingly crippling.
> Consider that talking to L1 is at least 1,000x faster than talking to the GPU.
This is largely true for "traditional" architectures, but s/GPU/TPU and s/L1/CMEM and suddenly this is no big deal anymore. I'd like Googlers to correct me here, but it seems well in line with classic MapReduce, and probably something that they're doing a lot outside of LLM inference... ads?
If a client wishes to use your GPU-based RDBMS engine, it needs to make a trip through the CPU first, does it not?
Send the query to both GPU and CPU pipelines at the same time and use whichever comes back first
GPUs can incur higher failure risks, and thus one will not normally find them in high-reliability roles. =3
Personally, I'd rather have another dual cpu Epyc host with maximum ECC ram, as I have witnessed NVIDIA GPU failed closed to take out host power supplies. =3
Maintainer of pg_analytics (now part of pg_search) here. I 100% agree that the statements against Postgres are often exaggerated. In practice, we see both the smallest and the largest companies "just use Postgres" while mid-scale companies often overthink their solution.
That said, there are indeed phenomenal "alternate" SQL engines. I've seen many users see great success on tools like ClickHouse, which ParadeDB is not yet competitive with, and sometimes (dare I say) even Elasticsearch. As for whether this one is one of them... That I couldn't say
Personally, the risk associated with GPU failure rates is important, and I have witnessed NVIDIA cards take out entire hosts power-systems by failing closed. i.e. no back-plane diagnostics as the power supplies are in "safe" off condition.
I am sure the use-cases for SQL + GPU exist, but for database reliability no GPU should be allowed in those racks. =3
It requires compute capability >= 7.0 (which is RTX 20xx and higher)
cpard•7mo ago
It has been supported by engines like duckdb but the main serious use case of it I’m aware of is from Apache gluten where it is used to add Velox as the execution engine of Spark.
It’s an ambitious project and certainly has limitations but more projects like this are needed to push it forward.
gavinray•7mo ago
We ended up having to roll our own [0], but I still think Substrait is a fantastic idea (someone has to solve this problem, eventually) and it's got a good group of minds consistently working on it, so my outlook for it is bright.
[0] https://hasura.github.io/ndc-spec/reference/types.html#query...
cpard•7mo ago
Substrait is the type of project that can only be built by trying to engineer real systems, just like you tried to do.