Thanks! I was also told to make a performance-focused demo... didn't do it in time, but was able to go from a 44-minute BigQuery job to a 5.5-minute ParaQuery job, with a similar dataset/query as the video here.

8x faster!

Still figuring out pricing! For our first customers, we're doing pricing as either bytes scanned or by compute time, similar to BigQuery. Also experimenting with a contract that also gives the minimum of the two potential charges (up to a sustainable limit).

However, for deployments to the customer's cloud, it would be a stereotypical enterprise license + support.

Can't wait to actually add an FAQ to the site, hopefully based off the questions asked here. Pricing is one of the things preventing me from just allowing self-serve, since it has to be stable, sustainable, and cheap!

Also, with the GPU clouds, pricing would have to be different per cloud, though I guess I can worry about that later. Would be crazy cheap(er) to process on them.

As far as I know, GPUs are definitely still being used in crypto/web3... and AI for that matter :P

After ETH PoS, GPUs are no longer used for wide scale crypto mining.

No relationship... yet! Hoping to have a good relationship in the future so I have a business reason to fly to Japan :D

Btw, interesting thing they said here: "By utilization of GPU (Graphic Processor Unit) device which has thousands cores per chip"

It's more like "hundreds", since the number of "real" cores is like (CUDA cores / 32). Though I think we're about to see 1k cores (SMSPs).

That being said, I do believe CUDA cores have more interesting capabilities than a typical vector lane, i.e. for memory operations (thank the compiler). Would love to be corrected!

Not sure if Spark is a classical portion for GPU compute ;) Well, outside of HPC and research.

SQL on GPUs is definitely a research classic, dating back to 2004 at least: https://gamma.cs.unc.edu/DB/

Came here to say the same thing.

Set Theory is the classical foundation of SQL:

https://www.sqlshack.com/mathematics-sql-server-fast-introdu...

It's analogous to how functional programming expressed through languages like lisp is the classical foundation of spreadsheets.

I believe that skipping first principles (sort of like premature optimization) is the root of all evil. Some other examples:

- If TCP had been a layer above UDP instead of its own protocol beside it, we would have had real peer to peer networking this whole time instead of needing WebRTC.

- If we had a common serial communication standard analogous to TCP for sockets, then we wouldn't need different serial ports like USB, Thunderbolt and HDMI.

- If we hid the web browser's progress bar and used server-side rendering with forms, we could implement the rich interfaces of single-page applications with vastly reduced complexity by keeping the state, logic and validation in one place with no perceptible change for the average user.

- If there was a common scripting language bundled into all operating systems, then we could publish native apps as scripts with substantially less code and not have to choose between web and mobile for example.

- If we had highly multicore CPUs with hundreds or thousands of cores, then multiprocessing, 3D graphics and AI frameworks could be written as libraries running on them instead of requiring separate GPUs.

And it's not just tech. The automative industry lacks standard chassis types and even OEM parts. We can't buy Stirling engines or Tesla turbines off the shelf. CIGS solar panels, E-ink displays, standardized removable batteries, thermal printers for ordinary paper, heck even "close enough" contact lenses, where are these products?

We make a lot of excuses for why the economy is bad, but just look at how much time and effort we waste by having to use cookie cutter solutions instead of having access to the underlying parts and resources we need. I don't think that everyone is suddenly becoming neurodivergent from vaccines or some other scapegoat, I think it's just become so obvious that the whole world is broken and rigged to work us all to the grave to make some guy rich that it's giving all of us ADHD symptoms from having to cope with it.

Large scale shuffles: Absolutely. One of the larger queries we ran saw a 450TB shuffle -- this may require more than just deploying the spark-rapids plugin, however (depends on the query itself and specific VMs used). Shuffling was the majority of the time and saw 100% (...99%?) GPU utilization. I presume this is partially due to compressing shuffle partitions. Network/disk I/O is definitely not the bottleneck here.

It's difficult to say what "workloads" are significant, and easier to talk about what doesn't really work AFAIK. Large-scale shuffles might see 4x efficiency, assuming you can somehow offload the hash shuffle memory, have scalable fast storage, etc... which we do. Note this is even on GCP, where there isn't any "great" networking infra available.

Things that don't get accelerated include multi-column UDFs and some incompatible operations. These aren't physical/logical limitations, it's just where the software is right now: https://github.com/NVIDIA/spark-rapids/issues

Multi-column UDF support would likely require some compiler-esque work in Scala (which I happen to have experience in).

A few things I expect to be "very" good: joins, string aggregations (empirically), sorting (clustering). Operations which stress memory bandwidth will likely be "surprisingly" good (surprising to most people).

Otherwise, Nvidia has published a bunch of really-good-looking public data, along with some other public companies.

Outside of Spark, I think many people underestimate how "low-latency" GPUs can be. 100 microseconds and above is highly likely to be a good fit for GPU acceleration in general, though that could be as low as 10 microseconds (today).

Just checked out Velox. It's awesome that you're reducing duplicate eng effort! What was your talk about?

I'm not too familiar with HeavyDB, but here are the main differences:

- We're fully compatible with Spark SQL (and Spark). Meaning little to no migration overhead.

- Our focus is on distributed compute first.

- That means ParaQuery isn't a database, just an MPP engine (for now). Also means no data ingestion/migration needed.

Is the shuffle the biggest issue? Not too sure about joins but one of the datasets we're currently dealing with has a couple trillion rows. Would love to chat about this!

You should try using an S3 based shuffle plugin: https://github.com/IBM/spark-s3-shuffle

Then mount FSX for Lustre on all of your EMR nodes and have it write shuffle data there. It will massively improve performance and shuffle issues will disappear.

Is expensive though. But you can offset the cost now because you can run entirely Spot instances for your workers as if you lose a node there's no recomputation of the shuffle data.

I was about to comment that Gluten is only targeting CPU vectorization, but then I found this (very cool!): https://github.com/apache/incubator-gluten/issues/9098

I'm not very familiar with Gluten, but I'll still comment on the CPU side though, assuming that one of Gluten's goals is to use the full vector processing (SIMD) potential of the CPU. In that case, we'd still be memory(-bandwidth)-bound, not to mention the significantly lower FLOPs of the CPU itself. If we vectorize Spark (or any MPP) for efficient compute, perhaps we should run it on hardware optimized for vectorized, super-parallel, high-throughput compute.

Also, there's nothing which says we can't use Gluten to have even more CPU+GPU utilization!

Apache DataFusion Comet also exists which is similar:

https://datafusion.apache.org/comet/

Both are stop gaps though since optimised SIMD accelerated Vector support is coming to the JVM albeit extremely slowly: https://openjdk.org/jeps/508

They have >1PB of data to ETL, with some queries hitting 450TB of pure shuffle.

It's very true that most users don't need something like BigQuery or Snowflake. That's why some startups have come up to save Snowflake cost by "simply" putting a postgres instance in front of it!

In fact, I just advised someone recently to simply use Postgres instead of BigQuery since they had <1TB and their queries weren't super intensive.

> A lot of BigQuery users would be surprised to find they don't need BigQuery.

No they wouldn't.

a) BigQuery is the only managed, supported solution on GCP for SQL based analytical workloads. And they are using it because they started with GCP and then chose BigQuery.

b) I have supported hundreds of Data Scientists over the years using Spark and it is nothing like BigQuery. You need to have much more awareness of how it all fits together because it is sitting on a JVM that when exposed to memory pressure will do a full GC and kill the executor. When this happens at best your workload gets significantly slower and at worst your job fails.

Yep, CPU has to transfer data because no RDMA setup on GCP lol. But that's like 16-32 GB/s of transfer per GPU (assuming T4/L4 nodes), which is much more than network bandwidth. And we're not even network bound, even if there's no warm data (i.e. for our ETL workloads). However, there is some stuff kept on GPU during actual execution for each Spark task even if they aren't running on the GPU at the moment, which makes handling memory and partition sizes... "fun", haha.

I've used GPU based Spark SQL for many years now and it sounds flashy but it's not going to make a meaningful difference for most use cases.

As you say the issue is that you have an overall process to optimise from getting the data off slow GCS onto the nodes, shuffling it which often then writes it to a slow disk before the real processing even starts then writing back to a slow GCS.

Nice! I attended a hackathon by Modular last weekend where we got to play with MI300X (sponsored by AMD and Crusoe). My team made a GPU-"accelerated" BM25 in Mojo, but mostly kind of failed at it, haha.

The software stack for AMD is still a bit too nascent for ParaQuery's Spark engine, but certain realtime/online workloads can definitely be programmed pretty fast. They also happen to benefit greatly from the staggering levels of HBM on AMD chips. Hopefully I can take a mini-vacation later in the summer to hack on your GPUs :)

> I started working on actually deploying a cloud-based data platform powered by GPUs (i.e. Spark-RAPIDS)

Based on this, the platform is using Spark-RAPIDS.

Good question -- it depends. For certain workloads, it might look exactly the same! For others, I found that the memory and VM constraints were creating large inefficiencies. Also, many teams simply don't want to manage that level of data infra: managing EMR, instance type optimization, spark optimization (now with GPU configs!), custom images, upgrades, etc.

We take care of that and make it as easy as pie... or so we hope! On top of that, we also deploy an external shuffle service, and deal with other plugins, connectors, etc.

I suppose it's similar to using Databricks Serverless SQL!

Another thing: we ran into an incompatible (i.e. non-accelerated) operation in one of our first real workloads, so we worked with our customer to speed up that workload even more with a small query optimization.

Indeed, Spark-RAPIDS has been around for a while! And it's quite simple to have a setup that works. Most of the issues come after the initial PoC, especially for teams not wanting to manage infra, not to mention GPU infra.