Marimo is pretty new (first release January 2025) but has a high rate of improvement. It's particularly good for WebAssembly stuff - that's been one of their key features almost from the start.

My notes on it so far are here: https://simonwillison.net/tags/marimo/

https://youtu.be/4fXLB5_F2rg?si=jeUj77Cte3TkQ1j-

disclaimer: I work for marimo and I made that video, but the gamepad support is awesome and really shows the flexibility

Our sandbox with public data is free for you to try, or just reach out and ask any question!

I'm a data engineer and make decisions around what software we use for pipelines. A lot of examples for these types of tools showcase the simple case, which is a handy intro, but I'd love to see a real world example of Bauplan scaling to interconnected pipelines!

We've done quite a lot of open source in our life, at Bauplan (you can check our github), and before (you can check me ;-)), so the comment seems unfair!

We understand the importance of being clear on how the platform works, and for that we have a long series of blog posts and, if you're so inclined, quite a few peer-reviewed papers in top conferences, ranging from low-level memory optimizations (https://arxiv.org/abs/2504.06151), columnar caching (https://arxiv.org/abs/2411.08203), novel FaaS runtimes (https://arxiv.org/pdf/2410.17465), pipeline reproducibility (https://arxiv.org/pdf/2404.13682) and more.

We are also always happy to chat about our tech choices if you're interested.

I think its a much better result to have data science prototype translated to a performant production version rather than have a databricks type approach or what bauplan is proposing.

"but it always takes forever and I'm never even able to articulate why." -> there are way more factors at play than DoFeatures unfortunately, see for example Table 1 (https://arxiv.org/pdf/2404.13682). Even knowing which data people have developed on is hard, which is why bauplan has git-for-data semantics built in: everyone works on production data, but safely and reliably, to avoid data skews.

Each computer is different, which is why bauplan adopt FaaS with isolated and fully containerized functions: you are always in the cloud, so no skew in the infra etc.

The problem of "going to production" is still the biggest issue in the industry, and solving it is not a one-fix kind of thing, but unfortunately the combination of good ergonomics, new abstractions and reliable infra.

This isn’t necessarily what you want in a daily production environment, let alone a real-time environment.

If you're worried about data movement or secure deployment, none of that is an issue because of Iceberg + BYOC option.

Databricks and Snowflake, just to mention two players in a similar space, are not OS: did you feel that would prevent you from adopting them as well?

If you want to dive deeper in one line reproducibility, you can chek our SIGMOD24 paper: https://arxiv.org/pdf/2404.13682. Let us know what you think!

As for all the other Python packages, including proprietary ones, the FaaS model is such that you can declare any package you want in a function as node in the pipeline DAG, and any other in another: every function is fully isolated, and you can even selectively use pandas 1 in one, pandas 2 in another, or update the Python interpreter only in node X.

If you're interested in containerization and FaaS abstractions, this is good deep dive: https://arxiv.org/pdf/2410.17465

If you're more the practical type, just try out a few runs in the public sandbox which is free even if we are not GA.