Marmot started as a sidecar project using triggers and polling to replicate changes over NATS. It worked, but I hit a wall pretty fast. Most people really want full ACID compliance and DDL replication across the cluster. I realized the only clean way to do that was to expose SQLite over a standard protocol.
While projects like rqlite use REST and others go the page-capture route, I decided to implement the MySQL protocol instead. It just makes the most sense for compatibility.
I’ve reached a point where it works with WordPress, which theoretically covers a huge chunk of the web. There are scripts in the repo to deploy a WP cluster running on top of Marmot. Any DB change replicates across the whole cluster, so you can finally scale WordPress out properly.
On the performance side, I’m seeing about 6K-7K inserts per second on my local machine with a 3-node quorum. It supports unix-sockets, and you can even have your processes read the SQLite DB file directly while routing writes through the MySQL interface. This gives you a lot of flexibility for read-heavy apps.
I know the "AI slop" label gets thrown around a lot lately, but I’ve been running this in production consistently. It’s taken a massive amount of manual hours to get the behavior exactly where it needs to be.
https://github.com/synopse/mORMot2
FreePascal ORM, so in an adjacent space (ORM, can work with both SQLite and MySQL).
I guess DB devs really love marmots? :-))
It was definitively a weird backend setup I had made but it just worked once set up so I didnt have to touch any of the frontend code.
geenat•1h ago
Now I'm curious how sharding/routing is handled- which seems like the final piece of the puzzle for scaling writes.
maxpert•53m ago
- Ability to launch a replica on selected databases from main cluster.
- Ability for replica to only download and replicate changes of select databases (right now all or nothing).
- Ability for replica to proxy DML & DDL into write cluster transparently.
- Custom set of commands for replicas to download and attach/detach databases on the fly.
This will instantly solve 80% of the problems for most of consumption today. I will probably go after on demand page stream and other stuff once core features are done.
Not to mention this solves majority use-cases of lambdas. One can have a persistent main cluster, and then lambda's coming up or going down on demand transparently.