Very excited to launch LiveStore in beta today after having worked on it over the past 4 years. I've built it for myself working on Overtone, an ambitious music client aiming for a native-grade high-performance app feel.
LiveStore embraces SQLite by adding a signals-based reactivity layer and combines it with event-sourced based syncing (similar to Git).
Happy to answer any question! Looking forward to thoughts and feedback!
I was just looking into LiveStore this past month for a hobby project, but it was behind a beta preview, so I hope to be able to dig in soon to see if it can be useful to me.
I love how you're pushing forward the localfirst conversation.
If you've spent any time building offline capable web apps with some sort of syncing mechanism, you begin to very quickly see the usefulness of a sync engine.
Few questions:
Would it not be possible to store and rely on larger amount of data on SQLite and keeping it on disk as an option? I see that there's a limitation of 1 GB. So maybe just a config flip to change the mode of persistence?
Also - multi-tenancy could be an interesting scenario. Think JIRA but each organisation needs their own namespace and each user/client wouldn't need all the tickets from all the organisation but only of their own (or even more scoped to department and team) therefore the local database would alway be a subset of the whole universe.
An out of the box sync server that could be run directly on Bun/Node (without Cloudflare) could be a nice addition.
But overall, really elegant solution. I like it, might give it a try in an idea I am evaluating various tools for. Unfortunately, the multi-tenancy story is critical for that.
tinyBase seems to be the only one that stands out in that regard or this LiveStore now.
PS: Thank you for putting this landscape. That really has enabled me or anyone else to look at what options are there. No amount of googling would do that kind of compilation. I find it very useful.
Is that a requirement or could there also be federated nodes, or a full p2p modus operandus? Think for use in decentralized social networking.
Thank you for championing SQLite and especially OPFS Wasm SQLite on the web — we (PowerSync) are clearly also big proponents of it, so love to see other projects having success with it too.
I was hoping the underlying Android web issue would have been fixed by now (as I first noticed it ~3 years ago with some indication for progress), but looks like LiveStore needs a custom workaround for it. You can track the progress here: https://github.com/livestorejs/livestore/issues/321
I hope you understand that bridging the gaps between various levels of supported web APIs take a lot of effort and is non-trivial when building ambitious systems like LiveStore.
https://developer.mozilla.org/en-US/docs/Web/API/Web_Locks_A...
It is supported by all browsers. You just have all the tabs try to lock the same resource and have them return a promise in the lock callback. The first one wins and when that tab closes the next one in line gets automatically elected.
The leader can then use a broadcast channel and act as a server for all the other tabs to serialize access to any shared resources.
What are your thoughts on:
- Handling compaction for long lived apps/pages?
- IMO events are nice, but also require discipline and good design w.r.t code (Thinking of how application layer evolves over time, possibility of maintaining code for older client versions and/or schema migrations.
- Overtone looks sick. I've been using Spotify less and less because of janky UI and constant UI changes, would love a replacement. I see that it supports multiple sources, will it offer offline playback?
(clarifying for non-native English readers)
> Handling compaction for long lived apps/pages?
That's a very common question and something I'm planning to ship a solution for soon. The basic idea is to give each event some more semantic "meaning" by annotating the event definition which allows you to express which events "semantically overlap". For example in a todo app you could express that the "todoCompleted" event for a given task id can compact other "todoCompleted" / "todoUncompleted" events for the same task id.
You can track the progress of this topic here: https://github.com/livestorejs/livestore/issues/254
> IMO events are nice, but also require discipline and good design w.r.t code
Yes, I agree with that. When it comes to data there is "no free lunch" - it's all about tradeoffs. I prefer the tradeoffs of event sourcing though for my own use cases such as Overtone.
That being said for many situations (like older client versions etc) there are pretty straightforward ways to address those concerns. Always depends on your application use case and possible tradeoffs though.
> Overtone looks sick. I've been using Spotify less and less because of janky UI and constant UI changes, would love a replacement. I see that it supports multiple sources, will it offer offline playback?
Very excited to hear! I'm sharing your frustrations which is why I'm building Overtone (next to many other reasons). Re "offline playback": That will depend on where your music is coming from. e.g. for your own music collection in Dropbox (or similar) it will be supported. For music streaming services like Spotify it will depend on their terms.
Hope that all makes sense?
I was experimenting with something similar for a custom local-first app with multi device support.
Would it be possible to add optional E2E encryption to this? While reading the docs, by adding encryption to the event payload you would be almost there already, even though it would prevent log compaction on the server
> Would it be possible to add optional E2E encryption to this?
Yes, that's something that should already be possible, though I haven't done this myself yet. Happy to help if you're running into any issues.
Will definitely keep this use case in mind while working on compaction. One solution could be that only clients could do the compaction.
Would this fit into the "1. Serialization" strategy described here? https://mattweidner.com/2024/06/04/server-architectures.html...
If so, would it run into the same issue ProseMirror-collab did as mentioned in the article? As in: A frequently updating low-latency client can lock out another high-latency client's updates.
The landing page draws comparison to Figma, Linear and Notion. But they are vastly different use cases.
Figma is document-centric, which means that:
- All data is tied to a single document, limiting their size.
- Requires that single backend session holds a document in-memory and deals with split-brain issues.
- Operational complexity arises around deployment.
- User interactions are often continuous (one event / frame), imposing tighter latency requirements.
- Generally requires that the document stays loaded in a memory in a stateful backend session.
- Technologies like Jamsocket are targeting such use cases.
Notion & Linear, on the other hand, have collaborative editing but not to the same degree of realtimeness. - Data is not tied to a single document.
- Data can grow unbounded.
- Data is typically more relational.
Presenting both these uses cases adds confusion. By using SQLite (emphasis: database and "SQL"), it makes the technology appear at first glance more suited for the Linear/Notion use case. However, the opposite appears to be true after reading https://docs.livestore.dev/evaluation/when-livestore/.
- "All the client app data should fit into a in-memory SQLite database".
- "Reasons when not to use LiveStore" -> "Your app data is highly connected across users".
The Figma-like document use case does seem like something that LiveStore could support. When it comes to designing a data model for collaborative apps (CRDT or CRDT-like), the most simple and flexible solution is to store every object as a map of [Object ID] -> [Property Name] -> [Property Value]. Assuming that the properties names & values come from a fixed set and are typed (this will generally be true unless you allow for arbitrary user-defined fields), that looks like a database row! So why not just store it in a database, indeed.
However, among databases, there are object stores and relational databases. The latter is useful, as applications often want to represent relationships between different objects. But if we support multiplayer-like use cases which implies the absence of server-side transactions, how should conflicting events involving the creation/deletion/relationships between objects be handled? Is it entirely on the application to think about it? This is an interesting topic that I believe LiveStore seems well-positioned to innovate in.
I also find the "local-first" emphasis to be at odds with all of Figma/Notion/Linear. Local-first software tends to have a limited number of concurrent users accessing the data in-mind. SaaS applications that are truly targeted at collaboration tend to have a lot of different requirements. On the other hand, those applications do tend to have relatively limited offline support, which a local-first application tends to emphasize more.
The Figma-like use cases also has additional requirements. But I understand this is beta and look forward to seeing further development!
For others looking for more details on how Figma's sync engines differ and why 2 sync engines emerged, I had a long thread about it here:
since sqlite now also provides official wasm builds why did you decided to go with wa-sqlite?
It would be really nice to have a SQLite3-like database with PG language functionality so that one could do local-first and remote-first as just a configuration setting.
On the main point around materialisation cost, I know more work is planned on compaction specifically but generally LiveStore is a framework that’s been highly crafted to address and optimise for all aspects of performance.
jessmartin•1d ago
I couldn't believe how robust LiveStore is already. I've been trying it out for a new project over the last few weeks and it's so smooth.
schickling•1d ago
apitman•1d ago