Looks very git-ish. But probably better equipped for large binary files.
echo "Hello, Lore" > hello.txt
lore stage hello.txt
lore status --scan
lore commit "Initial revision"
lore pushEDIT: and then an example for the merge stuff I couldn't find while typing before: https://mergiraf.org/ and HN discussion a few years ago: https://news.ycombinator.com/item?id=42093756
Lore is meant for situations where your repository is going to contain gigabytes of binary files, such as art assets for games. Git is technically great at everything but that, and even the external solutions for that situation still kind of suck.
I'm not just picking nits here. And this is not cynicism.
so there you go.
[0] https://en.wikipedia.org/wiki/List_of_minor_Star_Trek:_The_N...
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 10 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 290 bytes | 290.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
From the documentation, it looks like Lore does a bit better in this regard:
Pushing 1 fragment(s)
Pushed 1 fragment(s), 124.00 bytes
Pushing a3f8c2d1... to branch main
Pushed revision 1 -> a3f8c2d1... to branch mainStill the porcelain is more like cold stainless steel
Git is fine for text based files like code, but it's really bad at stuff like textures, 3D models, audio files, and other non-text files that game developers need to collaborate on. For example, one artist might need to obtain an exclusive lock on some art assets while editing them, because there is no sane way to merge two artists' async edits.
The SOTA in this area is Perforce (https://www.perforce.com/products/helix-core), a proprietary system. From what my gamedev friends tell me, when Perforce works it's great, but it hits enough snags that you need a tools engineer to manage it and occasionally fix issues manually. Git LFS is an alternative, but my gamedev friends all prefer Perforce especially when working on team projects beyond like 3-4 people.
Its remarkable that anyone thought this website was fit for release, and it gives off strong slop vibes
I also have absolutely zero trust in a product like version control being provided by a for-profit company. It seems like a terrible idea to tie your software stack to Epic Games of all people, given their track record
Anyway it's probably fine software and I am genuinely going to give it a shot for a usecase I have involving large image files. But the LLM-generated docs don't inspire confidence.
[1] https://epicgames.github.io/lore/explanation/system-design/
[2] They literally have a section header "10.1 Revision state as a 320-byte fragment". The byte size isn't even relevant in the code as an implementation detail, much less belongs in a design doc. No one read this doc before publishing it.
On the flip side, I expect the project itself will be workable - well, assuming they're actually using it themselves! UE is a big pile of Stuff on its own, and Fortnite must have god knows how much additional crap in there, so if this is (or will be) their replacement for Perforce internally, then it'll be getting a good deal of testing. (If they're just chucking it over the wall, though... well, sheesh, you first...)
(Perforce is the standard thing for games, and pretty well it works too, and hopefully this will deliver it a well-needed kick. It was sold to private equity about 10 years ago, and it feels like they've been coasting ever since. (Perhaps users in other sectors are happier though?))
I'm trying to figure out what Lore can accomplish that git+LFS can't. I've read about big binaries chunking, native interface and permission, is there anything else? Weren't those problems already solvable in the git+LFS ecosystem?
they do say they will open source it, but who knows:
"It isn’t open source yet—it currently depends on some proprietary components, including Epic’s internal design system—but we’re committed to open-sourcing it in the future"
> Open-source the desktop client so the community can build on its full graphical experience, not just download it. An early desktop client already exists as a binary download, but it isn't open source yet — it depends on some proprietary components, including Epic's internal design system. We're working to make all of it available in the open so that the client can ship as source alongside the rest of Lore. Lore is an open project, so it is important that the desktop client — which will be one of the main ways many people will interact with Lore — is also fully open so that the community is free to review, extend, and shape it.
- Pachyderm (Go): https://github.com/pachyderm/pachyderm
- XetHub (acquired by HuggingFace): https://huggingface.co/blog/xethub-joins-hf
- LakeFS (Go): https://github.com/treeverse/lakeFS
- Oxen (Rust): https://github.com/Oxen-AI/Oxen
I guess with AI, anyone can vibe code a content-addressed, chunk-level deduped, versioning system in Rust these days...
But jokes aside, Lore seems really cool! What's interesting is the realization that different domains/industries have similar problems, but they don't seem to be cross-polinating. In this case AI and Gaming both need a storage system that can version control large binary files at scale. I think there's lots of opportunities to share ideas here, but perhaps the lack of idea sharing (currently) creates opportunity!
If the roadmap's "Web client and code review tools" could replace gerrit for me, this would be a easy switch.
Moreover, it looks like they designed both the mutable store and immutable store to be able to easily store their state directly on an s3 like system.
There are a number of features that would greatly speed up CI/CD system operations I belive.
It’s like anything you do has to talk to the server
Even something as simple as diffing a file will just hang if there are server issues
[0] https://github.com/EpicGames/lore/tree/main/docs/developing/...
[1] https://github.com/EpicGames/lore/blob/main/docs/developing/...
Don't be too hard on Lore.
There are 1000 things that's true of about git. At a certain point that becomes a problem in and of itself.
I’d be a bit worried if git didn’t heave that particular contented sigh when I ask it to push
The objects are referenced by trees. A tree is just a directory.
The trees are then referenced by commits and/or tags into a DAG with named pointers into various parts of it (which are your branch and tag references):
https://git-scm.com/book/en/v2/Git-Internals-Git-Objects
Because it would be terribly in-efficient to have a bunch of loose objects, git periodically groups them together into packs. To save space, the objects are compressed against one another (delta compression) within the packs.
https://git-scm.com/docs/git-pack-objects
https://github.com/git/git/blob/master/Documentation/technic...
When pushing or pulling, the git transfer protocol basically enumerates what objects each side has so that it only needs to transfer the difference. On top of that, it delta compresses the objects on each side that aren't already grouped into packs against each other to save space.
https://github.com/git/git/blob/master/Documentation/technic...
Because git is an open-source project written by nerds, it shows you all of this information. Feel free to ignore it!
But if you really want to know, it's all documented both in the git book and git documentation directory, both linked above.
(Caveat: I'm working from memory and surely got some detail at least slightly wrong.)
https://epicgames.github.io/lore/explanation/system-design/#...
it's also tough when you have 1TB of data, over 1mm files and you might want to lock hundreds files in one go
Also git-lfs is a crutch that breaks more often than it works :/
(I agree though that for small game projects, git is mostly 'good enough', even without lfs).
For the longest time Git tools were really poor. In recent years there's a few ok ones, like Git Fork, though I wouldn't know if those tools scale to the level of a AAA team size repo and not fall over.
He’s in a category of influencers who post constantly about gripes and grievances and smug superiority. Some people like that content but I can’t stand it.
I really like hearing about indie development and small teams, but you don’t have to present everything as condescending superiority over the industry. That’s not the part I find interesting.
I think being influential just does that to people, with high regularity.
Huge teams are more often than not the sign of bloat and inefficiences.
Now there's an argument to be made that many don't need the remaining ones but to claim that you 'replaced PowerPoint' for anyone but yourself is ridiculous.
They're good at demos, I give them that.
Git enthusiast would often tell you to do this separately with a submodule, and set permission on the version control forge software level (which means Gitea/Github private RBAC access to certain repos for cloning), sure, but that is also painful as hell.
But my point is that all of this is exactly by design from Linus Torvalds's need for Linux Kernel to replace BitKeeper. Git simply isn't the tool for everything, it was developed for a software project with liberalism in mind, but corporate stuff is monoculture and prefers proprietary, shut-in model, and the eat your own dog food mindset, and no wonder it is so painful to deal with.
This was within a bank and the code in question was related to enabling Apple Pay from within the banking application. The consequences of that information and code leaking or being seen by anyone who had not signed the NDA were very serious (don't remember the details but it made the lawyers were extremely stressed about it).
Needing to figure out a way to protect those parts of the codebase it was decided in the end that the "easiest" way of doing this was to split the repository in half, with the actual artifact building taking place from the half that had the NDA code. The rest of the application (basically the whole application) was then used as a dependency by it.
Still didn't quite solve the issue, but access to that repository was heavily controlled.
Thats not how everyone else works.
We're all using package managers to help with massive amounts of 3rd party dependancies (why are you version pinning in any place other than your repo, why arent you pulling updates through your repo and reviewing them)
We're reliant on tools like artifactory to make sure those depedancys dont disappear or are not corrupted.
We use yet other tools to manage our binary files (this tool would fix that).
Github, gittea, gitlab, bitbucket... have all added piles of tooling around git, that are grafted on around its short comings.
> It's crazy to me that nobody has made anything better yet.
Because our entire industry has fallen into the rut of "more tools", of stacking turtles (https://en.wikipedia.org/wiki/Turtles_all_the_way_down ) rather than fixing the real issues that hold us back.
> Although I could start that myself and yet have not.
Because unless your a Google or a Linus, no one is going to look twice at your tool for something that is this important. Im not even sure that epic games has the good will, or trust to launch this.
I am going to give them the benefit of the doubt and take a long hard look at it, but my optimism is tempered. But unless it offers a LOT more than git, the extra overhead (lacking IDE support, deployment changes and all the other tooling in GIT's orbit) it isnt going to be a worth while change.
If perforce is the best there is out there for large binary asset management, then there is a blue ocean worth of potential improvement for git.
Perforce is a piece of crap, a relic of the 20th century that must die in a fiery inferno.
headwayoldest•1h ago
glouwbug•1h ago
tombert•1h ago
takipsizad•1h ago
tombert•32m ago
But Kojima games require me to be in a very specific mood and I have been trying to work my way up to it.
Tomte•1h ago
WillPostForFood•1h ago
It is interesting that people are so cynical about Epic giving out free games. I get that people love Steam, but competition in the storefront market is not bad.
falcor84•58m ago
RobotToaster•52m ago
rbits•36m ago