Git Can’t Handle Games… So Epic Built Lore
BBetter Stack
Computing/SoftwareSmall Business/StartupsVideo & Computer Games
Transcript
00:00:00Epic Games built its own version control system all because it got fed up with Git.
00:00:05They built it in Rust and then released it for free. It's called Lore. So yeah,
00:00:10the company behind Fortnite built a Git alternative. But Git is still great,
00:00:15obviously. But Git was built for code. Mostly text files, lots of small files,
00:00:21changes that are usually a few lines at a time. Games are basically the opposite of that. So
00:00:26how does Lore compare to Git and how do we even work with it? Let's find out.
00:00:35Now at this point, we've got massive textures, audio files, videos, 3D models,
00:00:41and all kinds of binary assets that can be hundreds of megabytes or even several gigabytes each. And
00:00:47once those files start changing, things get really messy. Your repository grows, clones get slower,
00:00:53history becomes huge. And eventually someone says, okay, maybe we should use Git LFS. Git LFS helps,
00:01:01but it also feels like a workaround added to a system that was never designed for this kind of data.
00:01:06Then you've got quotas, bandwidth limits, and old assets hanging around in history. So
00:01:12a lot of studios use Perforce. And to be fair, Perforce works. There's a reason so many game
00:01:18studios use it, but it's expensive. It can get complicated. And once your setup gets big enough,
00:01:24somebody usually ends up becoming the person who keeps it all alive. This is the thing that Lore was
00:01:29sort of designed to fix. If you enjoy coding tools to speed up your workflow, be sure to subscribe.
00:01:35We have videos coming out all the time. All right. Now, instead of just talking about Lore,
00:01:39it's cool. Let me get it running. Let me show you how it works. The demo starts with one install
00:01:44command and a demo flag, which we're going to run right here. And that's it. A couple of seconds later,
00:01:51I've got a Lore server running locally. No cloud account, no key, no cert setup. It's running here
00:01:57on these ports. And just to prove it's actually alive, I can just hit the health endpoint right
00:02:03here. And I'm going to do that over on this terminal. And there we go. It is alive. It is running.
00:02:08There are no background services I need to manually hook together. No auth token to generate. There's
00:02:14no setup wizard. It just starts. Now, let me create a repository. So I'm going to create a folder,
00:02:22right? And we are going to create this repository. And now I'm going to create a large binary file
00:02:27and commit it, right? This is just a dummy file, right? But let's create a larger file. I'm running
00:02:32DD here for data duplication. Now, instead of treating that 100 megabyte file as one huge object,
00:02:40Lore breaks into smaller chunks. Those chunks are hashed, compressed with Z standard, and stored in a
00:02:47content address Merkle tree. If most of the file stays the same, Lore doesn't need to save another full
00:02:52copy of the whole thing. It can reuse the chunks it already has and only store the parts that changed.
00:02:58That's a much better fit for a large binary asset. Right after the commit finishes, you can see the
00:03:04local state that was created on disk. There's now a Lore directory here with the config and the
00:03:10metadata. All right, now that we have that, let's make a branch. Okay, I can run Lore branch create
00:03:17name the branch. It works pretty much like Git. So let's switch to it. Let's actually make a small
00:03:24change here. I'll create a quick text file and commit it to this branch. So I modify, then we can stage it,
00:03:31and then we can commit it, right? The same flow is roughly the same as Git. Now I'm going to switch
00:03:37back. And that was basically instant. The other important thing is that none of that required a
00:03:44trip to any server. Staging, committing, branching, switching, diffing, all of that happens locally. So
00:03:50even though Lore has a central server, your normal day-to-day work still feels fast and you can keep
00:03:56working offline. It just makes it feel all lightweight. Now the question becomes, okay,
00:04:02where is all the data going? In this demo here, everything is temporary. So when I stop the server,
00:04:08the data disappears. That's because I'm just using demo mode. In a real setup, you would run Lore server
00:04:15with a config file and persistent storage. The same CLI commands and local workflow stay the exact
00:04:21same. You're just pointing the server at real directories or object storage instead of throwing
00:04:27away that temporary folder. Now with Git, Git gives you a history of project snapshots. Although
00:04:34internally it does a lot of clever optimization, Lore is designed around chunking and deduplication from
00:04:40the get-go. So for a large asset-heavy project, it doesn't have to treat every new version of the file
00:04:47as a completely separate giant object. Lore can also hydrate files on demand so you can work with
00:04:53the repository containing a huge amount of data without downloading every single asset from day one.
00:04:59You pull down what you actually need for the part of the project you're working on.
00:05:03Now, one thing that's a tad bit confusing here when Lore launched was whether it's centralized or
00:05:08distributed. It's centralized. There is one server of record, but most of the work that we're doing is
00:05:15happening locally. So in practice, it sits somewhere between perforce and Git. You get central control
00:05:22and access management, but local operations still feel quick and don't depend on the server being
00:05:28available every single second. There are some nice differences too. Lore is MIT licensed, the protocol
00:05:34is open source, and there are SDKs for multiple languages, which makes it much easier to script and
00:05:39build tooling around. Now, it's probably a good point to be realistic here. Lore is not something I would
00:05:45replace a production perforce setup tomorrow with. It's still pre-1.0. Epic Games says the APIs may change
00:05:52before the first stable release, and the project is clearly still evolving. There's also no Git
00:05:58interoperability right now, and you can't just point Lore at an existing Git repository and bring the
00:06:03full history across. It's self-hosted too. There isn't a hosted service where you create an account,
00:06:09push your repo, and you're done. And the desktop app you may see floating around isn't included in the
00:06:15open source release. What you get is the core library, the server, the CLI, and the SDKs. That GUI is not a
00:06:22part of it. Then, of course, performance. How does this perform? Epic says Lore can handle huge
00:06:28repositories without slowing down the way other systems do. And Epic obviously has experience with
00:06:33some very large projects. But right now, most of those claims are coming from Epic themselves. There
00:06:39isn't really solid independent benchmarks yet. So performance looks promising, but until we really
00:06:45start testing this out, it's hard to see how this performs. Should you use it? Well, it's fun to play
00:06:50around with. Are you making games? Are you building massive projects? Okay, for a new project, maybe.
00:06:56If you just want to see where version control for large binary assets could be going, it's worth a
00:07:01try. Test it on something non-critical, play around with it, see how it performs, see how the flow is.
00:07:07The bigger point here isn't whether Lore replaces Git or Perforce anytime soon. The bigger point is that
00:07:14version control stopped being a solved problem once projects started shipping with huge amounts of
00:07:19binary data. Git won for text, smaller files. Lore is trying to solve something that comes after that.
00:07:27And honestly, whether it wins, whether it doesn't, it's still a really cool direction. If you enjoy coding
00:07:32tips and tricks like this, be sure to subscribe to the Betterstack channel. We'll see you in another video.