Postgres was rewritten in Rust… and somehow passed every test
BBetter Stack
Computing/SoftwareSmall Business/StartupsInternet Technology
Transcript
00:00:00Someone rebuilt Postgres in Rust, and somehow the released version now passes all 46,000 Postgres regression queries.
00:00:08It works with normal PSQL. It can even boot an existing Postgres 18.3 data directory.
00:00:14That sounds like a production-ready replacement. It isn't.
00:00:17But the most exciting version of this project has yet to even be released.
00:00:21So what exactly are we even looking at here? Is this the future of Postgres or just an AI experiment?
00:00:30Now, to understand why this matters, you need to understand the problem.
00:00:36Postgres is one of the best databases ever built, let's face it.
00:00:40But it also carries nearly four decades of architecture and roughly a million lines of C.
00:00:46Every client connection gets its own backend process.
00:00:49That isolation is useful, but it also means more memory overhead, more pressure to use connection pooling,
00:00:56and more difficulty sharing state across parallel work.
00:01:00PGRust takes a different path.
00:01:02Keep the Postgres behavior, keep the client experience, keep the disk formats, but replace the engine underneath with Rust.
00:01:09This is not a Postgres extension.
00:01:11It's not a feature added on top.
00:01:13This is completely separate, an implementation trying to behave just like Postgres.
00:01:18We're seeing a lot of Rust rewrites now, just like the one we covered the other day on Bunn rewriting its entire codebase with Rust.
00:01:25I'll put it somewhere right here.
00:01:27And now, all this sounds good, but does it actually feel like real Postgres?
00:01:31Less tested properly.
00:01:32If you enjoy coding tools that speed up your workflow, be sure to subscribe.
00:01:36We have videos coming out all the time.
00:01:38Now, I'm starting with the official Docker image, then connecting with a completely normal PSQL client.
00:01:43Nothing custom.
00:01:44Now, I'm in.
00:01:46First, let's check the version.
00:01:48As you can see, it reports itself as PGRust with the latest version.
00:01:53Now, I can create a table, insert some data and look at a query plan.
00:01:59I'm just going to run this right here in my terminal.
00:02:01From the outside, this is completely indistinguishable from Postgres.
00:02:05Same client, same SQL, same output.
00:02:08You can even see the query planner chose an index scan and gave us real execution stats.
00:02:14Now, to make this a little more interesting, let's insert 100,000 rows and run another query.
00:02:20This is just a generated 100K rows of data.
00:02:23We're going to drop it in here.
00:02:25Now, what is the point of all this?
00:02:27Okay, well, good question.
00:02:28Well, in this current early version release, we're not going to see dramatic speed improvements over regular Postgres.
00:02:35The bigger performance claims come from an unreleased development version that switches to a thread-per-connection model.
00:02:43This all does prove, though, that this isn't just a partial implementation.
00:02:47It has passed the full official Postgres regression test suite, over 46,000 queries.
00:02:53It speaks the real wire protocol, and it has a working query planned in storage engine.
00:02:59This is a real database server.
00:03:01It's just written in Rust.
00:03:03So, with this progressing, we could see some serious performance enhancements on the speed side of things.
00:03:08So, now the question is, why not just create another Postgres extension?
00:03:12Because extensions sit on top of the original Postgres core.
00:03:16A fork can change that core, but then it inherits the same architecture and the permanent job of keeping up with upstream Postgres.
00:03:25You have databases like CockroachDB and YugaByte, but they are independent distributed databases.
00:03:31Exact drop-in compatibility is not their main goal.
00:03:35PG Rust is trying something else.
00:03:37It uses actual Postgres behavior as the specification.
00:03:41The current release targets Postgres 18.3.
00:03:44It passes the default regression suite and isolation tests, and it is discompatible enough to boot from an existing Postgres 18.3 data directory.
00:03:53The bigger experiment here is happening in a separate unpublished version, and that version reportedly replaces Postgres process per connection model with a thread per connection model.
00:04:05With the normal Postgres model, each connection gets its own process.
00:04:09With the new model, each connection gets a thread inside the same process.
00:04:14That can lower per connection overhead and make it easier for different parts of the database to actually share information.
00:04:21But with all this, there is going to be a trade-off, right?
00:04:24Separate processes also create useful walls between connections.
00:04:28If one process fails, that separation can help contain the damage.
00:04:32With threads, one unsafe extension or memory bug could affect more of the server.
00:04:38So thread per connection is not automatically better.
00:04:41It just opens new doors, but it may also remove some safety rails.
00:04:45Then there is the second major part of the story, AI.
00:04:49Michael Malice and Jason Siebel used coding agents heavily to speed up the rewrite.
00:04:54The published version intentionally follows the original Postgres structure in many places.
00:04:59The unpublished version is where they are trying larger architectural changes.
00:05:03So the real experiment is not simply, can Rust make Postgres faster?
00:05:08It's more like, can AI make a rewrite this large affordable enough that developers can actually rethink architecture underneath things?
00:05:16Because AI was heavily used here.
00:05:19Now, does this change things?
00:05:20Well, it might.
00:05:21It's also where we need to slow down a little bit.
00:05:25The released version is not heavily optimized.
00:05:28The major performance claims come from the unpublished thread per connection version, which right now we can't quite test.
00:05:36The devs claim roughly 50% better performance on transactional workloads.
00:05:40They also claim around 300 times Postgres' performance on analytical workloads.
00:05:45Those numbers are huge, but the code behind those results is not currently available for any type of inspection or benchmarking.
00:05:53So, hence this, there is still a lot of speculation.
00:05:57It seems there is a good 50-50 split here, at least reading online, with questions that look kind of like these.
00:06:03These are just the issues here on GitHub.
00:06:05Then you even get one issue like this one, right?
00:06:08Which is a completely reasonable question.
00:06:10As you read through this issue, the devs seem adamant about actually making this work.
00:06:15So, we don't have real stats just yet.
00:06:17That does not automatically mean the numbers are false, though.
00:06:20It just means we should treat them as promising claims, not settled facts.
00:06:24And honestly, the exact multiplier may not be the most important part.
00:06:28We don't need to change the entire Postgres project before learning whether an idea actually works or not.
00:06:34That freedom may be more valuable than any single benchmark we can get.
00:06:38Now, the reaction from devs with all these Rust rewrites, even on this PG Rust, has been massive.
00:06:44The main Hacker News discussion passed hundreds of points and comments, but again, the response is split.
00:06:50Both sides are making good points.
00:06:52First, passing every regression query is a serious achievement.
00:06:56A lot of projects claim to be Postgres compatible.
00:06:59That phrase can mean almost anything.
00:07:01PG Rust has a measurable target.
00:07:04The real Postgres tests are the judge of this.
00:07:07And the speed of this rewrite suggests that coding agents may completely change cost of large infrastructure experiments.
00:07:13Ideas that once looked too expensive to attempt are now seemingly becoming more possible.
00:07:19Now, for the other side of things, passing regression tests is not the same as earning production trust.
00:07:24Those tests don't replace years of crash recovery and replication testing, or databases that run for months without stopping.
00:07:31A project can pass every known test and still fail in a situation nobody even thought to test.
00:07:37Generating hundreds of thousands of lines of code is one challenge.
00:07:42Extension compatibility is another major gap.
00:07:45Now, should you replace your production Postgres cluster with PG Rust?
00:07:49No, absolutely not.
00:07:51The project itself is not production ready.
00:07:53It's stated.
00:07:54It's not fully optimized.
00:07:56In major compatibility areas, including the extension ecosystem, they're still unfinished.
00:08:02But should you try it?
00:08:03Sure.
00:08:03If you work with databases, Rust, query execution, compatibility testing, or AI development, why not give it a try?
00:08:10Run the Docker image, test your client library against it, read the source, and see what happens.
00:08:16So, drop your verdict in the comments.
00:08:18Where is this project going?
00:08:20Are we going to start rewriting more in Rust?
00:08:21We're going to find out.
00:08:23If you enjoy coding tips and tricks like this, be sure to subscribe to the BetterStack channel.
00:08:26We'll see you in another video.