Postgres was rewritten in Rust… and somehow passed every test

BBetter Stack
컴퓨터/소프트웨어창업/스타트업AI/미래기술

스크립트

00:00:00Someone rebuilt Postgres in Rust, and somehow the released version now passes all 46,000 Postgres regression queries.
00:00:08It works with normal PSQL. It can even boot an existing Postgres 18.3 data directory.
00:00:14That sounds like a production-ready replacement. It isn't.
00:00:17But the most exciting version of this project has yet to even be released.
00:00:21So what exactly are we even looking at here? Is this the future of Postgres or just an AI experiment?
00:00:30Now, to understand why this matters, you need to understand the problem.
00:00:36Postgres is one of the best databases ever built, let's face it.
00:00:40But it also carries nearly four decades of architecture and roughly a million lines of C.
00:00:46Every client connection gets its own backend process.
00:00:49That isolation is useful, but it also means more memory overhead, more pressure to use connection pooling,
00:00:56and more difficulty sharing state across parallel work.
00:01:00PGRust takes a different path.
00:01:02Keep the Postgres behavior, keep the client experience, keep the disk formats, but replace the engine underneath with Rust.
00:01:09This is not a Postgres extension.
00:01:11It's not a feature added on top.
00:01:13This is completely separate, an implementation trying to behave just like Postgres.
00:01:18We're seeing a lot of Rust rewrites now, just like the one we covered the other day on Bunn rewriting its entire codebase with Rust.
00:01:25I'll put it somewhere right here.
00:01:27And now, all this sounds good, but does it actually feel like real Postgres?
00:01:31Less tested properly.
00:01:32If you enjoy coding tools that speed up your workflow, be sure to subscribe.
00:01:36We have videos coming out all the time.
00:01:38Now, I'm starting with the official Docker image, then connecting with a completely normal PSQL client.
00:01:43Nothing custom.
00:01:44Now, I'm in.
00:01:46First, let's check the version.
00:01:48As you can see, it reports itself as PGRust with the latest version.
00:01:53Now, I can create a table, insert some data and look at a query plan.
00:01:59I'm just going to run this right here in my terminal.
00:02:01From the outside, this is completely indistinguishable from Postgres.
00:02:05Same client, same SQL, same output.
00:02:08You can even see the query planner chose an index scan and gave us real execution stats.
00:02:14Now, to make this a little more interesting, let's insert 100,000 rows and run another query.
00:02:20This is just a generated 100K rows of data.
00:02:23We're going to drop it in here.
00:02:25Now, what is the point of all this?
00:02:27Okay, well, good question.
00:02:28Well, in this current early version release, we're not going to see dramatic speed improvements over regular Postgres.
00:02:35The bigger performance claims come from an unreleased development version that switches to a thread-per-connection model.
00:02:43This all does prove, though, that this isn't just a partial implementation.
00:02:47It has passed the full official Postgres regression test suite, over 46,000 queries.
00:02:53It speaks the real wire protocol, and it has a working query planned in storage engine.
00:02:59This is a real database server.
00:03:01It's just written in Rust.
00:03:03So, with this progressing, we could see some serious performance enhancements on the speed side of things.
00:03:08So, now the question is, why not just create another Postgres extension?
00:03:12Because extensions sit on top of the original Postgres core.
00:03:16A fork can change that core, but then it inherits the same architecture and the permanent job of keeping up with upstream Postgres.
00:03:25You have databases like CockroachDB and YugaByte, but they are independent distributed databases.
00:03:31Exact drop-in compatibility is not their main goal.
00:03:35PG Rust is trying something else.
00:03:37It uses actual Postgres behavior as the specification.
00:03:41The current release targets Postgres 18.3.
00:03:44It passes the default regression suite and isolation tests, and it is discompatible enough to boot from an existing Postgres 18.3 data directory.
00:03:53The bigger experiment here is happening in a separate unpublished version, and that version reportedly replaces Postgres process per connection model with a thread per connection model.
00:04:05With the normal Postgres model, each connection gets its own process.
00:04:09With the new model, each connection gets a thread inside the same process.
00:04:14That can lower per connection overhead and make it easier for different parts of the database to actually share information.
00:04:21But with all this, there is going to be a trade-off, right?
00:04:24Separate processes also create useful walls between connections.
00:04:28If one process fails, that separation can help contain the damage.
00:04:32With threads, one unsafe extension or memory bug could affect more of the server.
00:04:38So thread per connection is not automatically better.
00:04:41It just opens new doors, but it may also remove some safety rails.
00:04:45Then there is the second major part of the story, AI.
00:04:49Michael Malice and Jason Siebel used coding agents heavily to speed up the rewrite.
00:04:54The published version intentionally follows the original Postgres structure in many places.
00:04:59The unpublished version is where they are trying larger architectural changes.
00:05:03So the real experiment is not simply, can Rust make Postgres faster?
00:05:08It's more like, can AI make a rewrite this large affordable enough that developers can actually rethink architecture underneath things?
00:05:16Because AI was heavily used here.
00:05:19Now, does this change things?
00:05:20Well, it might.
00:05:21It's also where we need to slow down a little bit.
00:05:25The released version is not heavily optimized.
00:05:28The major performance claims come from the unpublished thread per connection version, which right now we can't quite test.
00:05:36The devs claim roughly 50% better performance on transactional workloads.
00:05:40They also claim around 300 times Postgres' performance on analytical workloads.
00:05:45Those numbers are huge, but the code behind those results is not currently available for any type of inspection or benchmarking.
00:05:53So, hence this, there is still a lot of speculation.
00:05:57It seems there is a good 50-50 split here, at least reading online, with questions that look kind of like these.
00:06:03These are just the issues here on GitHub.
00:06:05Then you even get one issue like this one, right?
00:06:08Which is a completely reasonable question.
00:06:10As you read through this issue, the devs seem adamant about actually making this work.
00:06:15So, we don't have real stats just yet.
00:06:17That does not automatically mean the numbers are false, though.
00:06:20It just means we should treat them as promising claims, not settled facts.
00:06:24And honestly, the exact multiplier may not be the most important part.
00:06:28We don't need to change the entire Postgres project before learning whether an idea actually works or not.
00:06:34That freedom may be more valuable than any single benchmark we can get.
00:06:38Now, the reaction from devs with all these Rust rewrites, even on this PG Rust, has been massive.
00:06:44The main Hacker News discussion passed hundreds of points and comments, but again, the response is split.
00:06:50Both sides are making good points.
00:06:52First, passing every regression query is a serious achievement.
00:06:56A lot of projects claim to be Postgres compatible.
00:06:59That phrase can mean almost anything.
00:07:01PG Rust has a measurable target.
00:07:04The real Postgres tests are the judge of this.
00:07:07And the speed of this rewrite suggests that coding agents may completely change cost of large infrastructure experiments.
00:07:13Ideas that once looked too expensive to attempt are now seemingly becoming more possible.
00:07:19Now, for the other side of things, passing regression tests is not the same as earning production trust.
00:07:24Those tests don't replace years of crash recovery and replication testing, or databases that run for months without stopping.
00:07:31A project can pass every known test and still fail in a situation nobody even thought to test.
00:07:37Generating hundreds of thousands of lines of code is one challenge.
00:07:42Extension compatibility is another major gap.
00:07:45Now, should you replace your production Postgres cluster with PG Rust?
00:07:49No, absolutely not.
00:07:51The project itself is not production ready.
00:07:53It's stated.
00:07:54It's not fully optimized.
00:07:56In major compatibility areas, including the extension ecosystem, they're still unfinished.
00:08:02But should you try it?
00:08:03Sure.
00:08:03If you work with databases, Rust, query execution, compatibility testing, or AI development, why not give it a try?
00:08:10Run the Docker image, test your client library against it, read the source, and see what happens.
00:08:16So, drop your verdict in the comments.
00:08:18Where is this project going?
00:08:20Are we going to start rewriting more in Rust?
00:08:21We're going to find out.
00:08:23If you enjoy coding tips and tricks like this, be sure to subscribe to the BetterStack channel.
00:08:26We'll see you in another video.

핵심 요약

PGRust successfully replicates core PostgreSQL functionality while passing 46,000 regression tests, suggesting that AI-accelerated rewrites may enable radical architectural shifts like thread-per-connection models for legacy systems.

하이라이트

  • PGRust is an independent database implementation designed to mirror PostgreSQL 18.3, capable of booting from existing PostgreSQL data directories.

  • The current version of PGRust successfully passes all 46,000 queries in the official PostgreSQL regression test suite.

  • Unpublished development builds of PGRust report a 50% performance increase on transactional workloads and 300x performance on analytical workloads by utilizing a thread-per-connection model.

  • PostgreSQL currently relies on a process-per-connection architecture, which introduces memory overhead and complicates state sharing across parallel tasks.

  • AI-assisted coding agents were utilized extensively to make the architectural rewrite of the million-line C codebase economically feasible.

  • PGRust is not yet production-ready, lacking full optimization and extensive support for the broader PostgreSQL extension ecosystem.

타임라인

PGRust Project Scope

  • PGRust functions as a standalone database engine written in Rust rather than a standard PostgreSQL extension.
  • The system maintains wire protocol and disk format compatibility, allowing it to boot from existing PostgreSQL 18.3 data directories.
  • External behavior and output remain indistinguishable from native PostgreSQL during standard operations like table creation and query planning.

The project aims to replace the core engine of PostgreSQL while preserving established client experiences. It differentiates itself from existing forks by targeting exact compatibility through the official PostgreSQL regression suite. Testing confirms that standard PSQL clients interact with the Rust-based implementation without modification.

Architectural Changes and Performance

  • The standard PostgreSQL process-per-connection model incurs significant memory overhead and limits cross-connection state sharing.
  • Experimental builds replace the process-based architecture with a thread-per-connection model to reduce per-connection resource usage.
  • Transitioning to threads introduces new safety concerns, as memory errors in one connection may affect the broader server process.

PostgreSQL faces challenges managing memory and concurrency due to its long-standing architecture involving nearly a million lines of C code. The thread-per-connection model attempts to solve these limitations but risks the isolation benefits provided by separate process-based backends. Developers are evaluating these trade-offs to determine if performance gains outweigh the potential reduction in crash isolation.

AI Influence and Development Velocity

  • AI coding agents were central to scaling the effort required for a rewrite of this magnitude.
  • Unverified development claims suggest up to 50% faster transactional throughput and 300x faster analytical performance.
  • The project serves as an experiment in whether AI can lower the cost of complex infrastructure maintenance and modernization.

The rewrite demonstrates how automated agents can potentially change the economics of refactoring legacy systems. While the performance benchmarks for the thread-per-connection version are significant, the code powering these figures remains unavailable for public inspection or verification. Current testing remains limited to the public release version.

Production Viability and Future Outlook

  • Passing regression tests validates protocol compatibility but does not equate to the long-term reliability required for production environments.
  • The lack of comprehensive extension support remains a major barrier to widespread adoption.
  • The project is currently intended for exploration and testing rather than replacing production clusters.

While passing 46,000 regression tests is a notable milestone, production databases require extensive validation through crash recovery and long-term stability testing. The project represents a significant technical experiment that may influence future infrastructure development. Users are encouraged to test client libraries against the existing Docker image to aid in further development.

커뮤니티 글

모든 글 보기