Run S3 on Your Laptop? This Changes Everything (MinIO)

BBetter Stack
컴퓨터/소프트웨어창업/스타트업AI/미래기술

Transcript

00:00:00S3 bills can get way too expensive, apps generate so much data that every upload and download
00:00:05can just get heavy. And most devs think object storage means one thing, the cloud.
00:00:09But that's not always true. What if you could run an S3 compatible object store right on your laptop?
00:00:15That's where Mini I/O comes in. It's fast, S3 compatible, and runs pretty much anywhere,
00:00:20even locally on your Mac. In the next few minutes, I'm going to show you guys three things.
00:00:25How to set it up and run it, why devs are using it, and the reality of tools like this one.
00:00:30[MUSIC]
00:00:35If you're trying to cut cloud costs or make development easier, this could save you a lot
00:00:39of pain. Here's what this really solves. Most devs and teams are dealing with these same three
00:00:45problems. Cloud bills are hard to predict. Data that's too far away from the app using it,
00:00:51and AI workloads that need fast local storage. Mini I/O gives us another option. We get S3
00:00:57compatibility, strong performance, and full control without relying on the cloud.
00:01:01It's great for local development, prototypes, rag pipelines, media, backups, all that stuff. And yes,
00:01:07you can run the whole thing locally, no VPS, no cloud account. Now, quick disclaimer,
00:01:12the GitHub repo is now archived and it is in maintenance mode. So we can't just use this
00:01:17forever. They've shifted focus from Mini I/O to Mini I/O AI Store, but the I/O is still there too.
00:01:24We can use this for a good while with actual benefits. If you enjoy tools that speed up
00:01:29your workflow, be sure to subscribe. We have videos coming out all the time.
00:01:32All right, let me show you how simple this is. I've got Mini I/O running in one Docker container
00:01:37here on my M4 Pro. First, I connect it and I'm going to run MC alias set local. I'm going to add
00:01:44my local host and just use their given login credentials for now. Then I can create a bucket,
00:01:50MC, MB local demo bucket, we can call it. And then I'm just going to start uploading a few files,
00:01:55image, text, JSON. So first my image, then I could do a light text file. And finally,
00:02:02let's just add in some JSON data here. Now, if I list everything, there we go. Clean structure,
00:02:08just like S3. Now here's the part most of us actually care about. I run this Mini Python
00:02:14script right here. And it's still using Bato3, same code style as AWS. There's no changes here.
00:02:22And in the browser, it's just over on local host. Open the bucket. I can expand the folders. I can
00:02:27click the image and there's your preview. This feels like S3, except it's running on your laptop.
00:02:33So what is Mini I/O really? Basically, it's an object storage server that speaks the Amazon S3
00:02:39API. So you keep everything you already know. We can keep the same SDKs. We keep the same commands,
00:02:46the same mental model, but instead of being tied into AWS, it runs wherever you want. Docker,
00:02:53Kubernetes, bare metal, edge, or just on your laptop like I am here. And this matters more now
00:02:59than it used to because our apps are only getting more data heavy as the years pass. Because when
00:03:05your computer is here and your data is somewhere else, we end up paying a lot of money for that,
00:03:10in latency and in cloud costs. Mini I/O changes all this. It brings storage closer to where the work
00:03:17happens. And that's why this setup is so useful for development and testing. Not for production,
00:03:22but those two it's really good for. In my case, I'm running it locally, which makes it great for
00:03:27building and debugging data heavy apps before even touching production. Now we do have a lot of other
00:03:33options out there, sure. Like with AWS S3, you get fully managed storage and massive scale. But again,
00:03:40the trade-off is cost, fees, and then you're tied into AWS. With Ceph, you get something great and
00:03:46also scalable, but it's more complex to run. More moving parts, more overhead. Now a common pattern
00:03:52might look something like this. I can use Mini I/O locally for development, Mini I/O or S3 for
00:03:57staging in cloud in production. That works because your local setup still matches the S3 model your
00:04:03app already uses or will use. So if your app handles a lot of data, Mini I/O just makes sense
00:04:10as we're in development. All right, now let me be real for a second. Should you use this? Well,
00:04:15it's easy to deploy, it's single binary or Docker, strong S3 compatibility, and it's really fast.
00:04:21And if you're moving a lot of data, avoiding egress fees is a big win. But here's the thing,
00:04:26the open source Mini I/O you're seeing here is still free under the license AGPL, the version 3.
00:04:33And it works well for demos, local dev and prototypes. Now again, they've shifted their
00:04:38focus to Mini I/O AI store. That includes a free tier for single node use, but then
00:04:43they're now adding paid tiers for clustering, high availability, and all these other features.
00:04:48So for learning local dev and small projects, this is really cool. It's good. But for production
00:04:53clusters, you'll likely need AI store or alternatives like Ceph or Garage. So it's worth
00:04:59it if we're in local development or just experiments, but maybe not if you need fully managed infra or
00:05:04built-in HA from day one. We can still stick with other ones for this. Mini I/O or AI store free
00:05:11gives you S3 style storage, strong performance, and full control over your data without automatically
00:05:16paying the cloud premium just yet. And you can try it right now on your laptop like I've done here.
00:05:21If you enjoy open source tools and coding tips like this to speed up your workflow,
00:05:25be sure to subscribe to the Better Stack channel. We'll see you in another video.

Key Takeaway

MinIO enables full S3-compatible object storage on a local laptop via Docker to eliminate cloud costs and latency during the development and testing of data-heavy applications.

Highlights

MinIO provides an S3-compatible object store that runs locally on macOS and other platforms via a single binary or Docker container.

Developers can use the Boto3 Python library and standard AWS SDKs to interact with local MinIO storage without changing their code style.

The open-source version of MinIO operates under the AGPLv3 license and is suitable for local development, prototypes, and RAG pipelines.

MinIO AI Store has replaced the archived GitHub repository, offering a free tier for single-node use alongside paid tiers for high-availability clustering.

Running object storage locally eliminates cloud egress fees and reduces latency by keeping data close to the application logic.

Timeline

Challenges of Cloud-Only Object Storage

  • Cloud storage bills become unpredictable and expensive as applications generate more data.
  • Latency increases when data resides far away from the application using it.
  • AI workloads require fast local storage for optimal performance during development.

Most developers assume object storage requires a cloud provider, leading to high upload and download costs. MinIO offers an alternative by providing S3 compatibility and performance on local hardware, including the M4 Pro Mac. This setup grants developers full control over their data environment without a VPS or cloud account.

Local Setup and S3 Compatibility

  • The MC alias command connects the local MinIO instance using standard login credentials.
  • MinIO maintains a clean folder structure that matches the S3 organizational model.
  • Applications using the Boto3 SDK require zero code changes to point to a local MinIO bucket.

Setting up MinIO involves running a Docker container and using the MinIO Client (MC) to create buckets and upload files like images, text, and JSON. A browser-based interface on localhost allows for file previews and folder management. This workflow mirrors the AWS experience, ensuring that mental models and scripts remain consistent between environments.

Deployment Options and Architectural Benefits

  • MinIO runs on diverse environments including Kubernetes, bare metal, and edge devices.
  • Local storage facilitates building and debugging data-heavy apps before production deployment.
  • Developers can use MinIO for local development while keeping S3 for staging and production.

MinIO serves as a portable object storage server that speaks the Amazon S3 API. While AWS S3 offers massive scale, it introduces vendor lock-in and complex fee structures. Alternative tools like Ceph provide scalability but involve more moving parts and operational overhead compared to the single-binary simplicity of MinIO.

Licensing and Production Realities

  • The core MinIO software is free for local use and small projects under the AGPLv3 license.
  • Production clusters requiring high availability now fall under the MinIO AI Store paid tiers.
  • Small-scale projects can avoid the cloud premium by utilizing the AI Store free tier for single nodes.

The shift from the original MinIO repository to MinIO AI Store marks a change in focus toward AI-specific storage needs. While it remains a top choice for learning and experimentation, production environments with clustering needs may require paid versions or open-source alternatives like Garage or Ceph. MinIO effectively serves as a bridge for developers to build S3-reliant apps without immediate financial commitments.

Community Posts

View all posts