OpenCV 5 Is Here - The Biggest Update Since 2018 (We Tested It)
BBetter Stack
Computing/SoftwarePhotography/ArtInternet Technology
Transcript
00:00:00So we all know OpenCV, right? It is the ultimate computer vision library everyone's been using for
00:00:07resizing images, detecting objects, and performing all sorts of computer vision tasks. And it just
00:00:13got the first major release update since 2018. And this is a big one. OpenCV5 has a brand new
00:00:21deep learning engine that can run YOLO. It can also do stable diffusion style in-painting and
00:00:27even run a full vision language model all natively with no PyTorch, no ONIX runtime, and nothing else
00:00:35bolted on it. So in today's video, we'll break down what's actually new in OpenCV5, and then we'll test
00:00:41out the new features ourselves with some cool demos running locally on my own machine. It's going to be
00:00:47a lot of fun, so let's dive into it. So first off, here's why this update is a big deal.
00:00:57OpenCV is everywhere. It has more than 86,000 GitHub stars, over a million installs a day, and
00:01:05for two decades, it's been the foundation for robotics and AR and medical engineering, industrial
00:01:11inspection, basically any software that has some computer vision aspect attached to it. And for the
00:01:17last eight years, everyone's been building on the same version 4 line. So a major version bump here is
00:01:24genuinely a huge deal. And the headline feature of this new version is a complete rewrite of the DNN module,
00:01:32and that's the part of OpenCV that runs neural networks. So here's the most important number to pay
00:01:38attention to. In OpenCV4, the DNN engine supported only about 22% of ONIX operators. ONIX is the standard
00:01:47format you export a model to when you want to run it somewhere other than the framework you trained it on.
00:01:53And 22% coverage meant that more often than not, you'd grab a modern model, try to load it, and
00:02:01immediately hit a wall. Some operator wasn't supported, so you were stuck. But OpenCV5 brings that coverage
00:02:08up to 80%. So now this library runs most of these models out of the box. And the reason it jumped so
00:02:15much is because of how they rebuilt it. The old engine processed networks layer by layer. The new one is
00:02:22built around a typed operation graph. So it looks at the whole network as a graph first, and then does
00:02:29proper shape inference, constant folding, and operator fusion before it actually runs anything. So in plain
00:02:36terms, it understands the entire model before executing it, so it can handle dynamic shapes and
00:02:42modern transformer architectures that the old engine just couldn't handle. And here's the impressive part.
00:02:48With these new changes, not only is it more compatible, but it's also very fast. OpenCV benchmarked their new
00:02:56engine against Microsoft's Onyx engine. And on a CPU, OpenCV5 matched or even beat it on real models. So it
00:03:04was 11.5% faster than YOLO version 8, almost 37% faster than OWL version 2, and 30% faster than Xfeet.
00:03:15Now those are OpenCV's own self-reported numbers. So take them with the usual grain of salt and benchmark
00:03:22your own workload. But if they are true, that is pretty impressive. And there are more cool stuff in
00:03:27this new release like native FP16 and BF16 data types, real n-dimensional array support in CVMAT,
00:03:36a Python first core, and the legacy C API is gone and replaced with the C++17 now being the baseline.
00:03:44And one important catch you need to know about up front, the new engine is CPU only at the moment.
00:03:51GPU support is coming later in the version 5 cycle. So if you need GPU inference today,
00:03:58you will fall back to the classic engine, which still has CUDA and OpenVINO support.
00:04:03So everything I'm about to show you on this new version 5 runs on a CPU. So the biggest feature of this new
00:04:10version is the new DNN engine. So we'll test out some of the examples and see how it performs.
00:04:16All right, let's start with a fun example. So OpenCV ships a colorization example where you can take
00:04:22a black and white image and it predicts the color. And on a side note, I've been watching Spider Noir
00:04:28lately, and it's a pretty cool show. And it's also in black and white, so I thought it might be fun
00:04:34to colorize a shot from this show and see how it looks. So I'm going to take this image and run the
00:04:39colorization example, which is using the new DNN engine and boom, here it is. Look how fast that was.
00:04:47As far as the output goes, it's not perfect. We still see that it got the sky somewhat correct,
00:04:53but it failed to colorize some other parts of the image. And keep in mind that this is using an older
00:04:59colorization model, which plays it safe with the muted tones. But the point here isn't the model,
00:05:05it is the engine. So what I wanted to show you here that this image was produced using the native
00:05:11OpenCV neural network with zero extra dependencies, which is quite cool. Now let's try their object
00:05:17detection sample. And again, since we're on the Spider Noir theme, I'm going to be using a clip from the show
00:05:24and run it through the pipeline and see how it performs. And when I launched the script, look at
00:05:29that. It's doing object detection in real time, running on the CPU through the new DNN engine. And
00:05:36remember that benchmark from earlier? This is the kind of model where OpenCV is actually faster than the
00:05:43Onyx runtime. So you're not trading performance for convenience, you're getting both in this particular
00:05:49scenario. And you don't need to install PyTorch for this or a separate runtime. And there's no GPU required.
00:05:56This is all running in plain OpenCV. All right, now let's try their LDM in painting example.
00:06:03So LDM stands for latent diffusion and OpenCV 5 ships a latent diffusion in painting example here.
00:06:11So let's see how that works. So let's load up the same Spider Noir image we used earlier.
00:06:16And now I can paint over something I want to be removed from the image. It can be a person, an object,
00:06:23whatever. And the diffusion model fills in the blank space. And this is a bit slower to run because OpenCV's
00:06:31classic in painting uses a single forward pass, which is instant, but the LDM uses diffusion,
00:06:39which is iterative. So it starts from noise and then it denoises it step by step. So it's running this model
00:06:45multiple steps in a row. And because we're doing this on a CPU, it's even slower because diffusion
00:06:51is a task more suited for a GPU. But nonetheless, here is the result I got when running it on just a few
00:06:58steps. And I'd say it's done a pretty decent job. It's not perfect. We can still clearly see some diffusion
00:07:05artifacts. But this can be solved by increasing the step count or using a different diffusion model.
00:07:11And lastly, I want to show you the VLM inference example, which shows you how you can use vision
00:07:17language models or LLMs natively inside OpenCV to do things like image captioning or other sorts of
00:07:25things. Now, in this particular demo, we're using the Pali Gemma model to caption this spider noir image.
00:07:32And as you can see, it's painfully slow. And the end result is also nothing spectacular. So I would
00:07:39most definitely not use OpenCV for this. There are far better options for image captioning. For example,
00:07:45you can run a 12 billion Gemma 4 model locally on OMLX and get a hundred times faster output than this.
00:07:53I showed that particular example in one of my previous videos about the 12 billion Gemma 4 model.
00:07:58So check that out if you're interested. But the point of this demo is to showcase that OpenCV now has
00:08:04all the pieces you need to run LLMs. The tokenizer, the attention layers, and the KV cache,
00:08:11all built in. And the fact that it works at all on a separate runtime is genuinely signaling
00:08:18where this library is heading. It's going to have vision and language bundled all in one place. And
00:08:24once they ship the GPU update, it's going to be even better and even faster. So there you have it,
00:08:29folks. That is the new OpenCV version 5 in a nutshell. One library running on the CPU with no extra
00:08:37dependencies. And we used it for colorization, real-time object detection, diffusion in painting,
00:08:43and image captioning. Now you're still going to have to train your models on something like a Pytorch.
00:08:50That's not something OpenCV is designed to do. But when it comes to actually running those models
00:08:56inside the vision pipeline, OpenCV 5 is a great choice and a massive leap from where it was just
00:09:03a month ago with their new DNN engine. So what do you think about the new OpenCV 5? Are you going to
00:09:09use it in your projects? Let us know in the comment section down below. And folks, if you like these
00:09:14types of technical breakdowns, please let me know by smashing that like button underneath the video.
00:09:19And also don't forget to subscribe to our channel. This has been Andrus from BetterStack,
00:09:24and I will see you in the next videos.