Transcript
00:00:00This is LightPanda, a headless browser designed for AI agents written from the ground up in Zig, so no WebKit or Chromium, which makes it up to nine times faster and uses 16 times less memory than headless Chrome.
00:00:12It even exposes the Chrome Developer Protocol so it can be used with Puppeteer or Playwright, but it doesn't render any pixels or support common web APIs like service workers, IndexedDB and CORS.
00:00:22So why is it even used in OpenClaw and the cells agent browser? Hit subscribe and let's find out.
00:00:30LightPanda was released around 2024 by Pierre, Francis and Kate long before Clawed code was even a thing and around the time, ChatGPT started to get popular.
00:00:41It started out as a web scraping and automation tool, but soon after agents blew up in popularity, they focused to AI agents and managed to get a round of funding out of it.
00:00:51But was this just a quick rebrand because of the popularity of AI or is it actually good for agents?
00:00:56We're going to test this out by building our own agent using the Clawed SDK with a custom WebFetch tool, and we're going to use Chrome and LightPanda as the web browsing engines and compare the differences to see which one is better and why.
00:01:09Although LightPanda does have an MCP server and has its own cloud service, we'll try and do things as locally as possible.
00:01:16And we'll go through a basic example of how to use LightPanda first before expanding into making the WebFetch tool.
00:01:22So here is a script I have that essentially fetches a bunch of links from Wikipedia.
00:01:27And the way that works is it will first create a LightPanda server, then connect to that server using Puppeteer.
00:01:32We have some code here to check how long the process will take, and it will go to Wikipedia and go through all the links in the references section and print them out.
00:01:40If you want to picture that, the references section is down here for the web browser page.
00:01:44And as you can see, the first reference link is this Device Atlas one, which matches the one down here.
00:01:50So LightPanda found all the links in 344 milliseconds.
00:01:53And if we try the same thing with Chrome, it takes 392 milliseconds.
00:01:58So there's not that much of a difference, but things change dramatically when you have multiple pages you want to scrape.
00:02:04So here's a similar script with one major difference.
00:02:07Instead of getting links from one Wikipedia page, we're doing 100.
00:02:11So if we go to the definition, we can see all the pages it's going to fetch from the web browser to HTTPS, JavaScript, Mongo, Python, and so much more.
00:02:19So if we try that with LightPanda, the whole thing is done in 18 seconds.
00:02:23And if we try the same thing for Chrome, it takes about 30 seconds.
00:02:26And in some cases, when I ran this, it took 60 seconds.
00:02:29But let's see how the performance differs if it's running through an agent.
00:02:32So here I have a very basic agent using the Claude SDK and there's a webfetch tool over here that runs this prompt.
00:02:39So it summarises the difference between ArrayMap, Filter and Reduce in JavaScript and goes to these specific links in MDN to find information about them.
00:02:48Below we have our agentic loop and here, because there's only one tool, there's the if statement to check if that specific tool is being used.
00:02:54But if we had more tools, that would be the case.
00:02:56And here is the webfetch function being used whenever there's a tool call.
00:03:00Now, below that, we have some metrics and a function here to calculate the browser memory.
00:03:04This project is going to use LightPanda and Chrome using the Chrome DevTools protocol.
00:03:09So we're going to run it on a port and Puppeteer will connect to that port before running the agent.
00:03:14We can start the server using LightPanda serve and then we can run our agent, which fetches the three URLs from MDN, gives the information to Claude.
00:03:22And here it gives us a response telling us the difference between Map, Filter and Reduce, as well as returning the metrics.
00:03:29Now let's try that with Chrome, making sure to get rid of any default caching it has since LightPanda isn't a full on browser and doesn't have the same level of caching that Chrome does.
00:03:38So it's only fair to remove it from Chrome.
00:03:40And then we'll run the same thing in Chrome, which also fetches the URLs from MDN and also gives us an answer from Claude.
00:03:46But here are where things are drastically different.
00:03:48So if we put the results side by side, LightPanda here and Chrome on here, both of them do three fetches.
00:03:54But we can see the fetch time for LightPanda was way faster, almost twice as fast as Chrome.
00:03:59The wall time is the same.
00:04:01But look at here, the browser memory for LightPanda is only 66 megabytes, whereas for Chrome, it's 829.
00:04:07So way more than 10 times LightPanda and also the agent memory is a bit smaller for LightPanda, but the browser engine wouldn't really change this value too much.
00:04:17Honestly, you should never use Chrome ever again for headless browsing.
00:04:20I mean, why would you? LightPanda is up to 21 times smaller than Chrome, is faster and uses way less memory.
00:04:28But there is one scenario where you might want to use Chrome instead of LightPanda.
00:04:33So if we change the prompt from the JavaScript question to fetch me five properties from Airbnb in Tokyo, running that with Chrome gives us the expected results.
00:04:42But if you try the same with LightPanda, we see it isn't able to get the listings from Airbnb.
00:04:47And this is because Airbnb is a single page application, whereas LightPanda does have a V8 engine to execute JavaScript.
00:04:54This is just for language level execution.
00:04:57So async await, closures, promises and things like that.
00:05:00But for something more complex like this single page application for Airbnb, LightPanda does struggle to render this and get all the relevant information.
00:05:07So apart from not being able to render SPA's correctly, which they might change in the future, I don't know.
00:05:13LightPanda is a great tool or addition to add to a custom agent that needs to do any web search or any web fetching.
00:05:20And while we're on the topic of web browsing with an agent, if you've ever wanted clause code to control your actual running Chrome session safely.
00:05:27So with your logged in details and everything, then check out this video on remote debugging using the Chrome MCP server.