00:00:00I think the internet is desperately in need of some more whimsy, especially now that AI
00:00:03is good at one-shotting websites.
00:00:05So what if it was easier to just make the web more fun? Like this. Don't worry, I'm
00:00:09not holding you up or anything, just having a bit of fun on this website. You can see now
00:00:13there's a load of my eyes down in the bottom. And also by the way, if I can hit the subscribe
00:00:17button, so can you.
00:00:18Instead of that, maybe you want a more functional but beautiful interaction for your site, or
00:00:22you want to invent new dark patterns like making a user play pinball to unsubscribe. Or just
00:00:27maybe after a long day of work you want to sit down at your computer with a beer, to sit
00:00:30down at your virtual computer with a beer and browse Twitter or something. All of that is
00:00:34possible thanks to something called HTML in Canvas. Let's talk about it.
00:00:43So I only found out about this recently thanks to Matt Rothenberg on Twitter. And after I
00:00:46saw this post, it seemed to kick off a week on Twitter where there was just loads of cool
00:00:50demos popping up all the time. I saw that Wes Boss was giving this a go and having loads
00:00:54of fun. Then I also saw this one from AA which was a finger gun one. And both of our demos
00:00:58actually inspired that YouTube one I had in the intro. So I'll link all of the sources
00:01:02for the demo shown in the description below if you want to go and see more.
00:01:05For now though, what is HTML in Canvas? Well, in short, it just lets you drop in real interactive
00:01:10DOM elements straight into your WebGL or 2D canvases. And at the moment it is just a proposal.
00:01:15We actually got added to Chrome Canary as an experiment, which is why all of these demos
00:01:19have popped up recently and you can try it in Chrome Canary now by enabling this flag.
00:01:24Your next question though might be why? Well it's because Canvas can enable some awesome
00:01:28UIs and customisations like we've already seen. Things that are harder to do if not impossible
00:01:32in CSS. But what Canvas can't do is easily render complex layouts of text and HTML content.
00:01:38You tend to have to rebuild them from scratch internally. And this has meant that Canvas
00:01:42based content can suffer from things like accessibility, internationalisation, performance and quality.
00:01:47And these are things that HTML has largely solved. So HTML in Canvas is the best of both
00:01:51worlds.
00:01:52To show you how this works, let me take you through a demo of actually using it. And what
00:01:56I have here is a London underground timetable site that I made years ago before AI could
00:02:00probably one shot this. And I always thought it would be cool if we could have this info
00:02:02inside of a 3JS scene. Now yes, technically you can do this with 3JS, obviously 3JS supports
00:02:08text, but it's going to be much easier if I can just use HTML in Canvas and take the element
00:02:12that uses the timetable and just put it straight inside of my scene. So instead of this picture
00:02:16of Thomas the Tank Engine here, I just want to see my HTML. To do that, the first step
00:02:20is going to be taking the HTML that we want to actually render in the scene and putting
00:02:24it inside of the canvas. So this is the HTML that actually built out that board that we
00:02:28just saw. And we put it as a child element of the canvas itself. Now at the moment, this
00:02:32element is actually serving as the canvas fallback. So if the canvas doesn't load on the user's
00:02:36browser for whatever reason, they're actually going to see this element. And that's not what
00:02:39we want. To fix that, what we can do with HTML in Canvas is on our canvas element provide
00:02:44an attribute called layout subtree. This then tells the browser to treat any canvas children
00:02:48as real layout participants. So they get put into the accessibility tree, they can receive
00:02:52focus, but they're still not painted onto the screen. We can see that on my demo here
00:02:56where there's still nothing showing up. The element isn't going to display anywhere here,
00:03:00but we do have it in inspect element and we hover over it. It does show that it's technically
00:03:04rendering. It's just invisible. So to actually render on the canvas, we need to convert it
00:03:08into a texture, which we can then use in place of the Thomas the Tank Engine picture. And
00:03:12that's exactly what I'm doing with this function here. Now, most of this is actually three
00:03:15JS, so you don't need to worry about it. The first one is simply getting the texture, which
00:03:19is GL texture, and that is the Thomas the Tank Engine picture at the moment. But then what
00:03:22we're doing on this line is using a HTML in canvas function called text element image 2d.
00:03:27Now it looks pretty complex, but all we're doing is we're taking three JS texture that
00:03:30we want to apply the element to. We're providing some information about how it renders in like
00:03:34the color space and other things for the GPU, but we don't need to worry about that too much.
00:03:38And then we're also just providing the HTML element that we want to render. In this case,
00:03:42it's bored. And that simply comes from this line here where we're just using document dot
00:03:45get element by ID to get the element that we put inside of the canvas. With that back
00:03:49in our demo, you can see we actually now have the timetable in place of that image and it's
00:03:53live updating. The clock is updating and I've seen the times update as well. So this is now
00:03:57using our HTML element, but it's being provided as a texture onto this canvas. Now we're going
00:04:02to inspect element. You can actually see we hover over the element that is this board that
00:04:06is still being rendered in, in an invisible way. And that's because you can think of this
00:04:09as essentially just looking at what this element would be, taking a screenshot of this and then
00:04:14placing it in the textures place. So it's going to update each time any of those elements re-render.
00:04:19This is actually done with a paint event in HTML canvas that triggers the updates when
00:04:22it detects that any of the canvas children elements have been re-rendered, but you can
00:04:26also request a repaint if you want, similar to how you can with request animation frame.
00:04:30Now if you're still a little confused, I highly recommend checking out that proposal on GitHub.
00:04:34It has loads of information and demos. And to be honest with you, I did go down a little
00:04:38more of a complex route, choosing three JS and WebGL. But if you want to see the most
00:04:42basic example, all you need to do is inside of a canvas, put in something like a form element
00:04:46here. And if you want to render that onto the canvas, we can simply say context dot draw
00:04:49element image, then provide a form element and also where we want to put that. So that
00:04:54is the most simple form of HTML in canvas. I also just really quickly want to show you
00:04:58this demo. I think it's loads of fun. You can see it uses web GPU and the copy element image
00:05:02function of HTML in canvas to draw a div under a jelly slider. And it's just loads of fun.
00:05:07But the really cool thing about this is it's actually still using an input behind the scenes.
00:05:11It's just a very cool custom input. That is the type of stuff that HTML in canvas can enable.
00:05:16There is some drawbacks though that the proposal is trying to iron out. One of the key ones
00:05:19obviously being performance. It's a little bit wonky at the moment. There are also a few
00:05:24bugs like draw element image being a frame late compared to the DOM state. This is a little
00:05:28bit of visual desync. And apparently if you try and put a scroll bar inside the canvas
00:05:32children, it's just going to straight up crash. But this is why it's an experiment for now.
00:05:36This is exactly what they're looking for. There has also been some privacy concerns that if
00:05:40this can render in a HTML element, you don't want it leaking any more information than you'd
00:05:44usually be able to get through JavaScript. So they do have privacy preserving painting
00:05:48risk, excluding sensitive info like system colours, themes, preferences, spelling and
00:05:52grammar markers, visited link info and so on. And I just think the key worry here is that
00:05:56they don't want to add another point of data collection for fingerprinting. Either way,
00:06:00I like what I've seen so far. So I would love if this could break out of just being an experiment,
00:06:04but I'm super curious what you think. Let me know in the comments down below why they're
00:06:07subscribe and as always, see you in the next one.