Transcript
00:00:00Hey, thanks so much for joining us today.
00:00:02I'm Praneet from the workflow team here at Vercel.
00:00:05Hi, I'm Nate, also from the workflow team.
00:00:08Nate, you and I have been on the workflow team from the very beginning,
00:00:12and of all the things that we've shipped in the last six months,
00:00:15I think hooks and webhooks are one of my favorite features,
00:00:18and that's exactly what you're here to talk about today.
00:00:21Hooks and webhooks are also my favorite feature.
00:00:23They're incredibly powerful, and I'll show you a few demos to explain why.
00:00:28So the first demo is something we're probably all familiar with, Magic Links.
00:00:33Magic Link is a login form. You type in your email, receive an email in your inbox,
00:00:40and when you click that link, you're logged into the service.
00:00:44Yeah, and if I remember correctly, with Vercel, actually even before it was called Vercel,
00:00:48when it was still called Zite, Magic Links were the only way that you authenticated,
00:00:52and we kind of built this whole system ourselves at the time, too.
00:00:56That's right, and I still have ETSD.
00:01:01Because without workflow, implementing such a system is a lot more complicated than it seems like at face value.
00:01:08Logic ends up being scattered across multiple files.
00:01:12You need to involve a database to keep track of your state, and it gets messy quick.
00:01:19Yeah, I've already been thinking about sort of how I would structure this and what database I'd use,
00:01:24because this feels like a con problem that I've built stuff like this before.
00:01:28So yeah, I'd love to see what it looks like.
00:01:30Yeah, so just to demonstrate what I'm talking about, the pain points that I'm talking about,
00:01:38I started by implementing kind of a "traditional" without workflow version of a Magic Link login.
00:01:43And so there's three endpoints involved.
00:01:47First one is when the login form is submitted,
00:01:50and it needs to generate a session and store that session in a database, such as Redis.
00:01:57You need to implement a TTL, and you can't let data be hanging around forever, you need to expire it.
00:02:06And then send the email, this thing could fail, and then your login doesn't work, and that's a frustrating experience.
00:02:14Right, and then you have to go back and have a cron job or an intern clean up your database.
00:02:19I might have been the intern at the time.
00:02:22But then there's a second endpoint, this is the one that happens when the user clicks the link in their email.
00:02:28And this needs to basically query the database, restore the state that was constructed in the first endpoint.
00:02:36And we're already getting to really kind of spaghetti code.
00:02:38When I was trying to imagine what this would look like, this code looks so familiar and it's kind of how I would structure it too.
00:02:48We see that this is getting complicated quickly even though it's a very simple thing, a very simple concept.
00:02:54So let's look at how you would implement this feature in Workflow.
00:02:59The magic link implementation using Workflow SDK looks something like this.
00:03:05We can see we have our function, it has our useWorkflow directive, which means this is our Workflow function.
00:03:11And the very first thing we do is we call the createWebhook function, which comes from the Workflow package.
00:03:18And we are also using the respondWithManual option in this case, which means that our Workflow function is going to be responsible for writing or sending the response to the HTTP request that triggers this Webhook.
00:03:36And this is so you can do a redirect or something afterwards when they log in?
00:03:40Yeah, so if there's some information in our Workflow function that we need in order to know what kind of response to send.
00:03:51Similar to the first endpoint, we send the login email. This is a useStep function.
00:03:57So if something like this fails, Workflow SDK has automatic retries.
00:04:03The durability aspect is already providing some benefit over the traditional approach here.
00:04:10So sendLoginEmails is a step, and if it's doing one thing and if that email fails, you retry the sent email with the same URL that you've created already for the Webhook.
00:04:21And if we look here, this is a very interesting pattern.
00:04:26We're using promise race with a sleep five minutes.
00:04:30This is possible because this Webhook object implements a promise.
00:04:35So to await the request of this Webhook, you just await Webhook.
00:04:40Or in this case, you're doing it with the race. And this is cool because I kind of would have expected this Webhook feature to have a timeout or some sort of option as another argument.
00:04:50But I almost like that it's so much cleaner now that to do a timeout, you just model it as doing a race with the Webhook and a sleep.
00:04:58This feels like I can do a lot more with it. I could maybe race two different Webhooks with each other.
00:05:02There's not a lot that you can do when you have a couple of arguments in a function.
00:05:06But the fact that it's just a promise and I can do a promise.race against sleep, maybe get another step.
00:05:12I love this pattern. This looks like my mind is just racing on all the things I could build with it.
00:05:16Right, and that's the beautiful thing about the primitives that Workflow SDK offers.
00:05:21Everything is exposed as a promise.
00:05:23So standard JavaScript patterns like await promise.race just work.
00:05:28And then another thing to point out here is there's no Redis here. There's no database.
00:05:33In the traditional example, we were using Redis TTL in order to implement this timeout.
00:05:41And in this case, we're using the Workflow sleep primitive.
00:05:44And also no intern who has to go clean up a messy database afterwards.
00:05:50That's the best part.
00:05:51And so you can see that the Workflow responds to the public request by redirecting to the login success page.
00:05:59And then retrieves information about your user to return to the client that initiated the login page.
00:06:07And that's our entire Workflow. That's our MagicLink implementation is 50 lines of code.
00:06:12This is so incredible to see. Can we see this in action?
00:06:17So here's our MagicLink demo. So I'll just enter my email.
00:06:24And our Workflow got initiated there and sent the email. And there's a webhook just waiting.
00:06:31And in fact, our Workflow is suspended right now. So there's zero compute being consumed while waiting for the human to click the link in the email.
00:06:41Oh, and what does this look like on Vercel? Can I see a run that's hanging around?
00:06:47Okay, so we did get the email. And before I click that, let's take a look at the observability.
00:06:52I know I'm just like jumping around, but I love that we're looking at this.
00:06:57Okay, so we can see that our run is here and started 40 seconds ago.
00:07:02If we take a look, we have our standard observability features that Workflow provides.
00:07:08We can see the inputs to our Workflow run. You can see my email address that I typed into the login form.
00:07:13And interestingly, we can see our hook here is just waiting.
00:07:17And you said there's no compute running right now. This is the observability, but there's nothing actually setting in waiting for me to click that hook.
00:07:25That's right. So the hook is waiting, and the sleep is sleeping, and both of those things do not actually involve any compute.
00:07:39But we can see that our hook, and if you remember, both of these are racing in a promise.race.
00:07:46So one of these needs to finish first for the Workflow to continue.
00:07:50So if I go ahead and click that link, okay, so we can see that I was redirected to the login success page, which was one of the steps in our Workflow logic.
00:07:59And if I jump back to the login form...
00:08:01Right, and then back on the dashboard, that should be complete as well.
00:08:05That's right. So our Workflow did complete.
00:08:08And you can see the timer just stops as well once the hook's won.
00:08:11Yeah, so we were able to implement magic link with around 50 lines of code.
00:08:17That's really neat. It's so cool to see how, you know, if we just had to draw a whiteboard diagram of how magic link works and explain it to someone.
00:08:27So the steps that you have in code is exactly how you would map it out, but now that's what the final code is too.
00:08:34There wasn't an additional database in between. There wasn't multiple API routes. It was just the code you showed me read so cleanly.
00:08:41And I think that is the most powerful aspect of Workflow SDK, the way that it allows you to structure your application logic to flow logically as opposed to stretching itself to fit the infrastructure.
00:08:59Right. Also, I love the idea with webhooks here, naming it webhook because it gives me a whole different way of thinking about webhooks.
00:09:07It's just this ephemeral URL that I can create and suspend on.
00:09:10Actually, this is a good segue because I think about, you know, we build a lot of these agents at Vercel.
00:09:16We build a bunch of Slack agents and GitHub agents, and we often subscribe to webhooks from GitHub or Slack, right?
00:09:23Every single time there's a new comment in a PR and we want to kick off a Vercel agent, we want to do this based off of an event that GitHub sends these webhooks, right?
00:09:31Can we use Workflow webhooks to actually subscribe to events from GitHub, for example?
00:09:36Okay, so for something like a webhook dispatched from Slack or GitHub, typically you have to go into the dashboard manually and configure a static callback URL.
00:09:49Right. You can't create this like one-off, I can't give it a one-off URL the same way I can do with an email here.
00:09:54Right, right. So the create webhook feature is a little bit more high level in that sense, where it provides a randomly generated unique webhook URL.
00:10:04That maps back to one specific Workflow run.
00:10:07In the case of our GitHub or Slack webhook route, that might map to any number of Workflow runs.
00:10:14Right. You have to pre-configure something that, but you have multiple pull requests, but they all go to the same endpoint.
00:10:20So in order to implement that with Workflow SDK, we're going to drop down a level, we're going to use the more low-level hook primitive.
00:10:28And I have just a demo to show that to you.
00:10:31Let's take a look.
00:10:32Okay. So this is the storytime bot.
00:10:35This is the very first application that I wrote with Workflow SDK a little over a year ago.
00:10:40And how it works is you just type the storytime/ command and we're going to see this thread get created.
00:10:47Each thread is represented by an individual Workflow run.
00:10:52And so when we expand the thread, we see that an LLM started this story for us and you or me or whoever is in this channel is able to continue the story.
00:11:05And the LLM will help us get it to its final conclusion.
00:11:09Okay. So Luna has a magical seed and what happens next? She plants the seed.
00:11:13Okay. And we see some activity happening here.
00:11:17What happens next? Magical thing.
00:11:20Our story is finished and we have a final story and there's going to be a little image generated as well.
00:11:26But we'll jump to that, back to that.
00:11:28I'm actually really curious already because I noticed that I was expecting one webhook request but there was actually, there was two, at least two requests there because you had two messages.
00:11:35So I'm really curious to see what this looks like in code.
00:11:38Okay. So this is the Workflow function for our storytime bot.
00:11:44So we can see that it takes in the channel ID, which is the storytime bot channel.
00:11:50It has some configuration options that you can pass.
00:11:53But interestingly, we can see that there's this messages array, which if you're familiar with AISDK, this is the data format that you store your AI conversation in.
00:12:04And in a typical Slack bot application like we've created here, you would normally store this kind of thing in a database and on each iteration or each webhook event, each message is typed into the thread, you would restore your state, you would look up the conversation from a database.
00:12:23And that's not what's going on here. This is just an array in your function.
00:12:27Yeah. Actually, it's fine because I chuckled earlier because I saw the intro and you had this comment that said, "Look ma, no queues or KB."
00:12:34And there's no import here for a database. You're just importing Workflow.
00:12:40And back to the last message thing, this almost gets missed, but you really just have a variable here called final story that presumably over time, we're going to push messages to this array, I imagine,
00:12:55and the final story is going to show up as a string here, but there's no way that this, there's no database that this has to go to. So it's almost like let is your database here.
00:13:04Yeah, let is your database is a great term that we should, we're going to coin that.
00:13:10I may have stolen that from you, but.
00:13:14The interesting thing here, and the thing we're here to talk about is the hook feature that we can see that we're creating this hook here. And what's different from the web hook example that we saw with the magic link is that in this case, we're providing a token,
00:13:28which is a string that includes identifier information that is unique to this Workflow run.
00:13:35The TS is the thread ID. So this string is the token that uniquely identifies this Workflow run.
00:13:44So when we look at the code for the web hook route, we'll see that the event payload that Slack sends contains all the information that we need to deterministically recreate this identifier.
00:13:58And that's the magic of how the web hook maps back to an individual Workflow run.
00:14:04Yes, I was curious when I saw web hook because you're creating new URLs for the magic example, but because we've built these Slack bots before, you can't just have a, you can't give it a new URL in every single thread.
00:14:17So the way to understand the way you're doing it here is you have an API and applying this already hooked up to Slack, but every single time you get a message there, you're basically calculating the same token on the resumption side.
00:14:29So your Workflow can basically wait for this token and you can construct the same token from a message payload to resume this Workflow run.
00:14:37Exactly. Yeah. So the Slack bot was configured once in a manual click around in the dashboard of Slack fashion, and you need to statically define a webhook callback URL.
00:14:50So that's why the lower level hook primitive works better in this case, because we can dynamically recreate the token.
00:14:59So just to quickly look, this is the webhook route, and there's not a whole lot going on here actually.
00:15:07The main thing is how we recreate the token from data passed by Slack.
00:15:13And then we call the resume function and this resumes the Workflow run unique to that.
00:15:20So I actually imagine, that's really cool. And I guess I actually imagine that with webhooks, what you're doing is kind of the same thing.
00:15:28Is webhook basically just doing a random token and then you have an HTTP end point that just resolves the same random token?
00:15:35Yeah, well, so the difference with the webhook feature is that you don't need to define that API route in your code.
00:15:44The Workflow SDK actually implements a default route for the webhook feature for you.
00:15:50But yes, other than that, it's a randomly generated token unique to one specific Workflow run.
00:15:55But in this case, we have our hook with the token and back to something that you mentioned just a moment ago, this hook can receive data multiple times.
00:16:06This is different from the Magic Link example, which only needed to be triggered one time.
00:16:11In this case, we want the hook to fire for each unique message that somebody types into the Slack thread.
00:16:17So to do that, you use the 408 syntax in JavaScript, which is common with async iterators.
00:16:25But in this case, we're receiving multiple event payloads from the Slack webhook using our hook.
00:16:33This is so cool. I've never found a great use case for... I love async iterators and I love generators and I even did a talk about this a long time ago.
00:16:42But they've always been good for demos and I haven't been able to find a good way to use it.
00:16:46Here it reads to me like you just have a loop.
00:16:50But instead of looping on some fixed set of items or looping on a timestamp because you're using 408 and then doing it on the hook, here's a loop that maps exactly.
00:17:01Everything inside the loop maps to one user message.
00:17:05And that's a nice way to think about this with new user message will cause another iteration of this loop and it just queues up and keeps going.
00:17:12The beautiful part about this is that for each iteration of this loop, while we're waiting for the user to type in the next message, there is absolutely no compute being consumed.
00:17:22The workflow is completely suspended and the next message in the thread could arrive in minutes or days or maybe even never and that's totally fine.
00:17:33So there's probably like Slack threads in that same channel with Slackbot where I could go back now and there's still a run just sitting and waiting for a couple of weeks if no one responded.
00:17:42That's really cool.
00:17:43And so to touch back on that messages array that we mentioned earlier, now we're modifying the array.
00:17:48We're pushing the new user message and that's our database modification because our messages array is just a local variable.
00:17:57That's so cool. And then I can see you're doing more promise.alls to parallelize more steps in between.
00:18:03This reads so cleanly for every single loop, every single message on Slack.
00:18:08I like how this is exactly how I would model this if you were trying to build this in a hackathon or something.
00:18:12It's kind of like here's how I would write down what happens in every single message.
00:18:16Yeah, so then the promise.all model, these are just regular use step functions and the idea is to run them in parallel.
00:18:23And so something like adding a reaction to the Slack message is just to give more immediate feedback to the user that something is happening.
00:18:32But at the same time, we want to initiate the LLM to help move the move the story generation process along.
00:18:39I'd actually be so interested whenever to see what the observability looks like as well when we when we get around to it, because I can imagine those sort of spans starting at the same time and making it super obvious.
00:18:49So we do have the observability for our story time.
00:18:52It is completed, so we'll have to go back and check on that image.
00:18:56So we can see our hook.
00:18:58And what's interesting here is that in this case, we have two hook received events.
00:19:05So that maps to the two messages that I typed into our Slack thread.
00:19:10And the observability allows us to see the individual data that was provided to the hook.
00:19:14Oh, that's really cool.
00:19:16So that's basically the Slack payloads. And you didn't have to log this additionally. The Slack payloads just show up as events that I can go back and inspect on the dashboard.
00:19:25Right. And then we can see that each time the hook payload was received, that continues our workflow execution and our steps continue.
00:19:34And then we finally have our result of generating our storyboard image, which looks like there.
00:19:40So that's story time bot.
00:19:42That's really cool.
00:19:43I think seeing webhooks both generated from magic links and now seeing you use the lower level primitive with hooks and also do them in a loop so you could actually do multiple events.
00:19:54That's really cool.
00:19:55I feel like the model really clicks for how I do human operations with webhooks.
00:20:02Is there anything else you can use hooks for?
00:20:05Yeah, definitely.
00:20:06The last demo that I have planned for you here is a very similar pattern to responding to a Slack webhook.
00:20:17But in this case, we're going to be using webhook as a way of kind of handing off execution of our application code and then waiting for some compute to finish elsewhere while our workflow suspends and waits.
00:20:32And then using that webhook URL to call back into our webhook and then we can finish doing whatever we need to do with our application logic.
00:20:41For this example, we'll use Vercel Sandbox and we'll do some long running compute operation like using FFmpeg to do a conversion of a file.
00:20:51So this is our FFmpeg conversion workflow.
00:20:56One of the first things that happens is we create a Vercel Sandbox.
00:21:00What's interesting about this is actually that the Sandbox NPM package exposes functions that underneath the hood have used Step in them.
00:21:09So actually this operation is a Step.
00:21:12So you can ship an NPM package.
00:21:15Sandbox is basically just shipping an NPM package that has used Step, the directive, inside this function.
00:21:21So when you import it and use it inside a workflow, it automatically makes Sandbox a Step without you having to write any of that code.
00:21:29It doesn't mean you can still use Sandbox to create outside of workflow.
00:21:32What happens when you call this without a workflow?
00:21:35If you realize the directive is just a string and if you were just going to execute this without the workflow compiler, that string doesn't do anything.
00:21:47So this just works.
00:21:49Adding used Step in your NPM packages just works fine without workflow SDK.
00:21:55And then once you use that function inside of workflow SDK, you get the added benefits, durability benefits out of the box.
00:22:03Okay, so the Sandbox just does some typical things.
00:22:07It installs FFmpeg because that's not available by default.
00:22:11It downloads the URL of a file that we're going to specify.
00:22:14And each of these runs are just Steps as well right now?
00:22:17Yeah, so these run an individual command in the Sandbox and those are Steps. We'll be able to see them in the observability.
00:22:29And then we're back to calling create-webhook, which you might remember from the Magic Link demo.
00:22:36But in this case, we're just going to pass that webhook URL into our Bash script that we're going to run in the Sandbox.
00:22:43What's going on here is we're going to run FFmpeg and convert the file to the format that we request in the UI.
00:22:53And then when that's done, the Bash script is going to run a cURL against our callback URL from the webhook.
00:22:59And when that cURL request happens, our workflow logic resumes.
00:23:04Okay, gotcha. So that's cool. I can see already, I was keeping a little bit ahead, but I noticed that there's an AND on this run.
00:23:11So you're actually writing the script running this in the background because an FFmpeg Step like this could take a lot longer.
00:23:17You don't want a Step that's just sitting around and waiting for it.
00:23:20Right, right. So this line right here starts our FFmpeg version script in the background.
00:23:28And then our workflow function suspends and then we wait for the webhook to be resumed.
00:23:34And I see the promise race again with a one hour sleep. That's such a cool pattern.
00:23:40Right, and so this time, you know, our FFmpeg conversion process might take a long time.
00:23:46It could be a very large media file. So we're specifying a one hour timeout in this case.
00:23:51And that's totally fine. In workflow, you can sleep for essentially indefinite amount of time.
00:23:56And again, there's zero compute running while we're waiting for this webhook to be resumed.
00:24:01And can we see this? Can we see this run? Do we have a demo?
00:24:04We do.
00:24:05It's a little bit of a silly example.
00:24:07Yeah, no, I recognize the big bunny example immediately. That's from Blender.
00:24:12Yeah, I remember looking at these videos when learning Blender a long time ago.
00:24:16Oh wow, I'm jealous.
00:24:19So we got our media file URL pasted in. In this case, we'll just extract the audio layer from it.
00:24:26So once we click the button, that initiates a workflow and we should be able to go over to our observability.
00:24:33Oh, there it is. Yeah, so we can see our sandbox create.
00:24:37And that returns our sandbox instance. Pretty cool.
00:24:42And this is because sandboxes, everything in workflow has to be serializable.
00:24:46But like you said, sandboxes implement serialization. So they're actually serializable as well in the show up in workflow.
00:24:53Right. Yeah. So the Vercel sandbox and can package as a sandbox class and that class implements the workflow serialization functions.
00:25:03And so it just works in our observability.
00:25:06And so any package can do this, right? It's not just sandbox. There's nothing special about sandbox that any class that wants to work inside workflow could implement the same symbols and have you step directives.
00:25:17Yeah, that's right. So we can see that our hook ended up being called back in 20 seconds this time.
00:25:25A little bit faster of a conversion because it's kind of a small file, but that could have been any amount of time.
00:25:31We can see that after our sandbox got created and initialized, our hook got created and we passed that into the sandbox to initiate our FFmpeg command.
00:25:43And when that finished, we received a payload from our sandbox.
00:25:48And this is the curl that happened inside the bash script earlier. So it's writing the command and then literally just using curl in a sandbox to complete the webhook.
00:25:57Right. So our sandbox finished the work it was doing. So it's handing control back to our workflow.
00:26:04So the way I'm thinking about this now is with steps in workflow, you run a step, it runs a code in the background, and then continues the workflow.
00:26:13But like hook and webhook both feel like they're a lower level. I can just create a token or a URL and wait for anything.
00:26:21That could be a human magic link. It could be an email. It could be a sandbox. It could be any sort of computer, whatever that has to happen.
00:26:27And I have my workflow just like pause with all of its state till that event happens. It almost feels like it's lower level than step itself.
00:26:34Yeah. The way I think about it is webhook and hook are a way of passing in external payloads into your workflow.
00:26:42The way I think about this is step is a way that a workflow can suspend and then wait for some sort of compute to finish and then we resume.
00:26:50But hook and webhook both actually feel like they're even lower level because you're just making a token or a URL that you could send, in this case a sandbox, but you could send anywhere.
00:27:01It could be a human. It could be an email. It could be another workflow, for example.
00:27:05And whenever that completes, your parent workflow basically wakes up and resumes right where it left off.
00:27:12So it's somehow even lower level than step. It's a way to just suspend your workflow for any sort of external action.
00:27:19Yeah. The way I like to think about it is hook is a way of suspending your workflow and waiting for some external payload to be passed back into your workflow, which is very powerful.
00:27:31This is really cool. I know we're all out of time today, but with those demos you kind of validated again to me why hook is my favorite feature in workflow and I'm so excited to keep building with it.
00:27:42Awesome. Yeah. I'm glad you enjoyed.
Community Posts
No posts yet. Be the first to write about this video!
Write about this video