Ship 26 NYC - Workshop - Mini-workers
VVercel
Computing/SoftwareSmall Business/StartupsInternet Technology
Transcript
00:00:00Hi everyone. My name is Jonathan Clem, or you can call me Jay Clem, and I'm a software engineer at
00:00:11Notion where I work on our developer platform, but I'm particularly focused on a newer product
00:00:17that you may have heard about called Notion Workers. In this workshop today I'm going to give
00:00:23you a little bit of an overview of what Notion Workers are and why we built them using Vercel
00:00:28Sandbox. And then I'm going to kind of take you through a little bit of a mini version of Workers,
00:00:36so you can get at least a sense of how a product like that gets built using Vercel Sandbox.
00:00:43So if you're not familiar with what Workers are, they're an SDK and a runtime where you can write
00:00:50custom code and extend Notion with it. So you can do things like sync third-party data into Notion,
00:00:57you can write custom tool calls for your agents. We've seen people do fun crazy things like order
00:01:04their groceries with Notion Workers or control their smart home. We've also seen really complex workflows
00:01:11from like especially the IT and security areas. The nice thing about Workers is that there's no
00:01:17infrastructure for you to manage. You just write code or have a coding agent write it for you, and Notion
00:01:23handles making sure that it's always up and available and running. This has been a huge benefit to Notion
00:01:30users, especially developers. They're no longer having to wait for Notion to build, you know, first-party
00:01:36integrations for them. Anything that they kind of wish Notion could do but doesn't, they're able to just write
00:01:42that themselves. So when we first started setting out building Notion Workers, my primary concern was
00:01:53safety. I have a little bit of experience building platforms where you're running untrusted user code.
00:01:59I've worked on GitHub Actions, for example, for a long time. And I was primarily concerned about the
00:02:04difficulty in building infrastructure for a product like this and especially the safety. There's a lot
00:02:09of things you have to worry about. For example, you want to make sure that code that's written by users
00:02:14can't touch Notion databases, of course, or can't touch Notion services that they normally shouldn't be able
00:02:20to access. You also want to make sure that users can't impact other users. You don't want one user's
00:02:26code, you know, being able to access another user's code or obviously their secrets or anything like
00:02:31that. This isn't just about security. It's also about fairness and resource sharing. If there's a user
00:02:37doing something that's using up a ton of CPU and a ton of memory, you want to make sure that that's not
00:02:42unfairly impacting other users that are trying to do things at the same time. I'm going to take a minute
00:02:48and go a little bit off and tell a story here. This is like the resource sharing is especially a really
00:02:54tricky thing. This is probably going to eat into my time, but I like this story. When we were building
00:02:58like GitHub Actions, and this is something we thought about with workers too, as soon as you have an
00:03:02arbitrary code execution platform, people are immediately going to try to do crypto mining on it.
00:03:07And I was having a conversation with somebody a couple of weeks ago where they're like, well,
00:03:10how do you detect that? You can just tell if a CPU is pegged in at a hundred percent, right?
00:03:14Well, not really, because once you're a successful platform, immediately crypto miners are going to
00:03:20start doing things like sharing scripts with each with each other that sort of modify the way that
00:03:26they're running CPU instructions so that it looks like totally innocuous activity, but they're using
00:03:31like the absolute maximum amount of resources possible without having them flagged in your system.
00:03:37So it's extraordinarily difficult, and it takes many years and many layers of security and
00:03:42observability in order to get this right. Another thing you have to worry about is when you have a
00:03:48code execution platform, you've got to worry about people doing denial of service attacks using it,
00:03:54using it for botnet command and control networks. And we just didn't want to have to worry about all of
00:04:00this stuff right off the bat with Notion Workers. We wanted to focus on what we love doing, which is
00:04:07delivering a product that our users love using. So that's why we decided to build with Vercel Sandbox.
00:04:14Vercel Sandbox, we could see right off the bat, was really solid infrastructure that solved a lot of
00:04:18this problem, a lot of these problems for us out of the box. So I'm going to show you a super quick demo
00:04:25of what Notion Workers are. So I have a custom agent here and the job of this custom agent is to tell me
00:04:33whether or not this workshop is cursed. I wrote a worker called Mercury Retrograde. I don't know if anyone is
00:04:40familiar with this, but when Mercury is aligned in such a way that it's in retrograde, it's usually
00:04:47a bad omen. So this worker has a single custom tool call that uses an API I found that does nothing else
00:04:54but tell you whether or not Mercury is in retrograde. So I'm going to prompt my agent and say, "Is my
00:05:01workshop cursed?" and it's going to think for a minute and then we should, as long as the internet's behaving,
00:05:14we'll start to see it making a tool call. So it's using my Mercury in retrograde worker, calling the
00:05:22single tool that that worker exposes and finding out whether or not Mercury is in retrograde and then the
00:05:27agent's going to respond to us. I think I have this connected to like GPT-54 nano. So it's usually much
00:05:37faster. I think this is internet, unfortunately. If anyone happened to check whether or not Mercury is in
00:05:46retrograde, I know in advance that it is. And so that's probably why this is happening. I'm just going to
00:05:53skip it. We'll come back to it in a minute and see if we eventually got a response. But I think we already
00:05:59know the answer, apparently. Okay, so now you've got an idea of what notion workers are. I'm going to take
00:06:05you through like a little script I wrote that does a lot of the basic tasks that are involved in taking
00:06:13user code, deploying it, executing it safely, and sort of telling an agent about that code so that it can
00:06:20make toolcalls to it. Did this ever complete? Oh, yeah, here we go. Oh, I think maybe,
00:06:28I guess I must have taken the Mercury in retrograde API down because apparently the API didn't respond.
00:06:35Anyway, I'll have to open a ticket somewhere. Cool. So I have a basic script. I don't expect
00:06:40you to like totally follow along and write code or anything like that. I'm kind of going to jump through
00:06:44and give you an idea of some of the problems that we solved that you have to solve when you're building a
00:06:49product like this. But if you want, there's a repo make notion slash for cell ship 2026 workers
00:06:55that has all the code that I'm going to be running here.
00:07:01Cool. So let me open up my other slides here.
00:07:08So what are we working towards? We have a streaming chat agent. We're going to allow it to call tools that
00:07:16are defined by custom user code that we don't trust or we don't know what's in that code.
00:07:22And we're going to build this with Vercel sandbox and Vercel's blob storage service.
00:07:26So I'm going to run a quick demo here and hopefully we're not actually cursed.
00:07:31You can basically just pass it a single message. I'm going to say hi and hopefully we'll get a response
00:07:36back. It's going to be a little slow because you'll see it's doing some deploys in the background.
00:07:40So called a tool this time. I just called a tool called say hello and it decided to greet me.
00:07:47I'll send it a normal message here just asking it what's one plus one so that we can see it just
00:07:52streaming a normal response.
00:07:58If you've used the Vercel AI SDK this is just using the tool agent loop. Cool. So we got a streaming
00:08:04response back. You can also call much more complex workers. So I have another one here that I'll show you.
00:08:14With this one we're telling it that I'm at the address of this building.
00:08:17I have 90 minutes and I want to go see some historical site and I don't want to walk more than 15 minutes.
00:08:24So this is kind of illustrating why these are sometimes favorable over MCP servers. With an MCP
00:08:29server you have a bunch of disparate tool calls that you can make. And if you're trying to do something
00:08:33complex you have to describe those steps to the agent and it's going to like do a step, spend some
00:08:38thinking tokens, do a step. So this is going to call a worker that has a very large complex sort of like
00:08:45search algorithm that uses a bunch of New York City geolocation transit times and route planning APIs.
00:08:52It's called plan outing.
00:08:56And that worker is going to respond with a proposed historical site that we can visit.
00:09:00And it looks like it found the freedom tree marker which is a MIPOW memorial that's down near
00:09:07City Hall Park. Cool. So let's take a look at how this works. So what is a worker?
00:09:14It's user code that in this case is just defined in a file. Normally this would be you know defined
00:09:20by your users or on GitHub somewhere and deployed with a CLI. But we just have some examples checked
00:09:26into the repo here. And each of these workers needs to expose the tool name, a description of the tool so
00:09:34that the agent can route to the right tool, the input schema so the agent knows what inputs it needs to
00:09:38provide every time it calls that tool. And an execution function that says when the agent calls this tool,
00:09:44what's the actual code that gets run. So let's look at an example. Here's the greeter tool that you saw
00:09:51earlier. It's very simple. This worker is one JavaScript module here at this index.ts file. It exports a single
00:09:59worker called say hello. So you'll see how this build process works. But in this example,
00:10:04all of our module export keys are mapped to the names of our tools. So this tool is going to be called
00:10:09say hello. And it has a simple description, an input schema. In this case, I'm just using Zod to define
00:10:16the schema. And then I'm converting it into JSON schema. That's the format that these agents expect.
00:10:22And then there's a simple execution function. You can make much more complex ones, though. So if I were to
00:10:27open up the plan outing workflow, you can see it's got a much longer description telling the agent
00:10:34all about how this tool works, when to call it, what it does. And the script for this is much, much,
00:10:39much longer and more complex. I'm not going to go through all this. And I have only read small parts
00:10:45of this, but it works pretty well. That's the world we're in now. So these workers are going to get
00:10:54built and deployed to Vercel blob storage. Then we're going to learn about the contents of the
00:10:59workers somehow. And we're going to expose them to an agent. And then those tools are going to get
00:11:03executed safely in a sandbox. The build part, we're kind of going to step. With Notion workers, we have a
00:11:09cloud-based deploy and build process. In this case, I've just pre-built these workers on disk. So each
00:11:15worker has a tarball that contains all the compiled TypeScript code dependencies and things like that.
00:11:22It's not the super interesting part, so I'm just going to skip over it.
00:11:26So the question of the day is, how do we go from user code that we've never seen and that we don't trust
00:11:34to tools that are exposed and run safely by the agent? So there's kind of two parts of this.
00:11:39The first part of it is, once we have that user code in blob storage, let's say,
00:11:43how do we learn about what's in that code? Because we've got to be able to tell the agent before it
00:11:48ever calls the tool, what's the name of the tool, what's the input schema, and what's the description
00:11:53of the tool? And then the second question is, when the agent decides to call that tool,
00:11:58how do we actually execute it safely? So one of the things that I love about what we did with the
00:12:04Notion Workers SDK and that we've kind of done here is that the code describes itself. So what we didn't
00:12:11want when we were designing Notion Workers is that we didn't want users to have to write some TypeScript
00:12:16code defining their tool and then say, okay, well, now I have to create a static manifest file describing
00:12:21my workers and basically rewrite the same stuff. We didn't want an awkward local build process where
00:12:27they had to run a script, do some analysis, compile it on disk and then deploy it. So we wanted to start
00:12:34with what felt like the optimal developer experience where you just write the tool and then it's our
00:12:40problem to solve the difficult thing of figuring out how to get information out of that.
00:12:46So this is a super simple diagram, but before we go into the code, I'm going to tell you about
00:12:52kind of how we're going to be solving this. So we have the user's compiled code on blob storage.
00:12:59We're going to use that code to create a sandbox. So when we run commands in the sandbox, that user's
00:13:04code is going to be in the workspace root. We're going to import the user's index.js file that they wrote.
00:13:11So that's going to give us all the names of the exports and those descriptions and the input schemas.
00:13:19This is where things get a little bit weird, but it works really well. We're going to call
00:13:23json.stringify on that module and then we're going to log it to standard out. So that all happens in
00:13:29the sandbox and then our deploy script consumes that standard out and parses it and says, okay,
00:13:34so now I know the name of all the tools on this worker, the inputs and the description. And in this
00:13:40case, we're going to pass it directly to the agent. But in the case of notion workers, for example,
00:13:44that's part of our deploy pipeline. So we take all that information and store it in a database so that
00:13:49every time you run a custom agent, we're fetching those tool, those tool descriptions out of the
00:13:53database. Does that kind of make sense so far how that works? Okay. So let's take a look at the deploy
00:14:02script. I have to go back to my first commit here. So this script has all of our deploy and then calling
00:14:12the agent built into it. It's super simple. In this case, we're just iterating over all the directories
00:14:18and workers. Each of those subdirectories contains worker code like you saw a minute ago.
00:14:23Our job is to figure out how to extract all the tool information from each of those workers and populate
00:14:29this tools object. That gets passed to a tool loop agent. This is just part of the AI SDK that's made
00:14:35by Vercel. And then we're going to send the user's message to that agent, stream the output, and write
00:14:42it to standard out from the script. So the first thing that we need to figure out is how do we just upload
00:14:48the source code. That part's pretty easy. It's pretty fast. Instead of live coding, I'm just going to kind
00:14:53of like skip between each of these chunks so you're not having to watch me type. I promise I can write
00:14:59code. I just think no one wants to watch me do that. So skipping ahead a little bit, we have this function
00:15:06called upload source that I'll go into. But you can see I've imported the Vercel blob SDK. This is pretty
00:15:13simple here. So if we look at this upload source function, we're basically just calling this put
00:15:20function. And we're saying that we want to store this user's bundle under the worker name slash bundle.tar.gzip.
00:15:29We're going to stream the file from disk and we're going to store that in blob storage. So once that's done,
00:15:35we are able to create sandboxes from that blob. So that's the uploading part of it solved.
00:15:43The next thing that we need to do is we need to take that bundled source code and we need to create
00:15:49a sandbox from it. So to do that, I've imported the Vercel sandbox SDK. There's another helper function
00:15:56down here called create sandbox. I'll skip over a couple of parts of this. But essentially what we're
00:16:03doing is when we have that object in blob storage, we're using pre-signed URLs that we're going to
00:16:10pass off to the sandbox service. So that the sandbox service can for say, I think this is set to 10 minute
00:16:16expiry, it can pull in that blob from blob storage and use it to populate a sandbox. This is one of the
00:16:23features that I really like about Vercel sandbox is that it works with tarballs like this. It's easy
00:16:28to build a tarball of user code, a sandbox service that you want to create a sandbox from a tarball,
00:16:35it grabs it from storage or whatever URL you give it and it automatically extracts that tarball in the
00:16:40workspace route. So all the files are just there ready for you to work with. So that's just done by calling
00:16:46sandbox.create. We're not at the super complicated part of this, but we're going to get there soon.
00:16:52Another thing is that in this example, just for brevity, I'm creating fresh sandboxes every single
00:16:57time we run them. In reality, you want to use snapshotting so you're not constantly like every
00:17:02time a tool gets called, you're not like redeploying the sandbox or re-streaming it from Vercel blob
00:17:09storage or whatever other cloud blob storage you're using. There's basically a caching mechanism built
00:17:15into the platform. I've just skipped over it here for the sake of simplicity.
00:17:19So that's creating the sandbox and this is kind of where it gets interesting.
00:17:23The next thing that we need to do is that we need to extract information about the tools
00:17:31that are defined in that worker from that sandbox that we've just created.
00:17:36I'm going to pause really here really quick here and there's a couple of tips that I've scattered
00:17:40throughout. You want to make sure in general, if you're building like a production grade service like
00:17:44this, you always want to stop, make a best effort to stop your sandboxes. So in this case, you'll see
00:17:51what extract tools does, but when we're done with it, we make sure that we delete the sandbox explicitly.
00:17:56In reality, you also want to make sure you're probably enqueuing like an asynchronous cleanup
00:18:00job or something like that. You don't want these sandboxes just hanging out indefinitely.
00:18:06So let's take a look at what the extract tools function does. So we have our sandbox and I really like these
00:18:15examples because it looks so goofy how simple it is, but it works really, really well. So we're running a
00:18:23node command just using the node binary on the sandbox and in the script, we're importing the module that
00:18:30the user wrote, which is just index.js. We're stringifying that object and then we're calling
00:18:36console.log. The thing to keep in mind is that sandboxes aren't like a web server where you can
00:18:42like send a request and get a response back. All the input and output is via executing a command and then
00:18:48you could have it send a response to some service that you pull or in this case, you can just have it
00:18:52log to standard out. A couple of tips here. Normally you don't want to just like console.log and trust all
00:19:00the output that you get from the sandbox. There could be other packages that the user has installed or
00:19:07other code that they're running that can kind of like log things at the same time to the standard outstream.
00:19:12So one thing that's good to do is wrap your output in some sort of like tag that you can parse.
00:19:18This is like an XML like tag that we would be doing here. I'm just not doing it in the example.
00:19:23You also want to make sure that you're limiting the size of the logs that you consume. I'm just
00:19:29collecting all of the output in this example into a stream. In a production grade application, you don't
00:19:34want to do that because that could be, you know, a gigabyte of streams and you're going to OOM your servers.
00:19:39So there are also commands that the sandbox API has for streaming the logs. And so that's what we do
00:19:45with Notion work is we basically consume that stream until we see the start of the output sort of token
00:19:53we care about. Then we start buffering up that sort of description, the thing that we logged intentionally.
00:19:58And we stop processing the output stream as soon as we reach our closing tag. So that's how if something
00:20:04weird and going wrong on the sandbox and there's huge amounts of data being logged,
00:20:08it's not going to all collect in memory. We can just discard it and wait until we get the stuff that we care about.
00:20:14So once we have that command running, we wait for it to finish and we grab its standard out.
00:20:22And if you've used like Zod before, this should look pretty familiar. We're just parsing that string as JSON.
00:20:27And then we have a Zod type here that is
00:20:31validating it that it's the shape that we want.
00:20:34It's really important to just not trust this data because this is untrusted user code. You have no idea
00:20:39what people could be logging. So you want to make sure that you are
00:20:43validating the size and validating the general shape of it. So in this case, what we're expecting
00:20:50is a record whose keys, the string, are going to be the name of each of our tools.
00:20:57And the objects are going to be the description and then the input schema, which is that
00:21:02JSON schema. You'll notice that the execute function isn't here. That's just
00:21:06not getting returned which is nice because JSON does stringify just skips over things that aren't
00:21:11serializable. So it just kind of gets ignored. So we just get the description and the input schema out of it.
00:21:18So back up here, we have our worker tools that are that
00:21:23record of every work of every tool that was exposed in that worker.
00:21:28The next thing that we do is that we just kind of like get that data into the shape that the AI SDK
00:21:36expects. And we need to attach an execution function to each of those tools.
00:21:42So we didn't get an execution function obviously when we were just logging things to standard out.
00:21:46So the question now is now that we have this description of the tool, how do we provide
00:21:51a function that the SDK can call every time it wants to call this worker or this tool that's exposed by
00:21:58this worker? So I have this little wrapper here called execute tool. And let's take a look at what
00:22:04it does. It should look pretty familiar now. So it's calling that same create sandbox function. So it's
00:22:10creating a fresh sandbox. Even if you're using caching, there's a feature of recel sandboxes called
00:22:16persistence where every time the sandbox suspends or stops, it caches the state like everything that's
00:22:23on disk. For a feature like this, you don't actually want that. You want to snapshot your initial state
00:22:28where it has all the user code on it. But in general, after that, you want to make sure every single
00:22:33time that tool executes, you probably want a totally fresh instance. So that way, one run of a tool,
00:22:38if something goes wrong, it's not going to pollute the environment of the tool that runs after it.
00:22:43So we create a new sandbox, and we run another node script on it. So this should look pretty familiar,
00:22:49but it's a little bit different. So we're importing the module that the user wrote. We're grabbing the
00:22:56tool from that module, which is just the tool name that we got when we created this execute tool wrapper.
00:23:03We're calling that execute function on it. And we're passing that execute function the input that was
00:23:09provided by the model. Here I'm just typing this as unknown. This is pretty safe here, though, because
00:23:19we provided a-- did I not do it in here? I think I may have skipped over this in this example. But what
00:23:27you would normally do is our-- oh, no, I think I did it. Let me jump back up here. Yeah. So when we're sort of
00:23:34manipulating our tools to send them to the agent, we're taking that JSON schema, and we're turning it
00:23:39back into a Zod type. So we don't have to do any parsing ourselves. The AI SDK, the tool loop agent code,
00:23:46every time that tool gets called, it's going to validate the input that came from the agent for us.
00:23:52So we can more or less trust that this value is what we expect it to be. So we're going to pass it
00:23:56into the execute function here. Then when that asynchronous function completes, we're going to
00:24:03stringify it. We're going to log it to standard out. And then we're going to wait-- I'll get into this
00:24:10stuff here in a second. We're going to wait for that command to complete. And again, we're going to
00:24:14delete our sandbox and we're done with it. And then we're going to parse that JSON, which is the return
00:24:21value of that execution function that the user wrote. And we're going to send that back to the tool loop
00:24:26agent. So essentially the flow here is the tool loop agent says, I want to call the plan outing tool
00:24:35that we saw before where it uses all those transit APIs. That ends up calling this function here with
00:24:40whatever inputs the agent decides on. We create a sandbox using the blob that we uploaded from the
00:24:48user's compiled code before. Then we run a command on that sandbox where we're calling the user's
00:24:55execute function. We're waiting for that function to return. And then we're logging that return value to
00:25:00standard out, parsing it, and then sending it back to the agent, who's then going to continue the tool
00:25:05loop and either make another tool call or respond to the user. A couple of quick tips here. The same
00:25:13output verification rules apply in a production system. Again, you want to make sure that users can't
00:25:19return an object that takes like two gigabytes or something like that to parse. You also want to
00:25:28probably provide an SDK. We don't do this in this example, but in the Notion Workers SDK,
00:25:35the types are set up such that the return value of that execution function has to be JSON serializable.
00:25:41That's a really easy kind of like foot gun for your users is if you make it so that that function
00:25:47can return anything, they'll end up returning things that can't be serialized as JSON and then sent over
00:25:52the wire via standard out and then parsed. And they'll get very confused and not understand why the
00:25:57worker is not running. And then this is a little bit of a story that happened with Notion Workers. You also
00:26:05want to be always very careful to ensure that the process, the node process that you're spawning
00:26:11here exits no matter what. We had a bug with Notion where users code would run and it would kind of run
00:26:18to completion. But then for some reason, this node process wasn't exiting. It would just hang until the
00:26:25life cycle, the sandbox ran out, which was like five minutes or something like that. So it wasn't catastrophic,
00:26:30but it was wasting resources. And what we found out or remembered, I had totally kind of like forgotten that this
00:26:38is a property of the node runtime is that users were running code that was setting up timers with like
00:26:44set interval or set timeout. And especially with intervals or I think dangling promises will do the same thing.
00:26:52If the script runs and there are any intervals or anything like that running, the node process will
00:26:58never return. It's not going to return on its own until all of those timers run out. So if it's an
00:27:03interval, it's just going to run forever until the sandbox dies. So you always want to make sure that
00:27:08when the code or the function that you're trying to execute is actually complete, that you explicitly tell
00:27:14the process to exit. So that way it stops the sandbox, the process returns after that.
00:27:23So that is pretty much the entire workflow. I'm going to run it one more time and I'm going to turn
00:27:29debugging on so you can kind of see what happens.
00:27:42So we're deploying our departures worker first. I'll walk through this in a second. We deploy our
00:27:48departures worker by uploading that bundle first. We create a sandbox that we're going to use to extract
00:27:54information about that worker. We extracted these tools. So this is what we get when we log the
00:28:01modules contents to standard out. We get the keys for each tool, the description and the input schema.
00:28:08We do the thing for that simple greeter worker. Then we take all that and we pass it to the agent so
00:28:13that it can make tool calls and respond to the user. So that's pretty much it. That's a pretty simple
00:28:19example of how you can take untrusted user code, store it somewhere, learn about it in a way that's safe so
00:28:26that you can put it in durable storage or send it directly to agents and then have your agents safely
00:28:31execute that code. So we have about 10 minutes left. If anyone has any questions, if you want to ask about
00:28:39Notion workers or working with Vercel sandbox stuff in general or untrusted code execution,
00:28:46I'm happy to chat about that. Thanks.
00:28:56Oh yeah. Oh and afterwards, if you want to talk about Notion developer platform in general,
00:29:01you can speak to me or my colleague MJ over here. She's the product manager for the developer platform at Notion.
00:29:08Hi. This is an operational question, but how do you limit, because it's like users can do anything.
00:29:15Yeah. You might have multiple users writing a similar tool.
00:29:20Multiple users what? Writing a similar tool. For example, like planning a New York City trip.
00:29:24Yeah. You could have 10 different users having 10 different codes. Yeah. That do the same thing.
00:29:29Yeah. Is there something that you're doing to block that or it's just up to the agent?
00:29:33No. We just let them, if multiple users are all going to do the same thing, we let them do that.
00:29:37It's partially a product question. Like if it's multiple users in the same organization, you know,
00:29:42you want to make sure that, so it is an operational question, product question too.
00:29:46You want to make sure that like you have good sharing primitives and things like that. So I can
00:29:50look and search and say like, is there already a worker that does this job? And that way they're not
00:29:55rewriting it. But like broadly platform wise, we don't do anything to like, it's very unlikely users will
00:30:02deploy identical code. And it's just not worth it to try to like deduplicate that.
00:30:19Okay. Yeah. I think we have a couple more. I'm not sure who has the mic.
00:30:24I didn't hear it. Oh, oh, yes. I'm so sorry.
00:30:29I thought that it was picked up in the headphones.
00:30:33Oh, yeah. The question that was asked was, if you have lots of users deploying the same code,
00:30:40do we do anything to sort of like operationalize that? And we don't. It's more of a product question.
00:30:45Like we want to make sure that users aren't redoing the same work. So we want good sharing primitives
00:30:50that we're working on right now for Notion workers. But operationally on a platform level,
00:30:54if people deploy the same code 50 times, we don't care.
00:30:56Do you have problems with the workers running in timeouts because the sandbox is just
00:31:04they're doing their thing and just dies too soon?
00:31:07So what was the question exactly about timeouts?
00:31:10Are you having problems with timeouts between Vercel, like other products and workflows,
00:31:15for example, or functions and your sandbox? Or is it just like everything is fine?
00:31:20Yeah, we haven't had any issues with it. The platform has been super solid for us
00:31:25so far. I'm not shilling. I'm being serious. It's been really good.
00:31:32We run into a lot more problems with just like users accidentally doing the wrong thing. So over
00:31:36time, it's more about us eliminating those like foot guns and making the platform easier and easier to
00:31:42use for developers and non-developers alike. Cool. That was great. I wanted to ask about,
00:31:48I guess, when you stringify the initial user code. I assume you're trying to make sure
00:31:52either it's secure or you can run it. I believe -- I guess I wanted to ask a little bit more.
00:31:57You said you stringify the code and you get, I guess, the worker tags to get the desired inputs
00:32:04for the user's code and then a description of that tool. Is that for like the notion, I guess,
00:32:10like for your agent to execute the code? Because I noticed you're running the user's execution code
00:32:15or executor function already. So I was curious why you, I don't say cared, but are like doing this
00:32:21additional step to get the inputs and the description of the tool itself.
00:32:24That's a good question. It's a little clear in like the real product, but it's simplified here.
00:32:29The question is basically like, why am I executing the user code once to get information about the
00:32:34worker and then doing it again when the tool code is called? The reason is, is that before the agent
00:32:39can call the tool in the first place or know about the existence of the tool, we have to learn about
00:32:45what's in that tool and then expose it to the agent through the SDK. So what happens in Notion
00:32:50Workers, for example, when you run like NTN Workers Deploy, we go through the build pipeline,
00:32:55the sandbox that's executing the build takes the tarball, puts it into storage somewhere. We start,
00:33:02I think we do this in a new worker where we extract the name of the tools and the descriptions and the
00:33:09schemas and we store them in like DynamoDB. That way, like, you know, after that, we're never running
00:33:14sandboxes until the tool is actually called. You mentioned the tarball system. I guess that's what I'm
00:33:21curious the most about. In terms of distribution, like, how does that work exactly when it comes to the tarball?
00:33:26Like, do you have like a open marketplace right now or how does it work for distribution?
00:33:31Oh, like what's in the actual tarball? Yeah, yeah.
00:33:33We don't have, well, in terms of like mechanically what we do, what goes into that tarball,
00:33:38it's, we use just ES build right now to, the way that the whole real deploy pipeline works is that you run
00:33:47NTN workers deploy, you call an API endpoint, your computer gets a pre-signed URL, it bundles up all
00:33:52your source code, we create a tarball with the source code, we run a build process with ES build on
00:33:58that sandbox, and then the output gets put back into blob storage so that we can run the actual thing
00:34:04from there. We don't have a real like marketplace for workers yet. We're working on like sharing primitives
00:34:11within a notion workspace first, but there are definitely plans for like some sort of workers marketplace in the future.
00:34:18In the meantime, though, you could just distribute these on GitHub and it works great. That's what people do today.
00:34:23Yeah, just publish a repo and someone can clone it and run NTN workers deploy and use it themselves.
00:34:29Yeah.
00:34:34I think there's one in the back there.
00:34:37Yeah, so I have a question about billing.
00:34:40About billing?
00:34:40Yeah, so I just looked at, I mean, don't give away all your secrets, of course,
00:34:44but I did a quick Google search. Looks like custom agents work on some sort of credit system. I'm
00:34:48guessing that's tied to consumption of resources.
00:34:51Yeah, yeah.
00:34:51I guess like how does that work with the Vercel platform in like high level strokes?
00:34:54Okay, so the question is, how does billing for these work with the Vercel platform?
00:35:02So one of the benefits of workers is that you can write these like, a lot of teams have been able to go from
00:35:08like a huge instruction set using like MCP servers that they're giving to their agents. And every time they call a task,
00:35:16that agent is going to spend tons and tons of thinking tokens basically doing the same thing over and over again.
00:35:21And so if you can take those repetitive tasks that the agent is doing and deploy it as a worker,
00:35:33you're still billed for the execution time of your workers, but it's much, much less expensive than AI compute tokens.
00:35:40So if you're running a custom agent, it does some thinking, executes a tool, and then does some more thinking.
00:35:48You're billed for the token consumption in the custom agent that it's doing before it calls the tool.
00:35:53So when the tool runs, you're billed at a different rate that consumes your credits, your like Notion AI credits at a much, much, much lower rate.
00:36:01And then you're billed, you know, normal AI credits again when the, when the agent finally responds.
00:36:05So this is a way that if you do use Notion custom agents, you can cut down a lot on cost if you have repetitive tasks.
00:36:16That's also true. You also don't have to have Notion AI, like they're not, we're showing off tool calls here, which integrates with Notion AI.
00:36:24But workers also do like third party sync into Notion. So that doesn't require any AI features at all.
00:36:31So it's, it's not just an AI product. People are using that to sync.
00:36:36I have workers that sync like my letterbox feed into my Notion. That's my most important one personally.
00:36:46Okay. Anything else?
00:36:52Do you have one more?
00:37:06Oh, like expose them through your own user, like expose Notion custom agents through your own interface.
00:37:13It's not something that we have today.
00:37:16I'm sorry. Thank you, MJ. The question was, if you have like your own workers and custom agents,
00:37:22is there a way that you can expose them through your own application for your own consumers?
00:37:28And today there's not a way to do that. We do have like an alpha of a custom agent API where you could
00:37:35definitely do that. If you define custom agents and workers or custom agents in your workspace,
00:37:40you could use an API to call those agents and then get a streaming response. So it's possible.
00:37:46I don't think a ton of people are doing it yet. And that's a very like still alpha, public but alpha feature.
00:37:57Other questions?
00:38:00All right. Cool. Well, thank you all for spending the time
00:38:04it's time to come listen to this today. And yeah, if you have questions about Notion, Notion workers,
00:38:08or Vercel Sandbox, come find NJRI. Thanks.