00:00:00Hello, everyone. Welcome to this week's Vercel Community Livestream. I'm Amy, and this is
00:00:26Jacob. We are on the community team here at Vercel. Just a reminder, we are streaming
00:00:31this on X and YouTube, but if you want to make sure that we see your questions and comments
00:00:36in the chat, go to the community and sign in. It's at community.vercel.com/live and you'll
00:00:43see that as the first event.
00:00:44Yeah, at the end of the session, we'll have a little time for some questions. If you're
00:00:49going to participate in the chat while you're watching the session, just remember to follow
00:00:55our code of conduct and yeah, you can drop in any questions you have along the way and
00:00:59we'll make sure to ask. I'd like to introduce our guest. We have Jon Lindquist here to show
00:01:05us the new Vercel plugin for Cloud Code. Hey, Jon.
00:01:09Hey, Jacob. Hey, Amy. Thanks for having me. All right. Let's just go ahead and start sharing
00:01:16my screen right away so we can show what's going on here. So for a while now, skills have
00:01:23been all the rage and everyone's talking about which skills to use to level up your projects
00:01:29and to enable your agents to do things they normally aren't able to do. Now emerging from
00:01:37skills, the kind of next evolution of that is what we call plugins. And this is something
00:01:43that's still really catching on. There's not a lot of plugins out there and people are still
00:01:48exploring exactly how to build them. I built the first draft of the Vercel plugin and what
00:01:56I'd like to talk about is why build a plugin, when should you build a plugin, why a plugin
00:02:03instead of skills and how they complement each other and all those sorts of questions. So
00:02:09if you have any questions around what is, I'll be talking about it, but what a plugin can
00:02:16enable for you, if you should build one internally or personally, I would love to discuss that
00:02:22and we'll talk about that during the session today. So first of all, a plugin is initially
00:02:32kicked off by Clod Code and Gemini as well, but took very different approaches. And the
00:02:39standardization of plugins is an ongoing effort. And that's something that we hope to have,
00:02:44I can talk about more soon where a single plugin can work across all the editors. So
00:02:49right now we are talking about Clod Code and cursor supports as well. CodeX is coming very,
00:02:54very soon, if not already today and many of the others as well. There is a plugin standard
00:03:01being worked on. So we can expect plugins to be something that you can package up and share
00:03:08across whatever tooling you're using. So that being said, you may be familiar with skills
00:03:15as they are markdown files, which can be loaded by your agent to give it like additional instructions.
00:03:23If one, you manually invoke it, like you do a slash and invoke your skill or the agent
00:03:29detects based on the description that the skill should load in. So it's very user invoke or
00:03:36agent decides based on like a condition in the description. Now, the level up of a plugin
00:03:44is they have something called hooks. And if you think of a plugin, or you think of a session
00:03:53you have with Clod Code or with any of the agent harnesses, you can think they have lifecycles.
00:03:59So right now I'm in the Brazil plugin directory by spin up a just Clod Code here, I'm going
00:04:07to ask it are what are the hooks that this uses. And this is going to list out the hooks
00:04:18kind of like to find a lifecycle. And a lifecycle is the when one year session starts, when tools
00:04:27like before tools get called after they get called, when a user pushes text into it, and
00:04:33when your session ends. There are a whole bunch of other hooks in there. But for our purposes
00:04:39in the Brazil plugin, these are the ones we are currently using based on our like initial
00:04:44goals. So to be clear, the the initial goal of the Brazil plugin was to help people ship
00:04:51agents on Brazil. So the idea is how are we going to make agents aware of the Brazil ecosystem,
00:05:02like everything on the platform that has to do with the AI SDK, the gateway. And our workflows
00:05:11are are almost J here. So anything around workflows, and essentially, the versatile CLI has updated
00:05:19a lot. And all of these things around the a the agent SDK, or sorry, the around shipping
00:05:26agents on vercel. They're all things that have been released within the past days past weeks
00:05:33past months. And any of these agent harnesses like Clod Code, they have these knowledge cut
00:05:39offs from probably like six months to a year ago. So it depends on the model depends on
00:05:46a whole bunch of variables. But they're all things that all of these agents don't know
00:05:52about all of the latest, like amazing stuff that we shipped. So the goal of the Brazil
00:05:57plugin is to stop the agent from writing kind of outdated code, outdated practices, and bring
00:06:06it up to here's everything that's the absolute latest on the Brazil platform. So that's like
00:06:11the huge win there is models are smart, but they just don't know about all this stuff.
00:06:19And instead of asking them to do research every single time, or asking, asking, having a huge
00:06:25bundle of skills for every single thing, like if we can find some way of essentially forcing
00:06:32an agent to load in the best stuff about the Brazil platform, then that's a huge win for
00:06:38us. So if you look at these hooks in the lifecycle of an agent, and again, there are many, many
00:06:45more hooks that we can leverage in the future or that you can leverage. But the key ones
00:06:49for now are session start. And we use that a few different ways. P tool use user prompt
00:06:56submit postal use in session end. So session start and session end are you know, the beginning
00:07:03and end of a conversation with an agent. And you can think of your agents MD or cloud MD
00:07:12file is typically the thing you do at session start. So what can we do at session start for
00:07:19the resell platform. And what we could do is load in a resell platform file, meaning that
00:07:27if we look at, I'll just say describe resell.md. So this file is something that we can include
00:07:40in session start alongside a cloud MD. Essentially, you could have a plugin that is if all you
00:07:48want it to do is have like share a cloud MD file and have someone have an easy way to
00:07:52install it. You could build a plugin that all it does is just like inject a this is a knowledge
00:07:59graph of resell. And this is something that we're continuously tweaking. We're looking
00:08:04for ways to compress it and make it as small as possible but still have good performance
00:08:08and a a balance of what should be in there and what shouldn't be in there. It's very difficult
00:08:15to run evals against all the different models and all different harnesses. But we're slowly
00:08:21figuring those things out. So what's in there is essentially relationships in the resell
00:08:28ecosystem around AI, when to use, when to use which products, when to call which things
00:08:35like a structured understanding of the entire resell ecosystem. So the concept of a, I like
00:08:44how it calls us a table of contents. It's like the beginning of a book when you start a conversation.
00:08:50And the way that we structured this and thought about this is if you think of how do I talk
00:08:56to an agent about things, I have a certain vocabulary, they have a certain vocabulary.
00:09:02And if you can, if you can give it a glossary of terms, map those two docs, map those two
00:09:07skills, map those two things that it should load in. Once you hit those things, you can
00:09:12think that what are the things I'm going to say and what are the things that that the agent
00:09:19should load in and call and try and develop that like mind map up front so that I can just
00:09:24talk naturally to it and not have to worry about talking about like even mentioning the
00:09:30ASDK at all. And even mentioning any of the features at all. And if I can just find some
00:09:38way of saying just build me this app and ship it. And then it just handles everything in
00:09:45the plugin. Excuse me, handles everything in the plugin so that it gets a beautiful
00:09:52app gets shipped on Brazil and they can iterate from there. Similar to like how v zero and
00:09:57some other projects work for us. If we can just make it so that you can have a wonderful
00:10:02experience, but you don't have to think about the ecosystem. And that's a huge win. Does
00:10:09that make sense? Jacob, Amy, any any initial questions about that or anything in the chat?
00:10:16Jacob, I think you have a question for your meeting. Sorry, so sorry. I had a question
00:10:24for you, John. Yeah. Since this plugin operates as a table of contents, is that why it doesn't
00:10:30need to why the plugin doesn't need to constantly update every time a new new documentation comes
00:10:38out? Because it's basically a collection of URLs that the agents can then follow to find
00:10:43up to date the docs? Or does the plugin have a like a immediate update system? How does
00:10:49that approach work? Yes. So essentially, if you think of the the knowledge graph up front,
00:10:58a lot of what it can do is just point to the latest docs. So if you if you think of this
00:11:07is a question that comes up a lot with the skills and how we manage skills per libraries.
00:11:14And if you have someone on the version five of whatever SDK and someone else on the version
00:11:18six of the SDK, and you try and influence them with skills from different versions that are
00:11:22conflicting, like how do you do that? And a lot of it is like, based on what somebody
00:11:31has installed, you can check their package JSON file, you can check the version. And based
00:11:36on that come up with a smart way to look at which doxy rails that should look at and which
00:11:44skills should load and all those sorts of things. And so that that's another part of the kind
00:11:51of the preload phase that the session start is just like inspecting the person's project.
00:11:58And again, this is all local. It just it just looks at the project sees what are the libraries
00:12:02are using? What are the setups are using so that it knows to point you in the right direction?
00:12:07One of the most common errors is like, trying to use, you know, generate object on the latest
00:12:12version of a SDK when you know that API changed. So, so yeah, there's really no that the plugins
00:12:21just handle that, like, you shouldn't have to worry about if you're using this on a legacy
00:12:27project or if you're using on the absolute greenfield latest project, the plugin should
00:12:32be able to handle that you shouldn't have to worry about version numbers, you shouldn't
00:12:34have to worry about package names, you shouldn't have to worry about anything because just let
00:12:41just let Vercel do that for you. Right. And we're continuously working on weight, we have
00:12:49lots more ideas of how to do this better. And the plugin will continue to update and push
00:12:54this forward. So any other questions?
00:12:58Yeah, so what's the total, I'd say scope of the plugin? Does this handle say just Vercel
00:13:04services? Anything I may find in the dashboard? Or is it also like the the open source libraries
00:13:11that we support like Next.js, AISDK, workflows, and so on?
00:13:16Yeah, so the initial scope, what what we decided initially is to cover as much as possible,
00:13:25and like have every library, everything on the platform, like just everything in there,
00:13:32see what gets used the most, like internally, we have everyone turn their telemetry on, and
00:13:36you can like opt in to share telemetry. And we'll find which skills are actually being
00:13:40used and which ones aren't. So that way we can remove some things in there that we might
00:13:48not need as much information on or add more based on what seems to use the most, obviously
00:13:53Next.js, AISDK, and you know, the most popular downloaded things should have the most information.
00:13:59And so that's a it's a balance up front, where we intentionally went pretty big with the
00:14:08initial release, and we will pare it down to something much more svelte, I guess, going
00:14:15forward. So
00:14:16yeah, that makes a lot of sense. Thank you. Yeah. All right. So so yeah, if if you're
00:14:24building a plugin, and one of the reasons I talk about this in kind of like loose general
00:14:29terms is because I'm very much into like agentic driven development. And rather than show you
00:14:36exactly how everything works, I think it's much more important to like, learn how to talk
00:14:40about it, and take some of the terminology away from it. So you can just ask your AI assistant
00:14:45to do this similar stuff. If you tell it if you tell clod code to build a plugin for clod
00:14:52code, and you give it you give it the docs like it can do that. So it's much more important
00:14:58to discuss the concepts and ideas around this then like, dig into the code too much. With
00:15:05that being said, so session start is a great place for injecting a like a table of contents
00:15:11of platform like this is all the things that are out of date based on like the things that
00:15:19are important to me that are out of date, based on the models are for most models. So that
00:15:24is what brings everything up to like, here's what Brazil looks like today. And so that even
00:15:31if that's the only thing that happens in the plugin, it's it knows that it should go and
00:15:37check on things going forward. But what we're also able to do and just just kind of a side
00:15:44note a reason you do like each session in clod code has its own unique ID, which allows
00:15:52you to like as this goes through the session, it makes sure is it has like a skills budget,
00:16:01and it has the skills that have already been loaded and things of that nature. So we can
00:16:07put you can write things to like the sessions environment, or to like a session storage place
00:16:15where throughout that session, as you're going through that session, we make sure we don't
00:16:20load things that already been loaded, or like duplicate any efforts, because we can keep
00:16:26track of things that have already happened. So that's another great thing. Great part about
00:16:30the session is give you gives you a chance to grab a session ID so that throughout through
00:16:35any of the future hooks, anything that anything that gets run inside of the session, you have
00:16:40an ID to like latch on to and store things about the session for your individual plugin.
00:16:47Claude code also does that if you look in their dot cloud directory with the projects that
00:16:50you can see sessions and things in there. So you take that same concept for your own plugins
00:16:56and own tooling. Alright, so with that all being said, what pre tool use and we'll talk
00:17:05about pre tool and post tool at the same time, what these give you a, they give you a chance
00:17:11to see file changes. So in pre tool, you can get the content of the file like that's about
00:17:20to change. And so this is around like, reading files, writing files, or sorry, reading files,
00:17:29editing files, creating files. And pre tool also covers a lot of other any of the other
00:17:34tools that be used, such as like running things in bash. So you get this chance of like, before
00:17:40this thing happens, is there anything I want to do? And so say, for example, if we see someone
00:17:47running, trying to do a specific Vercel CLI command, and we noticed that there's a better
00:17:55way to do it now in the Vercel CLI, such as like Vercel CLI now has an API option where
00:18:00we can do much more advanced queries. And the Vercel CLI has been updated. If you haven't
00:18:05updated it recently, it has a lot of great new features. We can hint at those things and
00:18:11say that you're trying to do this or even check like, is your Vercel CLI out of date? It would
00:18:17be great to update it. So you have a lot more options here. So it gives you a chance to like,
00:18:21check to see if it's going to make some sort of tool call, whether it's a CLI call. This
00:18:27is super important for the sandbox CLI for the workflow CLI, for all these brand new
00:18:34CLIs that the models don't know anything about yet, because they've just released after the
00:18:39knowledge cutoffs to make sure that they're doing things correctly. So that's where for
00:18:47tool calls, like bash calls, it's for file edits, it gets really interesting with file
00:18:52edits. Something we're exploring more is diffing files like this file used to look like this,
00:18:59the agents trying to make it look like this. Is that the correct, based on our libraries,
00:19:06is that correct change? Or is there something that smells in there? And that gets into, I'll
00:19:11just lean back on the example of say you have the v6 SDK installed and it tries to use generate
00:19:18object as an API, which is a common one. And if you see that at that moment, like I see
00:19:25you're trying to use generate object, let's stop and remind them of what the AISDK is actually
00:19:32like so that before you even get to testing or deploying or anything, like you've caught
00:19:36it at the exact moment that the agent was trying to change the file. And this is one of those
00:19:46things where if you try to rely on a skill where skills are like textual hints of, if
00:19:54you ever tried to write into your cloud MD or into a skill, like never, never commit whatever
00:20:03files or never push without, never push unless I'm in a branch or something. You've probably
00:20:13experienced that like those don't always protect you, protect you from those. Pre-tool use gives
00:20:21you a chance to like literally like it can stop those things from happening in a programmatic
00:20:28and like engineered sort of way. And you can give it different instructions and different
00:20:32examples of ways it can do it. So that's the huge difference here between skills, between
00:20:41your, your cloud MD agents, MDs files is that you actually can control what's changing and
00:20:47if, or what's, what's happening. And if you want it to happen or what's changing, if you
00:20:51want that to change that's, that's the point where it gives you it's great. So from there
00:21:01you get similar capabilities on post tool. So the chance to check what exactly had changed
00:21:07after like a bash file, bash command had run, what are things that are different now that
00:21:13a tool has been called? So that this is, it doesn't allow you to prevent something from
00:21:19happening, but allows you to see the result of something that's happened. And if that,
00:21:25if if, if it changed something or did something unexpected, it's, it's another chance to get
00:21:31a diff and say, this looks kind of weird. Let's tell the agent that this smells weird to us.
00:21:38So those are like two huge life cycle hooks to know about when building. And for us, it's
00:21:43a, again, it's a lot of that, like it's all about that, that knowledge cutoff. That's the
00:21:49thing that Vercel has to kind of fight against because we ship fast, we shipped often and
00:21:55we're just like pushing all this new technology so quickly. And it's just this amazing stuff
00:22:00we want to surface to people. And that's where like a plugin can really really come in handy.
00:22:09Any questions about that?
00:22:11Yeah. So if it's using like file paths to decide which skills to inject, does that mean we actually
00:22:21stand to gain a lot from scoping the files in our projects down to be a little more single
00:22:27purpose so it can more accurately determine which, which skills it needs to which skills
00:22:34it needs to add. For example, if I have a big file and maybe it's using five or six of our
00:22:39libraries and now it can only add three up to three skills there, or it might not know
00:22:47based on the, based on the file path. So do you think that should be a consideration now
00:22:51when building software with these tools?
00:22:54I would say you shouldn't have to care about the code that the agent writes and that it
00:23:02is up to the plugin authors. It's up to the harness authors to, to make those things work
00:23:12well. Like the people who are actually running evals and seeing if seeing if that makes a
00:23:17difference or not. Whereas I think it's very, it's very easy to get back into engineer mode
00:23:27and think I'm going to solve this because this is what a human would do. And that's definitely
00:23:33what an agent would want. Like that's one of the greatest temptations of developers right
00:23:39now is problem solving, like finding problems that aren't problems. Because yeah, it's if,
00:23:50if you're, if you're trying to fix something and you don't know how to test it, like if
00:23:53you know how to test that against it against an agent, then go for it. If you have no desire
00:24:00to do that, then then just let the people who are building the plugins and the harnesses
00:24:05take care of it because the testing and evals it's expensive. Like it takes, it costs a
00:24:12lot of money to run all these models against those changes. It takes a long time and it's
00:24:17a huge pain. So like it's, it's one of those things. I hope nobody has to care about that.
00:24:23And I would spend I think I just spend my effort on other things for now. Yeah.
00:24:33It's, I see, I see. So no, no point changing the way I build things. If the next version
00:24:40of the model is going to be built in a way that would make all those alterations unnecessary
00:24:47essentially.
00:24:48Yeah. And I think, I mean, the end goal for all the harnesses and for Brazil is that that
00:24:56you really don't have to think or look at the code that much. I'm not saying that that's
00:25:01the case right now. I'm saying that that's like, that's the end goal. That's, that's what
00:25:07we're pushing towards is that you want to ship beautiful software. You want to be able to
00:25:11think you want to kind of free flow ideas and get variations and see which ones really resonate
00:25:19with you and then like really pare it down to something that is beautiful for you or your
00:25:25family or for your customers. Like that's just, we just want a beautiful experience where
00:25:33you're not, you're not thinking too much about how big your files are. If you're writing,
00:25:39you know, the right design patterns which libraries you're choosing. And it's, I totally agree.
00:25:48Like the temptation to say like, let's, let's use all these patterns because the agents are
00:25:53better with them. Let's do all these things. And it's definitely something we're, we're
00:25:58experimenting with the plugin and testing. But again, if you can't test it, then I it's,
00:26:07it's so tempting to say I made this change and it's obviously working better now. And
00:26:18that that's one of those like foot guns where, well, now that you've made this change, you're
00:26:23not looking at how it used to work and what, what that impacts in other places. And yeah,
00:26:28maybe it worked well for that one session and like, so engineering is different now. Sorry,
00:26:35I'm getting philosophical, I guess. Yeah. I think, I think we understand. Yeah. All right.
00:26:44So another hook is a user prompt submit. This is a really important one because it allows
00:26:50you to take the text of what the user typed. And if they mention a library, if they mention
00:26:55a concept, if they mention, you know, if they mentioned the word schedule, let's, let's bring
00:27:00in the cron skill. If this is very much similar to how like skills work, like if the user or
00:27:07if there's something in conversation detected that matches the description, but this actually
00:27:12gives you a chance to like run a regex against a user prompt. And if you detect certain keywords
00:27:20or what, or patterns or whatever, load those skills, do these things hint back to the agent,
00:27:26like maybe you should ask more follow-up questions. This, this is the interactive part of it where
00:27:33it's, it's your chance to say this, the user said this, maybe we should get more clarification
00:27:38or we should just load all these things and, and, and push it forward. You get to do there's
00:27:44some really fun things you can do with your user prompt submit where if you want to have
00:27:48your own custom, like glossary or language of commands for yourself, and you want to like
00:27:56prefix things with dollar signs. And if something starts with a dollar sign, then do this. Or
00:28:00if something has, you know it's kind of like writing like little bash scripts or something
00:28:07inside of user prompt submit that if you detect this, then you can run a tool right away and
00:28:13it doesn't like it can run it inside of the you can run whatever bash scripts, node scripts,
00:28:20whatever inside of that hook that's outside of the context of the session. So you can do
00:28:28all sorts of just wild quick things in there. If you want to like detect the word commit
00:28:37and then avoid having the agent take a few steps to commit something. And instead you
00:28:47just have a script that commits based on whatever it can do that. Like right after the prompt
00:28:53commit, you can tell the agent, you know, I ran this commit thing, you don't have to worry
00:28:59about it. And you can save the agent a few turns. And there's like some really interesting
00:29:03little tricks you can do in there to really accelerate if there's a lot of things that
00:29:09you do that are repetitive, rather than asking the agent to do it every single time. It's
00:29:15kind of a fun project if you want to like build a little user prompt submit for yourself and
00:29:19come up with your own like language of how you want to talk with the agent. But for us,
00:29:25it's more like, if they said this, this talked about this concept, what does that mean in
00:29:30Vercel terminology, so it's like matching words against like, specifically the like scheduling
00:29:37is crons and workflows. And, you know, words like parallel or performance or whatever, like
00:29:45you can find and direct the agent to specific things that you know about Vercel. Very, very
00:29:53fun to play around with user prompt submit if you have a weekend to mess around with that.
00:29:58And again, I'll just reiterate, it is totally possible to come in here and say, help me
00:30:03build a Claude code plugin with user prompt submit that will detect, you know, dollar signs
00:30:09and if it shows dollar sign, then run commit inside of the user prompt submit so that I
00:30:14like with that prompt, you know, I could have built up this so don't worry too much about
00:30:19the code, you can just start playing around with it. All right, so now that you have all
00:30:26this buttoned up, session end is a chance to clean up any files or anything that you wrote
00:30:32during the session. So if you started collecting which skills have been run, if you started
00:30:39collecting any information about results from tools that were called, or any sort of like
00:30:48budget or thing you're tracking throughout the session, it's a chance to go in there and
00:30:51clean it up. There's some fascinating things you can do in session end. Because it runs
00:30:57on essentially like the Ctrl C, like the exit of the session. You can spin up other agents
00:31:05to say, okay, look at all the files that have changed in the session. Do they match what
00:31:09everything we don't we don't do that. It's just it's really interesting things you can
00:31:13do with session end where my session is done. Does that session represent progress in this
00:31:20project? Does it represent cruft? Does it like look through all the files that were changed?
00:31:26Is this duplication of things that are already in the project? And if it is like go through
00:31:30and clean up and and alert, you'd have to alert the user some way like through a system notification
00:31:37or through some other like means since your session is gone. But really interesting, interesting
00:31:43things you can do in there. All right. So with all that being said, if I just I'm just gonna
00:31:50write a quick little demo here. So I'm going to spin up a version of Claude code running
00:31:56the for sale plugin versus when it's not running the for sale plugin. I'm going to use Claude
00:32:04dangerously skip permissions, I'm going to leave debug on I'll show that in a second.
00:32:08And this model is going to be set to haiku. So that's the fastest kind of dumbest model
00:32:13in for for Claude code. And I'm going to say like, just write a tutorial about the AI SDK.
00:32:26And just like we'll, we'll see what happens there. And going to compare it kind of side
00:32:30by side, you can see already it starts loading the skill. And this has the plugin loaded in.
00:32:37If I want to start a session without without any plugins loaded, there is a setting sources
00:32:46flag which gives you a chance to disable like user level sources or user level settings or
00:32:53project level settings, essentially, just ignore any settings. So this way I can load it without
00:32:59plugins. The no flicker whoops. This no flicker is new as of like yesterday, it's a nice way
00:33:05to do smooth scrolling inside of Claude code. So I'm enabling that as well. And then I'm
00:33:12going to set the model Opus being the smartest model. And if I just do the same prompt of
00:33:20it tutorial about the SDK. And this is like the vanilla version over here. And we'll kind
00:33:28of compare the results. Oh, whoops, I need to I didn't delete the previous one. Let's
00:33:36start again. Let me fire up this and run that in. This is a I was running this this morning
00:33:50and clean up before this started. So let me just launch this again.
00:33:56Kind of making a comparison. Yeah.
00:34:01All right. So
00:34:04the, the huge difference here is this is haiku 4.5 versus Opus 4.6, right? Like the most this
00:34:15is a million times cheaper, and much, much faster. And it's just loaded with a lot more
00:34:22vercel information. So even if you compare like the, the budget cost of the extra tokens
00:34:32we're including, including versus, you know, haiku versus Opus. While this is going into
00:34:38I should have said just set a markdown file. I guess it's going into actually writing code,
00:34:45but that'll be interesting enough. Let's see. So this is one thing to note about the one
00:34:59of the approaches we're taking for a lot of our skills is, as mentioned before, because
00:35:05of version number, because pack the different skills might change for we version number pinning
00:35:13the skills. Let me rephrase that. Well, different versions of our libraries have would require
00:35:21different skills. A lot of our skills say, please look at node modules, where we'll bundle
00:35:28a lot of the docs so that the docs are kind of version pin as well. So looking in here,
00:35:34you can see it was actually reading through the local docs. And it's going through here
00:35:40and putting together this tutorial. All right. Now, let's give this. This really didn't do
00:35:53any research at all. So this will be interesting. Last time I did a lot more research. And I
00:36:01can already see already like it's using an outdated model sonnet four instead of like
00:36:074.6. It's using an outdated. It mentioned generate object, which is not what we want to use. We
00:36:15use V six. And there's already a bunch of things that are just outdated. I wouldn't say they're
00:36:23wrong, but they're like very, very outdated for what the latest if you want the best performance
00:36:28and the best practices from the SDK. So all right. This was did this catch something in
00:36:37here? Yeah, it looks like Yeah, the SDK skill caught looks like it wrote a tutorial initially
00:36:51in here. And then the skill caught that it wasn't updated to be six, like without me saying
00:36:58anything at all. And now it's going and updating. It went from generate object to generate text.
00:37:03And it's going in updating everything to make sure that it's on the latest versions. Alright,
00:37:10so we I can compare these two. I'll just start a new session. Let's go in here and compare
00:37:23the tutorials from no plugin versus the tutorial in for sale plugin for accuracy. And this will
00:37:44be interesting to see like every time I've run this before Opus wrote a markdown file
00:37:48this time it did code snippets. It'll be fun to see the difference.
00:37:54When you're doing this, yeah, especially if you have other skills, maybe working alongside
00:38:03this plugin, is there a risk of them having a conflict and like resulting in outdated stuff
00:38:10being preferred or is the plugin going to kind of take like higher priority?
00:38:16The plugin will essentially, I would say forcefully inject skills based on its patterns. So if
00:38:27there is, there's kind of always that risk, like it's all it's all just kind of context,
00:38:34like all those are just text files that get shoved into a session. So I would always say
00:38:41only use the things you want for the project you're on as a like very, excuse me, very rarely
00:38:49install a skill at the user level unless you know you want that with every single project.
00:38:58And even for the for sale plugin, we're, we're working to find the best ways possible to like
00:39:06only only load what you need for the project you're in. And we have a lot of ideas around
00:39:11that. But one of the one of the like most probably controversial ideas is where, like, when would
00:39:23a user say they installed a versatile plugin. So what assumption could you make about that?
00:39:30If they install the versatile plugin, and they're on a competitor's platform, and they open a
00:39:35project and competitors platform? Should we tell them about our stuff? Like, that's, it's
00:39:42very, like, it's a very controversial thing, right? And if, if one person sees that, and
00:39:47they didn't want, and they're not actually interested on migrating, whatever to Brazil,
00:39:53they'll get very upset. If another person is, they'll be very happy. Like it's, so there's
00:39:59some really fascinating. If you take the, if you subscribe to the theory that everyone's
00:40:05just going to interact with agents. Like, what is the etiquette in a plugin or in informing
00:40:15agents of, like the new great thing on your platform without, like stepping on the feet
00:40:24of the other platforms. And so like, there's a lot of things. This is very relevant look
00:40:31to what I'm doing right now is we're going GA with workflow pretty soon. And so I'm writing
00:40:37migration guides. If someone wants to migrate from one of workflows competitors, that, you
00:40:43know, directly could use a skill to say migrate from this over to workflow. This is how you,
00:40:49this is how you do it. If you want to do that, like at what point in an agent would I inform
00:40:55someone of that? And that gets stuff is really, really interesting to me. Especially it's like
00:41:00super tough of mine since it's what I'm working on my like right now. But yeah, like context
00:41:07conflicts, like it's, it's going to be a problem going forward. I don't know what trying to
00:41:14infer what the user wants based on the context of their project of what they've asked so far.
00:41:19It would be great. Like we could technically look at past conversations and see what they've
00:41:27done. We could look at commit history. Like we could tell the GitHub CLI to like, go look
00:41:35at the anything in the project. Like there's lots of things you could do to go gather context,
00:41:40but that's also crossing some boundaries, like calling tools they didn't need for you to call
00:41:46lots of lots of really interesting stuff that if it it's, it's much easier just to do everything
00:41:53in Vercel and just have like, just rather than and like provide one agent with specific set
00:42:00of rules and a plugin that just like ships everything beautifully there and like not have
00:42:05to worry about all those conflicts. Anyway. I hope to mention, yeah.
00:42:14Kind of a plugin context, when our skills, like at what point in this process are skills
00:42:19normally at, for example, if I had like a migrate, create React app to Next.js skill, and I was
00:42:26just sitting in my skills folder, is that more likely to just automatically be invoked anytime
00:42:32I'm on a create React app project? Versus say, if I have the Vercel plugin here, I can then
00:42:39decide when it's going to be invoked and I can, if I built that same migration as a plugin,
00:42:47I can make it kind of only happen if the user is explicitly requesting it. Is that the right
00:42:52mental model? I think so. Like, we've been talking about
00:42:59the concept of like one shot skills or like single use skills that are more like, I want
00:43:06to do a thing like a migration, which is like, that's something that's a task you have, or
00:43:12like a task related skill that you don't want to load in any other time, except for the task.
00:43:17Like you want it, you want it scoped to the task. You don't want it scoped to the project.
00:43:22You don't want it scoped to the user, you want it scoped to the task. And those are discussions
00:43:27we're having. And I think that's, I don't have a great answer because right now, if you have
00:43:33a skill like that loaded, then it's going to influence it a harness in the models based
00:43:41on the model and based on other skills. Like it's so hard to say without knowing exactly
00:43:47what else is the project. But yeah, like that's definitely a, that's also top of mind is like
00:43:55a task scoped task scope skills. Another question on my mind here is how much
00:44:04of this plugin architecture is Claude code specific. So if I wanted to get similar behavior
00:44:12in cursor or codex or any of the others, do they have their own plugin format? Do I need
00:44:18a plugin for each of them? How does that work? Yeah. So we should have some announcements
00:44:25soon on the kind of the, the spec around plugins and everyone agreeing on like finally having
00:44:33plugins that will work universally and cross cross compatibility. Pretty much everyone's
00:44:39on board with that. I wouldn't say today, today, if you build a plugin in the Claude code format,
00:44:45you can, I would say you could expect it to work with the others in a short amount of time.
00:44:54But yeah, it's fantastic. Way too many skills folders already. I I'm really happy we're finally
00:45:00starting to standardize on something here. Yeah. Once the, the idea, like I love when
00:45:07I was talking to anthropic about plugins and the reason they chose the word plugin versus
00:45:13the word extension is that plugin implies that it's something you can plug in and something
00:45:18you can plug out. Like, so if you build a plugin with a set of skills, and you only want
00:45:24to use it again for like a single task, that's, that's something that should be very doable.
00:45:33If I'm in my project, and I'm a designer, and I want, I don't want all of my design skills
00:45:40to be interfering with all the developer skills or whatever. Like, you want to be able to plug
00:45:46in just those pieces. And I think, like, I strongly agree with those concepts is that
00:45:52individual people are going to have like their individual tasks and roles and projects. And
00:45:59they're going to want plugins more than skills, because they can, and they'll probably want
00:46:05to curate their own plugins as well, like build their own set of skills and, and hooks everything
00:46:10that are fully customized to the way that they work.
00:46:16Does this mean that I should be thinking of maybe disconnecting the versatile plugin if
00:46:21I'm working on a non versatile project?
00:46:25I would say like, as of today, yes, because of how, like, it's very aggressive for self
00:46:34forward. And hopefully, I don't get in trouble from someone by saying that. But like the the
00:46:39full intention of the Vercel plugin is like, this is something if you want to load up your
00:46:46agent and say, build me this thing. And then that thing just kind of almost magically appears
00:46:53on Vercel. Like that's the initial goal. And we're seeing how that works. And if that based
00:46:59on the feedback we're getting, we're pulling it back a bit seeing that we definitely need
00:47:04to do more detection of like what's in the project to disable and all these sorts of things.
00:47:09And that's, that's coming very, very, very soon. So, you know, probably probably keep
00:47:15it in like, keep it installed and enabled because we're like, we're actively working
00:47:18on that, like we were just talking about this morning. So
00:47:23I see. So it's it's kind of a trade off where maybe in the future, it'll understand a little
00:47:29better when it should invoke itself. But right now, the way the user controls whether this
00:47:36plugin is invoking or not is by plugging it in or unplugging it for a different session.
00:47:42Yeah. And right now, if you just type in slash plugin, you can go over to your installed plugins,
00:47:48just tab over, you can go down to whatever plugin you want to disable. And you can update
00:47:54you can uninstall you can do whatever. So you can disable it that way. And there's there's
00:48:03other like, more advanced tricks of like setting up custom customs, ESH functions or whatever
00:48:10that load specific sets of settings for whatever user like it's it's all very weird of like,
00:48:17the concept of a profile of the person who sits down at the computer. And like, what
00:48:23are all the plugins they want based on what they're looking at? And there's not a perfect
00:48:29solution for that. Hope to be able to figure one out, though.
00:48:34It's like when a Waymo pulls up and it adjusts all the seats ready for you the moment you
00:48:39walk in.
00:48:40Yeah, yeah. And I do think, like, at a certain point, I imagine we'll see agents asking you
00:48:49more about yourself. I feel like that's a huge like the the concept of profile identity role,
00:48:57which to me feels like a missing piece. Unless you want to assume that everyone is now like
00:49:05the full project manager for an entire thing. I just don't I don't see that anytime soon.
00:49:12I just I feel like some people are just have better ideas about some things and others.
00:49:19So
00:49:20yeah, let's let's review this though. So what's, what's interesting? Let's see, these are Yeah,
00:49:33it's interesting that I'm having Opus review Opus and it's still wrong, right? Like it's
00:49:37still recommending generate object and v6. Let's do verify against the docs. So it's trying
00:49:50to verify and it's saying that, you know, my, the one that I wrote is incorrect. This is
00:49:56still showing valid but dated. So like, our haiku recommended the most the latest models,
00:50:05and that's, to me, that's super important. I don't know how many times you've tried to
00:50:08use an agent to write it. Write something with AI. And it's like, let's use GPT for Oh, like,
00:50:14no, that's like, really old now. And it's nowhere near the quality that we need. Which
00:50:20is just one of those like things you can detect capture and say, like, check the latest models
00:50:25in the AI gateway. And like, you could say detect the latest models and ask the user which
00:50:31ones they want and describe each model and all that sort of stuff. Whereas right now,
00:50:36these agents always just like GPT for it has been interesting, just as like a side note
00:50:43seeing Claude code recommending the anthropic SDK more and more recently where it didn't
00:50:49before. And you're like, I think someone's tweaking some system prompts behind the scenes
00:50:55of like, recommend our own SDK. Which I don't blame them. It's business, right? All right.
00:51:03Yeah, so generate object is still in there. And so yeah, that was good. After checking
00:51:13the docs, tutorials more accurate. Several things I called fab created actually exist.
00:51:19So yeah, this is Opus, like, realizing that these are actual things like this is haiku
00:51:25with a plugin. And Opus, like, writing tutorial trying to validate the tutorial, like I had
00:51:32to tell it to go look exactly at the docs to find that something that haiku with the
00:51:39plugin kind of one did in one shot, right. And if you know the difference between Opus
00:51:44and haiku, that's that's a big deal. So here's a diff of like everything that it got right
00:51:53for the challenges or changes. Anyway, so hopefully that like demonstrates in a simple way. Just
00:52:04like one of the scenarios, the ASDK, this is much, much more. This for things like workflow
00:52:15and sandbox with the much, much newer versatile technologies that are incredible, but not within
00:52:21the knowledge cutoff of models. These distinctions become even greater because the models don't
00:52:28even know about them, right. So so that's what the Vercel plugin is for. If anyone has any
00:52:35questions, feel free to hit me up on on x or anywhere you can find me. I love chatting about
00:52:41this stuff happy to like address any concerns or this is definitely a work in progress and
00:52:48something we're going to iterate, iterate, iterate, iterate until it's like this amazing
00:52:52magical experience. And hopefully you can see the beginnings of that today. Anyone has any
00:52:59questions, I'm happy to answer them. Anyone has any plugin ideas, they want to brainstorm,
00:53:04I'm happy to talk about that as well. So that's unless unless anyone wants me to dig in something
00:53:14else. That's what I have to cover for today.
00:53:20Really appreciate you taking the time. It's a lot to take in. It's a big shift. We've been
00:53:25moving so fast with all of these AI tools.
00:53:28Yeah, it's not going to slow down, unfortunately.
00:53:33One question I have is if people want to contribute in any way, or if they find a bug and they
00:53:38want to report it, what's the best way to get that to you?
00:53:41Oh, yeah, the the repo for it. Vercel slash, Vercel slash plugin or is it Vercel slash
00:53:49Vercel plugin?
00:53:51Yeah, Vercel slash Vercel slash Vercel dash plugin.
00:53:58All right. I'll drop that in the chat as well so people have it.
00:54:03Thanks. Yeah, issues on there are great. It will be rapidly changing. I know I approved
00:54:10a big PR just right before this call.
00:54:14Awesome. All right, Jacob, any other questions from you?
00:54:20No, no, I think I think we covered everything on the stream. Yeah, thanks for going through
00:54:24such a in depth and great demo. I feel like I'm set up to build kind of my own cloud plugins
00:54:30here now. And instead of just yelling at the agent when it decides to force push to main
00:54:37on me, I now know which hooks I should be telling it to use to make sure it doesn't do
00:54:42that again. So that's that's super helpful.
00:54:45pre tools your friend is your best defense against things you don't don't want to happen.
00:54:51So nice. Thank you so much, john. Of course. Thanks, everyone.
00:54:56And thank you all for joining us. You can find upcoming sessions at community.vercel.com slash
00:55:01events. And we'll see you next time.
00:55:05Yep. Yep. I'll see you in the community. Have a great day, everyone.