Ship 26 NYC - Claude Managed Agents on Vercel's Agentic Infrastructure
VVercel
Computing/SoftwareSmall Business/StartupsInternet Technology
Transcript
00:00:00what's going on everybody my name is Harrison and I'm a member of technical staff working at
00:00:10Anthropic and my family and friends are just so excited that I don't have to talk to them about
00:00:15agents today and I get to hang out with people who are thinking about the same kinds of problems
00:00:19I think about on a daily basis so thank you so much for having me I'm so so so excited to be
00:00:24here and specifically I'm so excited to talk to you all about cloud managed agents which is a new
00:00:29suite of APIs that Anthropic has shipped and how you can use Vercel's agentic infrastructure to
00:00:34interop with your agents so without further ado let's get the show on the road so critically agents
00:00:41are only useful when they can do things I think this is probably no surprise to all of you in this
00:00:46room who are building and thinking about building and thinking about optimizing and thinking about
00:00:51finding more and more ways to apply agents to your day-to-day work this might mean that your agent needs
00:00:57access to read and write files needs to call APIs needs to generally just interact with the world around
00:01:03them and critically and perhaps most importantly we have to talk a lot about where the agents are
00:01:09actually executing so what is cloud managed agents before we go too deep into talking about execution I
00:01:16want to give a little bit of a lay of the land on what the actual agentic API is that we support out of
00:01:21the box are via Anthropics managed agents it's basically a suite of API's that make it extremely
00:01:26simple to get up and running running agents ASAP the first step is just to create an agent this is
00:01:32essentially a list of tools skills system prompt and permissions I think you're going to see that this
00:01:38is a very similar pattern to a lot of the other talks about agents that you've probably heard today but
00:01:42then that agent actually needs an environment to run in cloud especially is extremely good at getting work
00:01:49done when it's given access to a computer so the environment is the network allow list the packages
00:01:54that are installed in the container where cloud runs and generally is just how your agent interacts
00:01:58with the outside world then when you've got an agent and an environment you can kick off a session
00:02:03which is the agent going out into the world and doing actual real useful work and while all this is happening
00:02:10you need to be able to stream back what your agent is doing in real time and steer it if it's not doing exactly
00:02:15what you might want it to do but oftentimes we hear from our users that it's that they have to make a tough decision
00:02:22of where to actually run their agents and how to roll their agentic harness option one is to go with something
00:02:29like cloud managed agents off the shelf where you hand everything over to Anthropic and we're responsible
00:02:34for running the agent loop as well as for actually running the containers that your agent is running
00:02:39on top of but oftentimes this doesn't work for for all use cases in which case we've seen users who have
00:02:44decided to roll their entire agent to harness themselves and just integrate with an inference provider
00:02:49like Anthropics V1 messages or other inference providers out there the problem with this is that
00:02:54it's another thing you have to maintain another thing you have to keep up with anytime models come out you have
00:02:59to find a way to upgrade to those models and eval the whole thing end to end so we we often find that
00:03:06users are spending a lot of time building things that are not differentiated part of their products
00:03:10but are instead spending time building out this agentic harness this cost is nonzero from our users we
00:03:18hear that it sometimes takes between six and eight weeks just to get live into production you're having
00:03:23to build out tool dispatch retries to handle failures you're having to get the agent context when it needs it and where it needs it
00:03:30you're building out streaming infrastructure yourself and making it so that your users are able to interact with your agents in real time
00:03:36on top of all that you're evaluating the whole thing end to end and again spending time building things that don't make your product differentiated and
00:03:42once all this is done we're hearing from our users that you might need 20% or more of an engineer to just keep the thing rolling
00:03:50model upgrades generally new tools that are coming out and just keeping your agents up to date with the latest and greatest
00:03:57but one of the key insights that we uncovered as we've been building cloud managed agents is that we like
00:04:04to think about this entire set of agentic infrastructure as two totally separate components what we call the brains of the operation
00:04:12which is the inference provider it's the agent harness it's generally just the things that actually execute the brains of your agent
00:04:21but this is wholly separate from what we call the hands of your agent so the hands are the agent actually taking action in the real world
00:04:30running tools reaching out over a network making API calls reading and writing files and this is a key insight that totally changed the way that we built our own agentic infrastructure
00:04:39but it also makes things a little bit more easy to componentize if we think about it in these two separate mechanisms so in cloud managed agents you excuse me in cloud managed agents using self hosted sandboxes which is a new product feature that we have shipped in partnership with for sale sandboxes you're able to hand over the control plane to anthropic so we still own the integration with you on messages and the ability to actually keep
00:05:09keep up with the latest and greatest agentic harness developments while you using your vercel sandboxes can totally own the execution of your agent so this means that you bring essentially the container and the file system and all of the tools that clod is able to run within your sandbox so every bash command and file operation runs in a sandbox that you entirely control using your network policy potentially using your own internal services that you potentially want to expose
00:05:39to your sandbox and generally just you have the entire control over what this sandbox is able to access so to reiterate the anthropic hosting of the agent loop gets you a lot of things out of the box for free you're given a session abstraction that makes it extremely easy to get up and running with agents taking actions and seeing the durable history of what that agent does when it does it the tool calls that it makes generally the inference calls and the out there outputs and as well as
00:06:09as well as the uh just everything that's happening as the agent is going off and cooking also you're able to just quickly drop in new models as they come out and you can trust that we've evaled our harnesses our harnesses against all of the latest and greatest benchmarks that are coming out additionally we provide a set of monitoring as well as web hooks events that are coming through the event stream that will tell you exactly what's going on with your agent in real time so that's the anthropic side but let's go a little bit deeper into talking about what it might look like
00:06:39to hand over the sandbox execution to hand over the sandbox execution to you all via Vercel sandboxes here we have a hypothetical hospital where it's not necessarily acceptable to give anthropic access to all information that's sitting within your infrastructure in this case we have clawed up top that's running the agent harness but then in the middle we have your agent and its sandbox that's basically basically implies that you're able to execute all tool calls so all file rights all file reads
00:07:09and generally just any actions that cloud is taking inside of your network environment this is hugely powerful and it means that you're able to only give cloud exactly what it needs with the security controls around your infrastructure in a way that you feel comfortable with so for example if we have patient records that we want to get an anonymized version of to cloud it's very straightforward for us to use any existing service infrastructure or access policies that
00:07:39we've built in our infra infra or in your infra in this case to expose it to cloud to maybe in this case ship off billing codes to an insurer without cloud or anthropic ever having to know what's actually happening in your internal network this is a really really powerful way for all of you to use Vercel sandboxes to totally isolate yourself from anthropic's infrastructure in this case you can think about this environment that's a self-hosted environment so again as
00:08:09a self-hosted sandbox you can think of it as a work queue so anthropic publishes essentially a work item to you and then you're able to assign that work to something that's happening in your infrastructure
00:08:16essentially in this case that would be a Vercel sandbox so it's a couple step process the first is that you create a session using our API surface and the second is once work has been created you claim that work and essentially kick off the tool call on your infrastructure and execute it against whatever it is that you might have on your side that you want cloud to be able to interact with this is a pretty thematic from
00:08:46some of the prior conversations or the prior talks you had where the agent needs to meet you all where you are rather than having you push everything you need into some agentic infrastructure in the cloud then when you're ready to go you can actually execute the tools which cloud makes the decision to execute the tool and then you actually provide the response of the tool call and all this while it's happening leaves you with an intermediate file system of what the agent was doing on your file system any intermediate files that are created are totally
00:09:16essentially within your sandboxing infrastructure and do not live on a container anywhere in anthropics world but why does Vercel sandboxes fit this use case so cleanly well it's because Vercel sandboxes basically solve a lot of the core problems that come with running your own sandboxing fleet their sandboxes are ephemeral they're super fast and snappy to start up and the network call policies are trivial to edit yourselves on top of that you don't have a bunch of containers just sitting
00:09:45sitting around burning compute and running up bills that you don't want to pay so we found that Vercel has been an excellent partner to help us get you more power without having to roll entire sets of infrastructure yourself for running your own sandboxing fleet
00:09:52So what does this split actually enable what does splitting the brain and the hands and having you run your own sandboxing fleet actually do for you and what are our users doing in production we're really seeing three separate patterns the first is that it's extremely trivial to spin up sandboxes on a per user basis and ensure that files are never co-mingled with the
00:10:22where they should be and particularly for sensitive users your users can trust you to imply that you're not spreading their data anywhere on top of that as I was alluding to before your agents are now meeting you where you are and you can bring the agents to the data expose them private services private MCP servers potentially use any of the tools that were even announced today to expose your sandboxes to more information inside your own VPC without having to do it without having
00:10:52to do any fanciness of exposing things over the public internet in a safe way on top of that it's generally just a platform for you to do whatever you want with while still getting the power of cloud managed agents agentic harness so just to reiterate and really hammer home what this means we've essentially broken this down into three ownership categories the first is owned by anthropic and this is the agentic harness and the inference layer you know along
00:11:22you don't have to have to fiddle with any specifics of integrating with our inference layer and you can just rely on our eval harness in order to accomplish what you need to get done asap then bursell handles the execution of claude's tools using their sandboxing layer and you can focus on the right side on what makes your beer taste better and what makes your product generally differentiated from your competitors in the market so let's talk through what this actually looks like to get going because the hope is
00:11:52is that it's very straightforward to spin this stuff up so i was talking about before the first step is to create what's called a self hosted environment this is it as opposed to a cloud hosted environment that would handle the entirety of execution on your behalf so here we have a couple lines of code we're creating a versell sandbox it's configured with the type of self hosted indicating that it's using this work queue methodology then second step is using versell's awesome apis to quickly spin up a sandbox and
00:12:22here we've got a node 24 sandbox and we've allow listed api.anthropic.com as well as a fictitious internal billing url that's within our local vpc
00:12:29then the last step for getting the sandbox up and running is just quickly running a binary that we provide to you that essentially authenticates with our apis and enables you to receive work from anthropic's servers so that's this ant command down here and last but not least spin up a session from there you're good to go you're rolling you've got a session ready to receive inputs and boom anything you want to ask your agent to do
00:12:36it can just go off and accomplish it
00:12:43just go off and accomplish it using your versell's self hosted sandboxes as the compute infrastructure
00:12:50behind the whole thing so let's see if we can do a quick video showcase of what this actually looks like across a couple of the surfaces that are involved here so first i'm going to show off the console that's a cloud of surface in this case you can see
00:12:57ask your agent to do. It can just go off and accomplish it using your Vercel self-hosted
00:13:03sandboxes as the compute infrastructure behind the whole thing. So, let's see if we can do
00:13:09a quick video showcase of what this actually looks like across a couple of the surfaces
00:13:13that are involved here. So, first, I'm going to show off the console that's a claw to surface.
00:13:19In this case, you can see if I can click and get it playing. Excellent. We created an agent
00:13:25in our console. This agent is an EHR analyzer. It's using Claude Opus 4.8. It's doing some
00:13:31clinical analysis on our behalf. This is essentially the identity of the agent that's going to be
00:13:36taking action. Then we've also created an environment that is our Vercel self-hosted sandbox. You
00:13:42can see in the top that I've highlighted. I don't know if it's too small to see. Hopefully,
00:13:45you can see. You can see this is a self-hosted environment, and we've created an environment
00:13:50key that's used to authenticate with Anthropic servers. Then, on the right side, there's just
00:13:54a couple of instructions for how to get started to actually point your sandbox with Vercel at this
00:14:00environment. From there, we actually need to get our agent to go off and do something useful.
00:14:06Here, we're back in the Claude console, and we've got the ability to create a session. Again,
00:14:11that durable log of an agent taking action. Here, we're doing some EHR analysis. We're selecting our
00:14:17EHR analyzer agent, selecting our Vercel sandbox, and boom, we're off to the races. We've now got a session
00:14:24ready to receive inputs. Now, if we ask our agent to go and do some analysis on EHR records that are
00:14:30totally anonymized, we're able to do so. Kicking off a run, and here, what's that? A tool call where we're
00:14:37running a bash tool, and critically, this bash tool is not running anywhere on Anthropics infrastructure.
00:14:44This is on your Vercel sandbox, and that's where, again, the self-hosted sandbox piece of this feature
00:14:50comes from. So, we hope that this enables you to essentially do whatever you want while still giving
00:14:56Claude the power to request you to run tools on your infrastructure. Last but not least, I want to jump into
00:15:03a very quick demo of what it looks like on the Vercel side. So, we've jumped into our portal on Vercel,
00:15:10and you can see here that, indeed, all of these bash commands that we were just running are executing
00:15:15off of Anthropics infrastructure and on your Vercel self-hosted sandbox.
00:15:21So, how do you get started? We're so pumped to see what all of you are able to build. I'm way less
00:15:27creative than all of you in this room, but we hope that we've built a platform that makes it really
00:15:32easy to build agents that are useful for you while still adhering to any requirements that you feel
00:15:37like your business needs with respect to the data that Claude interacts with. So, we've got some docs
00:15:43pages. We've got one from Anthropic, one from Vercel, who has been an incredible, incredible partner
00:15:49as we've released this feature, and then also the announcement that it describes a little bit more
00:15:53about self-hosted sandboxes. So, if I could leave you with one thing, it's just to remember that in
00:15:58self-hosted sandboxes, Claude is still responsible for running the agent, but in this case, Vercel is
00:16:05the one running the code. Hopefully, we hope that you are all able to go out and build incredible,
00:16:11incredible things, and to scale your businesses, and to be extremely creative in the process.
00:16:17Thank you, all for having me, and I really appreciate you taking the time to come hang out.