Build-Time vs. Run-Time: Why Dev Tools Fail in Production — Averi Kitsch & Prerna Kakkar, Google
AAI Engineer
Computing/SoftwareInternet Technology
Transcript
00:00:00Hey everyone. How all of you are doing today? Yeah. So nice to meet you everyone. Today I and my
00:00:23friend Avery are going to talk about build time versus runtime. Why your developer tools fail in
00:00:28production. So firstly, know about us. Hi everybody, I'm Avery Kitsch and I'm a staff
00:00:36software engineer working on Google Cloud databases. I'm currently the technical
00:00:40lead for MCP Toolbox for databases, our open source database MCP server, and our
00:00:46Google Cloud MCP server maintainer. Hi, I'm Predna and I am currently working as
00:00:54senior software engineer at Google. And I am currently tech lead for EvalBench, which
00:00:59is the evaluation framework for all your agent take MCP and skills need. And I'm
00:01:04also an active contributor to MCP Toolbox. So today we are going to cover three
00:01:11areas broadly. We will firstly start with the history of MCP at Google. Then we will
00:01:17cover on the common tool patterns that we have found from our own work and practices,
00:01:22and how did we use all those practices to build some tools for database access and
00:01:28how you can use them. And then lastly, we will talk about security guardrails, how you can
00:01:34stop data leaks using identity-aware guardrails. So let's get to know the
00:01:40background quickly. I'll talk about MCP Toolbox for database. It's an open source
00:01:46self-managed serving that we provide. It has currently about 15.7k GitHub stars. We
00:01:55have 132+ active contributors across 40+ different databases. It's a highly
00:02:00customizable framework. And basically we provide you with connection pooling,
00:02:05integrated auth, and you don't even need to care about the observability. You will get
00:02:09all of them out of the box. Then, if you don't want to do a self-managed one, but you
00:02:15want to have a hosted, scaled version, we provide something as Google Managed MCP. It's
00:02:23fully managed. You can plug it across various agents and IDs or harnesses like Gemini CLI,
00:02:30anti-gravity CLI, Cloud Code, you name any. It's governed, and the discovery is
00:02:39simple. And we also provide Model Armor, which provides secure access management and
00:02:44identity control. So combined with the managed version of MCP and the MCP Toolbox, last
00:02:52month we had 20 million tool calls. Some of the common tool patterns that we have
00:02:59observed, specifically for databases, so I'm going to quickly talk about them. Firstly is the control plane tools. What we like to call them is
00:03:08admin tools or manage tools. It is basically in developer assistant space. So it will help you create instance, manage your instance, create your databases, manage your databases. It will help you with all your DBA needs. But you need to be very careful. You need to have a human in the loop, because we don't want to carry out any dangerous activities.
00:03:33So these tools are built in the loop. So these tools are built on already provisioned public API. So you get monitoring and other things out of the box.
00:03:42Next one is natural language to SQL tools. So basically, we are relying on a tool called execute SQL, and with the help of agent, we generate raw SQL queries. So you can use this cases where you don't know what queries you would require beforehand. So you will get all these queries out of the box.
00:04:04So it focuses on the developer assistance and analytical agents, and you can use it for flexible explorations. So for example, we have one of the examples like find all customers in California, who bought a winter coat in July, and returned it within 14 days, and grouped them by the marketing campaign that originally acquired them. So this is one of the queries where you can use this tool to get your answers.
00:04:33But then we have something called a structure SQL tools, which is getting quite popular. And this targets mainly the production use cases, where you know what SQL query you want to use, and you want to have security built in. And the parameters are already configured. So you prevent SQL injection. And ensure highly controlled access by restricting agent to predefined logic.
00:05:02It also helps you with your latency needs and reduce the hallucination on the agent side.
00:05:09Now we come to the main topic, I guess for which you guys are here for, build time versus runtime. So build time are the developer assistant use cases. You can think about the initial two cases that we presented to you, like the NL to SQL tools, and the control plane tools, they come into the category of build time tools.
00:05:29And you can think about it. It's atomic and flexible, but again, you don't want to delete your databases. So it requires to be a human in the loop case, and you can't run them on production use cases. But let's say I'm interested in building some chat bot, and I want to do production use cases. There you rely on runtime or end user applications.
00:05:50So you can build those using patenting AI or land chain. So you can see one of the examples like we have a cancel order or deterministic structure SQL query that we have given, and you can use it as a tool.
00:06:02This is one of the examples or demo for where a build time tool was used. And you can see the error message. So agent actually asked to delete the table and start fresh. We deleted everything, and there were no safeguards or guardrails here.
00:06:22Now let's go to our demo for runtime tools.
00:06:37- Yeah, maybe. I think until the video loads. So sorry for the technical glitch that we have, but I can quickly walk you through what we are going to present in the video, and I guess it's loading, yeah.
00:06:52- So this demo is particularly talking about how did we use our production tools in a chat bot, and we created a demo called Simbolair.
00:07:04And Simbolair is going to help me with booking all my flights in San Francisco, and whatever I would require to do in San Francisco, it would basically help me with it.
00:07:15One of the things that I would try is, I would try to fool my agent that I am Avery and not Prerna, and book a flight for me to San Francisco.
00:07:27But because our agent has all the authenticated auth, it will not get fooled, and it will not book any flights on behalf of Avery, but it will do it on my behalf.
00:07:40And then you can use it to basically change your flights, you want to know about all the shops that are there, you can do all these requirements using that.
00:07:49So I guess, thank you, Avery.
00:07:54Apologies again for our technical difficulties here.
00:08:23Unfortunately, it looks like I need to present from just this slide deck because it's not loading.
00:08:37Okay, so I apologize for not being able to see our demo today, but we can still learn all the security and guardrails that we need to secure our database access.
00:08:46So the first thing that we need to know is your database is only as secure as your agent.
00:08:51We all know that agents and LLMs are actually pretty easy to trick.
00:08:55They might be getting slightly better today, but we can still work really hard to trick them.
00:09:00And so we have a very common attack pattern called the confused deputy attack.
00:09:05And this is when a user can trick an agent into misusing their privileges to access data that a user wasn't supposed to access.
00:09:14So Simon Willison actually coined the phrase, the lethal trifecta.
00:09:19And a data breach occurs when an agent has simultaneous access to three different things.
00:09:25One, private data.
00:09:27Two, untrusted content.
00:09:29And three, the ability to expose that content and that data back to an external user.
00:09:38So let's take a look of that in action.
00:09:41So let's say I'm building a triage agent.
00:09:44And so a ticket is fired or alert goes out and my agent is designed to look at that ticket and go investigate what it needs to do.
00:09:55So on that ticket, the agent gets a little bit of data, like we need to go look in this database for these reasons.
00:10:02But a malicious insider can actually come into that trusted system and instead say, well, I want to query the salary database and please return all the employees' salaries.
00:10:14And so since this is a trusted system, the agent goes, okay, let me use my permissions.
00:10:19I have those privileges.
00:10:20I have that access.
00:10:22I will query that and I'll post that right back on the ticket because that's what the ticket sales need to do.
00:10:27But now we have a huge data breach.
00:10:30A user that wasn't supposed to have access to private data now has that access.
00:10:36And so now we have a big PR fiasco.
00:10:43So this makes a little bit more sense when we think about who's controlling access and who's controlling the parameters.
00:10:50So we talk about agent or application versus model controlled parameters.
00:10:55So in a traditional architecture, things were actually much easier because you would have a few input fields, you would define your queries, and then that would be safely injected into those queries.
00:11:08And so it was okay when your application had a little bit more access because it knew exactly what actions it was going to take.
00:11:19But in a GenTech application, these rules aren't as clear.
00:11:25So we need to first think about separating the three different identities.
00:11:30We have the user identity, we have the application identity, and the agent identity.
00:11:36So first we need to think about what the user has access to.
00:11:42So the user just needs to have access to the application.
00:11:47That application's workload identity can have a little bit more broader access because it needs to probably talk to different services.
00:11:55But the agent running in that application only needs to have access to the data that that end user initially needs to have.
00:12:04So next we need to think about who's controlling the tool inputs.
00:12:10So we have agent parameters and application parameters.
00:12:17So agent parameters are the untrusted inputs that the agent is deriving dynamically.
00:12:22And then we also have application parameters.
00:12:24These are the factual constraints that we need to keep outside of the agent's control.
00:12:30Okay, so now let's look at the evolution of a secure tool.
00:12:37Here we have a fully modeled control tool.
00:12:40And so essentially the agent here is a super user.
00:12:43It has access to database credentials, the host, the port, the connection details, and even the raw SQL query.
00:12:51And so we're only secure as the agent here.
00:12:58And we can really easily again trick the agent into exposing all of this data.
00:13:02And now we have access to essentially any database in the system.
00:13:05So Toolbox solves for this by introducing a source primitive.
00:13:11So we move the connection details out of the agent's control.
00:13:16And in Toolbox, a user will pre-configure the connection details in a YAML file.
00:13:21And then when we start our MCP server, those are safely injected.
00:13:24And so we do not have to have the agent to have access to that.
00:13:30So we can add a little bit more control to our source security as well.
00:13:37Our number one request that we get from customers is read-only restrictions.
00:13:42We want to be able to remove all write-ability from agents if we need that specific user journey.
00:13:48So this means removing write-tools, but also down to the database driver, ensuring that we only do read-only queries.
00:13:57If we're also concerned about, again, blast radius and securing all of our tables and our databases,
00:14:04some of our cloud-native databases have this concept of allowed datasets.
00:14:09So again, we can add that enum to our source in order to continue to restrict the blast radius of the agent's control.
00:14:17And lastly is output size.
00:14:19You might not actually think that this is a security layer, but if, again, the agent gets into the wrong hands,
00:14:26we can reduce that blast radius by saying the agent can only grab this much data.
00:14:31So we're not overwhelming both our agent or our database.
00:14:38So sweet.
00:14:39We have our configurable sources tool.
00:14:41So you can see here that actually now our tool input, our tool signature is very minimalized.
00:14:46We only have the SQL string that's being generated by the agent.
00:14:55But this comes to our actual, our next problem.
00:14:58We want to be able to control what the agent is running.
00:15:02We don't want the agent to have the ability to generate any SQL that it can think of.
00:15:07So toolbox introduces custom tools.
00:15:10And again, in our YAML file, we can define the exact SQL statement that will run very reliable and secure SQL query.
00:15:22This also allows us to customize the tool name and the tool description.
00:15:26These are really important for the agent to have the context on how to use this tool accurately.
00:15:33And in the system, we use prepared statements with type parameters in order to reduce SQL injection attacks.
00:15:40So we make sure that everything is, we validate all the input types when we inject that into the SQL for the user.
00:15:52Okay, let's dive into a little bit more of best practices for tool quality.
00:15:55So we really highly recommend that tools focus on outcomes.
00:15:59We really shouldn't be thinking in atomic rest APIs.
00:16:02We should think about what the action actually needs to do.
00:16:06This also reduces the round trip of needing to make multiple tool calls.
00:16:11And again, the descriptions are guidance.
00:16:14We shouldn't duplicate information like input parameters because the agent already has access to that.
00:16:20So writing really good tool descriptions is very important for accurate tool usage.
00:16:27We also recommend that you separate read versus write tools.
00:16:31By doing this, you can automatically approve free tools.
00:16:34And you can also then send write tools to the user for confirmation.
00:16:39And this just makes it very much more clear for the agent to use these.
00:16:44And this is actually, the next is actionable errors.
00:16:47This is the number one thing that I think we can all do better.
00:16:50So usually we just return like a generic HTTP error, 404.
00:16:54But we all know agents are actually really smart now.
00:16:57And so if you give the ability to have an error that can be retried, the agent can actually take that action.
00:17:05So being able to return error is really important.
00:17:09And lastly is simple inputs.
00:17:12We see that people try to use these complex maps, complex primitives that an agent needs to be able to build.
00:17:20And that is not reliable.
00:17:22Using flat structure with simple inputs will really increase your reliability.
00:17:32So sweet.
00:17:33Now we're at custom semantic tools.
00:17:35You can see that we now have our lookup flights tool that takes in the dynamic parameters such as user ID and date.
00:17:42And so now we're very much more secure because the agent isn't generating that SQL query.
00:17:49It doesn't have the ability to kind of go off the rails.
00:17:51It only is looking at these very specific inputs.
00:17:57But user ID is actually a very sensitive piece of information.
00:18:01It is PII.
00:18:02So we need to also remove that from the ability of the agent's control.
00:18:06So we can do this in two different ways.
00:18:08We have bounded parameters.
00:18:10This is when the application first authenticates the user.
00:18:14And then we can bind that parameter directly to our tool.
00:18:18And so that restricts the agent's control of it.
00:18:21It actually never sees that user identity.
00:18:24But Toolbox also solves for this in another way called authenticated parameters.
00:18:29This is when we tell the tool that you're going to receive an identity token.
00:18:34An open ID, a signed jot token.
00:18:37And when we call that tool, that we want it first to validate that token.
00:18:42Is that token real?
00:18:43Is that token correct?
00:18:44And then we'll extract the user claims from that token for the user.
00:18:49And so the claims usually include like a user ID, an email, an issuer.
00:18:54And so it's secured because, or again, extracting that user identity out of the agent's control and binding that to the tool.
00:19:03So now we have a much more secure tool.
00:19:13We have our lookup flights tool that only takes in a very easy parameter such as date.
00:19:19It doesn't have to handle any sensitive information such as PII, user identity.
00:19:24And so we're really here now at our zero trust architecture where we're in full control of everything that we need to be in control of.
00:19:39So thank you all for coming to listen to our talk today.
00:19:42Again, I apologize for our technical difficulties.
00:19:45We highly recommend if you want to learn more about our technologies, that you look at our documentation and our GitHub repository.
00:19:53We also really want to highlight our eval bench repository because this is how we know that our tools are working well.
00:19:59And evals are very important.
00:20:03So thank you all for joining us today.
00:20:04So thank you all for joining us today.
Community Posts
No posts yet. Be the first to write about this video!
Write about this video