MCP Apps: Give the Model Data, Give the User a UI — Dustin Mihalik, Indeed
AAI Engineer
Computing/SoftwareJob Search
Transcript
00:00:00Hey everyone, I'm Dustin Mihalik. I work at Indeed. We're the number one job site in the world.
00:00:19And I have to apologize for my voice. I'm recovering from a cold that I had last week.
00:00:24Yeah, so at Indeed we build job search and I also work on a team that does AI platform and I do like AI guardrails and gateways and compliance stuff.
00:00:41Occasionally my team gets cool projects to work on because we have relationships with the vendors.
00:00:47MCP apps is one of those and MCP connectors. So this is a little bit of like practical MCP apps.
00:00:55They gave a great introduction to MCP apps. This is a little bit of a lessons from the trenches.
00:01:01Which is a little bit of like what did we learn in building MCP and MCP apps for Claude, Chajubiti and our own internal career scout.
00:01:13Which is our agent that we have for job seekers. So this is the chat based interface.
00:01:21So a lot of my examples are going to be job search. I have a lot of screenshots of job searches.
00:01:26So this is a job search which is basically, you know, I'm looking for a barista in Austin and this is a text based response.
00:01:36And this works pretty well. As we kind of discussed, like there's no branding here. There's no Indeed branding.
00:01:43You know, Claude decided to say these are some jobs in Austin. These are some jobs in some suburbs of Austin.
00:01:50That's cool. That's probably good for the user. There may be some limitations of like what we can do for branding or how we can, you know, how we can control things.
00:02:01And you'd actually be surprised, unless you've tried to do this yourself, that it's really hard to get Claude or ChatGPT to link to things.
00:02:11Because they don't want you to leave their environment. It makes complete sense.
00:02:15But if you get back, here's five jobs that are somewhere on the internet without any links, that's a terrible user experience.
00:02:25So it took us a ridiculous number of hours in evals to make sure that like Claude would consistently link to things.
00:02:33So with MCP apps and with apps SDK, we could kind of control that. We can decide, you know, that we've got an apply button.
00:02:43We can decide what stuff is important that we want to highlight at the top.
00:02:48We can provide a link to view details so that when you click it, you get a pop-up, a modal that has all the job details so you don't have to leave the environment.
00:03:01So it's a win-win for both companies.
00:03:08So MCP apps is really good, but if you're thinking about, hey, I want to build an MCP app, or I want to take my website, I want to put it in ChatGPT or Claude.
00:03:20It's not quite just as easy as like dragging and dropping into a chat interface.
00:03:25You really want to think about how you are representing the data, how you're making it available to the user.
00:03:31So if you do a very naive thing, which is you still call your existing APIs for loading data, then it basically becomes a black box to the model.
00:03:43So you say, hey, I want to do something.
00:03:47The model says, cool, I'll call a tool.
00:03:49The tool says, okay, I'm going to show some stuff, but the model has no idea what you're displaying.
00:03:57So there's all these like follow-up questions, like tell me about the first result, or please rank this list of companies.
00:04:04The model has no idea what data is being displayed.
00:04:07So the very first rule for me building MCP apps, anything that you show to the user also needs to be provided as data to the model.
00:04:18I think this makes sense, but I've definitely seen some MCP apps where they just, you know, use it to inject some HTML on the page and then call some APIs.
00:04:28And that just makes a big black box for the model.
00:04:32So this is really easy to do using MCP spec and the MCP app spec.
00:04:37This structured content, this is what you would already be returning if you were doing just text-based MCP.
00:04:43And then this resource URI, that points to where the HTML is.
00:04:48You need to return both of these and you need to keep them in sync, right?
00:04:52If you add something new to the API, you need to make sure you add something, add that same data back.
00:04:59So kind of the next step that once you do that, you'll find is that now you've provided data to the model and you provided this black box that it has no idea about.
00:05:15It's still going to try and describe the, it's still going to try and take the output and describe it as it normally would.
00:05:22So you end up with like, here's your display and then here's the model doing basically the same thing that it would normally do.
00:05:28So what you need to do is you need to update your description in order to tell it that you're going to be displaying stuff in your MCP app.
00:05:36That way you get this like nice, you know, here's a list of, you know, here's a little summary of things.
00:05:42And you know, the results are, are, are, are shown above the links rather than it trying to like do a whole text-based display.
00:05:49You'll end up with this a little bit of a battle between like what gets displayed in UI and what gets displayed by the model.
00:05:57You can try and steer that with descriptions.
00:06:01So even something as simple as results were automatically displayed to the user as UI components at the top of your description, your tool description.
00:06:10That covers, that covers quite a bit of the cases.
00:06:14So that's one of the next things that you're going to want to do once you're providing both data and API access.
00:06:23Uh, the next thing is there's these interactable pieces, right?
00:06:27There's the apply button.
00:06:28There's a view details button, which pops up a big job description.
00:06:32Um, this is the same case where, you know, as you interact with those, the model's not going to know necessarily what you're looking at.
00:06:40So when you click view details, you get a big modal that's here's everything about the job.
00:06:45There's once again, a whole bunch of questions that the user could ask, uh, write a cover letter for this job.
00:06:52Summarize this job description.
00:06:55It has no idea because you've loaded that data in either via API or you loaded it in dynamically.
00:07:02Uh, you know, you've returned 10 jobs and user clicks on one.
00:07:07The model has no idea which one you've clicked on.
00:07:10Uh, so any information about user interactions, you also need to provide to the model.
00:07:18And, uh, once again, MCP apps spec has a pretty easy way to handle it.
00:07:24There's this update model context, um, method, which lets you pass in a string, uh, that is, so for MCP apps, there's a single string.
00:07:35Uh, so if you like want to track multiple events over time, you kind of have to append, uh, multiple things to the string.
00:07:42But this is, this is, this is, this is an example from the MCP apps, uh, documentation where basically, you know, this is a shopping cart application and they add the total cost and all the items that are on the shopping cart.
00:07:56So the user can ask for, uh, you know, tell me information about the items that are in my shopping cart.
00:08:03So these two things give you an app that like the model could kind of see what's going on.
00:08:12Um, but it doesn't necessarily make a really good MCP app yet.
00:08:17Uh, because it gives you, it gives you some UI that looks like what you want, but the thing that I usually do, I don't give, I don't give the, I don't, I don't give Claude, uh, my easy problems to solve.
00:08:33I give Claude my really hard problems to solve, right?
00:08:36Like if I just wanted to do one search, I would go to the web and do one search.
00:08:40I want to do a whole bunch of searches.
00:08:42Uh, this is, this is out of date because there's no Sonnet 5 here yet.
00:08:47But, uh, this is screenshot from two days ago.
00:08:51Uh, but yeah, so that, so like this job search is, hey, I'm looking for this, I'm looking for this title.
00:08:57Um, I'm willing to relocate.
00:08:59So I want to search across a whole bunch of different cities.
00:09:01Um, you know, I'm looking for the highest paying options.
00:09:04So I want you to just cherry pick a few out of there.
00:09:07There's some industries that I absolutely don't want to work in.
00:09:10Text based MCP does really well.
00:09:14Like we all see this, right?
00:09:16Claude will do 10 different searches, 15 different searches.
00:09:21It'll filter, it'll pull out all the individual pieces.
00:09:24Uh, and then it gives me a nice table at the bottom.
00:09:27Uh, which is, which is super nice.
00:09:29But with the MCP app that we were just discussing, you know, we, we said, hey, uh, you know, my
00:09:37results are going to be displayed in this UI.
00:09:40Uh, we don't, Claude, Claude will call it once.
00:09:45And then it'll be like, oh, I guess the results are already displayed.
00:09:48I'm not going to like do a deep dive, right?
00:09:50Like it doesn't, it's you as a user are not going to want 10 different carousels.
00:09:56And Claude will also notice that like it's already been displaying some stuff and it won't call to show 10 different carousels.
00:10:04Um, and so what you really want to do, and this is rule three, this like supersedes all the other rules, uh, which is basically you want to separate your data processing from your UI rendering.
00:10:16And this particular wording I stole, uh, from open AI, uh, open AI, uh, apps, SDK documentation.
00:10:28There's a couple of places where they say this.
00:10:31Um, but basically they want to, you want to separate your data processing from your UI rendering.
00:10:37So the job search that we had, we're just going to have that be a standard text based MCP application.
00:10:46Claude can call that as many times as it wants.
00:10:49And then we have a render tool that either you could pass, you could have the model basically pass all the data that it wants to render in, or you can do a reference, uh, to it.
00:11:01So in our particular case, uh, you know, we had search jobs.
00:11:06Now we have a search jobs that doesn't return in the UI and we have a render jobs widget that takes a list of IDs.
00:11:13Um, and so that list of IDs, uh, can be, you know, Claude can do a search.
00:11:19It can get a hundred different jobs that it cares about.
00:11:23It can filter those.
00:11:24It could find five that it cares about and it can show those five to the user.
00:11:29Um, now as you make these like render, uh, as you make these render calls, you need to update the descriptions to say like where they get the data, what format the data should be.
00:11:41Um, but it's a fairly easy, fairly easy mechanism to update your tool description to say you need to always call one of these three tools first in order to get, um, the data that you're going to be using for rendering.
00:11:55So search jobs, this is a pretty good, um, it's a pretty good example, uh, of, of how we, where we want to split data from rendering.
00:12:07I think there's a ton of other, like in most industries, you can kind of come up with like, Hey, where do I want to, you know, where do I want to split?
00:12:16Like I want to be, I want the model to be able to explore this data and then I want it to turn around and choose to be able to render it, right?
00:12:25Like there's, there's examples of, uh, you know, e-commerce, right?
00:12:30Like a bunch of e-commerce options.
00:12:33Uh, if you've got a map, you know, maybe you want to come up with like five different addresses and then you pass in addresses.
00:12:40Um, the other thing that you can do is you can let the model be a lot more creative.
00:12:45Like one of the things that we've seen in some of this text based, uh, stuff is that, you know, the model will say like, this is a, this is a reason why I picked this one.
00:12:56Or this is a reason why this is really good.
00:12:58And so, you know, potentially we can add something to the render jobs widget where you say, give us an ID and a reason why you think that this is a good fit.
00:13:09Um, or give us an ID and highlight a section of the job description that is really good.
00:13:15Um, so you can get really creative with your render tools to be able to have, to be able to give some extra character that the model can inject, uh, into those so that, uh, so that you've got a much better experience for the user.
00:13:32Um, so the key takeaways, um, right, when building MCP apps, you want to focus on the data before you focus on the UI, which sounds, sounds opposite of, you know, how we were thinking about it of like, hey, there's all these, there's these MCP apps.
00:13:48Like, it's how I put UI into ChatGPT.
00:13:52Um, I think if you want to really have a good MCP apps experience, you need to, you need to look at and see what data do I want, what data do I want to give to the model, what data do I want it to be able to do, and then rendering is a side effect of, of that, or it's a result of the, the model exploring the data.
00:14:14Um, and then I think small composable tools, right?
00:14:18So, uh, basically, you know, maybe there's like two or three different ways you can search for jobs.
00:14:26We can build two or three different search tools, and then there's one render tool.
00:14:30Or maybe there's like, maybe there's two render tools.
00:14:33There's one that's render a list of jobs, one that's, you know, highlight one particular job.
00:14:38So, you can build a bunch of much smaller tools.
00:14:42The descriptions can be fairly simple so you don't overload the model, but it gives the model flexibility about how it wants to explore the data and how it wants to render the tools.
00:14:52So that's, uh, basically, basically my talk.
00:14:57I'm a few minutes, few minutes fast.
00:15:00Um, but I, uh, don't have a booth that I'm gonna hang out in, but I'll, I'll hang out in the hall if anyone has any questions.
00:15:07Um, and I am either my last name or my first initial last name on most social media platforms.
00:15:15Thank you.
Community Posts
No posts yet. Be the first to write about this video!
Write about this video