Generative UI... in Python? — Jeremiah Lowin, Prefect
AAI Engineer
Computing/SoftwareInternet Technology
Transcript
00:00:00Thank you all for coming out. I'm going to talk today about one of the weirdest pieces of software
00:00:18I've ever written. It's sort of on the edge of a whole lot of stuff I've been putting forward
00:00:22into the world. So join me if you will. We're going to try and have the most reasoned approach
00:00:27to a very strange thing that agents in MCP and other things have enabled. And so to begin,
00:00:34I want to talk about MCP apps. I don't know if any of you were able to join any of the other
00:00:38talks earlier today, maybe the one that Ido and Liat just gave maybe an hour ago. Just a show of
00:00:44hands, MCP apps familiarity. Okay, this is probably the best crowd I've ever given this talk to,
00:00:52actually. So that's fantastic. For those that didn't put their hands up, MCP apps is an
00:00:57extension of the MCP protocol that was introduced, I think, in January of this year. And the idea is,
00:01:02this is a typical request response cycle for an MCP tool. The user makes a request to the agent. The
00:01:10agent in turn decides to use an MCP tool that's hosted on an MCP server. A tool result comes back
00:01:16into the agent's context. And the agent chooses to form some response and send it out to the user. And so,
00:01:23fundamentally, MCP servers are these fantastic ways of adding functions and business logic to your agents,
00:01:28but never a direct connection between a user and the MCP server. It always goes through the brain of the
00:01:34agent and, more importantly, through the context window of the agent. So, MCP apps are an extension of this,
00:01:40which allow us actually to bypass the agent. And instead, what happens is the following:
00:01:44the user requests something from the agent, the agent uses a tool, but instead of that tool request going
00:01:49back to the agent, it is sent out to the user. And it's sent out as HTML, CSS, JavaScript. It's a full UI.
00:01:55And it can be whatever you want it to be. And so, the idea is you have this way to make, to basically put
00:02:01the internet into your agent, so to speak. You can ship any custom branded useful UI that you want. You can
00:02:08let the user have any interactive experience that they want. And then the user, as you can see in the
00:02:12diagram, the user now can interact with the application. They can use the tools. They can send
00:02:17information back into a back-end, host on that app, and really get a full experience. So, you can imagine
00:02:21booking a table at a restaurant, or changing your seat on a plane, or interacting with a schedule.
00:02:28For AI engineers, there's a lot of things that you can do as a user now where the agent facilitated it,
00:02:33but you are going to interact as a human. And there's an extension coming now. This is going to
00:02:37come out in the July MCP release, where the agent can actually interact with the app as well. And this
00:02:43will tee up some really interesting use cases we're not going to talk about today, but you could
00:02:47hypothetically play a game of chess against the agent now in a visual app where you make a move,
00:02:52and then the agent interacts with the app as well. And so, I think that's going to open up a whole new
00:02:55world of possibilities. Now, some of you may know a framework that I'm the author of and my company
00:03:03maintains called FastMCP. FastMCP is one of the most popular ways of building MCP servers. And so, whenever new,
00:03:09cool things come to the world of MCP, the first thing I wonder is, how can I deliver this to our users?
00:03:16And one of the most important things I have to share with you about our user base is that they're
00:03:20mostly Python engineers. And so, and so, this is a little bit of a problem when we want to deliver
00:03:27front ends and UIs, because how are we actually going to do that? And this is the point in the talk
00:03:31where I reveal that I don't remember what the next slide exactly is, so we're going to take a peek at it.
00:03:35Nope, we're going to come back. We have a challenge now. How are we going to have Python engineers build
00:03:42UIs that are best practice, that are interactive, that are beautiful, that are useful, without
00:03:50pretending that we're going to do something silly, something that's been tried, and jam all of the
00:03:54front end, all of the ecosystem, everything into Python in some sort of like weird, compromised,
00:04:00haphazard, Frankenstein of a system. And so, I really struggled with this. I need,
00:04:04I really need, I feel an obligation to find a way to deliver this. But I can't, I can't pretend we're
00:04:10going to ship React in Python. It's not going to work. And so, we thought pretty hard about who are
00:04:15our users in the Fast MCP ecosystem? Who are these Python developers who tend to be in enterprises?
00:04:20What are they doing and what do they need these UIs for? What do they need these MCP apps for?
00:04:25What they don't need is consumer grade custom UIs that are fully branded. That's not what these folks are doing. What they are primarily
00:04:34charged with is sharing information throughout their, throughout their organization, for collecting
00:04:38information throughout their organization. And so, it changed the nature of what we expect them to do
00:04:43within MCP apps framework. And that constraint became really useful. So, fundamentally, we expect that
00:04:48they're going to do things like build tables, they're going to collect information through forms,
00:04:53and they're going to want to share charts. And so, fundamentally, with this constraint,
00:04:58we can introduce a piece of software that we open sourced a few months ago and has been surprisingly
00:05:03popular among this crowd called Prefab. And it's a scoped UI building framework for the purpose of
00:05:11delivering UIs through an agent for the set of purposes that I mentioned a moment ago. So, this is a hello
00:05:17world card. You might see this in any front-end framework. Literally any one. It'll have something
00:05:22that looks like this and it's on their website and you type your name in and it updates live.
00:05:27But, of course, the weird thing about this one is that the code that generated it is entirely written
00:05:31in Python. And so, I hope that you're feeling what I feel when I look at this, which is a really weird
00:05:38combination of, like, yes, that's cool and this really freaks me out. The yes, that's cool comes
00:05:44from the fact that I think there's something about this code, even if you can't see it up close, I can
00:05:48make it a little bigger, there's something about this that, like, kind of makes sense. You can see the
00:05:52structure of the UI in the code, but there's also something about it that's obviously alien and a little
00:05:58bit odd. And we come to this conclusion when you feel that, when you look at it, which is that
00:06:04when you compose a front-end in Python, it actually starts to feel good as long as we scope the
00:06:09challenge right. We are not trying to build a front-end from scratch. We are trying to compose a front-end
00:06:15from a bunch of world-class, well-designed components. And that's how we keep the guardrails,
00:06:20and that's how we keep the user in mind. The user here is not trying to do something arbitrary.
00:06:24They're trying to take a well-structured front-end and put it in front of whomever they're delivering
00:06:29it to. And so here's a little quick tour of that DSL. Primarily, we're using context managers. For
00:06:34those of you who do know FastMCP, you know that arguably you could reduce FastMCP down and say,
00:06:39the core innovation of FastMCP is that we used a Python decorator to build an entire MCP server.
00:06:44So if you want to take the same reductive approach to prefab, you could say, we use a context manager to
00:06:48build an entire UI. And by nesting components as context managers, as you see here, we are building
00:06:55up the exact same structure in the UI. It feels very natural when you read it. You can see how things
00:07:01are structured. Each element of the UI, each component, which is a beautiful shad-sian component
00:07:06when it's rendered, as you can see here, is a class that you instantiate. You can parameterize it. You can
00:07:11pass it stuff like CSS classes and make it look however you want. And then the last thing, which we're not
00:07:17going to have enough time to really explore today, is these reactive variables. I'll show you a demo
00:07:21of those in a moment. But essentially, we have a full way to create client-side interactivity
00:07:26and bind data between components that allows you to build these really rich experiences, again,
00:07:31without having to go fully into the JavaScript world and leave an ecosystem that my user base,
00:07:36at least, is extremely comfortable with. And this is the pipeline that prefab is essentially exposing.
00:07:42We use a Python DSL that I just shared with you. We use that to build a declarative
00:07:46representation of a UI. That then gets serialized into a JSON protocol. And that JSON protocol is
00:07:53ultimately rendered by a React app, which is hosted as the actual MCP app. And so this is going to open
00:07:59up a whole lot of possibilities for us that, again, I'm going to show you in just a second. But the key
00:08:02to this whole thing is the JSON in the middle. The Python is actually an accident that I discovered
00:08:07after the fact because it was a weird idiosyncratic thing that I wanted. The point of this was,
00:08:10"Can we create a serializable representation of a UI?" And that's the JSON protocol again.
00:08:17And because it's serializable, I can generate it from an agent. I can send it to an agent. I can generate
00:08:22it as a human and ask an agent to modify it. There's all this cool stuff that happens because of that
00:08:26intermediate representation in JSON. And then when the Python DSL just fell out of this and was really
00:08:31beautiful and easy to use, I kind of felt like we had something. So we have these docs. And sort of to
00:08:37prove the point, this was another constraint we took on. I don't even know what this is. These are the
00:08:42docs for the data table component in Prefab. I think there's 130 or 140 components that we ship that you
00:08:47can compose into an arbitrary form. The docs for Prefab are 100% rendered in Prefab. So the data table
00:08:55that's here in the basic usage, it is live rendered in Prefab. The Python code, you can see it sneaking in at the
00:09:01bottom of the screen. That Python code is being rendered live by the renderer to generate that.
00:09:05If you want, you can take any example in the Prefab docs, you can click a link, pop them into the
00:09:09playground and you can edit the code live, the Python code live, and you will see the UI update. And again,
00:09:15this is super weird. If you're feeling a little uncomfortable about this, that is okay. It makes
00:09:20a lot more sense when we constrain the problem and remember that we're composing a UI rather than
00:09:24building it. So I want to bring this back to the thing I opened with now, which is MCP servers and more
00:09:29specifically, MCP apps. You are welcome to use Prefab for any kind of front-end problem you have.
00:09:34My team has started using it for small interactive data apps and things to explore. They've been
00:09:39building presentations with it. We ship a dark mode theme that honestly looks kind of like the one I'm
00:09:43showing you right now to make slides and presentations. You can do a lot of stuff with it. But the reason we
00:09:48built it, the use case that it is satisfying is for MCP servers. And so I want to give you a quick tour
00:09:54of three ways that you can use it, three increasingly sophisticated ways that you can use it in your
00:09:58MCP server. The first is to build an interactive tool. As I showed you at the beginning of the talk,
00:10:04typically an MCP tool is something your agent calls and the agent gets the result and you don't get to
00:10:09interact with it at all. So what's the easiest way that we can advance that interactive functionality?
00:10:14I'm going to show you here. This is a fast MCP tool. It's been decorated with a tool decorator, as you can see,
00:10:20and it's just a Python function that returns some information. Bearing in mind, this information will go to the
00:10:24agent, not the user. If we want to turn this into a fully interactive tool with Prefab,
00:10:30we're going to make one change. Instead of returning a Python dictionary at the end, which will go to the agent,
00:10:36we're going to return a Prefab component. In this case, it's the data table. This is what I just showed
00:10:40you the docs for a moment ago. And when we return this Prefab component, Fast MCP will automatically
00:10:46detect that. It will automatically infer that you, in fact, want to return an MCP app, and it will spin
00:10:51up all the machinery to get the HTML, the JavaScript, the CSS, the render, everything in place so that your
00:10:55user will see a data table. Here's what this looks like in practice. This is using the Goose client,
00:11:00which is an excellent one. I asked a server that had the function I just showed you, show me the team
00:11:06directory, and what pops up, this would be better as a GIF, I apologize, but what pops up is a fully
00:11:10interactive data table component. It supports searching, and filtering, and sorting, and pagination,
00:11:16and all this stuff. And all it is, is what I showed you a moment ago, just return the data table class,
00:11:21and all this will be taken care of. We can go a step further. What if, in addition to the data table,
00:11:26we want to show a pie chart right next to the data table that breaks down this team directory?
00:11:30As you might imagine, very, very, very similar code. Instead of the data table alone, we're now going
00:11:35to import a grid and a pie chart, and if you look at the bottom, you'll see that we compose both the
00:11:40pie chart and the data table into a grid very naturally with a context manager, and this is the
00:11:45result. We now get a pie chart next to our data table. So this follows a principle that we really
00:11:51try to hold in a lot of our software at Prefect, which is one line of code, one big noticeable change.
00:11:55We try to keep that complexity incremental, and so this satisfies a lot of things that I think are really
00:11:59important about frameworks and DSLs. This is very quickly, because we won't have time to go into it,
00:12:05this is just an example I threw together and recorded of fully client-side interactivity,
00:12:09where all of these controls are linked, stuff's updating, text is updating, values are updating,
00:12:15no JavaScript was written. This is just a couple of classes composed that all have the same
00:12:19attribute assigned, so they all work together automatically.
00:12:24Oh, and I did throw in a quick code example of what that looks like. We have a class called Rx, which,
00:12:28as you may guess, stands for reactive. If you use these reactive variables, you can just reference
00:12:33them anywhere in your code, you can format them, you can make them the name of something, and it will
00:12:38automatically compile into the correct JavaScript implementation. The second thing that we can do is
00:12:43a FastMCP app. So if an interactive tool is sort of a one-shot, here's a user interface, and you can
00:12:49interact with it in the client, a FastMCP app is a full application with a backend, and in this case,
00:12:54the MCP server is going to be the backend. We don't have time to go through a full worked example in
00:12:59this session, but here's what the code looks like, just to give you a sense of the ergonomics.
00:13:03We're going to write a class, which is our FastMCP app, and then we're going to decorate at least
00:13:08two functions with app.ui. That's the entry point that's going to return the prefab components that
00:13:14form the base UI of that application, and then at least one, I guess this is optional, so zero or more
00:13:20app.tools, and these are essentially backend methods that you can now reference in the UI.
00:13:27So you could have a button that takes data that the user has entered into a form and sends it to a
00:13:31database using a decorated tool like this. One thing that we use, and we ship as a built-in
00:13:38component now in FastMCP, is an upload component. So as you can see, because only the agent has access
00:13:46to an MCP server, you can't simply upload a file to an MCP server. It has to go through the brain of the
00:13:51agent, and so what ends up happening is a lot of people create basically an upload tool on their MCP
00:13:56server, forget that the agent has to actually call it, and what you end up doing is the world's most
00:14:00expensive copy-paste operation. You give the agent a megabyte of text, the agent retypes it character
00:14:06by character into the MCP, and now, yes, in fact, you have uploaded it, but it's extremely, extremely
00:14:12inefficient. So this is a really good use case for an MCP app, where you ask the agent to bring up the
00:14:18app interface, you drag a file into it, and now the file bypasses the agent and goes right into the
00:14:23server, and we've made that a one-liner like this, along with a handful of other useful tools, and
00:14:30is this a GIF? This is not a GIF, or if it is, it's not rendering, but it would look like this in your
00:14:34client, and any client that supports MCP apps, if you ask the agent I need to upload something, it can now
00:14:38show you this, and you can upload safely, and most importantly, cheaply. Oh, I do have a GIF. Never mind.
00:14:45Know your own slides is a good lesson from this talk. So here's the agent is now interacting with
00:14:50a file that I just uploaded, that I dragged and dropped. I'll make these slides available later if
00:14:55you'd like to see this, or of course, it's a one-liner. You could try it in your servers this afternoon.
00:15:00The last thing that I want to talk about, which is sort of enabled by this architecture,
00:15:04is a fully generative UI. We're just going to skip and let this play while I talk. So this is a very
00:15:10simple demo where I asked Claude, "Hey, I'm giving a talk on this. Just start streaming the most
00:15:15interesting UI you can come up with." And so it just went. And what it's doing here is we exposed a tool
00:15:21that accepts the JSON, the protocol serialization of a UI that Prefab is based on. And so now as the
00:15:29agent is streaming that information over the wire, we are in real time rendering whatever we've got,
00:15:35healing that JSON and rendering it. And so this was a really cool demo, and it was really effective,
00:15:39and people like this because now you don't even have to define the UI yourself. All you have to do is
00:15:43use the skill we already ship, share it with your agent so it knows how to write a UI, and off it goes.
00:15:48It can make you whatever you want. There are some clients that have built-in versions of this. If they
00:15:52have a built-in version, you may prefer to use it, by all means. But this may be a way for you to build
00:15:56your own custom approach or limited set of components that are useful to you. Now, a really interesting
00:16:01thing happened when we spun this up. So as I mentioned, originally the plan was for the agent to send
00:16:07JSON over the wire and have it be rendered into this full React application. What we ended up discovering
00:16:13is that the Python representation of a UI is about 70% smaller than the JSON representation.
00:16:20So, we don't do this anymore. When I recorded this demo with streaming JSON, what we now do is we actually
00:16:26stream the Python over the wire. It's executed in a sandbox. It's turned into JSON on the server,
00:16:32and then that's rendered. And so this has a dramatic, dramatic token efficiency cost and latency benefit.
00:16:39So, it would work exactly the same as when I recorded this demo, but this is just one of those things
00:16:43that we've learned on the fly. And it's really fascinating that the Python representation is just that much more compact and ergonomic
00:16:49than the full JSON one. So, that's Prefab. If you'd like to check it out, if you're curious,
00:16:55if you want to see the weirdest thing I've ever built along with however other many people, you can see
00:17:00the docs at prefab.prefect.io. You can see the full library, which is on our GitHub here. And this is
00:17:06already fully baked into Fast MCP. So, if you're using a recent version of Fast MCP, you should be able to
00:17:11install this optional edition, import the components, return them, and start playing with these MCP apps.
00:17:17Thank you all for coming.
Community Posts
No posts yet. Be the first to write about this video!
Write about this video