Ship 26 NYC - Sandboxes in Production at FLORA
VVercel
Computing/SoftwareSmall Business/StartupsPhotography/Art
Transcript
00:00:00Hi, hi, hi. Thank you so much for my Vercel friends for having me back at SHIP. I'm super
00:00:12thrilled to be sharing some stuff with you today. My name is Charlton. I'm the CTO of Flora. Before
00:00:17that, I led engineering teams at Squarespace, Frame.io, and Adobe. Today, I want to share with
00:00:23you how we're using Vercel sandboxes in production and how we arrived at using them. First, a
00:00:31little bit about Flora. We're building tools for the creative professionals that adapt to
00:00:36the work in front of them. We think of it all as a creative operating system. Flora is a
00:00:42node-based canvas for text, image, audio, and video AI generations. We support over 200 AI
00:00:49models, and we're used by customers like Nike to take a choose sketch to a full 360 video,
00:00:55or Lionsgate films to spin up a spec video. We're rethinking the whole stack of creative
00:01:01production workflows through an AI-native lens. Here's a look at the product. You can see how
00:01:09the creative ideas are chained together in these little nodes. Each node is actually a generation.
00:01:15This one is taking an ad and reformatting it to different aspect ratios. We have customers
00:01:22across different creative industries from fashion, brand and design, architecture, film industry.
00:01:28We're used by hundreds of thousands of creatives across the world building real workflows. The
00:01:34real power of Flora, though, comes from building reusable workflows. We call these techniques. This
00:01:40was an evolution on the initial canvas interface that we introduced because customers really needed
00:01:47reliability and consistency. That's what we heard over and over and again. And this is how creative
00:01:52production teams of today actually scale. This is actually a technique for a brand campaign called
00:01:58repose. I think the guy looks super cool. But I want to start today with a tension that we kept running into
00:02:04when building Flora in this type of new agentic world that we're in. For decades, we, software engineers,
00:02:13shipped buttons. Buttons are rigid. Buttons like crop, blur, render. We had to decide what every single
00:02:21button did at build time, and then we shipped it. Code and buttons are rigid. But rigid is good. Rigid is
00:02:28predictable. Rigid is solid. It's trustworthy. AI, on the other hand, is squishy. I like that word. It came
00:02:36from one of our engineers, Kyle Shafer. It can adapt. It can morph. It can recover. It can rebound. It has to
00:02:43give a little. It can't just do exactly what you tell it to do. And here's the problem. Squishy and
00:02:49predictable are supposed to be opposites. Squishy means that anything can happen. And predictable means
00:02:57that you're not going to have any surprises. I found in my career that professional creatives,
00:03:03they have just an exquisite eye. These folks have spent decades looking at pixels so they can see a
00:03:09half pixel that's off. They can see a dropped frame. They will immediately churn from your app if they
00:03:16detect any jank. They want it to be polished. Polish isn't a nice to have for us at Flora. It's a must.
00:03:22We needed both. But the question we couldn't escape was, how do you build an app that's squishy enough
00:03:29and predictable enough that a professional trusts it? Because the irony is, as you reduce your constraint
00:03:36over AI, you and I have all felt this, it tends to get more powerful and creative. I almost shared a tweet
00:03:41from Matt Pocock that was like, you can just delete lines from your skills and it will probably keep
00:03:45working. For us at Flora, this meant giving the AI room to play and experiment, create new workflows,
00:03:52and even write and run code. But arbitrary code execution needs boundaries. That uncontrolled exploration
00:04:00needs to come from and go back into the deterministic side of the product. It needs to be protected and
00:04:07shaped, but that doesn't mean that it needs to be totally controlled in the same way that we used to
00:04:12control what every little button used to do. Code and agents, they kind of need different stuff, right?
00:04:18Code is consistent. It's predictable. It's fast. It's cheap. Whereas agents are smart and flexible,
00:04:26but slow and expensive. A lot of AI product work today, I've found, is deciding when to let the agent cook,
00:04:35when to let the code do the work, and when to let the agent write the code that does the work.
00:04:41Vercel sandboxes has really been that bridge for us. It's the place where we can give AI a room to
00:04:46explore. Because AI can easily write code, it just needs a place to run it cheaply,
00:04:51repeatedly, and it needs to know that the output is going to be valid. So, today, I want to walk you
00:04:57through three different production workflows, all of them used for Vercel sandboxes, but for completely
00:05:02different reasons. The first one is the little purple node. It's called actions. It's a way for users to
00:05:09run media operations on their assets in Flora. The second one is our Flora MCP server. It allows agents
00:05:16to compose Flora capabilities themselves with the whole SDK. And the third one is timeline editor.
00:05:23It's a heavyweight media workstation runtime that needs low latency and statefulness. So, three completely
00:05:30different reasons to use Vercel sandboxes, but really gives us that balance of the polished versus the
00:05:35squi -- or the predictable versus the squishy. First up is actions. While I talk, we'll have a little Ken
00:05:42Burns effect being built here. The purple nodes are going to do some media operations. This was built by
00:05:47our engineer Cameron Yi, who's in the audience today. Using actions, users can run custom media
00:05:53transformations on assets in their canvas. It runs against real Flora assets. You can see the images go in.
00:05:59There's more images that come out. And the result that comes back is normal product state. Assets on the
00:06:05canvas, metadata, errors, previews. And these aren't toy transformations. The code running inside has the
00:06:12real media stack. It has image magic, the full node and Python ecosystems. So, an action can, you know,
00:06:18transcode videos, pull frames, generate a mask, composite layers, re-encode to different codecs,
00:06:24everything you need for a real professional media pipeline. We started out with this feature with
00:06:30just like a small curated set of actions to make sure that it got it right every time. But to do that,
00:06:36we developed an internal agent that could write them on the fly, and we found that this was just too
00:06:41powerful to not give customers. So, I'm really excited. We're going to be releasing custom actions to users
00:06:46later this week. That lets users use our on-canvas agent called Fauna to generate actions for themselves,
00:06:54run those repeatable media transformations that you can use again and again, share them with your team,
00:06:59and package them into techniques, those repeatable workflows that allows you to just keep composing
00:07:04more and more of your media workflows. The point isn't that it's running code and generation by itself. It's that the arbitrary code
00:07:12execution just becomes another step in the flow of the node canvas and the creative workflow.
00:07:19Here's a little look under the hood. So, it starts with the Flora asset. And then we spin up a sandbox
00:07:26from a snapshot. We download the inputs. We create that trusted harness. We lock down the network. So,
00:07:32importantly, actions don't have access to any of the network. They can't get into too much trouble.
00:07:37And then we run that untrusted code on behalf of the users. This is the key part, though. After the code gets
00:07:43run, we validate that output, and the product gets to decide what's valid to get back into the product as a Flora
00:07:51asset. So, by the time it gets back to the canvas, it's a normal asset. The sandbox gives us isolation,
00:07:58but the product gets to decide what API the sandbox can interact with. And that's the difference between just, like,
00:08:04running a script somewhere and becoming a real Flora feature. In terms of the canvas and other features
00:08:09like techniques, these actions are just another step in the process. Next up, I want to show you our MCP
00:08:16server. While I talk, Claude is going to build us a little ad campaign for Kiwis. Our MCP lets you
00:08:24access all of Flora right from your own AI, whether it's in Claude or Codex or Hermes. For our first MCP
00:08:31server, we explored many different architectures. I went back and looked at the original PRD and we actually
00:08:37had the architecture that we landed on originally. But first, we ended up building the MCP server with
00:08:45one tool per capability. Every time we added a new tool, we just added it to the server. It was built by
00:08:50engineers Danny Aziz and Alec Harmon, who are also here. The first approach was super naive. But as anyone who works with
00:08:58context, you know that it would start to eat up the context window. We got up to 18 tools, and we noticed
00:09:07that the agent's recall was starting to go down. It wasn't picking up on the tools that it needed.
00:09:12It wasn't -- you know, it couldn't figure out which tools to use for the right job.
00:09:16So, we dropped the 18 tools and we switched to a two-tool model. There we go. We got some Kiwis.
00:09:21Search docs and execute. And now the agent isn't just calling one tool per turn. It's writing TypeScript
00:09:27against pre-authenticated Flora SDK client and running that code in the sandbox. Because we found -- I
00:09:35saw this when I was building the demo. Like, the agent's just really effective at writing parallelizable,
00:09:40organized code, and expressing that intent through code and running in a sandbox, rather than trying to do it
00:09:47one per turn. Again, a little look under the hood. Kind of the same format. This time, the user intent
00:09:55comes from outside Flora. The agent's writing the code. And then we spin up the sandbox. One thing
00:10:01that I've found and all the engineers working with it is that Vercel sandbox is just incredibly easy to
00:10:07work with from a developer experience. You can get something working in literally seconds from your dev
00:10:11environment because it's not tied to your deployments or anything like that. So, we spin up the snapshot.
00:10:17It takes a couple hundred milliseconds. We create a trusted harness again. This time, you can only
00:10:22talk to the Flora API and nothing else. We run that untrusted code. In this case, the agent was running
00:10:28the code. And then we validate the outputs again. So, we make sure that the output is not only what the
00:10:35agent expects, but also that it's a normal Flora asset that it can be used because all of this is just
00:10:40working as if it were on the Flora canvas like the other features. And so, notice the primitive. It's
00:10:45the same Vercel sandbox we use for actions. We have a squishy input. The agent gives us some arbitrary
00:10:51code, but we make it reliable by that boundary before it goes back out into the product.
00:10:57The third workflow I want to share with you is the real sleeper, I think. It's a timeline editor or a
00:11:02video editor. It was built by Eni Aji Biomo. We've been prototyping video editors for months now.
00:11:09It's something we knew that we were going to have to build at Flora, also coming out later this week.
00:11:13There are many very smart people working on this problem. I know from my time at Frame.io, it's a
00:11:18famously hard problem to get true video editing capability onto the browser. There's lots of cool
00:11:23teams working on fully featured browser-only editors all the way to agentic video editors.
00:11:29We knew we needed something like that in Flora, but our challenge was we needed a good solution that worked
00:11:34equally well both on the browser and in the server. If you go back to techniques, this has to work just
00:11:40like any other step in the workflow. We don't know where it's going to run. The problem was giving a
00:11:45professional video export of really fast reproducible runtime that we could do on the client or the
00:11:53server. The two unlocks for us here were Remotion and Vercel sandboxes. The snapshot becomes a little
00:12:01render workstation that can be called quickly and headlessly. We get the browser preview from Remotion,
00:12:06and then Remotion gives us a way to compose that full-frame render on the server in a really high-quality
00:12:14export. And again, it just becomes another step in the creative process on the canvas.
00:12:21Again, let's take a look under the hood. This time, the untrusted isn't really untrusted. We actually do
00:12:25trust it. So, we're using sandboxes for a little different reason. We get an intent from Remotion,
00:12:30we spin up the snapshot, we download the output, the input, we create that trusted harness so it can't
00:12:35get into too much trouble. In this case, it only has access to, like, dependencies and that type of thing.
00:12:40Then we render the composition using the Remotion export. We validate the outputs to make sure that
00:12:46it exactly matches what we can show back on the canvas, and we get it back in as a floor asset.
00:12:52You might say you could do this in other serverless primitives. There's nothing special about sandboxes
00:12:56here, but there's actually a few tailwinds that we found. One of them is that the sandboxes can be
00:13:02stateful, so we can actually pick back up a render where we left off if we want to. That helps with
00:13:07latency. And also, you get some asynchronous benefits. You don't have to manage the request life cycle.
00:13:11It just becomes another job in the queue. That's really helpful as we grow and scale this feature
00:13:17and get more and more complex video export workflows happening. So, across these workflows, the sandbox
00:13:27was never meant to be enough by itself. The shape that we kept discovering looked like this. We had this
00:13:34user intent. We spin up the sandbox in the snapshot in a few hundred milliseconds. We create that trusted
00:13:41harness. We're really specific about the network access that it has. We validate the outputs to make
00:13:47sure that the product, the deterministic side, is going to expect what comes out of it, and then we
00:13:51give it back out as product output. That unpredictability of untrusted code or user intent needs a place to
00:13:59run that squishy element. But we also needed to be able to give it a predictable boundary so it can get
00:14:04back into the product. And Vercel sandboxes was really the bridge for us that allowed us to do that.
00:14:10Squishy is really easy to demo these days. With everything going on in the industry,
00:14:15it's really easy to spin up a really quick demo that, you know, is very mesmerizing.
00:14:22But creative professionals need to be able to trust their tool. Like I said, they have a really exquisite
00:14:27eye. They need very high quality and consistent output. Vercel sandboxes gave us a way to combine
00:14:34those two needs into the polish that we need. They help us turn the arbitrary into the reliable,
00:14:40and they've been crucial in our evolution to create an app that is both squishy and predictable in a way
00:14:47that's polished enough for creative professionals. So, I encourage you to give Vercel sandboxes a spin.
00:14:53We've sure loved them, and they've been crucial in building what we're working on at Flora,
00:14:57and I'm sure we'll keep using them for ever more interesting things. And check out Flora,
00:15:02or come see us for a demo. We'll be around if you want to integrate AI into your creative
00:15:07production worthless. Thank you so much.