Transcript
00:00:00Boris Cherny, who is the creator of Claude Code, sat down for an interview.
00:00:04And interestingly, he said that even though we have capable models like Fable 5 now,
00:00:08people are still using them the same way they used Sonnet 3.5.
00:00:12Which is why he says you're carrying dead weight into Claude and you don't even realize it.
00:00:16So if you can't get the latest models working right, it isn't the model that's wrong,
00:00:20it's the way you're using it.
00:00:21Now these might sound like something you do once and forget about,
00:00:24but they're something you should be using every time a new model comes out,
00:00:28whether that's from Anthropic or OpenAI or anyone else.
00:00:31In the interview, Boris goes into detail about how his team works today.
00:00:34And honestly, who understands these workflows better than the person who created them in the first place?
00:00:39We've started applying the same approach ourselves and we've found it really helpful.
00:00:43Now if you're new here, we're a software company and this is AI Labs,
00:00:47where we show you how to optimize a business with AI using proven methods from our own team.
00:00:52And in this video, we're going over which of his tips to actually use and how to apply them.
00:00:56The first thing Boris says in his talk is that you should delete your setup every time a new model comes out.
00:01:03And when Opus 5 shipped, Anthropic did exactly that to Claude code and deleted about 80% of its system prompt.
00:01:09That sounds reckless right up until you look at what a setup actually costs you.
00:01:13Your setup is everything you've added to steer Claude, which include your Claude.md,
00:01:18all the skills you create, your hook setup and more.
00:01:20All of these are injected into the context window when you launch Claude and stay there the whole time.
00:01:25Now we've talked about our second brain before, which is a folder that contains all our research,
00:01:30strategies and more in one place and the whole team works out of it.
00:01:33There's the usage command.
00:01:35It shows how much of your 5-hour window you've burned through and it breaks down where that's coming from,
00:01:39which of your skills are eating the most and which habits are driving it.
00:01:43In our case, it was workflows that spin up sub-agents running several sessions at once
00:01:47and leaving sessions going without compacting them.
00:01:50The reason deleting works is that most of what's in your setup was never about your project.
00:01:54It was for correcting things the model used to get wrong.
00:01:57We've covered a lot of frameworks that use slash commands to control Claude on this channel,
00:02:01like BMAD, GSD and more, which fires the commands automatically at set points.
00:02:06All handed the model rigid steps so it couldn't wander off because models needed that much handholding.
00:02:12On a model this capable, the handholding isn't helping anymore.
00:02:15It's the thing that's getting in the model's way.
00:02:17So to find out whether your own setup is earning its place,
00:02:20you strip it all out and watch how the model does without it.
00:02:23Run the context command in an empty session before you've typed a single word
00:02:27and it already shows 3% of the million token window used up.
00:02:31That's the system prompt and the built-in tools loading by default,
00:02:34already shaping how Claude behaves before you've asked for anything.
00:02:37That 3% is Claude's own and you can't touch it.
00:02:40Everything on top of it is yours.
00:02:41But the better option is Safe Mode, which turns off all of your clutter,
00:02:45the Claude.md files, the skills, the plugins, everything you've installed
00:02:49and leaves you running on nothing but the base system prompt.
00:02:52Run the context command there and nothing of yours is loaded.
00:02:55Then you run the things you normally run and watch how the model does.
00:02:58If it's fine without your setup, that's your signal to start cutting.
00:03:02And this isn't something you do once.
00:03:03Every single lab ships a new model after a few months.
00:03:06So if your project isn't brand new, its setup is tuned for a model
00:03:10which is two generations behind.
00:03:11The rule for what survives is one question.
00:03:14Would Claude have worked this out on its own?
00:03:16If it would have, delete the line.
00:03:18You see this most with the command that starts your project.
00:03:20Every time you made a new landing page and ran the init command in that folder,
00:03:24the Claude.md ended up with a line that says use npm run dev to start it.
00:03:29You wrote that in because the previous models didn't know how your page starts
00:03:33and wouldn't run it unless you asked.
00:03:35That's over now because it works that out and runs it on its own with no Claude.md at all.
00:03:40But anything where your setup differs from what Claude already knows, that stays in the Claude.md.
00:03:45For example, you want it testing through the Versal's agent browser instead of the one it reaches for by default.
00:03:51It can't guess that or a test setup of your own or a service that has to be running first.
00:03:55So you need to add that in.
00:03:57You don't have to do this by hand either.
00:03:58Anthropic shipped the doctor command and it trims by exactly that rule.
00:04:02It cuts what Claude could work out from your project and keeps the areas where it made mistakes,
00:04:07the reasoning and anything that's different from what a tool does by default.
00:04:11We run it on that second brain every so often, especially after a new model release.
00:04:15It goes through everything you've set up and hands back a health report
00:04:18with a verdict on each piece and a list of things it wants to change.
00:04:21Nothing happens until you say so, so you can work through them one by one.
00:04:25One thing that makes rebuilding easier is that installing other people's skills isn't the risk it used to be.
00:04:30This is because Boris says that the models are less likely to get affected by prompt injections now.
00:04:36The obvious worry with any skill you pull in is what's hidden inside it.
00:04:40We installed a skill off GitHub and it had instructions inside it telling our project to send data out to a URL.
00:04:46Claude didn't follow them and it didn't quietly skip them either.
00:04:49It told us in the chat that the skill had asked and that it had skipped it because the skill didn't look trustworthy.
00:04:54Boris's line is that hidden instructions like that can't get through anymore, so you can install what you want without treating it as a risk.
00:05:01But before we move on to the next tip, it would be great if you subscribe to the channel and hit the hype button.
00:05:07This small gesture of support goes a long way for us.
00:05:10Once you've stripped all of that out, something else has to tell the agent when it's actually done.
00:05:15And Boris says this is the single most important thing people get wrong.
00:05:18That thing is an eval, which is basically a fixed check that decides whether the work is done and it has to be strict enough to fail.
00:05:25Saying run it and see if it looks better is just guessing with extra steps.
00:05:29A check you can't fail isn't a check.
00:05:31This idea isn't new.
00:05:33You write the check down before you build the thing, so there's never an argument later about whether it's finished.
00:05:38And it doesn't have to be a test, it just has to be something that can come back as a no.
00:05:43Some examples of these would be checks for seeing if a page fits on a phone screen, ensuring that wrong password doesn't get you in and the text has no em dashes.
00:05:51What the check looks like depends entirely on the job.
00:05:54Anthropic wrote up two of their own big rewrites, and between them they used three different kinds of tests.
00:05:59Bun is a framework Claude code is built upon.
00:06:02Its rewrite had a full set of tests that all had to pass, and it had the eight reviewer agents, each looking for one kind of problem instead of everything at once.
00:06:10The last one was a parity check, which ran every command through both the old version and the new one, and checked if the answers came back the same.
00:06:18The example worth copying is Boris's own.
00:06:20He gave Claude an empty codebase and told it to rewrite the Claude desktop app in Swift, run the old one inside a virtual Mac, screenshot it, compare that against its own version pixel by pixel, and not stop until it was done.
00:06:33That ran for over two weeks, and that comparison is the only reason it could keep going that long.
00:06:37Without something to tell itself it isn't finished, an agent stalls out in an hour, and you're back to watching it.
00:06:43The checking you used to write into the prompt yourself, it now does on its own.
00:06:47Whenever we build an app now, we give it a simple prompt telling it what we want, and Claude works through the parts it needs to build.
00:06:54You used to have to put a line in the prompt telling it to check how the page looked in a browser.
00:06:58Now it launches headless Chrome Sessions by default, which is a browser that runs with no window on screen, and it takes screenshots of different areas so it can see whether anything's overlapping.
00:07:08It also goes back over the code and flags anything that breaks a rule, so it catches what would have broken the app just by reading it.
00:07:15There is a limit though.
00:07:16All of that only catches what breaks loudly, but not the other issues that have to do more with just breaking the app.
00:07:21Only you can say what done and correct means for your task.
00:07:24And your checks expire too.
00:07:26Boris says they last two or three model generations, then everything starts passing and you need to throw the set out and write different ones from wherever it's struggling now.
00:07:34But before we move forward, let's have a word by our sponsor.
00:07:37If you've ever made an AI video, you know the real pain isn't the generating, it's jumping between one tool for the script, another for characters, another for scenes, storyboards and editing, and then watching your character change face from shot to shot.
00:07:50So we brought the whole thing into Buzzy Canvas, one visual workspace where the entire workflow lives.
00:07:56You start with a simple idea and Buzzy Canvas helps expand it into a structured video workflow from script and character concept art to scene planning or storyboard frames or local edits and final video generation, all inside one visual workspace instead of jumping between multiple AI tools.
00:08:13Because every step is visual, you actually control it, adjusting lighting, camera angles, and individual frames, while character and three-view references keep everyone consistent across a long video.
00:08:24Powered by first-look models like Seedance 2.5, Buzzy Canvas isn't just another AI video generator, it's a full production pipeline for films, ads, music videos, and more.
00:08:34Check out Buzzy Canvas using the link in the description.
00:08:37Knowing what done looks like also changes how much you have to write in the first place.
00:08:42We used to give it long prompts with each and everything in them, because if you left a gap, it went off the rails.
00:08:47Nobody trusted the model with more, and the prompts matched.
00:08:50Now you say what you want, and it works out the route.
00:08:53If you tell it how to do something instead of what you want, you'll never get better than your own approach.
00:08:58Boris says experienced engineers are the worst at this, because they're trying to get the model to do the task exactly the way they'd have done it.
00:09:05What works now is higher level, so you only need to describe the task, along with the limits it has to stay inside, and what finished looks like, then Claude will figure out the rest.
00:09:14The other half of the same habit is aiming higher.
00:09:16Give it tasks slightly harder than you think it can handle, because you only write all that detail when you've quietly assumed it can't take the whole thing, and that's actually out of date.
00:09:26And there's a second version of this that shows up the moment you try to get a design out of it.
00:09:30You take an inspiration image, hand it to your agent, and tell it to build a UI inspired by that design.
00:09:36What comes back is a near copy, so instead of handing over the image, describe it.
00:09:39Say the style you want and the look you're going for, instead of giving it the answer and it works out the approach itself.
00:09:45And it's not only design.
00:09:46The same thing shows up when you're using it to come up with ideas.
00:09:49You give it one that worked for you, ask for more like it, and everything that comes back looks exactly like the one you gave it.
00:09:56Anything concrete you hand it, it treats as something to copy instead of something to learn from.
00:10:00So describe it instead, and it has to work its way there itself.
00:10:04Now the rule at the start actually came from Boris, and it was to delete anything Claude can figure out on its own.
00:10:10But that's only half of it, because he also says every team should still be writing the Claude.mds and skills that let an agent work on their project without being told anything extra.
00:10:19So there's still stuff worth adding, and when the model keeps getting something wrong, the question is what kind of gap you're looking at.
00:10:26There are four answers, and they go from the cheapest to the most expensive.
00:10:29The cheapest one is that the instruction just isn't clear, and to fix that you just change the prompt.
00:10:34That's most of them, and it costs you nothing.
00:10:37The next one is something the model needs to know every single time, and that's what goes in Claude.md.
00:10:42You need to keep this lightweight.
00:10:43This could be a quick line on what the project's for, and then the things that would trip an agent up.
00:10:48Anything it can see for itself is just dead weight.
00:10:51Then there's the thing you're explaining for the fifth time, and that's when you make it a skill.
00:10:55Skills work best as lightweight guides that let Claude find information when it needs it,
00:10:59so you don't want to lock them down with rules unless the area actually calls for it.
00:11:03They're worth writing when they hold something that's particular to you or your team,
00:11:07because Claude already knows the generic best practices, so writing those down is just waste.
00:11:12And if a skill gets long, you split it across files, so only the part it needs gets loaded.
00:11:17The difference between Claude.md and skills is when they load.
00:11:20Your Claude.md gets loaded on every session start, but a skill only gets read when Claude actually needs it.
00:11:27So the short, always true stuff goes in Claude.md,
00:11:30and anything long or occasional goes in a skill that Claude.md tells it to go read.
00:11:35In ours, that's a short Claude.md at the top, with most of the detail in skills next to it,
00:11:39like the instructions for how to verify our work, because Claude only needs those when it's checking
00:11:44something. Same instruction, except one costs you every turn, and the other costs nothing until
00:11:49it's used. And the most expensive one is when the information somewhere Claude can't get to at all.
00:11:54That's the only one where you build it a way in, and this is where you use an MCP server,
00:11:58which basically connects it to whatever's holding that information. This one isn't for when it
00:12:03misunderstood you, and it's not for when you're repeating yourself. It's for when there's genuinely
00:12:07no way for it to know. That's everything you can do with a single prompt. When a job is too big for
00:12:12one, you split it across several agents, and there's more than one way to do that. You can hand the parts
00:12:17out yourself with sub-agents and decide what each one gets. The way Boris brings up is the dynamic
00:12:22workflow, and that's for the jobs that need hundreds of agents rather than a handful. You say use a
00:12:27workflow, and Claude plans the whole thing out itself, splitting the job up, sending a wave of
00:12:32agents out at once, waiting for all of them to report back, then sending out the next wave based on what
00:12:37they found. You don't plan any of it. We don't recommend actually using them though. The biggest
00:12:42example is that bun rewrite, where a million lines of code got rebuilt in under two weeks off a single
00:12:48prompt, with them steering it along the way. What nobody mentions is that it cost about $165,000 at API
00:12:55pricing, and once it was all added in, 19 things were broken that none of their existing checks had
00:13:00caught. And the money is the smaller problem here. When one agent goes wrong, you lose an hour. And when
00:13:05hundreds of them have been running for two weeks, you lose the whole two weeks. You get no result at the
00:13:10end of it, and you can't tell which one started it. That's too much to risk on anything you're actually
00:13:14trying to ship. What we use instead is Routines, and they're the ones you'll actually use day to day.
00:13:19They're recurring jobs that run in the cloud, so they keep going after you close your laptop.
00:13:24Anthropic runs 20 to 30 a day across their apps, and each one is a single sentence, which is something
00:13:30like "clean up all the stuff you don't need", including the stuff left over from features you
00:13:34already removed, because the agent still reads through it every time it goes looking around your
00:13:39project. Write tests for the parts that don't have any, so you're the one who finds out when
00:13:43something breaks instead of a user. Find the places where the same thing got built twice and merge them,
00:13:49because if you're working on a large project or second brain system, the models tend to forget
00:13:53that things already exist. Now all the skills and workflows and everything else we build and show you
00:13:58in all our videos can be found in AI Labs Pro, which is our community, so if you've found value in
00:14:04what we do and want to support the channel, this is the best way to do it. The link's in the description.
00:14:09That brings us to the end of this video. If you'd like to support the channel and help us keep making
00:14:14videos like this, you can do so by using the super thanks button below. As always,
00:14:18thank you for watching, and I'll see you in the next one.
00:14:21Thank you.
Community Posts
No posts yet. Be the first to write about this video!
Write about this video