Transcript
00:00:00if you work with good models like opus and gpt 5.6 you'll usually get good output but these models
00:00:06also burn through a lot of tokens and it gets even worse with the high-end ones like gpt astra
00:00:11and fable 5.1 because you'll hit your usage limit much faster than you would with other models
00:00:16they're also really slow so you end up waiting a long time for a single task to finish to fix this
00:00:21exact problem there's a tool that's been trending on github lately called graft it fixes the way
00:00:26these agents handle projects by default not only does it save you tokens but it also makes the
00:00:31agent much faster than when it works on its own and it has a really interesting way of working that
00:00:36fixes the core reason this problem happens in the first place it's not like there aren't already
00:00:40tools built to solve this issue but they all have a huge gap and that's exactly what this tool fixes
00:00:46if this is your first time here we're a software company and this is our channel ai labs in this
00:00:51video we're going to show you exactly how graft makes using these models cheaper but before we get
00:00:56into graft you need to understand why your agent uses so many tokens before it changes anything in
00:01:02your app the default method that agents like claude code and codex use to find files has a problem
00:01:07when you ask the agent to add anything to the app you're working on it first needs to find the parts
00:01:12of your app where it needs to write that change to find those parts it uses different commands in the
00:01:17terminal that search for words related to the thing you want to add models don't usually find the thing
00:01:23they're looking for on the first try they have to use multiple tools to narrow down what exactly
00:01:28they're searching for and every time the model needs to decide which tool to use next the agent
00:01:32sends the model your whole conversation so far including the responses from the tools it has
00:01:37already used the model then reads all that to decide its next step so it reaches the right file
00:01:42after multiple turns and each of those turns costs you usage for example when you ask the agent to make
00:01:47a button green it first needs to find which file even contains that button's code so it searches for
00:01:53the file containing that button and the results go to the model along with your message then the agent
00:01:58uses another tool to read the specific lines of code from that file and only after that does it actually
00:02:04make the change that back and forth is how every change you ask for gets made since all of your
00:02:08messages and tool results are sent again and again the context window keeps growing not only is it costing
00:02:14you context but it's also slowing the model down because this way the model is using multiple turns
00:02:19to narrow down the required file so every new search adds more information and the model has to use
00:02:25tokens to decide what to do with it that's actually one of the reasons why you hit your usage limit when
00:02:30you're working with agents on a lot of tasks in one session now hitting the usage limit is just one
00:02:36consequence you also get worse quality outputs because there is so much in the context window that
00:02:41the agent can't focus on one thing at a time now since this is a known problem there are tools out
00:02:46there already that try to solve it a common approach that these tools use is to turn sections of your
00:02:51code into numbers called vectors so that the model can compare them easily when you ask a question the
00:02:56search tool turns the question into vectors too then finds the sections with the closest matches
00:03:02that's called vector search which basically means finding information by how similar its meaning
00:03:07is to what you asked but similarity alone doesn't tell you how the parts of your app connect for
00:03:12example the code for creating an account and deleting an account can both match a question about
00:03:17accounts but they do two opposite things so picking the wrong one would be really costly
00:03:21that's why this search isn't as effective and most of the coding agents don't use it at all
00:03:26but before we move on to the tool it would be great if you subscribe to the channel and hit the hype
00:03:30button this small gesture of support goes a long way for us now graft is actually a terminal command
00:03:36that you install on your computer and it's built to change the default method your coding agent uses
00:03:41so that it doesn't have to search its way through your project before every change it does that by
00:03:46building a knowledge graph which is basically a map of all the parts in your project and it shows how
00:03:51the different parts connect the agent uses that map to find the code it needs and check which other parts
00:03:57depend on it it's actually free and open source and the main part of it is that it doesn't use models
00:04:02with a separate api key so it works on your usual subscription there's also an optional step that uses
00:04:07a model to write plain pages about your app which explain what each part of the code does and how the
00:04:12parts fit together you don't really need it because the map already tells the agent what connects to
00:04:17what this step just adds an explanation of what the code does so with those pages the agent gets to
00:04:22know what each part does instead of opening every file when the team behind graft ran this tool they
00:04:28found that it makes agents four times cheaper in terms of token usage and that's their best case in
00:04:33their own benchmark across 162 runs tasks took 60 less time on average the agent used its tools 46 fewer
00:04:42times and 42 percent fewer tokens and the cost came out 32 percent lower on average and since the saving
00:04:48comes from all the searching the agent doesn't have to do anymore it pays off on a bigger project because
00:04:53on a small project there isn't much searching to save in the first place graft works with claude code
00:04:58and codex as well as other coding agents that use terminal commands or mcp but the map it builds
00:05:04isn't like the vector search we talked about earlier because graft doesn't turn your code into numbers and
00:05:09match by similarity it reads the code and writes down which part actually uses which so when you
00:05:14change one part the agent can see exactly what else that change could break and when your code changes
00:05:20it just updates the parts that changed instead of building everything again so the map stays current
00:05:25on its own and the agent is never working off an old version of your project but first let's have a
00:05:30word by our sponsor hydra so we were adding an ai generation feature to our app which used to mean
00:05:35wrangling models infrastructure and job cues yourself that's where hydra's api comes in they've built the
00:05:41entire runtime and api behind their models so instead of standing up all that infrastructure yourself that
00:05:46whole layer is already handled for you we made one real call from our code and instead of an asset just
00:05:51appearing we got back a real job we could track and we watched it move from queued to processing to
00:05:56complete before the media came back that status readout is the part that matters because it's how you know
00:06:02this can handle real production work it takes only a bit of code to drop into your own app and there's
00:06:07a cli and sdk too so you can work right from your terminal or editor this is the kind of thing a solo
00:06:13developer just couldn't build before try hydra free at hydra.com and use our code for 50 off your
00:06:19first month the link and code are in the description below before we install anything you need to see
00:06:24how graft actually makes that map when you use it to build the map graft reads the code on your
00:06:29computer and writes down every part it finds with its name and where it is in the project
00:06:34then it writes down which parts are connected to which each part on that map is called a node and
00:06:39each connection between two parts is called an edge so when the agent asks what uses a particular part
00:06:44of your app graft follows those edges and gives it the code that's connected to what it was searching
00:06:49for so that it can see if that change affects anything else which is exactly what vector search
00:06:53couldn't do graft saves that map on your computer as a json file the reason it saves it as a json is
00:06:59because that format lets you write something down in a proper structure with details attached to it it
00:07:04also has a viewer that opens in your browser where you can see the map and explore the connections when
00:07:10graft is connected to claude code it gives the model the instructions for using the map at the start of
00:07:15every session then every time you send a prompt graft checks the words in your message against the map and
00:07:20attaches up to three matching locations to it so the model already knows which files and lines to read
00:07:26before it uses a single tool the code itself only comes into the context when the agent reads those lines
00:07:32so the model gets to the right file in fewer turns and a lot less gets added to the context window
00:07:37so it's faster and it uses up less of your limit there's also an mcp option and the difference is who
00:07:43starts the lookup with the setup we just covered graft guesses from your prompt and attaches those locations
00:07:48to every message whether the agent needed them or not with the mcp nothing gets attached to your
00:07:53prompts and the agent asks graft only when it actually needs something in graft's own tests the
00:07:58mcp version got a few more answers right than the cli version and the cli version was faster when you
00:08:04install it you actually get both and as you change your app graft keeps the map updated on its own before
00:08:10it answers a question it checks whether the code has changed since the map was built and if it has it
00:08:15updates the map first without using the model now to install graft you need to go to their site which
00:08:20we'll link in the description and from there you can either copy the install command or copy the setup
00:08:25prompt for whichever coding agent you're using and paste that straight into the agent that setup prompt
00:08:30has all the commands the agent needs to install graft and set it up in your project and if you'd rather
00:08:35do that part yourself you can do so by copying the install command from the site and running it in the
00:08:40terminal from any folder once it's installed the cli is ready to use in order to use it in a project you
00:08:46need to set graft up in that project by running the init command this command has to run in the terminal
00:08:52inside the folder you're working in because it adds some instructions for that project into the folder
00:08:57which would be lost if you ran it anywhere else when you run it it asks which coding agent you're using
00:09:02because each agent needs its own setup and since we were using claude code we selected it and
00:09:07proceeded with the installation once that's done you'll see a graph skill in your project folder
00:09:11which tells the agent how to use graft and which commands it has it also installs hooks and if you
00:09:16don't know what a hook is it's a small script that runs on its own at a set point these hooks enforce
00:09:22the agent to follow the graft workflow there are multiple hooks that get installed one gives the
00:09:26model instructions for using the map when a session starts another attaches the matching locations to each
00:09:31prompt you send and the last runs after claude edits a file so the map stays up to date then
00:09:37if you're setting this up in a project you've already been working on you need to run the graft build
00:09:41command in that same folder so that graft goes through all the code that's already there and
00:09:46builds the map from it and if you're starting in an empty folder there's nothing to map yet so the
00:09:50skill gets clawed to build it once there are files after that you just run clawed code normally and the
00:09:56map viewer shows the nodes and edges graft created in an empty folder the map starts at zero nodes and
00:10:01graft adds them as the files get created now we tested graft by building a booking and scheduling
00:10:06app similar to calendly but for independent providers with fable 5.1 since this model consumes tokens
00:10:13very quickly we didn't want it spending effort on the wrong tasks that's why we took a series of
00:10:18steps before working with it we wrote a prd first which is the document that lays out what the app is
00:10:23supposed to do so it knows every feature the app needs we also added a claude.md file which contains
00:10:29instructions tailored for this model to let it run for long turns without straying from the goal that
00:10:34claude.md file is what we uploaded as a template in our community ai labs pro so we use that same
00:10:40template we first asked it to fill in the gaps in that claude.md from the prd because the file was only
00:10:46a template and if you don't have a prd you can just tell it in your prompt what you're building instead
00:10:51once the claude.md was updated we switched the model to fable 5.1 and gave it the prompt to build
00:10:57the booking app along with which tools we wanted the app built with and you need to tell this model
00:11:02explicitly that it's working on its own which we covered in our previous fable 5.1 video so we told
00:11:07it not to stop and ask for permission with graft the build took 39 minutes and used around 31 of the
00:11:14context window without graft the same build took 47 minutes and used around 35 percent and both app
00:11:20practically had the same functionality so on that first build the difference was small because the
00:11:25graph wasn't built at this point but it got bigger once we started making changes because by then graft
00:11:30already had a map of the whole project to search a complete revamp of the landing page took less than
00:11:35two minutes and without graft the same change would have taken a lot more time than that after the change
00:11:40graft updated the map with the new files and it showed its own estimate of the tokens it saved on
00:11:46that turn and there's one last thing you need to know when you're working on an actual project it
00:11:50doesn't always just include code but it also contains other files that give agents context on what's being
00:11:56built these include the prd all your area specific files the learnings.md file and many others but
00:12:03graft only maps code so it doesn't map your prd or the notes which means that the agent uses the usual
00:12:09default method when it has to read those and aside from that a lot of people like us use claude code
00:12:14for a lot of non-coding tasks as well so in order to make the tool cater to those as well we modified
00:12:20it a bit so that we can use it in our actual projects where we have multiple plan files and we've added
00:12:25that version in ai labs pro which is our community so if you find value in what we do and want to
00:12:31support the channel this is the best way to do it the links in the description that brings us to the end of
00:12:36this video if you'd like to support the channel and help us keep making videos like this you can do so by
00:12:41using the super thanks button below as always thank you for watching and i'll see you in the next one
Community Posts
No posts yet. Be the first to write about this video!
Write about this video