How To Make Claude Code Tokens 20x CHEAPER (& 4 More Usage Hacks)
CChase AI
Computing/SoftwareSmall Business/StartupsInternet Technology
Transcript
00:00:00You might be paying 20 times more for your Cloud Code tokens than you need to and not even know it.
00:00:05Understanding how to get the most out of your Cloud Code tokens is one of the most important
00:00:08skills you can master, and it goes well beyond simply putting a line in your Cloud.md file
00:00:14saying, be brief. Because if you don't understand things like how prompt caching even work,
00:00:19you can be paying way more than you need to. So in this video, I'm going to give you five
00:00:23different ways to better manage your Cloud tokens so you can actually get the most out
00:00:27of these top tier models. Now tip number one is the most important. This will save you more money
00:00:31and usage than every other tip combined by far, and that is understanding how prompt caching actually
00:00:37works. Now to understand prompt caching, you need to understand how tokens work at all. So we're going
00:00:43to do a very quick review, like 60 seconds, so we're all on the same page. Now tokens are the currency of
00:00:47large language models, and for all intents and purposes, this is somewhat simplified, is every
00:00:52word equals one token. So when you, the user, on your first message to Fable say, how are you doing today,
00:01:00you have sent five input tokens. Now when Fable responds to you and says, I'm doing great, thanks,
00:01:09it has given you four output tokens. These are priced differently. In fact, output tokens, by and large,
00:01:16cost five times what an input token costs. Now these five input tokens and these four output tokens now
00:01:23accumulate inside the context window. If tokens are the currency, then the context window is our budget.
00:01:31Opus, Fable, Sonnet, they all have a budget of one million tokens, so at this point we have used
00:01:37nine of our one million, not a whole lot. Now what gets interesting is when we send follow-on messages
00:01:43after the first one. So in our second message, I, the user, say, build me an app, no mistakes,
00:01:50six input tokens, right? I'm just going to send Anthropic and Fable just six input tokens. Well,
00:01:56not quite. What I'm actually going to be sending it is everything. I'm going to be sending Anthropic
00:02:04this entire conversation up until that point. So I'm not really sending six input tokens, I'm in fact
00:02:11sending 16 input tokens because I'm going to send them the five from here and the four from here because
00:02:16it's going to get the entire message so that it can understand the context. And for every single
00:02:20follow-on message that will be the case, I am always sending the entirety of the conversation
00:02:26before that latest message. And you can quickly see how this compounds very, very quickly to where
00:02:32you're talking about 5,000, 10,000, 100,000 input tokens being sent every single message. And you are
00:02:37paying for that. Now, if that is the case, then why don't we just run through all of our usage
00:02:43immediately? Why doesn't everyone pay a billion dollars every time they try to use these AI systems?
00:02:47Because obviously we're sending message after message. So there's a whole lot of like
00:02:51compounded messaging being sent to the servers. Well, the solution here is the cash system. So yes,
00:02:57when I send message number two over here, I am in fact sending all of this. But what has been created
00:03:05at this point is the message cash. And for our second message, that message cash are these first
00:03:12two messages. I want you to think of the message cash is simply a document that Claude has in front of
00:03:17it that has your entire conversation up until that point. So we got you talking to Claude right here,
00:03:23you are now sending message number two, which is build me an app, no mistakes. But we have the cash
00:03:30system. So this whole section, these first two messages are now sitting in this document. So what
00:03:36Claude and Anthropic is able to do is it is able to read that document, it's able to read the cash,
00:03:42take a look at the entire message history up until that point at a way cheaper rate than if you were
00:03:47just sending it all at once without the cash. And that's how it calculates the cost. And this cost
00:03:54is important because this cash system does not last forever. What you need to understand is that when
00:04:00you and Claude are talking to one another, and we have this cash document between us, which has all
00:04:06of our conversation history, which you can read very cheaply, it only holds on to that for one hour.
00:04:11So if you and Claude are having this discussion, you're having a back and forth over and over again,
00:04:16you know, just back, back, back, back, back, and then you walk away for an hour. And you have a
00:04:21document that is 500,000 tokens long, a 500,000 token conversation. After an hour, this disappears.
00:04:29This is gone. And so when you send a message, you know, 500,000 in one, well, guess what, you are now
00:04:38going to be charged for a full rate 500,000 token message, even if all you said was, "Hey, what's up?"
00:04:46Now understand when I say this cash only lasts for one hour, I mean an hour with no activity. So it's
00:04:52refreshed every single message. So if it's been 59 minutes since our last message and I hit it again,
00:04:56well, that one hour counter resets. Now, why is it so important? Well, it's the cost. I talked about
00:05:01in the intro, the difference between a cash read, AKA reading this whole history versus reading the 500,000
00:05:08message without a cash is quite literally a 20 X difference. And this is reflected in the price
00:05:16increasing documentation. So remember output tokens, what Claude is giving us, that never changes.
00:05:22For Fable, we're looking at $50 per million tokens and $25 when it comes to Opus. But the input tokens
00:05:27is where this whole conversation has been living. So we always talk about base input tokens of being
00:05:33$10 per million, but that's kind of a misnomer because in reality, we are always doing cash writes.
00:05:40So a one hour cash write, which is what you are doing on a subscription plan is in fact double the cost.
00:05:47So when we are writing to that cash for the first time for like that first message, it is $20 per
00:05:54million tokens. You'll notice a five minute cash right here, but that is really just for people who
00:05:59are on the API. Now compare that to the cash hits, AKA Anthropic is just reading that document that's in
00:06:05front of it, that it's been accumulating every hour and refreshes. It's $1, 20 times cheaper. That's
00:06:13a wild difference. And that's why this whole thing, this particular tip, understanding prompt caching and
00:06:18how the token system works is so important. There is nothing I'm going to talk about beyond this point
00:06:22that is going to give you any sort of, you know, efficiency gain close to this. Nothing.
00:06:29Now let's go back to our example of that 500,000 token conversation we are having
00:06:35with Claude code and we're about to send a follow on message. In scenario number one, we're going to
00:06:40imagine we have the cash system. So this 500k conversation history is cashed and I'm sending
00:06:46that new message. The way the pricing is going to work is it still has to read this whole conversation
00:06:51history. And that is at the $1 per million token rate. So it's going to cost me 50 cents on that
00:06:58next message for it to read everything plus the new message. So imagine that new message was a thousand
00:07:05tokens long. That thousand token follow-on message is not charged at $1 per million. It's charged at $20
00:07:14per million. So for a thousand tokens, that's what, like two cents or something like that? I don't know.
00:07:18My math is probably off, but point being the history is at $1. The new one's at $20 a month because this is cached.
00:07:26Now, the next time we send a message, same scenario, except this thousand tokens will now be part of that
00:07:32cash document. Sort of makes sense. Okay. That's scenario one. Scenario two, we have no cash. We waited an hour
00:07:40to send it. That's right to send it. Well, instead of this being 50 cents, we instead now are going to get
00:07:47charged at a cash read, I'm sorry, at a cash write rate, which remember is $20 per million. So what is
00:07:54this going to cost? Well, now this one message is costing us like 10 bucks. So we went from 50 cents to
00:08:0010 dollars just because we waited an hour. That's the like sort of consequences of not understanding
00:08:05this. Now, time isn't the only thing we have to think about when it comes to losing the cash. There
00:08:10are other things that will reset it completely. And this is straight from the Cloud Code documentation.
00:08:14If you switch your model, you know, you are on Fable and you switch to Opus and you're 500,000 tokens in,
00:08:20that is gone. Cash is reset. Effort level, going to change it. Fast mode, connecting or disconnecting
00:08:26an MCP server, plugins, denying tools, compacting the conversation, or just upgrading Cloud Code in
00:08:32general. Any of these things will reset the cash and your next message will be 20 times more expensive
00:08:38than it needs to be. So with all that being said, what can you actually do with this information?
00:08:42What do you do when you're in a position where you need to step away from a conversation
00:08:46that has a lot of important information in it, but you're going to be gone for more than an hour,
00:08:50or you stepped away for an hour plus and you're just coming back and realizing, oh, shoot, like
00:08:54I kind of didn't think about this ahead of time. Well, that's what we will talk about in tip number two.
00:08:59But first, a quick word from today's sponsor, me. So this week, I am releasing a completely updated
00:09:06version of my Cloud Code Masterclass inside of Chase AI+. A ton has changed since I first came out
00:09:12with this. I update it all the time. But fundamentally, we've come a long way since March when I came out
00:09:19with this thing. So if you're someone who is trying to level up their AI game, someone who definitely
00:09:24doesn't come from a technical background and you want a roadmap to actually know how to use this tool
00:09:28from the ground up and something that focuses on real use cases, then this is for you. It's inside
00:09:33of Chase AI+. There is a link to it in the pinned comment. Now, tip number two is all about your
00:09:37options when we lose the cash. We've stepped away for too long. We have some conversation that's two,
00:09:42three, four, five hundred thousand tokens long. And we want to know what the next step should be
00:09:47instead of paying those outrageous costs. And it kind of just depends on what your scenario is.
00:09:52Now, our first option is sort of the nuclear option, and that's just to do forward slash clear.
00:09:58Forward slash clear is just going to wipe the entire conversation history. And this isn't always a bad
00:10:04thing. In fact, if you have some sort of code base that you're working in, some sort of project with
00:10:10a bunch of files and a bunch of context, you probably just want to forward slash clear. Whatever
00:10:15you were working on, whatever you were talking about, chances are there is evidence in the project itself
00:10:20as to what has happened. Cloud Code can just take a look at that. And when you start a new conversation
00:10:25from scratch, it can pick up the pieces and get you back to where you were. You don't need to be a slave
00:10:31to the previous conversation. Now your second option is to use compaction. Now there is an auto compact
00:10:38feature inside of Cloud Code once you hit a certain amount of tokens. I suggest not waiting to get to
00:10:42that point because when we're working in the six, seven, eight hundred thousand token range, we're
00:10:47starting to deal with context rot. Still a problem with these bigger models, these more powerful models,
00:10:51but we can forward slash compact anytime. And what that is going to do for us is it's going to create a
00:10:57summary of what we've talked about. And then for all intents and purposes, it's going to do forward slash
00:11:03clear, but it's going to start a new conversation with that summary, kind of like in the memory. So
00:11:10if you did have important stuff in that conversation, you don't think whatever's in the code base is going
00:11:15to be enough for it to understand. Well then just do compaction, forward slash compact. It will start a new
00:11:20conversation with that summary. And that summary just lives in the message history. Now your third option
00:11:26is very similar to compact and that is doing some sort of custom handoff tool. There are a bunch of custom
00:11:32handoff skills out there. I have one myself. You can get that inside of my free community. And the difference
00:11:38between handoff and compaction is where that summary lives. So if I'm doing handoff, what that's going to do is it's
00:11:46actually just going to put it on my disk. It's going to create an actual markdown file with the summary with
00:11:51whatever I want in it. And then I can start a new conversation and say, Hey, Claude code, take a look
00:11:59at that handoff document on the disk so you can get spun up on what you need to know. Versus compact,
00:12:04it doesn't create any sort of file. It's just a message in the message history in that particular
00:12:10conversation you're having subtle difference. But for some people, they want some sort of file on the
00:12:15disk. And oftentimes it's a living, breathing document that's constantly being updated.
00:12:20So these are really your three options. Do you want to just clear everything and start from scratch?
00:12:26Oftentimes, this is totally fine. Do you just want to use the Claude native feature and compact it and
00:12:31have the summary injected right in? Or do you want a summary to be an actual document that Claude can look
00:12:37at? In that case, use handoff. Those are your three options. And oftentimes they are better than just
00:12:42sending a new message because by and large, you really shouldn't be operating in 400, 500, 600,000
00:12:48token ranges anyways. Now, tip number three is all about model routing. How do we choose the right model
00:12:53for the job? So we aren't just using Fable for everything. Can we use a smaller, cheaper, less smart
00:12:59model for the simpler tasks? The answer is yes. And we can approach this in a number of different ways.
00:13:04We can use outside models. We could go to GPT, Sol. We could go to GPT, Luna and Terra, which just got
00:13:10super cheap. We can use local models, or we have options if we want to stay in the anthropic ecosystem.
00:13:16And the easiest way to do this, I think, is advisor mode. Now, what you see here is from the original
00:13:22advisor blog post that came out several months ago. And so it shows Opus and Sonnet, but the same system
00:13:29remains with models like Fable. And the idea is we have a smart model like Fable or even Opus advising a
00:13:37smaller model like Sonnet when it comes to executing tasks. So the big model comes up with a plan, the
00:13:43small model executes it, and the small model is able to share its context with the bigger model whenever
00:13:50it runs into issues. And this model has boasted better outcomes at lower costs. And to bring in our
00:13:54previous discussion about prompt caching, both the advisor and the executor have their own prompt cache
00:14:01working at the same time. Now, your second option is to delegate tasks to models outside of Claude
00:14:06code. An easy one to do is Codex. There is a Codex plugin for Claude code, which makes it very simple
00:14:12to call on Codex from the Claude code interface. So you could have Fable essentially doing that same
00:14:18sort of advisor mode, but instead of calling on Opus or Sonnet, it calls on the GPT models. There's
00:14:24other repos like this Fable advisor that do exactly that. And ultimately it's pretty trivial to set up your
00:14:31own skill that does this exactly. And if you're looking for those cheaper models, again, I really suggest
00:14:36looking at the GPT ones, specifically Luna and Terra, because A, their costs got reduced significantly,
00:14:41and B, there really isn't any model in the, you know, Anthropic family that does what they do
00:14:48at that price point. It can even take it a step further and bring in some local models if the
00:14:53tasks make sense for that. Now, tip number four is all about your Claude hygiene. You might have seen
00:14:57this video recently making the rounds of Boris Cherney, the creator of Claude code saying,
00:15:01you need to delete your Claude.md file. Well, do you need to really delete that Claude.md file? Well,
00:15:07not necessarily, but what you do need to do, and this has gotten some recent changes over the last
00:15:12couple of weeks is run the forward slash doctor command. And what does this have to do with tokens?
00:15:18Well, first of all, what this command is going to do amongst other things, but we're kind of keeping
00:15:23this token related is it's going to take a look at your Claude.md and it's going to trim this thing down.
00:15:30The way these models work, especially these five series models, is they do not need as much
00:15:36instruction. If you look at what people were creating for the Claude.mds three, six, nine months
00:15:42ago, they were very prescriptive and extremely detailed. And perhaps at the time, an argument
00:15:46could be made that they needed that. That is not the case anymore. And so not only is a bloated Claude.md
00:15:53making it slower, it is literally costing you tokens. And forward slash doctor is going to take a look at that
00:15:59and get rid of things in your Claude.md that just don't need to be there. Secondly,
00:16:04it's going to take a look at things that are harming your context or bloating your context window.
00:16:10Because even when you start a new conversation and you run forward slash context, there are things
00:16:15filling it up. Now I've recently run forward slash doctor, so I've gotten rid of a lot of my bloat,
00:16:20but this is what my context window looks like right at the beginning of a conversation with no messages
00:16:25sent. We've already used 40,000 tokens. And what is using up a lot of this? Well, some of it comes from
00:16:31things like skills, like you can see here. Some of it includes stuff like the system prompt as well as
00:16:36memory files. What forward slash doctor is going to do is it's going to take a look at things like your
00:16:41skills, like your MCPs and start trimming the ones that you just haven't been using. Is this a huge token
00:16:47save? No, but it's something, it's something on the margins and it's such a simple fix. There's no
00:16:53reason not to do it, especially if you're someone who's been just accumulating 10 million skills over
00:16:58the last six months and you haven't actually gone through and started cutting them because doing so
00:17:03will also make your skills fire more effectively. And there won't be any confusion on Claude's part as
00:17:08to which skill it needs to call. I bet you probably have like 10 skills sitting there that all have to
00:17:12do with front and design and you don't need all those. So this very simple and easy to execute tip
00:17:17when it comes to sort of your Claude hygiene is one you shouldn't pass up. Just run doctor. And so that
00:17:22brings us to our final tip, which I think is the least effective out of the bunch these days, but that is
00:17:28sort of the additional skills and scaffolding that you see running around all over the place.
00:17:33The most popular one these days is ponytail and essentially it reduces the amount of code
00:17:38Claude writes while maintaining its effectiveness and therefore making it cheaper and faster. Now I
00:17:44did a video on this comparing these numbers with what actually happens in reality because the GitHub
00:17:51repo only shows haiku 4.5, which obviously is very outdated. When I ran this using Fable, the numbers
00:17:56did hold up. In fact, the numbers look even better with better models. Now I use the benchmarks that were
00:18:01provided with this GitHub repo. What works for you in reality may be a little bit different depending on the
00:18:06complexity of your project. But this is something you can tack on to everything we've talked about up
00:18:11until now if you want to continue to reduce your token usage. Another one you will see a lot is Caveman,
00:18:18which these days is claiming it reduces your output tokens by 65%. There's a lot of people out there who
00:18:25also talk about just doing one-liners in Claude MD, something as simple as just saying be brief,
00:18:30which will also reduce your output tokens. But remember, output tokens are just one piece of the puzzle.
00:18:36And that puzzle to bring it back to our original discussion is dominated by prompt caching. So if
00:18:41you got nothing else from this video, I hope you were able to get that because that is where we're
00:18:46going to be ending it today. So as always, let me know what you thought. Make sure to check out
00:18:50ChaseAI Plus if you want to get your hands on the Claude Code Masterclass. Again, dropping a huge
00:18:55update on that this week. And besides that, I'll see you around.