This Finally Makes Our Hermes Agent Setup 90% Cheaper
AAI LABS
Computing/SoftwareManagementInternet Technology
Transcript
00:00:00Ever since we got our hands on Hermes, we've built a ton of workflows on it and for the longest time,
00:00:04we ran all of it on GPT models through our OpenAI subscription. That worked fine at first.
00:00:09But our whole team uses Hermes and everyone's got their own flows set up and with that many
00:00:14people using it at once, we kept running into the usage limits. That would make our work just stop.
00:00:18So we moved over to OpenRouter. That did fix the limit problem we were facing but it led to a new
00:00:24one. OpenRouter charges for every single token and for the first time we could see exactly what
00:00:29Hermes was costing us. The number was way higher than we expected and the strange part was that
00:00:34most of it wasn't coming from us actively using it. Most of it was coming from tasks people had set up
00:00:39incorrectly. So we set out to cut that bill down but we weren't ready to compromise on output quality.
00:00:44So we went through all the settings and found the ones that cut the cost without making Hermes any
00:00:49worse. And if this is your first time here, we're a software company and this is our channel AI Labs
00:00:54where we show you how to optimize your processes with AI just like we've optimized our own.
00:00:59So in this video, we're going to go over every single setting we changed, how that changed our
00:01:03workflow and how much it actually saved us. Before we get into saving costs on Hermes,
00:01:08we first need to figure out what's actually eating the tokens. What you pay for Hermes comes down to
00:01:12how many tokens it uses. And tokens are basically the building blocks of words the model reads and
00:01:17writes. Every model builds on the tokens you put in and the tokens you get back. But the input isn't just
00:01:22your prompt. It also covers the system prompt and the whole conversation you've had so far,
00:01:27along with a few other things that get sent with every message. That includes the header of each skill
00:01:31you've added, which is basically its name and description that's always loaded, plus the MCP
00:01:36tools you've set up and the memory and user files. All of this adds to your input tokens and the skills
00:01:41are a big part of that. Hermes comes pre-installed with 90 skills and that number only grows the longer
00:01:46you use it. That's because it turns any workflow worth reusing into a new skill. So every one of them
00:01:51adds to your per message cost. And it isn't only the skill headers. Hermes scans your conversations
00:01:56for chances to build a new skill and that burns tokens too. Hermes also has a self-evolving memory.
00:02:03It's constantly going through your conversations to pull out specific details about you and it writes
00:02:07those to memory. It updates that memory on its own so it can hold context about you and tailor its future
00:02:13answers. And none of this is free because the evolving skills and the auto-updating memory are both
00:02:19burning tokens that you end up paying for. Another big one is that Hermes runs 24-7. It isn't an agent
00:02:25you start once and let run a single time, the way we do with Cloud Code. You can run it on a local
00:02:30server like the one we've already set up for our team or on a VPS, which is basically a server you
00:02:35rent and run yourself so the rest of your team can reach it. Because Hermes is always on, there are a lot
00:02:40of tasks running in the background. Those are what pushes your usage up if you don't keep them in check.
00:02:45A single run of one of those tasks is cheap, but once it keeps repeating or you've got a lot of similar
00:02:50jobs set up, the cost piles up. On top of that, the MCP tools and hooks you've set up are also loaded
00:02:56into the context window, so they run up tokens too. And if you're running goals on Hermes, that costs you
00:03:01money too. The good thing is Hermes gives you a way to track all of this. It stores every bit of token
00:03:07usage data in a database in the root folder. And you can ask any agent you're working with to go through
00:03:13that database and hand you a detailed breakdown. The agent can go all the way back through the
00:03:17database to when you first installed Hermes. From there, it can tell you how many sessions you've
00:03:22run, how many tokens you've used, and what it's all cost. It'll lay out your spending in detail,
00:03:27so you can sit with Claude and figure out how to manage your budget better. Hermes also has an
00:03:31insights command that does pretty much the same thing. It gives you a cost breakdown of the last 30 days,
00:03:36and it tells you what each part is using and which tools and skills you've used the most. It even shows
00:03:42your activity patterns and your longest sessions. So now you know what's eating the tokens. Let's
00:03:47start with the single biggest one, which is the model itself. The model you've connected Hermes to
00:03:52drives most of your bill. Hermes gives you a lot of models you can connect to, so if you already pay
00:03:56for a subscription, like the codec subscription we have, you can run Hermes straight through it. Once
00:04:01that's set up, you're paying through the subscription you already have, with no extra cost stacked on top.
00:04:06Now you might want to do the same with Anthropic or Gemini's subscription, but even though Hermes
00:04:11lists them as available, they actually aren't. And you can't use Claude code directly. You need a
00:04:16separate API for that, because both of them count using their subscription this way as breaking their
00:04:21policy. So for now, the codec subscription is your best option. We started out on the OpenAI
00:04:26subscription ourselves, but we ended up switching to OpenRouter, because it let us reach a lot more
00:04:30models under a single API key, and it was best for company-wide use. So if you're on OpenRouter
00:04:36like us, you can save costs by using the Pareto router. It looks at what your task actually needs
00:04:41and routes it to the right model for the job. It's got 13 models split across different tiers,
00:04:46from cheap and basic, up to expensive and powerful. Beyond the provider, there are settings in your
00:04:51config.yaml file that help you save tokens. Hermes runs a lot of background tasks, and by default,
00:04:57the model for these is set to auto. That means it falls back to your main model, even for the small
00:05:02stuff. These are called auxiliary tasks, basically the small background jobs Hermes runs, and they
00:05:07don't need the heavy reasoning your main model is built for. Things like reading images, searching
00:05:11across your skills, loading MCP tools, and writing a profile description all run fine on something
00:05:17lighter. So you can point those tasks at a cheaper model, and that way your main model isn't burning
00:05:21expensive tokens on simple stuff. The same idea applies to sub-agents which we covered before.
00:05:26Hermes can spawn multiple sub-agents and hand them tasks. Each one runs in its own context window and
00:05:32works on its own. But that's token heavy, because every sub-agent is basically its own session that
00:05:38only reports its findings back, so the cost adds up fast. So you can set a cheaper model for sub-agents and
00:05:43you'll save every time one gets spun up. Whatever model you use, there's one more thing worth tweaking,
00:05:48and that's the effort level. It's basically how much the model thinks before it answers. If you turn it
00:05:53all the way up to max, you do get better output, but you also burn through more tokens, so it ends up
00:05:58costing you more. So you want to match the effort level to the task you're doing, and when that task
00:06:03is a simple one that doesn't need much reasoning, you can switch thinking off completely. But before we
00:06:08move forwards, let's have a word from our sponsor, Luma. If you do creative work, the real bottleneck isn't
00:06:13ideas. It's juggling eight different AI tools for one project and losing half your day just managing them.
00:06:19Luma fixes that with agentic AI. Regular AI just assists you, but Luma's agents actually create
00:06:24with you, and they understand the physical world, how things move, behave, and exist in space, not just
00:06:30pixels. You bring the taste and the direction, and the agents orchestrate everything under the hood,
00:06:35holding context across the whole project and pulling in the right models for each step. From the first
00:06:40idea to the final cut, the agents keep the whole project moving, and it's never one prompt, one output.
00:06:45You shape the work with them turn by turn, refining as you go, so you can explore 10 times more ideas
00:06:51without 10 times the work, with the best models all in one place instead of 10 scattered tabs. Create with
00:06:56Luma, get early access at the link below, or scan the QR on screen. But the model isn't the only thing
00:07:02running up your tokens. The context window does too. Your Hermes agent comes with a whole set of tools and
00:07:07skills, and every message you send carries all of that along with it, plus the entire conversation
00:07:12history you've built up so far. The longer a conversation runs, the more history there is
00:07:16to send each turn, and the context window keeps growing fast, so it's a good habit to hit compress
00:07:21often. What that does is start a fresh session built on a summary of everything that's happened so
00:07:26far. You save a lot of tokens, and the model still keeps the context of the conversation, just in a much
00:07:31leaner form. By default, the compression threshold is set to 50%, which means Hermes will compress the
00:07:37conversation once half of the context window has been filled. You can adjust this value, and in most
00:07:42cases, it's better to keep it on the lower side. Doing that reduces the number of messages that need to
00:07:47be sent with each turn, helping keep context usage efficient and lowering the cost of every message.
00:07:52After compressing, Hermes also leaves a bit of the tokens uncompressed and adds them on to keep more
00:07:58context, and that's what it calls the target ratio. You can set it to a lower percentage so less of the
00:08:03old conversation carries over into the context window and fewer messages get sent with each turn.
00:08:08You can also control how much of each tool result actually enters the context. Back when we were on
00:08:13the OpenAI subscription, we set those values high so Hermes wouldn't miss details when tool output
00:08:18got cut off, like we covered in the previous video. But once we moved to OpenRouter, we had to watch
00:08:23cost more closely, so we brought those values back down. If you've got a one-time instruction that you
00:08:28only need for a single session, don't write it into the Hermes context files. Instead, you can use an
00:08:34ephemeral system prompt, which basically just adds it to that one session and nothing more. That way,
00:08:39you can use the instruction without it wasting space in the context files. And if you're running local
00:08:44systems like a second brain, as we showed you before, you want that information organized properly. That way,
00:08:49the agent can load it bit by bit as it needs it, instead of pulling in extra tokens it doesn't need.
00:08:54Beyond the files you create yourself, you can also trim down the files Hermes leans on, like the
00:08:59memory files and the agent files. These sit in your context window the entire time, so the smaller
00:09:05they are, the less they're putting in front of the model on every message. Hermes also lets you switch
00:09:10off its auto memory feature, which stops it from collecting memory and stops the memory files
00:09:14from landing in your context window. That saves you money, since fewer tokens go out with every message.
00:09:20But turning auto memory off means giving up one of the features that makes Hermes so good in the
00:09:25first place. So in our own workflows, we keep it on even though it costs us. We want it pulling in
00:09:30details about our company and sharing that context across the whole team, since everyone's using it
00:09:35through Slack. When Hermes does get something wrong, don't just prompt it again. Use the undo command to
00:09:40step back one message. It isn't a full rewind that walks back through several messages, but it does
00:09:45let you undo the most recent one. That's the better move, because from there you can give it a fresh
00:09:50prompt that spells out what went wrong and what to avoid. And if you're enjoying the video so far,
00:09:54subscribe to the channel and hit the hype button. This small gesture of support goes a long way for us.
00:10:00Trimming all of that brings your usage down and your tools are the next thing to look at. Every tool your
00:10:05agent has access to is also sent with each message as part of the context window, so it pays to cut the
00:10:10tools you don't actually use. Hermes comes with more than 17 tools, and you can see all of them with
00:10:15the Hermes list command. To disable one, you can either go through the desktop app or run the tools
00:10:20disable command with the name of the tool you want to turn off. For example, the AI labs profile we set up
00:10:26for the whole team wasn't doing any work on code bases, so we didn't need code execution there at all.
00:10:31We keep a separate profile dedicated to that. We disabled it along with any other tools we weren't
00:10:36using, so none were left in the context for no reason. Hermes also ships with a lot of skills,
00:10:41and most of them are ones you'll probably never use, so you can turn off every skill you don't need.
00:10:46Your skill list ends up holding only the ones you actually use, instead of a long list that just sits
00:10:51there and bloats the context. The same goes for your MCP servers. Each one you connect brings its own set of
00:10:56tools into the context window, so disconnect any servers you're not actually using, and for the
00:11:01ones you keep, make sure tool search is set to auto. It works like the tool search feature in Claude. It
00:11:06loads a tool only when it's actually needed, instead of keeping all of them in the context window the
00:11:11whole time. If yours isn't on auto, switch it over, so you're not wasting tokens and you get the most out
00:11:16of the ones you do spend. Hard limits are another way to keep your token usage down. They cap how much the
00:11:22agent can do before it has to stop. The first one is setting the model's max tokens to a specific number
00:11:27which controls how many tokens it can produce as output. That saves you on output costs and pushes
00:11:33the model toward tighter answers instead of rambling. By default, the agent's max turns are set to 150.
00:11:39That means, while it's working on a task, it can take up to 150 turns to think, call tools, read the
00:11:46outputs, and weigh the results before it's done. The trouble is that when the agent gets confused,
00:11:51it can burn through all of those turns, resending the whole context while it tries to solve something.
00:11:56So you can drop this to something lower. We set ours to 60, so the agent doesn't waste turns
00:12:01spinning on a problem it's stuck on. There's another setting worth turning on that guards against
00:12:06looping. You can switch the hard stop setting from false to true, which keeps the agent from looping
00:12:10for no reason. So when it gets stuck and isn't making progress, the hard stop kicks in and stops it
00:12:15before it keeps grinding away. Then there are your cron jobs. Basically tasks that run on their own,
00:12:21on a schedule, which have no limits set on their max turns by default. You can set that to a specific
00:12:26number, so there's a cap on how many turns a cron job can take. That keeps background jobs from running
00:12:31up tokens and it stops them from eating up cost forever. Now in order to get started with Hermes
00:12:36Agent, we have curated a whole starter pack that's available in AI Labs Pro, which is our community.
00:12:41That's where you'll get the resources, the starter packs, and more, along with a place to interact
00:12:45with a bunch of like-minded people, including our team. So if you've found value in what we do
00:12:50and want to support the channel, this is the best way to do it. The link's in the description.
00:12:54That brings us to the end of this video. If you'd like to support the channel and help us keep making
00:12:59videos like this, you can do so by using the super thanks button below. As always, thank you for
00:13:04watching and I'll see you in the next one.