Cutting AI Cost in Hermes Claw: Model Mixing, Local Models, and Context Control

Where the money goes

Hermes Claw, the long-running agent from Nous Research, is built for tasks that stretch across many steps and a lot of back-and-forth. Two things drive the bill: the model you pick for each step, and the size of the conversation the agent re-reads on every step. Control both and the cost falls hard.

The work itself is rarely the expensive part. The expense is the conversation around the work, carried and re-read step after step, and the habit of sending every step to the most powerful (and most expensive) model whether the step needs it or not. Hermes is unusually good at fixing both, because it can route across many providers, including open models running on your own machines, from a single configuration file. That makes it the best place to cut cost not just per token but per machine you already own.

The three biggest levers

This report walks through every major lever in order of impact: first the big structural wins (model mixing and local models), then the context engineering that Hermes is famous for, then output control, tool-output reduction, migration cleanup, and the governance that makes it all stick at team scale.

1. What you are paying for

Hermes runs on AI models from many providers, and you pay for text: both what you send in and what comes back. The longer and busier the session, and the more powerful the model, the more you pay.

AI reads and writes text in small chunks called tokens, which are roughly three-quarters of a word each. You never count them yourself; just remember the rule: more text, more tokens, more money. The text you send in (your requests, files, command results) is cheaper per token; the text the model sends back (answers, code, explanations) usually costs several times more per token.

On top of the per-token rate, the choice of model swings the price enormously. A top-tier model can cost ten to thirty times more per token than a strong mid-tier model, and an open model running on your own computer costs nothing per token at all. So there are two questions on every step: how much text is moving, and which model is reading and writing it. This report attacks both.

2. Why long sessions are the core cost

The key fact: the model has no memory between replies. To stay coherent across a long task, it re-reads the entire conversation each time. The conversation does not just grow; you pay for the whole thing again on every step.

Each reply, the model is handed the standing instructions, every file it has seen, every command result, and the complete back-and-forth so far. The total amount of text a model can hold at once is its context window, measured in tokens. On a short task, filling a fraction of that window is no big deal. On a task that runs for hours and dozens of steps, that pile grows toward the window limit, and it is re-read and re-charged every single step.

This is why the structural levers come first. If you move most steps to a cheaper or local model, the growing conversation is read by something far cheaper. And if you keep the conversation small, even the expensive model reads less. The two strategies multiply: a smaller conversation read by a cheaper model is the cheapest possible step.