Researchers’ TokenEconomizer reports coding-agent cost cuts of up to 30.9%
TokenEconomizer is an open-source controller developed by researchers affiliated with UC Berkeley, Microsoft Research and Microsoft Security. In the team’s benchmark tests, it cut reported costs by 18.4% to 30.9% by carrying useful repository knowledge between coding tasks and trimming tools agents rarely use.
- Reported costs fell in all 30 repository-and-agent-setting comparisons.
- Pass@1 improved in four of five settings and fell by 1.0 point in one.
- The results come from related but independent benchmark fixes, not everyday developer sessions.
How TokenEconomizer works
TokenEconomizer doesn’t change a model’s weights or rewrite an agent’s code. It works through the agent’s existing interfaces, using information from completed tasks in the same repository to guide later sessions.
One mechanism creates repository-specific “micro-skills”: short facts that could help with future work, such as a useful test command. They aren’t universal instructions. What works in one codebase may be irrelevant or wrong in another.
The project says proposed micro-skills are reviewed and confirmed before they’re added, then removed if later evidence contradicts them. That review matters. Stale guidance can waste time or send an agent in the wrong direction.
Another mechanism, tool pruning, temporarily removes tools that seem unnecessary from the agent’s available list. This can reduce the need to repeatedly supply descriptions for tools that go unused. Pruning isn’t permanent: the project describes ways to restore tools and review what the agent needs.
A third mechanism looks for signs that an agent is stuck. It can redirect the agent, then alert the user if the problem persists. It does not stop the agent’s turn itself.
What the researchers report
The project’s paper and GitHub repository report an evaluation of 277 SWE-Bench tasks across six repositories and five agent-model settings. The reported cost reductions compare each setup with and without TokenEconomizer. The researchers say they included the controller’s own model calls in the cost figures.
TokenEconomizer cost less in all 30 repository-setting cells: six repositories multiplied by five settings. The reported mean cost reductions ranged from 18.4% to 30.9%:
- Claude Code with Sonnet 5: 23.6% lower cost; Pass@1 rose by 1.3 points.
- Claude Code with Opus 4.8: 30.9% lower cost; Pass@1 fell by 1.0 point.
- Codex with GPT-5.6 Terra: 24.7% lower cost; Pass@1 rose by 0.9 points.
- Codex with GPT-5.6 Sol: 26.1% lower cost; Pass@1 rose by 1.6 points.
- GitHub Copilot CLI with Sonnet 5: 18.4% lower cost; Pass@1 rose by 2.6 points.
These are the project team’s results, not independent confirmation. The figures reflect the researchers’ cost metric, not dollar totals. The project materials don’t provide enough detail to translate the percentages directly into a developer’s bill.
Performance didn’t improve across the board. The reported Pass@1 measure rose in four settings and fell by 1.0 point with Claude Code and Opus 4.8. The materials don’t explain the drop, so it shouldn’t be brushed off as rounding or assumed to be inconsequential.
What the benchmark shows, and what it doesn’t
SWE-Bench tasks ask a model to tackle software issues in a codebase and produce a patch. In this evaluation, the tasks were related but independent fixes, and the controller could use experience from earlier tasks on later ones. That makes the setup relevant to TokenEconomizer’s core idea: reuse repository knowledge rather than start from zero each time.
Still, the benchmark isn’t a continuous simulation of a developer working with an agent. The project materials don’t describe a separate leakage audit, so the results are evidence about the reported task setup, not a guarantee of savings in every real-world workflow.
Repeated work in one repository is the clearest potential fit. Developers who use an agent across many projects, or return to a given codebase only occasionally, may have fewer chances to benefit from repository-specific learning. The controller’s own calls and management overhead also need to be outweighed by later savings.
The project lists support for Claude Code, Codex, GitHub Copilot CLI and Pi, but the paper says its experiments covered three agents and didn’t include Pi. Listed support doesn’t mean benchmark validation.
Questions developers are likely to ask
-
How much did TokenEconomizer save in testing?
The researchers report mean cost reductions of 18.4% to 30.9% across five settings, with lower costs in all 30 repository-setting comparisons. The figures apply to their evaluation and cost metric.
-
Did performance improve too?
Reported Pass@1 rose in four settings and fell by 1.0 point with Claude Code and Opus 4.8. The results weren’t uniformly positive.
-
Was Pi included in the tests?
No. The project lists Pi as supported, but the reported experiments covered three agents and excluded Pi.
-
Will the savings carry over to my repository?
The benchmark doesn’t establish that. The approach is best suited to recurring work in the same codebase, so developers would need to assess its net cost and performance in their own workflows.
TokenEconomizer targets a real source of waste: making coding agents repeatedly load repository context and tool descriptions that earlier sessions may already have clarified. The results look promising for repeated work, but they don’t amount to a universal discount. Developers can inspect the project, its methods and its reported results in the repository before deciding whether the trade-offs suit their workflow.
