Changed a Few Lines, Burned 120K Tokens: Brutal Optimization

Ever run into a trap like this: you only need to tweak a few lines of code, but because the long context keeps getting carried along, you burn through hundreds of thousands of tokens in one go? DevFlow from Tencent Cloud’s Lightweight Cloud team is a textbook example.

Here’s what happened. In multi-agent development workflows, context balloons as the process moves forward. At one point, a Developer’s context had swollen to 120K tokens just to handle four tiny tasks: change a handler, add an import, modify another handler, and update a report. If those four moves are split into four separate requests, each one has to carry that 120K “historical baggage” all over again. The actual new code might only be a few dozen lines, but the cost gets amplified exponentially. That’s the classic “long context × extra model calls” trap.

So how do you break free? Tencent Cloud shares three cost-cutting tactics. First, “decluttering”: hand off code exploration to short-lived, temporary Agents, passing only the conclusions to the main flow while the raw search logs disappear with the Agent. Second, “load on demand”: templates and Skills are no longer jammed into the prompt upfront; instead, they’re only read when it’s truly time to generate the report. Third, “batching”: multiple independent read-write operations are bundled into a single response round, supported by a dedicated `replace_batch` tool that handles several edits in one call, sidestepping repeated consumption of the long context.

With this combo, you not only slash a lot of pointless model round trips, but more importantly, you keep context bloat in check. If you’re building AI-assisted development workflows or multi-agent systems, this is a must-read field guide. Want to dig into the LoopForge project behind it all? Check it out on GitHub!

Source: https://mp.weixin.qq.com/s/1jTuuEZzOryliS1C87Jy1g