The engineering team at Cloudflare has managed to reduce the operating cost of its AI-powered code review tool by 68%, while simultaneously significantly accelerating review times. This transformation followed a sharp rise in costs during previous months, where the average cost per review jumped from $1.19 in April to approximately $2.46 by the end of August without initial structural changes.
Internal analyses conducted by the team clarified that the rise in costs was not solely due to the volume of tasks, but rather the shift toward using more capable and larger models, such as those from Anthropic, specifically the use of the Claude Opus 4.7 model for the internal standards reviewer, Codex. This shift led to more than a threefold increase in token consumption, with the average tokens per review reaching approximately 2.99 million, increasing the financial burden despite higher utilization rates of Prompt Caching.
What changed practically to reduce costs?
The company divided the achieved improvements and financial savings into two main aspects:
- About 40% of the total savings came from direct improvements to the programmer's code and updating it to the second version, OpenCode v2.
- The remaining percentage was achieved through experimenting with different AI models and balancing the effort levels required for each review task.
Before fully adopting the second version, the company subjected the system to rigorous testing by running it alongside the first version in Shadow Mode, where it showed a 98% match in approval decisions, allowing for a full transition to the new version, which completes full reviews about 13% faster.
Why does this news matter?
This achievement highlights the economic and operational challenges organizations face when deploying AI agents in large-scale software production environments. Scaling the use of large language models consumes a massive amount of context and repetitive tokens across multiple calls within a single session. By re-engineering how the tool interacts with models and improving caching efficiency and task routing, Cloudflare has proven that cost and performance optimization can be achieved together without sacrificing the quality of code reviews.