Sam Altman, CEO of OpenAI, revealed that the company was on the verge of making a strategic decision to change the pricing model for its upcoming models, as it considered canceling the token-based pricing system and replacing it with a flat-fee per job system. The developer was supposed to send the prompt, and the company would determine the final price for the operation, a model similar to the billing mechanism in cloud computing services like AWS.
Why Did Developers Reject This Change?
Altman explained during his talk on the "Cheeky Pint" podcast that the company backed down from this step after consultations with developers, where the idea was met with very negative reactions. Developers expressed an urgent need to understand how costs are calculated, emphasizing that they need to know exactly what they are paying for.
The problem lies in the fact that developers currently rely on tracking input and output tokens to manage their budgets and estimate the costs of scaling model usage. Without this detail, it would become impossible to predict costs, raising concerns among developers about the possibility of being subjected to unjustified or ambiguous costs in the long run.
Transparency vs. Cost
In a striking irony, Altman noted that the developers he consulted expressed a willingness to pay higher amounts to maintain the token pricing system, as long as it provides them with transparency and predictability. Altman added: "Developers prefer to pay more for a clear and tangible unit of measurement, rather than an ambiguous pricing system that raises doubts about how the cost is calculated."
Challenges of Measuring AI Usage
Altman admitted that the "tokens" system is not perfect, describing it as a "fake unit" that does not accurately reflect the true value or the computational effort expended, but he acknowledged at the same time that the company has not found a better alternative yet. He pointed out that developers currently face an additional cognitive burden when choosing between cheaper and more capable models, which does not add real value to the final product.
This reversal comes at a time when OpenAI continues to rely on the token pricing system, while the company seeks to address challenges related to inference costs, which decrease periodically, while continuing to search for a fairer and more transparent pricing model for developers of AI applications and intelligent agents (AI Agents).