OpenAI announced the availability of the Decisions API to general users and developers for the first time, marking an official entry into the growing market for ultra-fast, low-cost artificial intelligence—a field recently dominated by startups like TypeSafe and its model known as Jev.
This category of deep learning models functions as super-powered classifiers, allowing users to select a specific option from a set of choices at extremely high speed while providing a confidence score for the decision made. Software developers have found that these decision models, or what are known as System One systems, are capable of replacing large language models in many operational workflows, at a significantly lower cost and with much lower latency.
From ChatGPT to the Decisions API
Nikunj Handa, API Product Lead at OpenAI, explained that his team noticed the great success of the Jev model and rushed to develop the company's own alternative. Rather than building a custom decision model from scratch, the team relied on reconfiguring the ChatGPT Luna model to accelerate inference operations and reduce operational costs.
Handa noted that the most prominent improvement in the Decisions API was reducing latency from a range of one to three seconds down to between 100 and 180 milliseconds. Although the company may later resort to training fully dedicated models for this purpose, the modified Luna version currently performs the task efficiently.
Availability and Practical Uses
The company announced the availability of the API to all customers after it was first revealed at the developer conference, with the addition of new audio and visual features, and the geographic distribution of inference resources to ensure reduced latency worldwide. During initial trials, companies such as Box, Datadog, Ramp, and Speak used the new API to increase processing efficiency; Speak, a company specializing in building language learning tools, managed to reduce the time gap between actions from 1.4 seconds to just 0.13 seconds.
Performance Standards and Market Competition
Performance metrics between different decision models vary depending on the nature of the workloads. While benchmarks such as the AIM System One Index and the Artificial Analysis Decision Index top other models like D1 Ultra and Elephant-1, the Jev model and OpenAI's API continue to compete in advanced ranks according to various testing environments, including gaming tests that measure decision-making speed under pressure, such as PacMan.
Regarding competitive advantages, OpenAI officials believe their API enjoys superiority in multimodality, particularly in image classification and processing visual outputs with high accuracy and reliability, with the team looking forward to further reducing latency to reach around 60 milliseconds to process individual video frames directly via the programming interface.