The French startup Kog is betting that the traditional graphics processing units used in data centers have not yet exhausted their capabilities, and that low-level software improvements can increase AI model inference performance without the need for new specialized hardware. The bet comes at a time when inference speed and cost have become a major bottleneck to the expansion of AI applications, particularly agent-based workflows.
Kog appeared on the front page of Hacker News in May, when it presented an early technical version intended to demonstrate the possibility of achieving very high single-request decoding speeds using standard GPUs already found at enterprises, such as the AMD MI300X and Nvidia H200. The demonstration was not aimed at the graphics processors found in laptops, a point that disappointed some followers, but it attracted clear commercial interest; the company's CEO and sole founder, Gaël Delalleau, said Kog had received 200 concrete business opportunities.
Engineering Software as an Entry Point into the Inference Market
The company expects software engineering to be among the first use cases for Kog Inference Engine, known as KIE for short. Professional users of tools such as Claude Code sometimes face waits lasting hours before receiving results, while Anthropic recognizes that speed has financial value and charges a premium for its fast mode.
Kog is targeting customers whose use of AI for professional tasks is hindered by these delays. The company also has design partners that allow users to create games and applications through a single prompt, where faster results can translate into higher revenue, according to Delalleau.
But the company's market observations revealed that potential customers are not yet ready to tune small models themselves. Therefore, since its launch, Kog has focused on accelerating larger models in response to the demand it observed, putting it before a much greater challenge than the initial demonstration.
From 3,000 Tokens per Second to Larger Models
Kog says it aims to achieve inference that is approximately 30 times faster in large language models. However, the demonstration it has presented so far achieved 3,000 tokens per request per second using a small custom model of approximately two billion parameters, Laneformer 2B, which has become open source.
Delalleau believes the same approach can work with large language models, although the size of these models presents a challenge for inference hardware. In his view, the belief that GPUs are unsuitable for decoding has become a misconception, because newer generations of these units provide greater memory bandwidth that merely needs to be exploited through software.
Kog is not working in this direction alone. The French company ZML has launched hardware-independent software that goes beyond Nvidia's CUDA to support fast inference on competing chips. But Delalleau describes Kog's approach as closer to the research of Stanford University's Hazy Research laboratory, with a deeper focus on accelerating GPUs.
A Low-Level Approach with High Engineering Costs
Kog's focus is connected to its founder's background in solid-state physics at France's École Polytechnique, followed by work in offensive cybersecurity, or ethical hacking. Delalleau said that studying physics reinforced in him the idea of understanding the laws of nature and the laws governing how a GPU works in order to exploit it to the fullest, while his work in hacking taught him to reverse-engineer systems at a very low level, down to assembly language and binary code, to achieve goals for which the system was not originally designed.
But this approach is both practical and slow. Delalleau says the company devotes several weeks or even months to each new GPU to delve into its details and conduct engineering research on it. With a team of 11 people, that limits the number of chips Kog can support in the foreseeable future.
The Next Crucial Step
In the long term, Kog wants to turn its methodology into agent-based processing pipelines, allowing it to support a larger number of chips and models. This could give it a boost connected to European technological sovereignty, at a time when Europe is seeking to build its own capabilities in this field. The company is backed by Scaleway, as well as by Bpifrance and the French Tech 2030 program. Varsity VC, which includes Kog founder's former partner Kamel Zeroual, co-led the company's seed funding round.
Nevertheless, the most important test remains Kog's ability to transfer its results from the Laneformer 2B model to an actual large language model. Delalleau said the company expects to run its first major model at ten times the speed in September, and after beginning its customer-acquisition push, it hopes to raise a Series A funding round. Kog's promises therefore still require practical validation on the models customers actually want to use, not on a small model customized for a demonstration.