Breaking Performance Restrictions on Tensor Cores
A series of open-source projects have revealed the existence of deliberate software restrictions on NVIDIA GeForce graphics cards (RTX 20, 30, and 40 series), where Tensor cores operate at half their maximum speed when performing certain calculations based on FP32 accumulation. This restriction, imposed by NVIDIA via software to differentiate gaming cards from professional workstation cards, limits the efficiency of the cards in AI model training and inference tasks.
How Do the Modifications Work?
The initiative began with a GitHub user named Zagitta, who developed a Linux kernel module called gpu-unlock. This tool operates before the official NVIDIA drivers are loaded, rewriting the card's speed selection registers and resetting them. Since the modification is done at the kernel level, it does not require modifying the VBIOS or making permanent changes to the drivers.
Tests on an RTX 3090 Ti card showed a significant increase in matrix processing performance, with BF16 processing speed rising from 87.5 to 157.4 TFLOPS, and TF32 performance nearly doubling. These solutions have also been adapted for other generations, such as gpu-unlock-ada for the RTX 40 series and turing-unlock for the RTX 20 series.
What Changes Practically for Users?
- Performance Improvements: Users primarily benefit in image generation tasks and fine-tuning processes for models, as these tasks rely heavily on FP32-accumulate calculations.
- Limited Benefit for Gaming: Gamers will not notice any difference in performance, because games do not rely on this type of intensive Tensor core calculation.
- Chat Applications: Token generation speed does not see a significant improvement, as it is primarily limited by memory bandwidth rather than raw computing power.
Important Considerations and Risks
Although developers point out that the card's thermal and power protection mechanisms remain effective, there are warnings that running Tensor cores at full capacity may lead to high power consumption, raising concerns about long-term hardware stability. Furthermore, current support is almost entirely limited to the Linux environment, with very limited experiments on Windows.