Build note
Gpu Unlock My RTX 3090 was doing tensor math at half speed
The project page says up to 60 percent more tensor throughput on an RTX 3090. I installed it, ran my own training loop before and after, and measured about three percent. That gap is the interesting part of this story, and it is not a complaint about the project.
The cap is real
NVIDIA’s own specifications say it. On GeForce cards, FP16, BF16 and TF32 matrix multiply with FP32 accumulation runs at half the rate of the same multiply with FP16 accumulation. The professional cards do not have that limit. The GA102 die in an RTX 3090 is the same silicon family as the one in an RTX A6000, so this is a firmware decision about market segments, not a missing piece of hardware.
The project is gpu-unlock. It removes the limit at boot, before the NVIDIA driver loads. No VBIOS flash, no patched driver.
What the numbers on the box mean
Dense BF16 matrix multiply, FP32 accumulation, stock power limit:
| Card | Before | After | Gain |
|---|---|---|---|
| RTX 3090 24 GB | 70–75 TFLOPS | 112–125 TFLOPS | +60–65% |
| RTX 3060 12 GB | 27 TFLOPS | 51 TFLOPS | +87% |
Those are peak dense GEMM numbers. They are true, and they are also the best case the hardware can produce. A training step is not a single large matrix multiply, which is where the story turns.
How the unlock works
The module runs from the initramfs, before nvidia binds the card. It uses the SEC2 booter overflow that the CMP unlock work made public, builds a return-oriented-programming chain inside the GPU’s most privileged mode, and uses it to open two privilege masks. With the masks open, the host can write two firmware registers:
SS0at0x0082381c, set to0x88888888.SS1at0x00823820, set to0x8.
Each 4-bit field in those registers has an override bit and a speed selector: 00 full rate, 01 half rate, 10 quarter, 11 eighth. Writing 8 into a nibble means “override enabled, full rate”. The module then issues a function-level reset, confirms the values survived, and hands the card to nvidia.
Nothing is written to flash. A power cycle restores the stock selectors, and the module rewrites them on the next boot.
Installing it on Ubuntu 24.04
The install is three steps: build two Debian packages, install them, and add one kernel parameter. DKMS builds the module for every installed kernel, and the initramfs hooks put the module and its 16 firmware blobs into each boot image.
gpu_unlock.targets=0000:3b:00.0
I targeted only the 3090. The display is on the 3060, so a problem with the unlocked card would not take the screen with it.
The first boot logged the whole chain (device=0x2204, both writes ret=0, ss0=0x88888888 ss1=0x8 before and after the reset) and ended with gpu_unlock: unlocked. The driver bound the card normally, and there were no Xid or PCIe errors.
One detail about this machine: the monitor is on the 3060 while the firmware boot GPU is the 3090, so the GRUB menu is invisible. That is why I made the parameter permanent instead of relying on editing the menu at boot.
The measurement
I used ghedi-c, a GPT implementation in C and CUDA. The model is 14.5M parameters, d_model=128, 8 layers, GPT-2 vocabulary. Two shapes, 200 steps, seed 42, stock power limits, medians of 5 runs at 512 tokens and 3 runs at 2048.
| Shape | Backend | Before | After | Change |
|---|---|---|---|---|
| B512, WikiText-103 | cuBLAS FP32 | 15.837 ms | 15.919 ms | +0.5% |
| B512, WikiText-103 | cuBLAS TF32 | 15.808 ms | 15.352 ms | −2.9% |
| B2048, PG-19 | cuBLAS FP32 | 81.729 ms | 82.733 ms | +1.2% |
| B2048, PG-19 | cuBLAS TF32 | 81.424 ms | 80.656 ms | −0.9% |
The FP32 rows are the control. They barely move. The TF32 rows use the tensor cores that the unlock affects, and they improve by a few percent.
The cleaner way to read it is the gap between the two backends in each state:
| Shape | Before unlock | After unlock |
|---|---|---|
| B512, TF32 vs FP32 | −0.2% | −3.6% |
| B2048, TF32 vs FP32 | −0.4% | −2.5% |
That is a real effect. It is also nowhere near 60 percent.
Why only three percent
The step time says it directly. At 512 tokens, Adam takes 5.0 ms of a 15.8 ms step, about 32 percent. At 2048 tokens, the backward pass takes 44.7 ms of an 81.7 ms step, about 55 percent, and most of that is elementwise work rather than matrix multiply.
The matrix multiplies in this model are narrow. With d_model=128, the inner dimension of each GEMM is K=128, which is small. The one genuinely large GEMM is the 50,257-token output head, and it is only part of the step.
This is Amdahl’s law with receipts. Doubling the rate of the part that is 15 percent of the work buys you single digits. The peak number on the spec sheet describes a workload made almost entirely of large GEMMs. A small transformer in a training loop is not that workload.
What it does and does not buy
It makes the same model train faster. It does not make a bigger model fit. The 24 GB of VRAM is unchanged, so a model that ran out of memory before still runs out of memory.
The gain shows up in work that is genuinely tensor-bound: LLM prefill, LoRA and full fine-tuning, and diffusion image generation. It does nothing for token-by-token decode, which is limited by memory bandwidth, and almost nothing for quantized INT4 or GGUF inference, which uses different kernels.
The risks I accepted
- A boot hang is possible, because the module runs before the display driver. Recovery is a power cycle and a bootloader edit.
- The current release leaves two GPU privilege masks open until power-off. On a single-user desktop that is a small risk. On a shared, rented or passthrough host I would not run it.
- The project was 18 days old when I installed it, with 121 open issues. There are no permanent-brick reports, and the card’s own power and thermal limits still apply.
- The module is unsigned, so Secure Boot has to stay off, or the module has to be signed.
Would I keep it
Yes. It is free at runtime, reversible with a power cycle and a package removal, and it will matter more when I add BF16 tensor-core kernels or measure prefill-heavy inference. Three percent is three percent.
But the honest lesson is the one I keep relearning: the number on the box describes the best case, and the profiler describes your case. The next 10 to 20 percent in this training loop is in the Adam step and the backward pass, not in the tensor cores.
How this was made
- Written by: Rupesh, from the
gpu-unlockrepository (its README,docs/How Ampere was unlocked.mdandkernel/gpu_unlock/recipes.c), the kernel log from this machine, and benchmarks run againstghedi-c, with AI drafting and editing. - Benchmark method: 200 steps, seed 42,
GHEDI_TRAIN_TIMING=1, no checkpoint writes, medians of 5 runs at B512 and 3 at B2048, stock 350 W power limit. Before was kernel 6.8.0-147 on the stock driver; after was kernel 6.8.0-149 with the unlock active. The FP32 rows are the control for the kernel change. - AI-assisted sections: structure, technical explanation and copy editing.