Home lab serverPart 2 of 13

Build note

Two GPUs, two jobs

The machine has two cards. The 24 GiB one is the LLM card, the 12 GiB one is the display card. The 24 GiB one is the headroom, and the 12 GiB one is the display, and the split is the reason the display stays usable while a model runs.

The two cards are the same generation

Card Memory PCI address Compute capability
RTX 3090 24 GiB 3b:00.0 8.6
RTX 3060 LHR 12 GiB af:00.0 8.6

Both are 8.6, so the difference is memory, not the instruction set. The 3090 has the same instructions as the 3060 and twice the memory. A kernel that runs on the 3060 runs on the 3090, and the driver is 595.99.02 with CUDA 12.8.93 for both.

The 3090 for LLM work

The 3090 is the headless card. It is the model card, and it is the headroom:

  • A 7B model at 16-bit is about 14 GiB of weights, which leaves more room for context and cache on the 3090 than on the 3060. Actual use also depends on the model format and runtime overhead.
  • A 13B model at 16-bit is about 26 GiB of weights, so it exceeds the 3090’s 24 GiB before context memory is counted. Quantization is needed to run it here.
  • A 20B model at 4-bit may fit on the 3090, depending on its quantization, context length and runtime overhead. The model file size alone is not the total memory requirement.
  • The 3090 is not the display card. It is headless, so the compositor, the browser and the desktop never touch the model’s memory. A display reset cannot take the model with it.

The 3060 for display and gaming

The 3060 is the display card. It has the connectors, and it is the card that drives the monitor:

  • The 3060 can handle many games at 1080p and 1440p, while the 3090 runs the model. Exact frame rates depend on the game and settings. The cards have separate VRAM pools, though they still share system power, CPU and PCIe resources.
  • Many 7B models at 4-bit fit on a 12 GiB card, depending on context and runtime overhead. The same 7B model at 16-bit needs about 14 GiB for weights alone, so it does not fit in the 3060’s VRAM.

The split, the default and the exception

The default is the model on the 3090 and the display on the 3060. The split is the exception, and Part 3 covers the measured run. LM Studio put qwen3.5-9b across the 3090 (12.2 GiB) and the 3060 (8.6 GiB), at 20.7 GiB combined, with the 3060 not running the display. When the display is live, the 3060’s 8.6 GiB is shared with the compositor, so the split is not the default. The rule is: the model takes the 3090, the display takes the 3060, and the split only when the display is idle.

The cost

The 3060 is a 12 GiB card and the 3090 is a 24 GiB card. A model that needs more than 12 GiB may fit on the 3090; that is the point of the larger card. A model that needs more than 24 GiB may require quantization, a shorter context or a multi-GPU split. The 3090 is headroom, not an automatic second card.

The rule that would have prevented the next week

Pin the card, not the index. The index is a number the driver decides, and the driver changes it when a card is added. The UUID or the PCI address is the card, and it stays the card when the index moves. Part 3 is the index, and the split is the same rule.

How this was made

  • Written by: Rupesh, from his own nvidia-smi output, GPU role notes and LM Studio run, with AI drafting and editing.
  • Numbers: nvidia-smi (24576 MiB and 12288 MiB, driver 595.99.02, CUDA 12.8.93), and the LM Studio split for qwen3.5-9b.
  • AI-assisted sections: structure and copy editing.