Build note
Two serving stacks, one card game
Thuruppu talks to its model server through an OpenAI-compatible chat-completion endpoint. That gives the game a common provider interface, but it does not make the serving engines identical. SGLang and vLLM have their own Python dependencies, GPU-memory controls and launch options.
The first practical choice in this repository is isolation. SGLang and vLLM
have incompatible Torch requirements, so they use separate environments.
SGLang lives in .venv; vLLM is installed into .venv-vllm by its wrapper.
That costs some disk space, but it avoids asking one environment to satisfy
conflicting inference stacks.
The wrappers protect the runtime environment
Both shell wrappers remove Conda paths from PATH and LD_LIBRARY_PATH before
starting Python. The SGLang script explains why: an older libstdc++ from
Anaconda can interfere with FlashInfer’s compiled CUDA kernels. The wrappers
also create clean Python 3.12 environments, keeping the server launch separate
from the web application’s Node.js dependencies.
The SGLang launcher calls its server with a configurable GPU-memory fraction and optional quantization. The vLLM wrapper passes native vLLM server options through to its Python entry point. Both scripts expose a model and port, so the game can point its agent profiles at the local endpoint rather than changing the decision code.
GPU memory is weights plus room to work
The default model in both scripts is Qwen3-8B. The vLLM wrapper defaults to
CUDA device 0, allows an override through THURUPPU_GPU or
CUDA_VISIBLE_DEVICES, and limits max-model-len to 16,384 unless the caller
supplies a value. The script comment intends the default device to be the 24
GiB RTX 3090, because its 8B model does not fit on the 12 GiB RTX 3060. Device
ordinals depend on the machine’s mapping, so I would verify the selected card
instead of assuming that 0 always means the same GPU.
The vLLM wrapper comment also explains the context trade-off: the model’s 40,960-token default would need an estimated 5.62 GiB for KV cache in the noted setup, alongside about 15.27 GiB of weights at 0.8 GPU-memory utilization.
Those are configuration estimates in the script, not a promise that every machine will have the same free memory. A long context reserves room even when one card-game decision uses much less text. Quantization, model format, runtime overhead and the memory left for KV cache all affect whether a model fits. The launcher exposes controls so the setup can be adjusted rather than assuming parameter count tells the whole story.
Keep the comparison honest
The README recommends SGLang for this project because it expects repeated prompt prefixes and wants guided JSON output. That is a choice based on the workload and available features, not a benchmark result in this repository. There are no recorded side-by-side measurements for the same model, prompt, context limit, quantization and cache state.
To compare the servers, I would hold those inputs constant and measure the same repeated game decisions. I would also confirm that the selected GPU and cache settings were actually applied. Otherwise, a faster result could come from a shorter context or different memory budget rather than from the serving engine itself.
One deployment setting deserves attention before exposing either server. The
SGLang launcher defaults its host to 0.0.0.0, which listens on all network
interfaces. If the endpoint is intended only for the game on the same
machine, bind it to 127.0.0.1 and keep it behind the intended access
boundary.
For this game, the useful abstraction is the shared endpoint. The app can ask for a decision without caring which server answered. The environment around that endpoint—Python packages, GPU memory, context length, cache settings and network binding—is still part of the application and needs its own checks.
How this was made
- Written by: Rupesh, from the
thuruppuREADME, provider code and SGLang and vLLM launch scripts, with AI drafting and editing. - Memory figures and launch defaults are taken from the checked-in vLLM wrapper comments and scripts. They are configuration-specific, not independent performance measurements.
- AI-assisted sections: structure, explanations and copy editing.