Thuruppu: An Agentic Card GamePart 3 of 4

Build note

Two serving stacks, one card game

Thuruppu talks to its model server through an OpenAI-compatible chat-completion endpoint. That gives the game a common provider interface, but it does not make the serving engines identical. SGLang and vLLM have their own Python dependencies, GPU-memory controls and launch options.

The first practical choice in this repository is isolation. SGLang and vLLM have incompatible Torch requirements, so they use separate environments. SGLang lives in .venv; vLLM is installed into .venv-vllm by its wrapper. That costs some disk space, but it avoids asking one environment to satisfy conflicting inference stacks.

The wrappers protect the runtime environment

Both shell wrappers remove Conda paths from PATH and LD_LIBRARY_PATH before starting Python. The SGLang script explains why: an older libstdc++ from Anaconda can interfere with FlashInfer’s compiled CUDA kernels. The wrappers also create clean Python 3.12 environments, keeping the server launch separate from the web application’s Node.js dependencies.

The SGLang launcher calls its server with a configurable GPU-memory fraction and optional quantization. The vLLM wrapper passes native vLLM server options through to its Python entry point. Both scripts expose a model and port, so the game can point its agent profiles at the local endpoint rather than changing the decision code.

GPU memory is weights plus room to work

The default model in both scripts is Qwen3-8B. The vLLM wrapper defaults to CUDA device 0, allows an override through THURUPPU_GPU or CUDA_VISIBLE_DEVICES, and limits max-model-len to 16,384 unless the caller supplies a value. The script comment intends the default device to be the 24 GiB RTX 3090, because its 8B model does not fit on the 12 GiB RTX 3060. Device ordinals depend on the machine’s mapping, so I would verify the selected card instead of assuming that 0 always means the same GPU.

The vLLM wrapper comment also explains the context trade-off: the model’s 40,960-token default would need an estimated 5.62 GiB for KV cache in the noted setup, alongside about 15.27 GiB of weights at 0.8 GPU-memory utilization.

Those are configuration estimates in the script, not a promise that every machine will have the same free memory. A long context reserves room even when one card-game decision uses much less text. Quantization, model format, runtime overhead and the memory left for KV cache all affect whether a model fits. The launcher exposes controls so the setup can be adjusted rather than assuming parameter count tells the whole story.

Keep the comparison honest

The README recommends SGLang for this project because it expects repeated prompt prefixes and wants guided JSON output. That is a choice based on the workload and available features, not a benchmark result in this repository. There are no recorded side-by-side measurements for the same model, prompt, context limit, quantization and cache state.

To compare the servers, I would hold those inputs constant and measure the same repeated game decisions. I would also confirm that the selected GPU and cache settings were actually applied. Otherwise, a faster result could come from a shorter context or different memory budget rather than from the serving engine itself.

One deployment setting deserves attention before exposing either server. The SGLang launcher defaults its host to 0.0.0.0, which listens on all network interfaces. If the endpoint is intended only for the game on the same machine, bind it to 127.0.0.1 and keep it behind the intended access boundary.

For this game, the useful abstraction is the shared endpoint. The app can ask for a decision without caring which server answered. The environment around that endpoint—Python packages, GPU memory, context length, cache settings and network binding—is still part of the application and needs its own checks.

How this was made

  • Written by: Rupesh, from the thuruppu README, provider code and SGLang and vLLM launch scripts, with AI drafting and editing.
  • Memory figures and launch defaults are taken from the checked-in vLLM wrapper comments and scripts. They are configuration-specific, not independent performance measurements.
  • AI-assisted sections: structure, explanations and copy editing.