Thuruppu: An Agentic Card GamePart 4 of 4

Build note

Tracing the game, scraping the model

When an agent turn takes longer than expected, I want to know whether the time was spent in the game server, in the model request, or inside the inference server. Thuruppu uses traces to follow that path. vLLM exposes a separate Prometheus endpoint for measurements such as request latency, token counts and prefix-cache activity.

Those are two different signals. A trace follows one operation across services. A metric is a number sampled over time. The Collector and Jaeger handle the trace path; Prometheus scrapes vLLM’s /metrics endpoint.

Start tracing before the first request

Next.js loads src/instrumentation.ts when its server starts. Its register() function calls registerOTel() with the service name thuruppu. The app can disable this bootstrap with OTEL_ENABLED=false. The example environment file points the OpenTelemetry Protocol (OTLP) exporter at the Collector’s HTTP receiver:

OTEL_ENABLED=true
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://10.0.0.213:4318/v1/traces
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf

The host address is specific to this setup. The Collector listens for OTLP HTTP on port 4318 and for OTLP gRPC on port 4317. The Node application uses HTTP/protobuf, so it sends traces to the HTTP port with the /v1/traces path.

Follow an agent turn through its spans

The tracing helper in src/lib/tracing.ts creates spans and makes each one active while its function runs. When a function throws, it records the exception, marks the span as failed, and then closes the span.

The agent path adds a few useful layers:

  • agent.turn wraps a single non-human player’s turn. It records the player, phase, trick number and agent profile. The game identifier is shortened in the trace attributes.
  • agent.decision wraps an LLM decision. It records the phase, profile, provider, model and temperature.
  • llm.call wraps the HTTP request. It records the provider, model, base URL, attempt number, response status and request duration.

Card play in the default search mode can be decided by the local Monte Carlo search without an LLM call. Bidding, rebidding and trump selection still use the LLM. So a trace can show an agent.turn without an llm.call; the span tree reflects the decision path that actually ran.

The provider also injects W3C trace context into the request headers. That context carries the trace identifier to a backend such as vLLM. The fetch instrumentation has an allow-list for the local SGLang and vLLM ports, Ollama, and the configured backend host. When the backend’s own tracing is enabled and it accepts that context, its spans can be correlated with the app’s trace.

The trace path and metrics path stay separate

The Collector receives application traces over HTTP on 4318. vLLM’s own OpenTelemetry exporter can send inference-server traces over gRPC on 4317. The Collector batches the spans, exports them to Jaeger, and writes a JSON copy to /tmp/otel_trace.json. Jaeger’s UI is available on port 16686.

Thuruppu (Next.js)
  spans: agent.turn ──> agent.decision ──> llm.call
  request: POST /v1/chat/completions + traceparent ────────────────> vLLM :8000
  exporter: OTLP/HTTP :4318/v1/traces ────────────────────────────> Collector

vLLM internal spans ── OTLP/gRPC :4317 ───────────────────────────> Collector
Collector ── OTLP ──> Jaeger (UI :16686)
         └── file ──> /tmp/otel_trace.json

Prometheus ── GET /metrics ──> vLLM, on the same HTTP port as its API

The Collector configuration also declares generic metrics and logs pipelines. That does not mean Thuruppu currently emits application-level metrics. The application code uses the tracing API; it does not create custom metric counters or histograms. The Collector Compose project also does not start Prometheus.

vLLM serves /metrics on its API port

The native vllm serve command exposes a Prometheus-formatted /metrics endpoint on the same port as the OpenAI-compatible API. There is no separate metrics port to open, and no --metrics-port flag is needed. If the API listens on 8000, the metrics URL is http://127.0.0.1:8000/metrics.

The vllm-ai repository’s run.sh wrapper sets the trace service name and exporter environment, then src/vllm_ai/main.py passes the endpoint to vLLM’s --otlp-traces-endpoint option. A direct launch with a local-only bind looks like this:

export OTEL_SERVICE_NAME=vllm-server
export OTEL_EXPORTER_OTLP_TRACES_INSECURE=true

vllm serve Qwen/Qwen3-8B \
  --host 127.0.0.1 \
  --port 8000 \
  --otlp-traces-endpoint grpc://10.0.0.213:4317

curl -fsS http://127.0.0.1:8000/metrics

Thuruppu’s default vLLM base URL is http://localhost:8000/v1, so this port matches its default provider setting. If the vLLM learning wrapper is used instead, its default port is 30001; point VLLM_BASE_URL at that port or start it with 8000. The wrapper binds to 0.0.0.0 by default, so keep that listener on a trusted network or use a loopback bind when only the local game needs it.

Prometheus must scrape this endpoint separately. For a Prometheus process on the same host, the scrape job can be as small as:

scrape_configs:
  - job_name: thuruppu-vllm
    scrape_interval: 15s
    metrics_path: /metrics
    static_configs:
      - targets: ["127.0.0.1:8000"]

The target address must be reachable from Prometheus. If Prometheus runs in a container, 127.0.0.1 points to that container, not the host running vLLM.

The prefix-cache lesson already has a small live check: learn/03-prefix-caching/run-server.sh in vllm-ai calls /metrics and reads the prefix-cache query and hit counters. That is server-level evidence. It is not a custom Thuruppu metric, and its names can vary between vLLM versions.

One documentation trap is worth recording. vllm-ai/docs/METRICS_AND_TRACING.md contains examples for vllm_ai.main_enhanced, which is not present in the current source tree. This note follows the actual run.sh wrapper, its main.py, and the native vLLM server endpoint instead.

How this was made

  • Written by: Rupesh, from the Thuruppu instrumentation and tracing helpers, the OpenTelemetry Collector configuration, and the vllm-ai serving and prefix-cache examples, with AI drafting and editing.
  • Collector ports and span attributes come from the checked-in configuration. The /metrics behavior is vLLM’s built-in endpoint. This note distinguishes it from metrics that would require additional Thuruppu instrumentation.
  • Reference docs: Next.js instrumentation, vLLM metrics and vLLM OpenTelemetry tracing.
  • AI-assisted sections: structure, technical explanation and copy editing.