Build note
Tracing the game, scraping the model
When an agent turn takes longer than expected, I want to know whether the time was spent in the game server, in the model request, or inside the inference server. Thuruppu uses traces to follow that path. vLLM exposes a separate Prometheus endpoint for measurements such as request latency, token counts and prefix-cache activity.
Those are two different signals. A trace follows one operation across
services. A metric is a number sampled over time. The Collector and Jaeger
handle the trace path; Prometheus scrapes vLLM’s /metrics endpoint.
Start tracing before the first request
Next.js loads src/instrumentation.ts when its server starts. Its register()
function calls registerOTel() with the service name thuruppu. The app can
disable this bootstrap with OTEL_ENABLED=false. The example environment file
points the OpenTelemetry Protocol (OTLP) exporter at the Collector’s HTTP
receiver:
OTEL_ENABLED=true
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://10.0.0.213:4318/v1/traces
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
The host address is specific to this setup. The Collector listens for OTLP
HTTP on port 4318 and for OTLP gRPC on port 4317. The Node application uses
HTTP/protobuf, so it sends traces to the HTTP port with the /v1/traces path.
Follow an agent turn through its spans
The tracing helper in src/lib/tracing.ts creates spans and makes each one
active while its function runs. When a function throws, it records the
exception, marks the span as failed, and then closes the span.
The agent path adds a few useful layers:
agent.turnwraps a single non-human player’s turn. It records the player, phase, trick number and agent profile. The game identifier is shortened in the trace attributes.agent.decisionwraps an LLM decision. It records the phase, profile, provider, model and temperature.llm.callwraps the HTTP request. It records the provider, model, base URL, attempt number, response status and request duration.
Card play in the default search mode can be decided by the local Monte Carlo
search without an LLM call. Bidding, rebidding and trump selection still use
the LLM. So a trace can show an agent.turn without an llm.call; the span
tree reflects the decision path that actually ran.
The provider also injects W3C trace context into the request headers. That context carries the trace identifier to a backend such as vLLM. The fetch instrumentation has an allow-list for the local SGLang and vLLM ports, Ollama, and the configured backend host. When the backend’s own tracing is enabled and it accepts that context, its spans can be correlated with the app’s trace.
The trace path and metrics path stay separate
The Collector receives application traces over HTTP on 4318. vLLM’s own
OpenTelemetry exporter can send inference-server traces over gRPC on 4317.
The Collector batches the spans, exports them to Jaeger, and writes a JSON copy
to /tmp/otel_trace.json. Jaeger’s UI is available on port 16686.
Thuruppu (Next.js)
spans: agent.turn ──> agent.decision ──> llm.call
request: POST /v1/chat/completions + traceparent ────────────────> vLLM :8000
exporter: OTLP/HTTP :4318/v1/traces ────────────────────────────> Collector
vLLM internal spans ── OTLP/gRPC :4317 ───────────────────────────> Collector
Collector ── OTLP ──> Jaeger (UI :16686)
└── file ──> /tmp/otel_trace.json
Prometheus ── GET /metrics ──> vLLM, on the same HTTP port as its API
The Collector configuration also declares generic metrics and logs pipelines. That does not mean Thuruppu currently emits application-level metrics. The application code uses the tracing API; it does not create custom metric counters or histograms. The Collector Compose project also does not start Prometheus.
vLLM serves /metrics on its API port
The native vllm serve command exposes a Prometheus-formatted /metrics
endpoint on the same port as the OpenAI-compatible API. There is no separate
metrics port to open, and no --metrics-port flag is needed. If the API listens
on 8000, the metrics URL is http://127.0.0.1:8000/metrics.
The vllm-ai repository’s run.sh wrapper sets the trace service name and
exporter environment, then src/vllm_ai/main.py passes the endpoint to vLLM’s
--otlp-traces-endpoint option. A direct launch with a local-only bind looks
like this:
export OTEL_SERVICE_NAME=vllm-server
export OTEL_EXPORTER_OTLP_TRACES_INSECURE=true
vllm serve Qwen/Qwen3-8B \
--host 127.0.0.1 \
--port 8000 \
--otlp-traces-endpoint grpc://10.0.0.213:4317
curl -fsS http://127.0.0.1:8000/metrics
Thuruppu’s default vLLM base URL is http://localhost:8000/v1, so this port
matches its default provider setting. If the vLLM learning wrapper is used
instead, its default port is 30001; point VLLM_BASE_URL at that port or
start it with 8000. The wrapper binds to 0.0.0.0 by default, so keep that
listener on a trusted network or use a loopback bind when only the local game
needs it.
Prometheus must scrape this endpoint separately. For a Prometheus process on the same host, the scrape job can be as small as:
scrape_configs:
- job_name: thuruppu-vllm
scrape_interval: 15s
metrics_path: /metrics
static_configs:
- targets: ["127.0.0.1:8000"]
The target address must be reachable from Prometheus. If Prometheus runs in a
container, 127.0.0.1 points to that container, not the host running vLLM.
The prefix-cache lesson already has a small live check: learn/03-prefix-caching/run-server.sh
in vllm-ai calls /metrics and reads the prefix-cache query and hit counters.
That is server-level evidence. It is not a custom Thuruppu metric, and its
names can vary between vLLM versions.
One documentation trap is worth recording. vllm-ai/docs/METRICS_AND_TRACING.md
contains examples for vllm_ai.main_enhanced, which is not present in the
current source tree. This note follows the actual run.sh wrapper, its
main.py, and the native vLLM server endpoint instead.
How this was made
- Written by: Rupesh, from the Thuruppu instrumentation and tracing helpers,
the OpenTelemetry Collector configuration, and the
vllm-aiserving and prefix-cache examples, with AI drafting and editing. - Collector ports and span attributes come from the checked-in configuration.
The
/metricsbehavior is vLLM’s built-in endpoint. This note distinguishes it from metrics that would require additional Thuruppu instrumentation. - Reference docs: Next.js instrumentation, vLLM metrics and vLLM OpenTelemetry tracing.
- AI-assisted sections: structure, technical explanation and copy editing.