Build note
The repeated prompt at the card table
An agent turn is a fresh request to the model, but much of the instruction text can be the same as the last turn. In Thuruppu, every decision needs the game rules and a strategy prompt. The player’s hand, trick history and remaining cards change as the round moves forward.
That shape is why KV cache and prefix caching are relevant to this game. It is not proof that every request gets faster, and the repository does not include cache-hit counts or latency measurements. The useful experience here is understanding which parts of the prompt are stable, then checking that the serving configuration really reuses them.
KV cache is working memory for generated tokens
During inference, a transformer computes key and value tensors for tokens it has already processed. The KV cache keeps those tensors so the model does not have to recompute the whole preceding sequence for every generated token. That cache uses GPU memory, alongside the model weights and other runtime needs.
Prefix caching reuses KV cache from an earlier request when a later request starts with the same token prefix. It is not fuzzy matching: two prompts that mean the same thing but tokenize differently do not share that prefix. vLLM’s Automatic Prefix Caching documentation describes reuse of matching prefix blocks. SGLang’s RadixAttention design documentation describes finding and reusing shared prompt prefixes.
So there are two related but different questions. How much KV cache can the server hold? depends on the memory left after loading the model and setting the context and memory limits. How much prompt work can it reuse? depends on whether another request has a matching prefix that is still available in the cache.
What the system prompt actually contains
Every decision is built from two strings: a system message and a user message.
The system message is assembled in src/agents/prompts.ts from three parts,
always in the same order.
The first part is the game rules, produced by buildGameRulesContext(). It is
a long, static block: the card ranking, the point values, the bidding range,
the trump and calling-trump rules, the scoring table, and a long strategy
section. The last line is always “Respond with valid JSON only.”
The second part is one line naming the agent and its strategy, such as
You are Raman, with playing style: Supportive — plays to help partner win tricks. Conservative bidding, cooperative play.
The third part is the phase instruction. Bidding, play, rebid, trump selection and explain each add their own paragraph.
An abridged system message looks like this:
You are playing Twenty-eight (Thuruppu/Irupathiyettu), a 4-player Indian
trick-taking card game.
RULES:
- 32 cards: J, 9, A, 10, K, Q, 8, 7 of each suit (♥♦♣♠)
- Card ranking (high→low): J > 9 > A > 10 > K > Q > 8 > 7
- Card points: J=3, 9=2, A=1, 10=1, K/Q/8/7=0. Total: 28 points.
… (trump, calling-trump, scoring, and a long strategy section) …
Respond with valid JSON only.
You are Raman, with playing style: Supportive — plays to help partner win
tricks. Conservative bidding, cooperative play.
PLAY PHASE: You must play a card (or call trump in Phase 1 if you cannot
follow suit). Play your strongest card to win the trick if beneficial, or
discard low cards if you cannot win.
The rules block is the largest part, and it is word-for-word identical on every request. That is the reusable front of the prompt.
What the user message carries
The user message holds the current state, rebuilt fresh for every turn. For
the play phase it includes the hand, the trump and lead suit, the cards
already on the table, the bid and score, and the card-memory block built by
buildCardMemory().
The card-memory block lists the tricks already played, the points won by each team, and the cards still unaccounted for, grouped by suit. It changes after every trick, so it sits at the tail of the prompt where the cache does not help.
An abridged play-phase user message looks like this:
CURRENT GAME STATE:
Your hand: J♥, 9♥, A♣, 10♠, K♦, Q♦, 8♣, 7♠
Trump is hearts. Phase 2: trump is active.
Lead suit: clubs
Trick so far: Player 1: 9♣, Player 2: Q♣
YOUR TEAM is currently winning this trick (Player 2 with Q♣).
Trick number: 3 of 8
Bid: 20 by Player 0
…
CARD MEMORY (tricks already played this round):
Trick: J♥ 9♥ 8♥ 7♥ → won by Player 0 (3 pts)
…
CARDS STILL UNACCOUNTED (in other players' hands or not yet played):
hearts: A, 10
diamonds: (none)
…
RESPOND with JSON: { "reasoning": …, "action": …, "cardSuit": …, "cardRank": … }
Notice the direction. The stable rules come first, and the changing state comes last. A prefix cache works on the beginning of the token sequence, so putting the stable text at the front is what makes reuse possible. If the rules were appended after the hand, there would be no shared prefix to reuse.
Four layers, four lifetimes
The prompt is really four layers, and each one repeats on a different schedule.
| Layer | What it holds | How often it repeats |
|---|---|---|
| Game rules | ranking, points, bidding, trump, scoring, strategy | identical for every decision, every player, every phase |
| Persona line | You are <name>, with playing style: … |
identical for one player across all phases |
| Phase instruction | bidding / play / rebid / trump / explain | identical for one phase across players |
| User state | hand, bid, trick, card memory | rebuilt each turn, so it always differs |
The whole system message is a shared prefix for one player in one phase. The rules block alone is a shorter prefix shared even across players and phases. The cache can only reuse what a later request actually matches, so the longest useful prefix is whatever stays byte-identical from the start.
The application does not keep a conversation going. It does not ask the model to remember the previous trick. It rebuilds the full rules-and-state prompt each turn from the current game view, which is exactly the pattern prefix caching likes: the same front, a fresh tail.
The same prompt twice: how a prefix hit works
Raman, play phase, turn 1:
[ rules ] [ You are Raman… ] [ PLAY PHASE… ] [ hand, trick, card memory ]
└── all computed on turn 1
Raman, play phase, turn 2:
[ rules ] [ You are Raman… ] [ PLAY PHASE… ] [ NEW hand, trick, memory ]
└─────────────── identical prefix ────────────────────┘ └────── recomputed ───────┘
The rules, persona, and phase instruction tokenize identically, so the server
reuses their cached key-value pairs and only processes the changed tail.
This is the mechanism the README is pointing at. The repository’s prompts.ts
already lays the prompt out in this order, so no code change is needed to make
reuse possible. What is missing is confirmation that the server is configured
to cache the prefix, and a measurement of whether it helped.
The README claim needs a configuration check
The README says these prompts are nearly identical and recommends SGLang for
prefix caching. That is a reasonable workload hypothesis, but it is not a
measurement. The SGLang launcher passes its model and memory settings and does
not disable the radix cache. SGLang’s server arguments
document RadixAttention as enabled unless --disable-radix-cache is passed.
The vLLM wrapper forwards extra command-line options, but its example commands
do not explicitly include --enable-prefix-caching. The vLLM documentation
describes Automatic Prefix Caching as a feature to enable, so I would verify
the exact installed version and launch arguments before assuming it is active.
The checked-in code also does not record cache hits, prompt-token counts or time-to-first-token for a repeated decision. Without those measurements, I cannot say how much the cache saved, or claim that one engine was faster on this workload. A useful comparison would pin the model and server versions, repeat the same tokenized prefix, keep the changing state controlled, and record both cache behavior and latency. It should also distinguish a cold first request from the later repeated ones.
The practical lesson is to separate the mechanism from the result. The prompt layout makes reuse possible. The server configuration decides whether it is enabled, and measurements decide whether it mattered.
How this was made
- Written by: Rupesh, from the
thuruppuprompt builders and serving scripts, with AI drafting and editing. - The project README describes similar prompts, but the repository contains no cache-hit or latency report. This note does not claim measured speedups.
- KV-cache behavior is linked to the official vLLM and SGLang documentation.
- AI-assisted sections: structure, technical explanation and copy editing.