What Token Telemetry Measures — and What It Cannot Prove

What token telemetry is

Token telemetry is the on-device measurement of AI coding activity through token counts. Every interaction with an AI coding tool — a prompt sent to Claude, a completion from Copilot, a chat turn in Cursor — moves tokens through a cascade: you send some in, the model sends some back, and prompt caching reuses or writes context for the next turn. Telemetry records the counts at each stage. Four numbers, per session, per platform. That is the entire data layer.

It is the foundation everything else in SigRank is built on. Without telemetry there are no metrics, no leaderboard, no operator scoring. With it, you get a complete, privacy-preserving picture of how efficiently you drive your AI tools — measured not in hours or keystrokes, but in the actual currency of LLM compute: tokens.

The four pillars

Every token in an AI coding session falls into exactly one of four buckets. Together they describe the full flow of the cascade.

Why token counts, not prompt content

The central privacy design choice in SigRank is this: measure the counts, never the content. Four integers — input, output, cache_read, cache_write — fully describe the architecture of a token cascade. They tell you whether signal is compounding (high cache reuse, high output per fresh input) or tokens are burning (low cache, low output). You do not need a single word of the prompt or the response to know that.

This is what makes token telemetry privacy-preserving by construction. The scanner reads counts and content lengths locally. Only the resulting numeric scores, signed with ed25519, leave your device. Server-side verification operates on integers — replay guards, plausibility checks — never on text. Your prompts, your code, your conversation history never leave your machine.

The tradeoff is deliberate: you give up the ability to analyze what someone said to their AI, and in exchange you get a measurement system that is safe enough to run continuously, across 15+ platforms, and publish to a public leaderboard. Token counts are the unit that makes a global operator ranking possible without a privacy scandal.

A worked example

Consider two operators using the same model for a 30-minute coding session. Operator A sends 12,000 input tokens, gets 8,000 output tokens, reuses 45,000 cache-read tokens, and writes 5,000 cache-write tokens. Operator B sends 40,000 input tokens, gets 3,000 output tokens, reuses 2,000 cache-read, and writes 1,000 cache-write.

Operator A's yield is (45,000 × 8,000) / 12,000² = 2.5. Operator B's yield is (2,000 × 3,000) / 40,000² = 0.00375. Same model, same clock time, wildly different cascades. Operator A compounds signal; Operator B burns tokens. Four integers per session are enough to see the difference — no prompt content required.

What token telemetry cannot prove

Token telemetry measures the architecture of token flow. It does not measure productivity, work quality, task correctness, professional skill, employee performance, or business value. A high yield means an operator ranks highly under the stated metric and time window — nothing more. Conflating token-cascade efficiency with any of these is a category error.

Telemetry also cannot tell you why an operator's cascade looks the way it does. A low yield might reflect a difficult task, a new codebase, an unfamiliar model, or a deliberate exploration phase — not poor operator skill. Context that telemetry does not capture (task complexity, domain familiarity, model differences) can explain cascade shape as much as operator behavior. Use telemetry as a signal worth investigating, not a verdict.

Explore the category

FAQ

What is token telemetry?

The on-device measurement of AI coding activity through four token counts — input, output, cache-read, and cache-write. It records how many tokens flowed through your sessions without ever reading the content of your prompts or the model's responses.

Why token counts instead of prompt content?

Four integers fully describe the architecture of a token cascade without revealing a single word. Token counts are the privacy-preserving unit that makes a global, continuous operator ranking possible without reading anyone's prompts.

What are the four token pillars?

Input (fresh tokens sent), output (tokens generated back), cache-read (cached tokens reused from prior context), and cache-write (new tokens written to cache for future reuse). Together they describe the full flow of tokens through a session.

How do I collect token telemetry?

Install the SigRank CLI (npm install -g sigrank), run sigrank enroll, and submit a snapshot. The on-device scanner reads token counts locally and publishes a signed snapshot. No message content leaves your machine.

Is token telemetry private?

Yes. The scanner reads token counts and content lengths only — never the words of your prompts. Only ed25519-signed numeric scores leave your device, and server-side verification operates on integers, not text.