AI Operator Scoring — The New Performance Layer

What operator scoring is

AI operator scoring is the systematic ranking of humans who drive AI coding tools by the efficiency of their token cascade. It does not rank which model is smartest. It does not rank who spent the most hours at the keyboard. It ranks who uses their AI most efficiently — who compounds signal, who reuses context, who turns a small fresh input into dense output. The unit of measurement is the token, and the architecture of the cascade is the score.

It is a new performance layer because it sits between two existing layers that each miss half the picture. Model benchmarks rank the AI but ignore the operator. Time trackers rank the operator but measure the wrong currency — hours, not tokens. Operator scoring ranks the operator in the currency that actually matters for LLM compute: token-cascade efficiency.

How it differs from model benchmarking

Model benchmarking asks which AI is best? and ranks models by test-suite scores (MMLU, HumanEval, SWE-bench) or preference votes (LMSYS Chatbot Arena). It holds the operator as a constant — the same harness, the same prompts, the same evaluators — and varies the model. Operator scoring inverts that: it holds the model as a constant (you pick one and drive it) and varies the operator. The data is not a synthetic test or a vote; it is real token telemetry from live coding sessions, captured on-device.

The two are complements. A model benchmark helps you choose a model. Operator scoring helps you measure whether you are driving the model you chose well. Most developers have never had the second measurement. That is the gap SigRank fills.

How it differs from time tracking

Time trackers — WakaTime and its peers — measure hours spent coding. They reward presence: the more time you log, the more productive you appear. But time is a proxy for productivity, not a measure of it. A developer who compounds signal in two hours outscores one who burns tokens for eight — but the time tracker ranks them in the opposite order. Operator scoring measures the actual currency of LLM compute: tokens. It rewards the architecture of the cascade, not the duration of the session.

This matters more as AI coding becomes the dominant mode of development. When the bottleneck is not how fast you type but how well you drive an AI, hours become a misleading metric. Token efficiency is the number that tracks the skill that actually matters.

The SigRank scoring system

Scoring starts with four on-device token pillars: input, output, cache-read, cache-write. The yield metric Υ = cache_read × output / input² is the headline — it measures the architecture of the cascade in one number. The composite SIGNA rate blends yield with signal-force and drift components to produce the final operator score. Proprietary weights (RS.xx) govern the composite and remain server-side — the scoring shape is public, the exact weights are not, to prevent gaming.

Operators are classified into tiers — IGNITER, SEEKER, BUILDER, TRANSMITTER — and ranked over 7-day, 30-day, 90-day, and all-time windows. Snapshots are ed25519-signed on-device and verified server-side with replay and plausibility guards. The system is platform-neutral: it works across Claude, ChatGPT, Gemini, Copilot, Cursor, and 15+ platforms. The foundation is a published conservation law for language under compression (DOI: 10.5281/zenodo.20029607).

Privacy preservation

Operator scoring is privacy-preserving by construction. The on-device scanner reads token counts only — four integers per session. It never reads the content of your prompts or the model's responses. Only ed25519-signed numeric scores leave your device. Server-side verification operates on integers with replay and plausibility guards, never on text. Your prompts, your code, and your conversation history never leave your machine.

This is what makes a global, continuous operator leaderboard possible. You cannot publish a ranking built on prompt content without a privacy scandal. You can publish one built on four signed integers. Token counts are the unit that makes operator scoring safe enough to run at scale.

Explore the category

FAQ

What is AI operator scoring?

The systematic ranking of humans who drive AI coding tools by token-cascade efficiency. It ranks who uses their AI best — measured by yield and the composite SIGNA rate — not which model is best or who spent the most hours.

How does it differ from model benchmarking?

Model benchmarking ranks models by test scores or votes — it asks “which AI is best?” Operator scoring ranks humans by real token telemetry — it asks “who uses the AI best?” One holds the operator constant; the other holds the model constant. They are complements.

How does it differ from time tracking?

Time tracking measures hours — it rewards presence, not efficiency. Operator scoring measures token-cascade efficiency — it rewards the architecture of how you use the AI. A developer who compounds signal in two hours outscores one who burns tokens for eight.

What is the SigRank scoring system?

Four on-device token pillars feed yield (Υ = cache_read × output / input²) and the composite SIGNA rate. Weights (RS.xx) are server-side. Operators are tiered (IGNITER to TRANSMITTER) and ranked over multiple windows. Snapshots are ed25519-signed and verified.

Is operator scoring private?

Yes. The scanner reads token counts only — four integers, never prompt content. Only ed25519-signed numeric scores leave your device. Server-side verification operates on integers, not text. Your prompts, code, and conversation history never leave your machine.