Rank the Driver, Not the Car
LMSYS ranks models by human preference. SigRank ranks operators by token cascade efficiency. Models don't drive — operators do.
The short version
LMSYS Chatbot Arena is the gold standard for one question: which AI model is best? It collects blind pairwise votes, computes Elo, and ranks GPT-4, Claude, Gemini, and the rest. That is a model-ranking problem, and LMSYS solves it well.
SigRank solves a different problem: which operator drives best? Give ten operators the same model and the same task and you get ten different token cascades. The model didn't change — the driving did. LMSYS ranks the car; SigRank ranks the driver. The leaderboard that was missing was the one that scores the human in the human-AI loop.
Feature comparison
| Feature | LMSYS Arena | SigRank |
|---|---|---|
| What gets ranked | AI models (GPT, Claude, Gemini…) | AI operators (the humans driving) |
| Ranking signal | Human preference votes (Elo) | Token cascade efficiency (Υ Yield) |
| Measurement method | Blind pairwise voting | On-device token telemetry (ed25519-signed) |
| Scores the model or the operator | The model | The operator |
| Objective vs subjective | Subjective (human taste) | Objective (token counts, no content read) |
| Privacy-preserving (no prompt content) | N/A (votes on outputs) | Yes (token counts only) |
| Reproducible from your own logs | No (centralized vote corpus) | Yes (on-device scanner) |
| Class tier (IGNITER → TRANSMITTER) | No | Yes |
| Operator profiles + head-to-head compare | No | Yes |
| Platform-neutral (15+ AI tools) | Models only | Yes |
| Published science (Conservation Law, DOI) | Elo methodology papers | Yes (DOI: 10.5281/zenodo.20029607) |
| MCP server for agent integration | No | Yes |
The driver, not the car
A leaderboard that ranks models tells you the ceiling — the best engine you can put in the car. It tells you nothing about who is extracting the most from the engine they have. That is the gap SigRank fills. The cascade metric Υ = cache_read × output / input² is model-agnostic: it measures how well the operator reused context, compressed input, and converted tokens into output — regardless of which model produced them.
Same model, different drivers
Ten operators, all on Claude. Same task. Ten different Υ scores — because one reused cached context (high cache_read, low input), one re-sent everything every turn (low cache_read, high input), and one wrote tight prompts (high output, low input). LMSYS would rank the model identically for all ten. SigRank ranks the operators differently — because the driving was different.
Objective cascade vs subjective votes
LMSYS's Elo is built from human preference votes — which response "feels better." That is a subjective signal, and it is the right signal for model ranking (you want models humans like). But it carries known biases: longer responses win, confident tone wins, style wins. SigRank's Υ is built from four token integers read on-device — no judges, no content read, no opinion. The score is the arithmetic of the cascade, reproducible from your own logs. Anyone can verify it; no one can vote it up.
SigRank's scoring is grounded in published science — the Conservation Law of Commitment (DOI: 10.5281/zenodo.20029607) — with a governance framework (MO§ES™, patent pending) enforcing submission integrity.
Frequently asked questions
- What is the difference between LMSYS Chatbot Arena and SigRank?
- LMSYS Chatbot Arena ranks AI MODELS — GPT-4, Claude, Gemini — by collecting blind pairwise human preference votes and computing an Elo score. SigRank ranks OPERATORS — the humans driving the AI — by measuring token cascade efficiency (Υ = cache_read × output / input²) from on-device, signed telemetry. LMSYS answers "which model is best?"; SigRank answers "which operator drives best?" Models don't drive — operators do. The leaderboard should rank the driver, not the car.
- Is SigRank an LMSYS alternative?
- They are complementary, not replacements. LMSYS is the gold standard for model ranking — it tells you which AI to use. SigRank is the standard for operator ranking — it tells you how well you used it. You pick the model with LMSYS; you measure your skill with SigRank. If you want an AI benchmarking leaderboard that ranks the human side of the human-AI loop, SigRank is the one that does that.
- Why rank operators instead of models?
- Because the model is a constant across operators, but the outcome is not. Give ten operators the same Claude model and the same task and you get ten different token cascades — different input sizes, different cache reuse, different output. The model didn't change; the driving did. LMSYS controls for the operator to isolate the model. SigRank controls for the model to isolate the operator. Both are valid; only SigRank answers "how well did I drive?"
- How is SigRank objective while LMSYS is subjective?
- LMSYS uses human preference votes — which response "feels better." That is a subjective, taste-based signal, vulnerable to length bias and style preference. SigRank reads four token integers (input, output, cache-read, cache-write) from your local logs and computes Υ Yield = cache_read × output / input². No human judges, no prompt content read, no opinion — just the arithmetic of the cascade. The score is reproducible from your own logs; anyone can verify it.
- Does SigRank ignore model quality?
- No — it normalizes across it. SigRank is platform-neutral, so operators on different models are comparable on the cascade axis. A strong operator on a weaker model can still achieve high Leverage and cache reuse; a weak operator on the best model can still burn input tokens. The cascade measures driving skill, which is partly independent of engine power. LMSYS tells you the engine's ceiling; SigRank tells you how close you got to it.
You picked the model. Now measure the driving.
LMSYS told you which AI to use. SigRank tells you how well you used it. Install the CLI, submit a signed snapshot, and see where you rank among operators — not models.