Qwen-Plus

Crowdsourced pairwise human preference rankings of LLM responses. Higher Elo means more frequently preferred by users.

Median output speed in tokens per second over the last 14 days on the model maker's own API, measured nightly with a fixed ~800-token prompt and ~250-token answer. Models not yet probed fall back to a dated manual figure. Higher is faster.

Median time from request to first streamed token in milliseconds over the last 14 days on the model maker's own API, using the same nightly probe prompt. Lower is snappier; reasoning models are penalised here because they think before talking.

Mathematical research problems spanning analysis, algebra, combinatorics and number theory. Tiers 1-3 are progressively harder; even frontier reasoning models only solve a small fraction. The hardest publicly reported benchmark for general mathematical reasoning.

AIME-style competition problems written specifically for the OTIS mock contest, then run as an evaluation by Epoch AI. Closer in spirit to the public AIME but with novel problems unlikely to appear in training data.