About
AI Model Analyzer is a free, ad-free tool for comparing AI models. There is no backend, no database, no tracking. The whole site is a static bundle that recomputes every ranking in your browser.
Methodology
For each benchmark, raw scores are min-max normalised to a 0–100 scale across the participating models. This makes scores from very different benchmarks (Elo points, % pass rate, % accuracy) directly comparable.
Scenario scores are a weighted average of the normalised benchmark scores. If a model is missing data for some benchmarks, the weights are renormalised over what's available — we never penalise a model just because a score hasn't been reported yet.
Cost is folded in via a separate cost-vs-quality slider. Cost is converted to a 0–100 score on a log scale (because pricing spans four orders of magnitude) and combined with the quality score to produce a composite ranking.
Benchmarks are flagged with a contamination-risk indicator — low means a live, continuously refreshed eval, high or saturated means the question set is fixed and well known. Separately, we check the data for benchmarks where many models tie at the top score; see benchmark health.
Why composite benchmarks?
Individual benchmarks saturate. MATH and HumanEval used to spread the field; today every frontier model clears 90%, so the score stops discriminating. Composite indices fix this by stitching many benchmarks together and weighting harder ones more.
We surface one such composite as a single benchmark row, Frontier Composite. It is computed upstream using Item Response Theory — each benchmark gets a fitted difficulty and discrimination, and a model's capability is the value that best explains its observed pass rates across the whole battery. Doing well on hard, contamination-resistant evals (FrontierMath, ARC-AGI 2, Humanity's Last Exam) moves the score; doing well on a saturated benchmark barely does. We import the composite as a number per model and treat it like any other benchmark — you can weight it into scenarios from the wizard.
One caveat: the upstream composite uses an anchored scale (e.g. Claude 3.5 Sonnet at 130, GPT-5 at 150). Our pipeline min-max normalises every benchmark across the models we track, so that anchoring is squashed; raw values are still visible in tooltips.
What is reliability monitoring?
Quality benchmarks tell you how smart a model is on a frozen test set. Reliability metrics tell you how the deployed API is behaving this week — whether it's refusing more, drifting downward, or occasionally producing malformed output.
We surface four reliability axes as benchmarks: output stability (variance across re-runs), recovery rate (does it self-correct after a wrong step), format adherence (does it obey output formats), and safety handling (refuses unsafe prompts and is robust to jailbreaks). Coverage is partial — only the models the upstream monitor tracks — and the scenario engine re-weights gracefully for missing data.
A simple drift signal is computed at ingest time by z-scoring the most recent 7-day mean for each metric against the prior 21-day mean. Models that have visibly degraded on any axis are tagged in the leaderboard. AI Stupid Meter (the upstream monitor we ingest from) uses more sophisticated change-point detection (CUSUM, Mann-Whitney U); ours is the cheaper, more conservative cousin.
Data pipeline
Data is refreshed nightly by an automated pipeline. Each ingester has a fallback dataset baked in so the site keeps rendering when an upstream is briefly unavailable. Last refreshed 1m ago.
The Analyzer Index (v1.0)
One general-capability number per text model. It is a weighted average of normalised scores from 13 benchmarks, all with low or medium contamination risk. Saturated or widely leaked sets are deliberately left out.
Missing results are skipped and the other weights scaled up, the same rule scenarios use. A model needs at least 3 index benchmarks covering 30% of the weight before we show a number; the leaderboard shows how many it has (for example “7/13”). 107 models currently qualify. Weights change only with a new version number, effective 2026-10 for this one.
- Humanity's Last Exam10%
- Rolling Contamination-Controlled Average10%
- GPQA Diamond5%
- ARC-AGI 25%
- SimpleQA Verified5%
- OTIS Mock AIME 2024-20258%
- FrontierMath Tiers 1-37%
- SWE-bench Verified10%
- Terminal-Bench 28%
- Aider Polyglot6%
- LiveCodeBench6%
- Frontier Composite10%
- Chatbot Arena Elo10%
Disagree with the weights? Fork this index in the scenario builder and share your own version by URL.
How we measure speed
Every night a small script sends the same fixed request to each model: an ~800-token passage and an instruction to write about 250 tokens. Requests are pinned to the model maker's own API with fallbacks disabled, so a slow host cannot hide behind a fast one. Open-weight models also get one request on each of their two cheapest third-party hosts, which fills the per-host speed column on model pages.
Time to first token is measured from sending the request to the first streamed token of any kind, including visible reasoning. Output speed is completion tokens divided by the time from first token to the end of the stream. API success rate is the share of requests that streamed a complete answer with no error, timeout or empty reply; we show it only after at least five attempts.
We publish 14-day medians (plus the 90th percentile for latency) so one bad night does not swing a ranking. About a third of models are probed each night on rotation, under a hard spend cap. Requests run from a US-hosted CI runner, so latency includes that network path. Models the probe has not reached yet show a hand-maintained figure carrying its own date; speed data is current as of 2026-10-25.
Tracked benchmarks
We track 38 benchmarks across 16 categories. Each one carries the date its question set last changed, the date of its newest score, and a contamination label.
The benchmarks page lists them all with result counts, which scenarios use them, how much of each scenario's weight is backed by data, and which ones no longer separate the top models.
By the numbers
Data sources & licenses
We aggregate publicly available benchmark data from the projects below. Per-row attribution is intentionally omitted from the leaderboard so the site stays neutral, but the contributing projects are credited here. If you maintain one of these projects and would like a different attribution, please open an issue on our repository.
Crowdsourced human-preference Elo, pulled from the open Hugging Face dataset. Used with attribution under CC-BY 4.0.
Open-weights leaderboard. Source for IFEval, MMLU-Pro, and BBH on community models.
Continuously refreshed coding benchmark. Source of our coding pass-rate scores.
Contamination-controlled rolling benchmark. Source of our rolling-average and data-analysis scores.
Frontier benchmarks (FrontierMath, ARC-AGI 2, Humanity's Last Exam, SimpleQA Verified, OTIS Mock AIME) and the Epoch Capabilities Index used for our Frontier Composite row. Used under Creative Commons Attribution 4.0; full citation and modifications notice below.
Polyglot coding benchmark. Per-model pass rates re-published by Epoch AI; we credit the original Aider project here as the upstream of the questions and grading harness.
Real-world terminal-tool benchmark. Per-model accuracies re-published by Epoch AI; we credit the original Terminal-Bench authors here.
Source of our reliability metrics (output stability, recovery rate, format adherence, safety handling) and the time-series we use for drift detection. Code is MIT-licensed; data is used with attribution.
Decision Index scores for Jev and pplx-decider v1 and v1.1, hand-recorded from the card's evaluation table and dated 5 October 2026. The figures are self-reported by a model maker; decision models pick one answer from a fixed set, so these scores are shown on their model pages only and are not part of the Analyzer Index, the leaderboards or the frontier.
Output throughput (tok/s), time-to-first-token, and image-arena Elo are hand-maintained from publicly observable sources -- provider documentation, OpenRouter, model cards, and community measurements. PRs welcome.
Citation for Epoch AI: Epoch AI, “AI Benchmarking Hub”. Published online at epoch.ai. Retrieved from https://epoch.ai/benchmarks/use-this-data. Used under the Creative Commons Attribution 4.0 International license.
Citation for LMArena: based on the open lmarena-ai/leaderboard-dataset published on Hugging Face under the CC-BY 4.0 license. We pull the latest snapshot of the “overall” category and re-publish per-model Elo scores with attribution.
Modifications notice: upstream values are transformed before being shown here. Specifically, every benchmark score is min-max normalised to 0–100 across the models we track, multi-source rows are deduplicated to one canonical model id, attribution and source URLs are stripped from the public bundle, and reliability scores are aggregated to daily means for drift detection. Raw values remain visible in tooltips.
License coverage: Epoch AI and LMArena data are used under CC-BY 4.0. Apache-2.0 sources (Aider Polyglot, Terminal-Bench) keep their permissive terms; MIT sources (LiveCodeBench, SWE-bench, AI Stupid Meter source code) are used per their license terms.
Speed and image-arena metrics: output speed (tok/s), time-to-first-token, and image-generation arena scores are hand-maintained from public provider documentation, model cards, OpenRouter, and community measurements. They are not derived from any single proprietary leaderboard; the YAML files backing them are released under CC-BY 4.0 and PRs are welcome.
Common questions
How is the Analyzer Index computed?
The Analyzer Index is a weighted average of low- and medium-contamination benchmarks covering reasoning, math, coding, and breadth. Each benchmark is min-max normalized to 0–100 across the models we track. Missing scores are skipped and the remaining weights are renormalized, so a model is not penalized for a benchmark that has not been reported. A model needs enough coverage before we publish an index. The weights are listed on this page and can be forked in the scenario builder.
How often is the data updated?
A nightly pipeline refreshes benchmark scores, prices, and speed measurements, then rebuilds the static site. The date on each page is the generation time of the bundle you are reading. The What changed page summarizes week-to-week movement from those nightly snapshots.
Are the scores independent?
We do not run the models. Scores come from public benchmarks and are normalized here so different scales can be compared. We have no commercial relationship with any provider. Pricing is best-effort and should be confirmed on the provider's own page.
How is cost included in a ranking?
Scenario rankings can blend quality with cost. Price is converted to a 0–100 score on a log scale, because API prices span several orders of magnitude, and then combined with the quality score using that scenario's cost weight. The cost calculator estimates spend for a specific workload, including caching, batch discounts, and the cheapest host.
Do coding benchmarks measure code quality?
Not directly. Coding benchmarks check whether a model's patch or program passes tests. They do not measure whether the change is easy to review, fits the existing codebase, or adds complexity that slows down later changes. Recovery Rate, which tracks whether a model corrects its own mistakes, is the closest signal we track. Trying a model on your own repository is still the best test. The benchmarks page lists what each ranking rests on.
Disclaimers
- Benchmark scores are summary statistics. They don't predict how a model will do on your task.
- Pricing is best-effort and changes constantly. Always confirm on the provider's page before relying on a number.
- Open-source pricing reflects a median of common hosted endpoints — your self-hosted cost will differ.
- We don't run any of these models ourselves and have no commercial relationship with any provider.