The Aggregate — LLM benchmark aggregate

Tools

Token cost is the o200k count of the tools list JSON, including schemas, before any tool is called.

• get_leaderboard

  Top of the cross-benchmark aggregate ranking: every model placed on one rating scale by an IRT model fit over public benchmark leaderboards (call about_the_aggregate for the current coverage counts).

• search_models

  Find ranked models by (partial) name or provider. Returns rank, Aggregate Rating and the model page URL. One row per model by default, fused across reasoning-effort settings.

• get_model

  One model in depth: aggregate rank, Aggregate Rating with standard error, provider, what it is, cost per task where known, and its most notable benchmark results (with percentiles).

• compare_models

  Head-to-head between 2-4 models: aggregate ranks, rating gap with a significance note based on the standard errors, and notable benchmarks they share.

• search_benchmarks

  Find benchmarks in the aggregate by (partial) name. Returns model coverage, difficulty on the rating scale, and the benchmark page URL.

• get_benchmark

  One benchmark in depth: what it measures, the original source leaderboard URL, IRT stats (difficulty, noise, model coverage), skill weights, and the current top models on it.

• get_prediction_duel

  Guesswork — the public prediction duel: every day frontier LLMs and The Aggregate's own IRT model predict newly scraped benchmark scores before seeing them, and the errors are scored. Returns the curr

• about_the_aggregate

  What this data is: how the IRT fusion works, current coverage counts, update cadence, and how to cite it.