Source priority
- Official provider APIs and pricing pages
- Official documentation, model cards, and announcements
- Original benchmark repositories and peer-reviewed publications
- Trusted aggregators used as a cross-check
The catalog combines canonical model facts, provider offers, benchmark links, and hosted catalog coverage. Aggregator-only records are labeled source-linked. Hermes records become cross-checked only when two distinct source organizations agree.
Ingestion and failure safety
The scheduled ingestion job validates complete catalogs, hashes responses, records changed values, and commits each source update atomically. Invalid or empty payloads are rolled back.
Benchmarks
Raw scores retain their benchmark name, version, metric, harness, evaluation date, settings, and original source. Rankings compare one benchmark version at a time. ModelBench does not infer performance from price, model name, or context size.
Price semantics
Positive values are documented USD rates per million tokens. Zero is shown only when a source explicitly marks a model as free;
missing and internal sentinel values remain undisclosed and are returned as null by the public API.
Verification labels
Source-linked means a record has a cited source without independent confirmation. Cross-checked means matching evidence was found across at least two source organizations. Model pages show the available evidence and confidence label.
Corrections and independence
Anyone can submit a correction without an account. Reports remain pending until a maintainer checks the cited evidence. ModelBench does not sell rankings, sponsored placement, or locked reports.
Known limitations
Provider-direct policy, latency, reliability, regional availability, and historical coverage are incomplete. Missing values remain undisclosed instead of being estimated.