How CompareLLM updates every day
Plain-language walkthrough of the daily ingest: OpenRouter, LiveBench, SWE-bench, Arena Elo, then pages refresh.
One job, once a day
A Vercel cron hits /api/cron/ingest at 06:00 UTC. Hobby plans cannot run hourly. That job is the only automatic writer. Humans do not retype Elo cells.
If nothing material moved, the site stays as it is. If a price, Elo, or new model name crossed a threshold, we write a changelog row and revalidate the affected pages.
Where the numbers come from
OpenRouter gives us new model names, list prices, and context windows. LiveBench and SWE-bench official JSON overwrite seed cells when the name matches an alias. Arena Elo does the same for preference scores.
A brand-new OpenRouter id lands as preview. It is not in the sitemap until a second source matches or an admin promotes the alias. That stops one marketplace listing from creating 40 thin compare pages overnight.
What you still do by hand
Publishing /news posts. Grok can draft them in /admin, but nothing goes live until someone hits Publish. Seed snapshots also stay until ingest replaces them — they are a bootstrap, not a lab result.
Ready to evaluate your stack?
Calculate your optimal model weights with Stack Engine or compare top models head-to-head.