Automated Website Updater — Research, Structure, Publish
Sheet AWU-01 · Rev A · Status: Live in Production · 2026-08-06
An autonomous agent that keeps a live page current without a human touching it. Running in production today on this site's own LLM Cost Calculator, updating pricing data on a daily schedule with no manual edits.
01 — The problem it solves
The LLM Cost Calculator on this site compares what it costs to run different AI models, based on input and output token pricing across the major providers. The trouble with any page like that is the data goes stale fast — providers change prices on a rolling basis, new models launch, older ones get cheaper, and a pricing tool that's even a little out of date stops being useful.
The old way to keep it fresh is manual and doesn't scale: open every provider's pricing page, read the numbers, notice what's changed, update a spreadsheet, reformat, publish. It's boring, easy to get wrong, and nobody has time to do it every week — exactly the kind of dull, repetitive, high-friction task an autonomous agent is a good fit for.
02 — What's actually running
The agent runs a three-step pipeline — Research, Structure, Publish — once a day, with no human in the loop.
Research ──> Structure ──> Publish
│ │ │
reads each normalises to POSTs to
provider's one consistent pricing-update-api.php
pricing page schema (API key, validated,
atomic write)Research — the agent visits each provider's pricing page and pulls out what actually matters: input and output token cost, per model.
Structure — every provider formats pricing differently: per-million or per-thousand tokens, different model naming, different currencies. The agent normalises all of it into one consistent schema — same shape, same units, every time. That's the difference between raw information and usable data.
Publish — rather than editing the website's files directly, which would be fragile and risky, the agent sends a finished data payload to a dedicated API endpoint built on the site. The endpoint validates the payload, keeps one prior generation as a rollback point, and writes the new data atomically — so the calculator never reads a half-written file. It's a clean handoff: the agent produces data, the website consumes it.
Because the two halves only ever talk through the API, they're fully decoupled — the agent can change without touching the website, and the website can change without touching the agent.
03 — What ships from one run
| Normalised pricing data | Input/output token cost per model, per provider, in one consistent schema. |
| Source + verification metadata | Each provider entry carries its source URL and the date it was last verified. |
| Atomic publish | Validated payload written via temp-file-and-rename, so the live calculator never serves a partial update. |
| One-generation rollback | The previous data file is kept alongside the new one, so a bad payload can be reverted by hand. |
| Audit trail | Every publish attempt — success, validation failure, or auth failure — is logged. |
04 — Where the value actually comes from
A single successful update is a nice demo. The real case for this pattern is what it removes on an ongoing basis.
01 — The data stays current without anyone remembering to check. The tool is trustworthy because it maintains itself — freshness stops being a task on someone's list and becomes a property of the system.
02 — It gives time back. A task that's manual, tedious, and easy to skip becomes zero ongoing effort. That time goes into actual work instead of data entry.
03 — It's more consistent, not just faster. A well-built agent applies the same rules and validation every time — no typos, no copy-paste into the wrong cell. Data quality goes up, not just publishing speed.
04 — It's genuinely cheap to run. The whole pipeline runs within existing subscription costs — no dedicated infrastructure spend to justify.
05 — The API boundary makes it flexible. Because the agent and the site only ever meet at the API, either side can change independently — swap the research method, point another consumer at the same endpoint, or upgrade the site, without the other half breaking.
The pattern generalises. Nothing about Research → Structure → Publish is specific to AI pricing. Any data that changes over time and needs to live somewhere public fits this shape — market or industry data, regulatory and compliance updates, product availability, event or fixture listings, currency or commodity rates. It doesn't have to publish externally either; the same pipeline works for an internal dashboard just as well as a public page.
05 — What this is not, yet
It's running in production for one data feed, and the gaps are about breadth and hardening, not the core loop:
| Deferred | Why it's fine today |
|---|---|
| HMAC-signed requests / short-lived tokens | Currently authenticated with a dedicated, rotatable API key over HTTPS with rate limiting and auth-lockout alerting — solid for one trusted agent, worth revisiting before adding more publishers. |
| Multiple data feeds | Proven on LLM pricing; the pipeline and API pattern are directly reusable for the other data types above, but each needs its own schema and validation. |
| Scheduling/observability dashboard | Runs are logged individually today; a run-history view would make "did today's update happen" a glance instead of a log search. |
| Automatic rollback on anomaly | A bad payload is rejected at validation, but a plausible-looking but wrong payload would still publish — anomaly detection against the previous generation is future work. |
None of these block the pattern — the agent, the API contract, and the atomic publish path are proven and running unattended today.
Automated Website Updater · Live — LLM Cost Calculator feed, daily run — Sheet AWU-01, Rev A