feat(models): browse and price every OpenRouter model from APPE's catalogue #4

Merged

gabrielvidal merged 1 commits from or-models into main 2026-08-09 23:51:22 +02:00

Conversation 0 Commits 1 Files Changed 18 +907 -53

gabrielvidal commented 2026-08-09 23:50:20 +02:00

Owner

Copy Link

Summary

The pi.dev harness could only be pointed at two hand-written qwen entries, and
every pi session was costed at one flat qwen rate whatever it actually ran on.
OpenRouter serves several hundred models, and picking between them is a price
decision — so the composer now browses the whole catalogue with the three rates
that bill an agent run on every row, and the backend prices a session at the
model's real rates.

None of those numbers are ours: they come from APPE
(appe.dev.gabvdl.xyz/api/models/openrouter.json, a daily models.dev sync), so a
re-priced model follows automatically instead of drifting against a hard-coded
table. Companion PR: GabrielVidal1/appe#1, which publishes that endpoint.

Key changes

• backend/openrouter.py — new: fetches APPE's OpenRouter slice, shapes it into
  {id, name, vendor, inputCost, outputCost, cacheCost, contextWindow, modelSize, tags, tier, license, speedTps}, cheapest first. In-process cache
  (6h) + a disk copy under /data so a restart with no network still lists
  models; warmed in a background thread at startup.
• GET /api/models/openrouter (main.py, schemas.py) — the catalogue, kept
  out of /api/models on purpose: that is the composer's short chip list, this
  is a few hundred rows only the browser renders.
• backend/conversations.py — rates_for(model) replaces price_for as the
  pricing primitive and returns (input, output, cache-read). A vendor-
  prefixed id is priced from the catalogue; unknown ids keep the old estimate;
  Claude models keep the built-in table and the flat 10% cache-read cut. pi's own
  costUSD still wins over any estimate, unchanged.
• frontend/.../composer/components/OpenRouterBrowser.tsx — new modal: a
  FuzzyList over the catalogue, each row = name + id, a parameter-size chip, a
  context chip and IN / OUT / CACHE $/Mtok, with capability filters (tools,
  reasoning, vision, open weights, free).
• frontend/src/lib/models.tsx — the pi harness's model select gains
  Browse OpenRouter…; a model picked there gets its own option + chip;
  isOpenRouterId() (a slash — Claude ids never have one) routes any such id to
  the pi harness, so an arbitrary pick survives a resume.
• frontend/src/lib/costAnalysis.ts — a vendor-prefixed id no longer splits its
  cost at Opus rates in the dashboards.
• Mock backend seeded with a 5-model catalogue; CLAUDE.md + runner/README.md
  updated.

Key decisions

• APPE is the price source, not a table here. A second copy of OpenRouter's
  prices is exactly the thing that goes quietly wrong; APPE already syncs
  models.dev daily for this.
• Cache-read is per-model, not a fixed multiple. Anthropic's 10% cut doesn't
  hold on OpenRouter, so rates_for carries a third rate. A model with no prompt
  cache reports its cache-read rate as the input rate — what those tokens would
  actually cost — and the row greys it to say so.
• The pricing lookup never fetches. Transcript parsing prices every turn
  through it, so it reads an in-process index (falling back to the disk copy
  once); a cold, network-less start prices at the estimate rather than stalling a
  parse on HTTP.
• A second endpoint rather than a longer /api/models. Every composer paints
  the chip list; only the modal needs 340 rows, and it fetches them on first open.
• No allow-list of pickable ids. The runner already routes by id shape
  (slash ⇒ OpenRouter, bare ⇒ local EVOX2), so the browser can offer the whole
  catalogue without a second list to keep in step.

Changelog

• Added: OpenRouter model browser in the composer — search every model the pi.dev
  harness can run, with its input / output / cache price per million tokens,
  parameter size and context window, plus capability filters.
• Changed: pi.dev sessions are now costed at the model's real OpenRouter rates
  (cache reads included) instead of a single flat qwen estimate.
• Fixed: a conversation on a vendor-prefixed model is attributed to the pi
  harness on resume, and no longer has its cost split at Opus rates in the
  dashboards.

Test notes

• npx tsc --noEmit clean; npm run build clean.
• Backend, against APPE's freshly built openrouter.json: 339 models load,
  rates_for returns meta-llama/llama-3.1-8b-instruct → (0.05, 0.08, 0.025),
  openai/gpt-oss-20b → (0.03, 0.13, 0.03), claude-opus-4-8 → (5, 25, 0.5)
  unchanged, an unknown vendor id → the qwen fallback, bare qwen3.6-35b-a3b →
  free. Same usage block costs $0.0121 vs $0.0135 vs $0.46 respectively.
• UI driven in mock mode (screenshots below): harness → pi → model select →
  Browse OpenRouter… → the list, filters and fuzzy search.
• Not verified: a live billed OpenRouter run. The OPENROUTER_API_KEY in
  .env.claude is currently rejected by OpenRouter (401 User not found), so
  the two runs attempted (meta-llama/llama-3.1-8b-instruct,
  openai/gpt-oss-20b) came back with an empty assistant message and zero usage.
  Nothing in this change is involved in that path — the runner already routed
  vendor-prefixed ids to OpenRouter — but the end-to-end run should be repeated
  once a valid key is in place.

Screenshots

## Summary The pi.dev harness could only be pointed at two hand-written qwen entries, and every pi session was costed at one flat qwen rate whatever it actually ran on. OpenRouter serves several hundred models, and picking between them is a **price** decision — so the composer now browses the whole catalogue with the three rates that bill an agent run on every row, and the backend prices a session at the model's real rates. None of those numbers are ours: they come from APPE (`appe.dev.gabvdl.xyz/api/models/openrouter.json`, a daily models.dev sync), so a re-priced model follows automatically instead of drifting against a hard-coded table. Companion PR: GabrielVidal1/appe#1, which publishes that endpoint. ## Key changes - `backend/openrouter.py` — new: fetches APPE's OpenRouter slice, shapes it into `{id, name, vendor, inputCost, outputCost, cacheCost, contextWindow, modelSize, tags, tier, license, speedTps}`, cheapest first. In-process cache (6h) + a disk copy under `/data` so a restart with no network still lists models; warmed in a background thread at startup. - `GET /api/models/openrouter` (`main.py`, `schemas.py`) — the catalogue, kept out of `/api/models` on purpose: that is the composer's short chip list, this is a few hundred rows only the browser renders. - `backend/conversations.py` — `rates_for(model)` replaces `price_for` as the pricing primitive and returns **(input, output, cache-read)**. A vendor- prefixed id is priced from the catalogue; unknown ids keep the old estimate; Claude models keep the built-in table and the flat 10% cache-read cut. pi's own `costUSD` still wins over any estimate, unchanged. - `frontend/.../composer/components/OpenRouterBrowser.tsx` — new modal: a `FuzzyList` over the catalogue, each row = name + id, a parameter-size chip, a context chip and IN / OUT / CACHE $/Mtok, with capability filters (tools, reasoning, vision, open weights, free). - `frontend/src/lib/models.tsx` — the pi harness's model select gains `Browse OpenRouter…`; a model picked there gets its own option + chip; `isOpenRouterId()` (a slash — Claude ids never have one) routes any such id to the pi harness, so an arbitrary pick survives a resume. - `frontend/src/lib/costAnalysis.ts` — a vendor-prefixed id no longer splits its cost at **Opus** rates in the dashboards. - Mock backend seeded with a 5-model catalogue; `CLAUDE.md` + `runner/README.md` updated. ## Key decisions - **APPE is the price source, not a table here.** A second copy of OpenRouter's prices is exactly the thing that goes quietly wrong; APPE already syncs models.dev daily for this. - **Cache-read is per-model, not a fixed multiple.** Anthropic's 10% cut doesn't hold on OpenRouter, so `rates_for` carries a third rate. A model with no prompt cache reports its cache-read rate as the input rate — what those tokens would actually cost — and the row greys it to say so. - **The pricing lookup never fetches.** Transcript parsing prices every turn through it, so it reads an in-process index (falling back to the disk copy once); a cold, network-less start prices at the estimate rather than stalling a parse on HTTP. - **A second endpoint rather than a longer `/api/models`.** Every composer paints the chip list; only the modal needs 340 rows, and it fetches them on first open. - **No allow-list of pickable ids.** The runner already routes by id *shape* (slash ⇒ OpenRouter, bare ⇒ local EVOX2), so the browser can offer the whole catalogue without a second list to keep in step. ## Changelog - Added: OpenRouter model browser in the composer — search every model the pi.dev harness can run, with its input / output / cache price per million tokens, parameter size and context window, plus capability filters. - Changed: pi.dev sessions are now costed at the model's real OpenRouter rates (cache reads included) instead of a single flat qwen estimate. - Fixed: a conversation on a vendor-prefixed model is attributed to the pi harness on resume, and no longer has its cost split at Opus rates in the dashboards. ## Test notes - `npx tsc --noEmit` clean; `npm run build` clean. - Backend, against APPE's freshly built `openrouter.json`: 339 models load, `rates_for` returns `meta-llama/llama-3.1-8b-instruct` → (0.05, 0.08, 0.025), `openai/gpt-oss-20b` → (0.03, 0.13, 0.03), `claude-opus-4-8` → (5, 25, 0.5) unchanged, an unknown vendor id → the qwen fallback, bare `qwen3.6-35b-a3b` → free. Same usage block costs $0.0121 vs $0.0135 vs $0.46 respectively. - UI driven in mock mode (screenshots below): harness → pi → model select → `Browse OpenRouter…` → the list, filters and fuzzy search. - **Not verified:** a live billed OpenRouter run. The `OPENROUTER_API_KEY` in `.env.claude` is currently rejected by OpenRouter (`401 User not found`), so the two runs attempted (`meta-llama/llama-3.1-8b-instruct`, `openai/gpt-oss-20b`) came back with an empty assistant message and zero usage. Nothing in this change is involved in that path — the runner already routed vendor-prefixed ids to OpenRouter — but the end-to-end run should be repeated once a valid key is in place. ## Screenshots  

or-modal-browser_20260809-234650.jpeg

58 KiB

or-browser-models_20260809-234632.jpeg

101 KiB

gabrielvidal self-assigned this 2026-08-09 23:50:20 +02:00

gabrielvidal added 1 commit 2026-08-09 23:50:21 +02:00

feat(models): browse and price every OpenRouter model from APPE's catalogue ... 14aca8ea09

The pi.dev harness could only be pointed at two hand-written qwen entries, and a
pi session was costed at one flat qwen rate whatever it actually ran on.

- `backend/openrouter.py` pulls APPE's published catalogue
  (appe.dev.gabvdl.xyz/api/models/openrouter.json — a daily models.dev sync) and
  serves it at `GET /api/models/openrouter`: ~340 models with $/Mtok in / out /
  cache-read, context window, parameter size and capability tags. Cached 6h
  in-process, with a `/data` disk copy so a restart with no network still lists
  models, and warmed at startup so nothing waits on it.
- `conversations.rates_for()` prices a turn from that catalogue — per model,
  cache-read included (OpenRouter's cache price is per-model policy, not
  Anthropic's flat 10%). Unknown ids keep the old estimate; pi's own `costUSD`
  still wins over any of it.
- The composer's model select gains `Browse OpenRouter…` on the pi harness: a
  FuzzyList of the whole catalogue, each row stating the three rates that bill an
  agent run plus size and context, with capability filters. Cheapest first.
- Any vendor-prefixed id now resolves to the pi harness frontend-side
  (`isOpenRouterId`), so an arbitrary pick survives a resume.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

gabrielvidal merged commit 6f2c69724f into main 2026-08-09 23:51:22 +02:00

gabrielvidal referenced this issue from a commit 2026-08-09 23:51:25 +02:00

Merge pull request 'feat(models): browse and price every OpenRouter model from APPE's catalogue' (#4) from or-models into main

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

gabrielvidal

No Assignees gabrielvidal

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: gabrielvidal/ai-agent#4