Gabriel Vidal b07e93246e perf(backend): unstick the conversation list under live traffic ...

The list rebuilt every card after each transcript write, and every client
the SSE bus woke up missed the cache at once and ran the same full build in
parallel on one GIL; FastAPI then encoded the big bodies on the event loop.
Live, that pinned a core with /api/conversations at 16-80 s and even
/api/health at 43 s.

- _Memo: version-keyed, single-flight memo for the card list, agent runs
  and agents-by-conversation (cards also key on a 15 s time bucket, since
  a card's running/finished state reads the clock)
- lru_cache on _conv_id / _parent_path_of / _is_sidechain_path:
  pathlib.relative_to was 0.24 ms a call, half the list's cost
- MetaStore.peek no longer takes the lock (convoyed thousands of peeks
  per build); MetaStore.items() replaces the 1.4 MB deep copy in
  _build_agent_runs
- @_json_in_worker: the eleven heaviest GETs return a ready JSONResponse
  from their worker thread instead of leaving jsonable_encoder (5-10x
  slower than json.dumps) to run on the event loop

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

2026-09-29 19:22:55 +02:00

9.9 KiB

Raw Permalink Blame History

View Raw