mirror of
https://github.com/multica-ai/multica.git
synced 2026-08-02 18:13:27 +02:00
c25a82eee0
* perf(agents): make runtime model discovery fast on runtime switch (MUL-5444) Switching runtime in the agent creation form left the model picker spinning for ~8-20s. Two costs stacked up: - the list-models request sat in the store until the daemon's next scheduled heartbeat (0-15s, avg 7.5s of pure dead wait), and - the daemon then enumerated the catalog locally (static for claude, but a CLI/ACP round trip up to ~15s for everyone else). Both are addressed with the two standard techniques for a slow, low-frequency, read-only operation: push instead of poll, and stale-while-revalidate. Push (removes the heartbeat wait): - new additive `daemon:pending_work` hint, runtime-scoped, delivered through the existing daemon WS hub and the Redis relay so the API node holding the socket does the delivery. - the daemon answers a hint with ONE immediate heartbeat and dispatches what it claimed. The hint deliberately carries no work, so nothing has to be un-claimed when delivery fails and a duplicate hint cannot duplicate work - PopPending stays the atomic claim. - per-runtime coalescing plus a 1s floor keeps a caller-triggered hint from becoming a heartbeat amplifier. Cache (removes the discovery wait on repeat opens): - server-side per-runtime catalog cache (in-memory single-node, Redis multi-node) written on every successful report. - a snapshot younger than 15min answers the POST immediately as an already-completed request; older than 60s it also enqueues a background refresh that only warms the cache. - only supported, non-empty catalogs are cached; a completed-but-empty report invalidates instead, while a failed report keeps serving the last known good list. Frontend: staleTime 60s -> 5min and gcTime 30min, so a runtime revisited in the same session renders from cache and revalidates in the background instead of showing the spinner again. Compatibility: every wire change is additive. Old daemons ignore the unknown hint type and keep using the scheduled heartbeat; new daemons against an old server simply never receive one. The cached response is shaped exactly like a completed live discovery apart from the optional `cached` / `cached_at` markers. Co-authored-by: multica-agent <github@multica.ai> * fix(agents): address review on model discovery SWR (MUL-5444) Sol-Boy's review on #6098 found the client cache could outlive the server's own staleness promise, and that the two changed endpoints were still cast rather than validated. Must-fix 1 — client freshness now derives from the served answer. `staleTime` was a flat 5min, so a 14-minute-old snapshot (which the server returns while queueing its own refresh) was held as fresh for another 5min: observable staleness became server window + client window, and the refreshed catalog never reached the tab that triggered the refresh. `staleTime` is now a function of the query data: a `cached` answer is stale on arrival (bound stays the server's window alone, and the next mount/focus picks up the refreshed snapshot), while a live discovery — which just measured the truth — is trusted for the full 5min so a cold runtime is never re-enumerated inside one form session. `gcTime` stays 30min, so a revisited runtime still renders from cache and revalidates in the background; the pickers gate their spinner on `isLoading`, which stays false throughout. Must-fix 2 — both model-discovery responses go through a zod schema. `POST /api/runtimes/{id}/models` and its poll companion were casting network JSON to `RuntimeModelListRequest`, which the root CLAUDE.md API-compatibility rules forbid. Added a lenient schema (`status` stays `z.string()`, `supported` defaults to true, `.loose()` keeps unknown fields) plus a fallback record whose `status` is `failed`: a malformed body now surfaces "discovery failed" with manual entry still usable instead of a fabricated empty catalog or an endless spinner. `resolveRuntimeModels` was tightened to match — only an explicit `completed` is a catalog, so an unrecognised status is an error rather than a silent empty list, and `supported` can no longer be `undefined`. Nit — the in-memory catalog cache now deep-copies each entry's `Thinking` (and its level slice) and `ServiceTiers`, so it delivers the independent value its comment promises and matches the Redis backend's JSON round-trip semantics. Tests: staleTime policy for cached/live/no-data; a QueryObserver test proving the refreshed catalog reaches the same client with no blank loading state; unknown-status and omitted-`supported` handling; schema tests for live, cached, old-backend and nine malformed shapes; client tests that both endpoints degrade to an explicit failure; nested-field mutation isolation for the cache. Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Eve <eve@multica-ai.local> Co-authored-by: multica-agent <github@multica.ai>