3 Commits

Author	SHA1	Message	Date
Bohan Jiang	44ce16d9b8	MUL-5549: fix(agent): stop reporting a failed model discovery as a real catalog (#6196) ... * fix(agent): stop reporting a failed model discovery as a real catalog (MUL-5549) Selecting the CodeBuddy runtime showed a model list that shares no IDs with what the CLI actually supports, so every pick was an ID codebuddy rejects (GH #6180). The list in the report is codebuddyStaticModels() verbatim: the daemon had fallen back, but nothing downstream could tell. discoverCodebuddyModels returned (staticModels, nil) on all three failure paths, and copilot/cursor/grok do the same. A failed discovery therefore arrived as a successful one, which defeated every guard built to catch it: the daemon reported status "completed", the picker's discovery_failed hint only renders on isError, and cacheableModelCatalog — whose own comment says an empty list means transient failure — waves through a non-empty stand-in and stores it as last-known-good for the full 24h serve window. One blip got pinned as the answer for a day. Discovery now returns a Catalog carrying a Fallback marker, which the daemon forwards as an additive `fallback` field (older servers ignore it; an older daemon omitting it keeps the previous behaviour). A fallback catalog is still rendered — the picker stays populated and manual entry still works — but it is kept out of both the daemon's 60s discovery cache and the server's catalog cache. On the server it maps to Keep rather than Drop: a stand-in is no grounds to evict a real catalog, matching how a `failed` report is treated. Also stop codebuddyHelpOutput swallowing the exec error. CombinedOutput folds in stderr, so a codebuddy whose `#!/usr/bin/env node` interpreter is missing from a GUI-launched daemon's PATH had `env: node: No such file or directory` parsed as help text — and cached as such for 60s. Verified against CodeBuddy CLI v2.130.0: the parser itself is fine (16 models from real --help), so this fixes the reporting of the failure, not the parse. Co-authored-by: multica-agent <github@multica.ai> * fix(agent): run codebuddy --help at most once per model-list request (MUL-5549) Review catch on the previous commit. Model discovery and effort discovery both read `codebuddy --help`, and the effort pass called it independently. That was free while a failed --help was (wrongly) memoised, but once failures correctly stopped being cached, the failure path ran the 35s command twice in a single request — past the server's 60s running timeout, so the request timed out and the late report was then discarded as stale. The user got nothing, not even the fallback list the previous commit exists to preserve. discoverCodebuddyModels now owns the thinking annotation, so the one help capture feeds both catalogs, and the failure path uses codebuddyFallbackCatalog to apply the static effort levels without exec'ing at all: whatever broke --help for the model catalog breaks it for the effort catalog too. Also strengthen the handler tests. They decoded into a struct declared in the test rather than calling ReportModelListResult, so a wrong JSON tag or a mis-wired cache branch would have passed. They now drive the real endpoint with daemon auth and chi params, covering: a fallback report leaving a previously discovered catalog intact, an older daemon omitting the field still warming the cache, and an authoritative empty catalog still dropping the snapshot. Both fixes are mutation-tested — reverting either makes the new tests fail. Co-authored-by: multica-agent <github@multica.ai> * docs(agent): correct codebuddy --help comments after the single-capture refactor (MUL-5549) Review nit. The comments still described the pre-refactor call graph, where both discoverCodebuddyModels and codebuddyEffortSuperset called codebuddyHelpOutput and the cache was what stopped the duplicate run. The effort parser now takes an already-captured string, and the single-invocation guarantee is structural rather than cache-dependent — which matters, because a failed --help is deliberately not cached, so a second caller would re-run the full 35s timeout. Also note on codebuddyHelpOutput that it has exactly one caller and why a new one would reintroduce the bug. Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Bohan-J <bohan@devv.ai> Co-authored-by: multica-agent <github@multica.ai>	2026-07-30 20:34:01 +08:00
Multica Eve	5f820a1d22	perf(agents): raise the model catalog serve window to 24h (MUL-5444) (#6115) ... The serve window shipped at 15 minutes, which was calibrated for a constraint that no longer exists. In the first cut the browser held a flat 5-minute staleTime decoupled from the server, so the two windows compounded and a short server window was the only bound on observable staleness. Since the review round, a server-cached response is stale on arrival client-side, so nothing compounds. What 15 minutes actually cost: no background job keeps a snapshot warm — only a picker open reads or refreshes it — and the browser's react-query cache dies with the tab. So opening the agent form once today and again tomorrow was guaranteed to be a cold miss both times, paying the full 1-15s daemon round trip. The cache only helped inside a single session. Guarding a change that happens on a scale of days (a CLI upgrade) with a minutes-scale window bought no freshness for that price. Freshness is not traded away, because the two constants do different jobs and only the other one governs it: any served snapshot older than modelCatalogRevalidateAfter (60s, unchanged) queues a background refresh, and the client treats a cached answer as immediately revalidatable. The observable behaviour stays "this open shows the old list, the next open shows the new one" at any window length. The window stays bounded rather than infinite because a *failed* discovery report deliberately leaves the snapshot in place — a transient failure must not empty the picker — so for a runtime whose CLI was uninstalled or logged out, expiry is the only thing that eventually retires the stale catalog. 24h covers the dominant daily-usage pattern while keeping that backstop inside a day. Both backends derive from the one constant (the in-memory retainFor and the Redis TTL), so there is no second place to keep in sync. Storage is not a factor: a measured snapshot is 206 bytes for a Claude catalog and 6.4KB for a codex catalog with full thinking levels and service tiers. Adds a behavioural regression test instead of restating the constant: an hours-old and a just-inside-the-window snapshot are still served, a past-the-window one is a miss, and serving an aged snapshot still enqueues the refresh plus its wakeup hint. It also fails loudly if the window is ever dropped back to minutes or if the revalidate threshold stops being well below it. Co-authored-by: Eve <eve@multica-ai.local> Co-authored-by: multica-agent <github@multica.ai>	2026-07-29 17:25:20 +08:00
Multica Eve	c25a82eee0	perf(agents): fast model discovery on runtime switch (MUL-5444) (#6098) ... * perf(agents): make runtime model discovery fast on runtime switch (MUL-5444) Switching runtime in the agent creation form left the model picker spinning for ~8-20s. Two costs stacked up: - the list-models request sat in the store until the daemon's next scheduled heartbeat (0-15s, avg 7.5s of pure dead wait), and - the daemon then enumerated the catalog locally (static for claude, but a CLI/ACP round trip up to ~15s for everyone else). Both are addressed with the two standard techniques for a slow, low-frequency, read-only operation: push instead of poll, and stale-while-revalidate. Push (removes the heartbeat wait): - new additive `daemon:pending_work` hint, runtime-scoped, delivered through the existing daemon WS hub and the Redis relay so the API node holding the socket does the delivery. - the daemon answers a hint with ONE immediate heartbeat and dispatches what it claimed. The hint deliberately carries no work, so nothing has to be un-claimed when delivery fails and a duplicate hint cannot duplicate work - PopPending stays the atomic claim. - per-runtime coalescing plus a 1s floor keeps a caller-triggered hint from becoming a heartbeat amplifier. Cache (removes the discovery wait on repeat opens): - server-side per-runtime catalog cache (in-memory single-node, Redis multi-node) written on every successful report. - a snapshot younger than 15min answers the POST immediately as an already-completed request; older than 60s it also enqueues a background refresh that only warms the cache. - only supported, non-empty catalogs are cached; a completed-but-empty report invalidates instead, while a failed report keeps serving the last known good list. Frontend: staleTime 60s -> 5min and gcTime 30min, so a runtime revisited in the same session renders from cache and revalidates in the background instead of showing the spinner again. Compatibility: every wire change is additive. Old daemons ignore the unknown hint type and keep using the scheduled heartbeat; new daemons against an old server simply never receive one. The cached response is shaped exactly like a completed live discovery apart from the optional `cached` / `cached_at` markers. Co-authored-by: multica-agent <github@multica.ai> * fix(agents): address review on model discovery SWR (MUL-5444) Sol-Boy's review on #6098 found the client cache could outlive the server's own staleness promise, and that the two changed endpoints were still cast rather than validated. Must-fix 1 — client freshness now derives from the served answer. `staleTime` was a flat 5min, so a 14-minute-old snapshot (which the server returns while queueing its own refresh) was held as fresh for another 5min: observable staleness became server window + client window, and the refreshed catalog never reached the tab that triggered the refresh. `staleTime` is now a function of the query data: a `cached` answer is stale on arrival (bound stays the server's window alone, and the next mount/focus picks up the refreshed snapshot), while a live discovery — which just measured the truth — is trusted for the full 5min so a cold runtime is never re-enumerated inside one form session. `gcTime` stays 30min, so a revisited runtime still renders from cache and revalidates in the background; the pickers gate their spinner on `isLoading`, which stays false throughout. Must-fix 2 — both model-discovery responses go through a zod schema. `POST /api/runtimes/{id}/models` and its poll companion were casting network JSON to `RuntimeModelListRequest`, which the root CLAUDE.md API-compatibility rules forbid. Added a lenient schema (`status` stays `z.string()`, `supported` defaults to true, `.loose()` keeps unknown fields) plus a fallback record whose `status` is `failed`: a malformed body now surfaces "discovery failed" with manual entry still usable instead of a fabricated empty catalog or an endless spinner. `resolveRuntimeModels` was tightened to match — only an explicit `completed` is a catalog, so an unrecognised status is an error rather than a silent empty list, and `supported` can no longer be `undefined`. Nit — the in-memory catalog cache now deep-copies each entry's `Thinking` (and its level slice) and `ServiceTiers`, so it delivers the independent value its comment promises and matches the Redis backend's JSON round-trip semantics. Tests: staleTime policy for cached/live/no-data; a QueryObserver test proving the refreshed catalog reaches the same client with no blank loading state; unknown-status and omitted-`supported` handling; schema tests for live, cached, old-backend and nine malformed shapes; client tests that both endpoints degrade to an explicit failure; nested-field mutation isolation for the cache. Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Eve <eve@multica-ai.local> Co-authored-by: multica-agent <github@multica.ai>	2026-07-29 16:03:26 +08:00