mirror of
https://github.com/multica-ai/multica.git
synced 2026-08-05 09:30:05 +02:00
main
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Bohan Jiang
|
79c5832e1d |
MUL-5708 fix(taskfailure): classify response-side context-window overflow (#6366)
* fix(taskfailure): classify response-side context-window overflow (MUL-5708)
Claude Code 2.1.x reports an exhausted context window on the response,
not as a 400 on the request: the turn ends with stop_reason
"model_context_window_exceeded" and the CLI prints
API Error: The model has reached its context window limit.
That string carries none of the phrases rule 1 matched and no "token",
so it classified as agent_error.unknown. Unknown is absent from the
resume blacklists (resumeUnsafeFailureReason, GetLastTaskSession,
GetLastChatTaskSession), so the over-full session stayed pinned as the
resume pointer for the (agent, issue) pair and every later comment on
the issue resumed the same transcript and overflowed again.
Match the CLI copy and the raw stop reason so the failure lands in
agent_error.context_overflow, which those blacklists already exclude —
the next comment then starts from a fresh session instead of replaying
the overflow.
Co-authored-by: multica-agent <github@multica.ai>
* fix(taskfailure): upgrade an old daemon's catchall on context overflow (MUL-5708)
Installed daemons update on their own cadence, and FailTask only
re-classifies when the caller supplied no reason. A daemon whose rule 1
predates the response-side wordings reports agent_error.unknown, which
is on no resume blacklist — so until every host updates, the over-full
session stays pinned as the (agent, issue) resume pointer and every
later comment replays the same overflow. One un-upgraded host means a
permanently stuck issue, not just a missing label.
Recognise the two witnesses server-side in NormalizeDaemonReason, next
to the MUL-5370 skill-bundle rule it mirrors, so the retirement lands
the moment the server deploys. The accepted legacy set is narrower than
that rule's: only the catchall and the pre-MUL-1949 coarse agent_error.
A refined reason means the old daemon matched an earlier rule on the
same text, which says more about what ended the run than a witness
appearing somewhere in the blob does.
The witnesses move into one shared var so Classify and the normalizer
cannot drift apart.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: Bohan-J <bohan@devv.ai>
Co-authored-by: multica-agent <github@multica.ai>
|
||
|
Multica Eve
|
4fe94a6d40 |
revert: "MUL-5637 fix(mcp): agent mcp_config is an authoritative allowlist (#6292)" (#6314)
This reverts commit
|
||
|
Multica Eve
|
aa349fed02 |
MUL-5637 fix(mcp): agent mcp_config is an authoritative allowlist (#6292)
* fix(mcp): treat agent mcp_config as an authoritative allowlist
An agent's saved mcp_config was silently widened with the runtime host's
own user-level MCP servers, so an explicitly empty `{"mcpServers":{}}`
resolved to the COMPLETE host set instead of no servers at all — the
opposite of what the operator configured (GitHub #6283).
`--strict-mcp-config` was being passed correctly; the merge happened
before it, in the daemon, so strict mode constrained an already-widened
set. Introduced by #5277 and present in v0.4.16 through main.
Restore the three-state contract in resolveEffectiveMcpConfig:
null / unset -> inherit the provider's native MCP configuration
{"mcpServers":{}} -> strict empty, no host servers
non-empty object -> strict allowlist, exactly those servers
Two explicit inherit paths keep the additive behaviour reachable without
weakening the default:
- runtime_config.mcp.inherit_runtime = true opts an agent back in.
- The claim response now carries mcp_config_overlay_only so the daemon
can tell an agent-authored config from a per-task Composio overlay.
Without it, enabling an integration on an agent that never configured
MCP would have stripped the host servers it was already inheriting.
Both decode paths fail closed: malformed runtime_config never enables
inheritance, and a failed runtime merge falls back to the agent's own
config.
The web MCP tab and the `agent create/update --mcp-config` help text
described the old additive behaviour, which is how a tightened config
could look correct while exposing every host server; both now state
which mode is in effect.
Note for rollout: the fix lives in the daemon, so self-hosted users must
upgrade the daemon — a server/UI upgrade alone does not apply it.
Co-authored-by: multica-agent <github@multica.ai>
* fix(mcp): close review gaps in the authoritative mcp_config change
Addresses the four must-fix findings from review of #6292.
1. Deleting the last managed server no longer widens access.
removeManagedMcpServer cleared the config to null, which now means
"inherit the host's MCP servers" — so a delete took the agent from one
allowed server to every server on the host. It now leaves an explicit
`{"mcpServers":{}}`. Restoring inheritance moved to a separate
clearManagedMcpConfig action behind its own confirmation that states the
widening. The delete dialog no longer claims "Runtime servers are not
affected", which was the opposite of the truth.
2. The UI no longer promises a boundary an old daemon does not enforce.
The strict semantics live in the daemon, so a config saved against an
older daemon is not yet in effect. Adds the authoritative-mcp-v1 daemon
capability:
- The daemon advertises it and reports authoritative_mcp on the
runtime-capabilities response.
- The claim path fails closed: a managed, non-inheriting mcp_config
claimed by a daemon without the capability cancels the task and
returns 412 with an actionable message, instead of letting that daemon
merge the host's servers in. runtime_config.mcp.inherit_runtime is the
documented escape hatch, and it is honest — it declares that the
operator accepts the host's servers.
- The MCP tab shows "needs upgrade" rather than "Not exposed" while the
bound runtime lacks the capability.
3. Saving OpenClaw settings no longer drops the inherit opt-in.
parseOpenclawRuntimeConfig discarded unknown keys and the tab persisted
the result as the whole runtime_config, so one unrelated routing save
silently deleted mcp.inherit_runtime. Unknown keys now round-trip
through OpenclawRuntimeConfig.passthrough, excluded from the dirty check
so they cannot make the form look edited.
4. Documents the new semantics in the built-in creating-agents skill and
its source map: the three states, the persisted
runtime_config.mcp.inherit_runtime field, and the claim-time capability
gate.
Also corrects the PR's rollout claim: there is no database migration, but
this does add a persisted JSON field and change the meaning of an existing
one.
Co-authored-by: multica-agent <github@multica.ai>
* fix(mcp): stop the authoritative-daemon gate from blocking valid claims
CI's backend job failed three handler claim tests with the new 412. Two
distinct problems, both real:
1. The gate fired for a non-object mcp_config. 66 handler fixtures seed
`[]`, which is not a valid MCP config and cannot carry `mcpServers`, so
it expresses no boundary to protect. An old daemon does not widen it
either: mergeRuntimeAndAgentMcpConfig fails to unmarshal a non-object
and falls back to the agent config alone (verified directly). Gating
these blocked tasks with no security benefit, so the gate now requires a
JSON object.
2. The shared daemon test-request helper advertised no capabilities, so
every claim test was accidentally simulating a pre-#6283 daemon. It now
defaults authoritative-mcp-v1 on, matching what every current daemon
sends. Only that capability — skill-bundles / coalesced-comments / rpc
are feature negotiations whose absence tests real legacy behaviour, so
they stay opt-in per test.
Adds claim-level coverage for the gate itself, which is what the unit tests
alone could not catch: an outdated daemon gets 412 with an actionable
message and the task is cancelled; a capability-advertising daemon gets
200; the inherit_runtime opt-in lets an outdated daemon through; and an
unmanaged or non-object config is never gated.
Verified against a real migrated schema this time (throwaway Postgres),
which is how the three failures were reproduced locally and confirmed
fixed: `go test ./internal/handler ./internal/daemon` both ok.
Co-authored-by: multica-agent <github@multica.ai>
* fix(mcp): surface the daemon-upgrade refusal and stop gating safe providers
Addresses the second review round on #6292.
1. The refusal is now visible wherever the operator looks. The default
claim path is the machine-level BATCH endpoint, which skips build
failures and still answers 200 {"tasks":[]}, so the previous bare
CancelTask showed a task that vanished with no stated reason — turning an
explicit upgrade requirement into an unexplained failure. The claim path
now fails the task with a new classified reason,
mcp_config_daemon_outdated, plus the actionable message. That reaches the
user on all three claim paths and on any daemon version, which a new
response field could not: the audience is by definition a daemon too old
to read one. The per-runtime path keeps its 412.
The reason is deliberately not auto-retryable — the same outdated daemon
would claim the retry and fail it again.
2. The gate no longer cancels safe tasks. It applied to every provider, but
only claude / codebuddy / codex / cursor / opencode / openclaw were ever
merged with host MCP by an old daemon (loadRuntimeMcpServerConfigs).
Qwen was never merged and already had strict semantics, so its tasks were
being failed for a risk that does not exist. Scoped via
providersOldDaemonsMergedRuntimeMcp; an unknown provider does not gate,
because the gate should only fire where the old behaviour is concrete.
3. The new authoritative_mcp flag now goes through the API schema layer.
Both local-skills responses were returning raw network JSON, so the flag
that decides whether the UI may assert an MCP boundary rested on an
unchecked type assertion. Adds RuntimeLocalSkillListRequestSchema with
authoritative_mcp and mcp_supported defaulting to FALSE — the fail-closed
direction — and a MALFORMED_ fallback that cannot express a guarantee.
Claim-level tests now cover all three paths, which is what the previous
helper-only tests missed: per-runtime 412, batch recording the refusal on
the task while still delivering the healthy tasks in the same batch, WS RPC
refusing and accepting, the qwen negative case, the inherit_runtime escape
hatch, and unmanaged / non-object configs.
Verified against a real migrated schema (throwaway Postgres):
go test ./internal/handler ./internal/daemon ./pkg/agent ./pkg/taskfailure
./internal/service all ok.
Co-authored-by: multica-agent <github@multica.ai>
* fix(mcp): register the new failure reason and wire its copy into the UI
Addresses the third review round on #6292.
1. mcp_config_daemon_outdated was declared but never registered in
taskfailure.allReasons, so metrics.NormalizeFailureReason missed the
known-value map and fell through to free-text Classify() — relabelling a
platform-side refusal as `agent_error.unknown` (verified directly) and
leaving the Prometheus series un-pre-warmed. Registered it, canonical
count 22 → 23 (platform 8 → 9), with the wire value and IsAgentError split
pinned. New test pins the WHOLE canonical set through
NormalizeFailureReason so forgetting the next reason fails a test instead
of quietly mislabelling a metric; NormalizeFailureReason had no coverage
at all before.
2. The upgrade copy was dead. The locale strings landed last round but
neither consumer mapped the reason: chatFailureCopy fell back to generic
failure text with the actionable detail buried in the collapsed raw
error, and task-failure.ts rendered the bare wire value
`mcp_config_daemon_outdated` in the agent activity list and issue
execution log. Both are mapped now, with regression tests, plus the
runtime class pinned in failure-class.test.ts. This directly contradicted
the claim in the claim-path comment that every path reaches the user, so
that is now actually true.
3. providersOldDaemonsMergedRuntimeMcp is documented as what it is: a FROZEN
record of what pre-capability daemons merged, not a mirror of the daemon's
current provider switch. The old "keep the two lists in lockstep" note was
actively harmful advice — runtime MCP discovery for a new provider can only
ship in a daemon that already advertises the capability (never gated), so
adding it here would fail tasks on old daemons that never merged for it,
re-creating the qwen false-positive. Pinned with a test.
Also corrects a stale count in task-failure.ts (7 → 9 platform reasons).
Verified against a real migrated schema (throwaway Postgres): full backend
suite green apart from the pre-existing environmental cmd/multica guard; all
9 TestMcpGate_* integration tests pass.
Co-authored-by: multica-agent <github@multica.ai>
* docs(taskfailure): correct taxonomy counts and finish the reason registration
Non-blocking nits from the fourth review round on #6292.
- Taxonomy counts now say 23 reasons / 9 platform-side. Registering
mcp_config_daemon_outdated last round updated the assertions but not the
prose. Swept the whole repo rather than only the flagged lines, which
turned up four more that were already stale at 21 and drifted further:
handler/dashboard.go, daemon/poisoned.go, core/types/agent.ts, and the
db/queries/task_usage.sql comment sqlc copies into the generated file.
The generated file's comment was updated by hand to match its source.
Running `sqlc generate` churned 58 lines across 47 unrelated files — the
local sqlc version differs from the one that produced the checked-in
output — so that churn was reverted and only the one intended line kept.
- failure_test.go's `required` list now includes
ReasonMcpConfigDaemonOutdated. Length and label assertions already covered
the reason, but the list is documented as the complete canonical set, so
the omission contradicted its own comment.
- Restored the line break in chat-message-list.test.tsx that a previous edit
of mine collapsed.
Comment, test-fixture and formatting only; no behaviour change.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: Eve <eve@multica-ai.local>
Co-authored-by: multica-agent <github@multica.ai>
|
||
|
Bohan Jiang
|
30318b79bc |
MUL-5426: fix(daemon): retire sessions whose history the provider refuses to replay (#6083)
* fix(daemon): retire sessions whose history the provider refuses to replay A run killed mid-reply (machine shutdown, force-quit, SIGKILL) can leave an empty assistant message in the agent CLI's transcript. Every later resume replays it, the provider rejects the request, and the (agent, issue) pair is bricked with no self-healing and no user-facing recovery. Multica already has the mechanism for this — poisoned-session classification — but its detector paired "400" with "invalid_request_error", which is the Anthropic wire shape. The same defect reported by any other provider carried neither token, so it classified as agent_error.unknown: resume-safe by omission. GetLastTaskSession kept handing back the dead session on every follow-up, manual Rerun resolved it through the same predicate, and the in-turn fresh-session retry never fired because ResumeRejected is false here (nothing rejected the resume — the transcript loaded and the provider refused to replay it). Add taskfailure.UnresumableHistory, which recognises the defect by what the provider says is wrong — some content is empty, and here is which message in the history — rather than by status code or provider name. Both signals are required, so a tool reporting "field must not be empty" does not match. Wire it into the four places that decide whether a session survives: - classifyPoisonedError, so the task is written as api_invalid_request - shouldRetryWithFreshSession, so the turn recovers on all 17 backends instead of the subset whose adapter learned to detect it; the tools == 0 gate is unchanged, so a run that already used a tool is never re-run - ResumeUnsafeFailure, covering the manual-Rerun path - both resume queries, as defense-in-depth for hosts whose daemon predates this (self-host daemons upgrade on their own cadence) Fixes #6066. Also covers the daemon half of #5760. Co-authored-by: multica-agent <github@multica.ai> * fix(session): close the Chat and fresh-retry paths that resurrect a poisoned session Review found the previous commit stopped short in two places, both of which put the dead transcript back in play. Chat never consulted the guarded query. The claim handler reads chat_session.session_id first and only falls back to GetLastChatTaskSession when it is empty, so a poisoned pointer there bypasses every filter that query applies. The fail path merely declined to OVERWRITE the pointer, leaving it in place. It now clears it in the same transaction, matched on session and runtime so a concurrent turn's newer pointer survives. The promote guard moves to ResumeUnsafeFailure as well — the reason-only check passed an un-upgraded daemon's agent_error.unknown row and re-pinned what the clear had just removed. GetLastChatTaskSession also kept the row-level filter the issue query dropped in GH #5975: it discarded the newest poisoned row and fell back to an older completed row carrying the same dead session. It now judges each session by its latest terminal state, matching GetLastTaskSession. A recovered turn could not retire anything. A terminal report carried one session_id, and an empty one meant both "nothing to report" and "forget the old session", so a fresh-session retry that SUCCEEDED left the id it retried away from selectable — through an older completed row on the issue, or through the chat pointer. agent_task_queue.retired_session_id records the abandonment itself, reported on every terminal path including completed, and both resume lookups exclude it. This is the contract gap the previous PR deferred; the fresh-retry path now runs on all backends, so deferring it is not safe. Also narrows what the cross-backend test claims: it pins the shared decision, not that all 17 adapters surface the error into Result.Error (#5760 is the counter-example), and says so. Co-authored-by: multica-agent <github@multica.ai> * test(session): require pgx.ErrNoRows in the resume-exclusion assertions The `if err == nil && prior.SessionID.Valid` form these tests shared is false-green: any real fault — undefined column, syntax error, dead connection — makes err non-nil, so the condition is false and the test passes. Run against a database missing this branch's new column, the exclusion tests reported PASS on a SQLSTATE 42703, meaning they could not have caught a broken query. requireSessionExcluded demands pgx.ErrNoRows specifically and fails loudly on anything else, so a green run now means the filter worked rather than the query never ran. Applied to all nine sites, not just the four this branch added: the other five guard the same GetLastTaskSession exclusion behaviour that this branch changes, so leaving them false-green would leave the change under-tested. All nine pass on a correctly migrated database. Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Bohan-J <bohan@devv.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Bohan Jiang
|
c271f80999 |
MUL-5370 fix: label stalled skill-bundle downloads, align failure-reason copy with the backend taxonomy (#6001)
* fix(daemon): label stalled skill-bundle downloads and make them retryable A skill bundle that could not be downloaded during task preparation surfaced as the bare string "resolve skill bundles: context deadline exceeded". taskfailure.Classify has no rule for a Go context deadline, so it landed in agent_error.unknown — a bucket that is NOT on the server's retry allowlist. A transient stall therefore became a terminal chat failure carrying a label nobody could act on, and the failure was invisible on the Usage page's Errors breakdown. (MUL-5370) - Add the platform-side reason skill_bundle_unavailable and put it on retryableReasons. Retrying is cheap and safe: the agent process never started, and bundles that did arrive are already cached on disk, so successive attempts converge. - Carry a sentinel error from the resolve loop so the reason is derived structurally rather than by matching the wrapped transport error's text, and name the skill, its declared size and the elapsed wait in the wrap — enough to tell "this bundle is too big for the link" from "the link is dead" without reading daemon logs. - Normalise the wire shape an OLD daemon produces (a non-empty catchall plus the previous "resolve skill bundles:" wrapper) on the server side. Installed daemons upgrade on their own cadence, and FailTask only classifies when the caller supplied nothing, so without this the fix would reach only hosts that happened to update — while the un-upgraded hosts most likely to be hitting the bug kept failing terminally. - Teach Classify about "deadline exceeded" and net/http's "Client.Timeout exceeded while awaiting" so any other Go-side deadline that reaches it as text stops falling into the unknown bucket too. - Backfill historical rows in both agent_task_queue and chat_message. Scoped to agent_error.unknown alone — the old wrapper string postdates the in-flight classifier by three weeks, so no row carrying it can hold the legacy coarse value — which keeps the down migration an exact inverse. Co-authored-by: multica-agent <github@multica.ai> * fix(chat): give chat its own failure copy for the refined reasons #5991 rebuilt the operator-facing failure labels around an open wire string with a raw-value fallback, but the chat bubble kept its own exact-key lookup against the six coarse values from migration 055. So all 14 agent_error.* values still missed and rendered the generic "Something went wrong and the agent couldn't finish replying" — the classification the backend had already computed was discarded at the last step, and that is the message the MUL-5370 reporter saw. - Add resolveFailureReasonKey in packages/core: exact match, else degrade an `agent_error.*` value to its family, else undefined. A reason newer than the shipped client now lands on the family line instead of the fallback. - Rekey the chat copy map by wire value and route it through the helper. Chat deliberately degrades to friendly copy rather than adopting the operator surfaces' raw-value fallback: it is read by the person who just sent a message, and the raw error is one click away under the collapsible. - Add refined chat copy (en / zh-Hans / ja / ko) only where it can say something the family line can't — a different next step: network, auth, quota, rate limit, context overflow, missing/outdated CLI, skill download. - Give skill_bundle_unavailable a label on the web and mobile surfaces and a class on the Usage page's Errors breakdown (runtime — the operator response is "check the daemon's link to Multica", the provider is not involved). - Mobile's two label maps were still coarse-only for the same reason; rekey them by wire value and fill in the refined taxonomy. Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Bohan-J <bohan@devv.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Bohan Jiang
|
6dba74c3c1 |
fix(task): auto-retry transient "Connection closed mid-response" like chat (MUL-4910) (#5565)
* fix(task): auto-retry transient provider stream cut like chat (MUL-4910) Claude Code's "API Error: Connection closed mid-response" is a transient network cut. In unattended issue runs it fell through to agent_error.unknown / process_failure — neither in retryableReasons — so the task terminated. Interactive chat only appeared resilient because the CLI's own in-process retry usually recovers first; there is no chat-specific retry in Multica. Both paths share the same finalizeStreamResult -> Classify -> retryableReasons pipeline. - classify: route "connection closed" / "mid-response" to agent_error.provider_network, before the exit-status rule so the "exited with error: exit status N" variant also lands here. - retry: add provider_network to retryableReasons. It is resume-safe, so the retry child inherits the session and continues the truncated conversation instead of restarting. Tests cover the new classification (incl. exit-status variant) and the retry/resume flags. Note: mirror the substrings into the MUL-1949 offline backfill SQL to keep in-flight and historical taxonomies aligned. Co-authored-by: multica-agent <github@multica.ai> * feat(task): defer provider_network's final retry ~5s — three-tier (MUL-4910) Follow-up to the immediate-retry fix: make the connection-closed retry a three-tier schedule — first run + immediate retry + one retry deferred ~5s — so a blip that survives the immediate retry gets a short cooldown before the final attempt instead of firing back-to-back. Reuses the existing deferred/fire_at primitive (the comment-routing escalation mechanism) rather than adding new infrastructure: CreateRetryTask gains an optional fire_at; when set, the child is inserted 'deferred' and the existing PromoteDueDeferredTasksForRuntime sweeper — already run promote-first on every claim poll — flips it to 'queued' at fire_at. No migration, no claim-query change, no daemon change. - retryAttemptCeiling: raise provider_network's ceiling to 3 (other reasons keep max_attempts=2); applied in both retryEligible and MaybeRetryFailedTask's budget pre-check so the primary and sweeper paths agree. - retryDelayForAttempt: only provider_network's final attempt is deferred (5s); every other retry — including provider_network's first — stays immediate. - FailTask + MaybeRetryFailedTask pass fire_at and skip the queued broadcast/notify for a deferred child (promotion emits them at fire time). Timing: the deferred child fires on the first claim poll at/after fire_at, so >= 5s; on an otherwise-idle runtime it can stretch to the poll interval — the same behaviour deferred escalations already have. Tests: pure schedule/eligibility coverage (TestProviderNetworkRetrySchedule) plus a DB test asserting fire_at controls deferred-vs-queued, attempt=3, and resume-safety. Co-authored-by: multica-agent <github@multica.ai> * fix(task): persist reason-aware retry budget; respect max_attempts=1 disable (MUL-4910) Addresses the pre-merge review must-fix: retryAttemptCeiling raised provider_network to 3 unconditionally, which (1) overrode the max_attempts=1 "auto-retry disabled" contract and (2) persisted a self-contradictory child (attempt=3, max_attempts=2) that leaks to the task API, so a naive attempt < max_attempts consumer would misjudge the budget. - retryAttemptCeiling now returns taskMaxAttempts unchanged when it is <= 1 (disabled stays disabled) and only ever WIDENS otherwise (max(col, 3) for provider_network) — a higher configured budget is kept. - CreateRetryTask takes an optional max_attempts; FailTask and MaybeRetryFailedTask write the reason-aware effective ceiling into the child so the whole retry chain self-describes (attempt=3, max_attempts=3). NULL inherits the parent column, so non-provider_network reasons are unchanged. Tests: - pure: ceiling widens to 3, keeps a higher budget, and never revives a disabled (max_attempts=1) task; eligibility rejects the disabled case. - DB (end-to-end FailTask): default budget → deferred final child at attempt=3/max_attempts=3; first failure → immediate child; max_attempts=1 → no child. Plus CreateRetryTask persists the passed budget / inherits on NULL. Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Bohan-J <bohan@devv.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Rusty Raven
|
7356b56a10 |
fix(taskfailure): anchor HTTP status-code matches to digit boundaries (#5275)
Anchor the 401/402/403/429/529 HTTP status-code matches to digit boundaries so embedded numbers (e.g. "402913 tokens", "15290ms") no longer misclassify process/unknown failures as provider errors and skew failure observability. Mirrors the existing 5xx anchoring (providerHTTP5xxRe). Fixes #5271 MUL-4422 |
||
|
Multica Eve
|
10afd1af1b |
feat(server): introduce pkg/taskfailure classifier and switch in-flight failure_reason writes (MUL-2946) (#3693)
Lift MUL-1949's offline backfill failure_reason taxonomy into a shared in-flight classifier so the agent_task_queue.failure_reason column is written with refined values (provider_auth_or_access, context_overflow, provider_capacity_or_rate_limit, …) at write time rather than waiting on SQL backfill to re-classify after the fact. PR1 of the Grafana board plan in MUL-2328 — the upcoming PR2 reuses pkg/taskfailure.AllReasons() to pre-warm the Prometheus failure_reason label set. * server/pkg/taskfailure: new package with the canonical 21 Reason constants (7 platform-side + 14 agent_error.* sub-reasons), AllReasons() returning a defensive copy, IsAgentError() prefix check, and Classify(rawError) Reason mirroring the SQL CASE rules from MUL-1949 (db-boy's analysis). 100% statement coverage. * server/internal/daemon/daemon.go: route the 'agent_error' coarse fallback paths (StartTask error, runTask early-return error, CompleteTask permanent rejection, reportTaskResult default branch) and the executeAndDrain default error case (chained after classifyPoisonedError) through taskfailure.Classify so blocked / timeout / unknown-status results all carry a refined reason on the wire. * server/internal/service/task.go: FailTask classifies errMsg when the daemon-supplied failureReason is empty, eliminating the legacy COALESCE(.., 'agent_error') landing. * server/internal/daemon/poisoned.go: alias FailureReasonIterationLimit and FailureReasonAPIInvalidRequest to the canonical taskfailure constants. agent_fallback_message and codex_semantic_inactivity are pre-existing operational reasons not in the canonical 21 — kept as literals for now and revisited in a follow-up PR. Backfill SQL from MUL-1949 stays as the authoritative offline source of truth; this PR keeps the in-flight classifier in lock-step with the SQL CASE expression so historical and future rows share the same taxonomy. No behavior change for the platform-side reasons (queued_expired, runtime_offline, runtime_recovery, timeout, etc.) which already align with the canonical set. Co-authored-by: Eve <eve@multica-ai.local> Co-authored-by: multica-agent <github@multica.ai> |