Lift MUL-1949's offline backfill failure_reason taxonomy into a shared
in-flight classifier so the agent_task_queue.failure_reason column is
written with refined values (provider_auth_or_access, context_overflow,
provider_capacity_or_rate_limit, …) at write time rather than waiting on
SQL backfill to re-classify after the fact. PR1 of the Grafana board
plan in MUL-2328 — the upcoming PR2 reuses pkg/taskfailure.AllReasons()
to pre-warm the Prometheus failure_reason label set.
* server/pkg/taskfailure: new package with the canonical 21 Reason
constants (7 platform-side + 14 agent_error.* sub-reasons),
AllReasons() returning a defensive copy, IsAgentError() prefix check,
and Classify(rawError) Reason mirroring the SQL CASE rules from
MUL-1949 (db-boy's analysis). 100% statement coverage.
* server/internal/daemon/daemon.go: route the 'agent_error' coarse
fallback paths (StartTask error, runTask early-return error, CompleteTask
permanent rejection, reportTaskResult default branch) and the
executeAndDrain default error case (chained after classifyPoisonedError)
through taskfailure.Classify so blocked / timeout / unknown-status
results all carry a refined reason on the wire.
* server/internal/service/task.go: FailTask classifies errMsg when the
daemon-supplied failureReason is empty, eliminating the legacy
COALESCE(.., 'agent_error') landing.
* server/internal/daemon/poisoned.go: alias FailureReasonIterationLimit
and FailureReasonAPIInvalidRequest to the canonical taskfailure
constants. agent_fallback_message and codex_semantic_inactivity are
pre-existing operational reasons not in the canonical 21 — kept as
literals for now and revisited in a follow-up PR.
Backfill SQL from MUL-1949 stays as the authoritative offline source of
truth; this PR keeps the in-flight classifier in lock-step with the SQL
CASE expression so historical and future rows share the same taxonomy.
No behavior change for the platform-side reasons (queued_expired,
runtime_offline, runtime_recovery, timeout, etc.) which already align
with the canonical set.
Co-authored-by: Eve <eve@multica-ai.local>
Co-authored-by: multica-agent <github@multica.ai>
* fix(daemon): treat upstream API 400 invalid_request_error as poisoned session
A markdown-linked image in an issue description that the agent downloads as
a tiny CDN auth-error file and Read's as a PNG poisons the conversation:
the LLM API rejects the bad image with 400 invalid_request_error, the
session_id is pinned mid-flight, and every follow-up task on the issue
(comment-trigger, auto-retry) resumes the same poisoned conversation and
hits the same 400 — the issue can no longer be executed even after the
description is cleaned up.
Mirror the existing fallback-output classifier on the error side: detect
"API Error: ... 400 ... invalid_request_error" in the agent error string,
persist failure_reason='api_invalid_request', and add it to the
GetLastTaskSession exclusion list so the next task starts a fresh
session that re-reads the (now-clean) description.
Co-authored-by: multica-agent <github@multica.ai>
* fix(daemon): unblock issues already poisoned by API 400 invalid_request_error
The forward-only classifier from the previous commit only tags new failures.
Issues like MUL-1918 already have multiple failed-task rows whose
failure_reason is the pre-fix default 'agent_error', and GetLastTaskSession
falls back to those legacy rows on the next claim — so deploying the
classifier alone leaves existing poisoned issues stuck (GPT-Boy review
on PR #2314).
Two complementary changes:
- Migration 079 backfills failure_reason='api_invalid_request' on every
pre-existing 'agent_error' row whose error text matches the canonical
Anthropic 400 invalid_request_error shape. Keeps observability
consistent (multica issue runs / UI now report the right reason).
- GetLastTaskSession adds a defensive ILIKE clause on error text. Closes
the deploy-window gap where the old binary could write a new
'agent_error' row between the migration running and the new code
taking over, and protects against future error-format variants the
daemon classifier might miss.
Plus regression tests covering the legacy + new coexistence case GPT-Boy
flagged, and a guard rail asserting benign 'agent_error' failures
(timeouts, tool errors) still resume their session.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: multica-agent <github@multica.ai>
* fix(task): rerun starts a fresh session, skip poisoned resume
When a task ended in a known agent fallback ("I reached the iteration
limit and couldn't generate a summary.", "Put your final update inside
the content string. Keep it concise.") the (agent_id, issue_id) resume
lookup would still pick that session, so a manual rerun inherited the
poisoned state and reproduced the same bad output.
Two complementary guards:
1. Daemon classifies poisoned terminal output and routes it through the
blocked path with failure_reason set ('iteration_limit' /
'agent_fallback_message'). GetLastTaskSession excludes failed tasks
with those reasons, so even comment-triggered tasks no longer resume
them. Tasks that failed mid-flight (timeout, runtime_recovery, etc.)
are still resumable, preserving MUL-1128's auto-retry contract.
2. Manual rerun marks the new task force_fresh_session=true. The daemon
claim handler skips the resume lookup entirely when the flag is set,
capturing the user-intent signal that "the prior output was bad" even
when poisoned classification misses a future fallback wording.
Auto-retry of orphaned mid-flight failures (MaybeRetryFailedTask →
CreateRetryTask) does not take this path, so it keeps resuming.
Tests: classifyPoisonedOutput unit test; integration tests assert the
SQL filter excludes poisoned classifiers, RerunIssue flips the flag,
and the normal enqueue path leaves it false.
Co-authored-by: multica-agent <github@multica.ai>
* fix(daemon): cap poisoned-output matcher to short trimmed text
GPT-Boy review on MUL-1630: the previous strings.Contains match would
classify any output that quoted the marker substring — including a
review/analysis that simply discussed the marker itself. Real fallback
messages are short single-sentence affairs, so cap the candidate at
~one paragraph and trim whitespace before matching. Adds regression
tests covering a long quoting review and a marker buried in a long
real conclusion; both must stay classified as completed.
Co-authored-by: multica-agent <github@multica.ai>
* fix(migrations): rename 065 force_fresh_session → 066 to clear collision
main introduced 065_project_resources after this branch was cut, so
both files shared the 065_ prefix. The readiness check
(server/cmd/server/health.go → migrations.LatestVersion) takes the
last entry by lexical order, which is 065_project_resources, leaving
this branch's 065_force_fresh_session unguarded — a deploy that
applied project_resources but not force_fresh_session would still
report ready, and the next enqueue / rerun / claim would crash on
"column force_fresh_session does not exist".
Renaming to 066_force_fresh_session puts it strictly after
project_resources so readiness blocks until it's applied.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: multica-agent <github@multica.ai>
