Bohan Jiang 6dba74c3c1 fix(task): auto-retry transient "Connection closed mid-response" like chat (MUL-4910) (#5565) ...

* fix(task): auto-retry transient provider stream cut like chat (MUL-4910)

Claude Code's "API Error: Connection closed mid-response" is a transient
network cut. In unattended issue runs it fell through to
agent_error.unknown / process_failure — neither in retryableReasons — so
the task terminated. Interactive chat only appeared resilient because the
CLI's own in-process retry usually recovers first; there is no
chat-specific retry in Multica. Both paths share the same
finalizeStreamResult -> Classify -> retryableReasons pipeline.

- classify: route "connection closed" / "mid-response" to
  agent_error.provider_network, before the exit-status rule so the
  "exited with error: exit status N" variant also lands here.
- retry: add provider_network to retryableReasons. It is resume-safe, so
  the retry child inherits the session and continues the truncated
  conversation instead of restarting.

Tests cover the new classification (incl. exit-status variant) and the
retry/resume flags. Note: mirror the substrings into the MUL-1949 offline
backfill SQL to keep in-flight and historical taxonomies aligned.

Co-authored-by: multica-agent <github@multica.ai>

* feat(task): defer provider_network's final retry ~5s — three-tier (MUL-4910)

Follow-up to the immediate-retry fix: make the connection-closed retry a
three-tier schedule — first run + immediate retry + one retry deferred ~5s —
so a blip that survives the immediate retry gets a short cooldown before the
final attempt instead of firing back-to-back.

Reuses the existing deferred/fire_at primitive (the comment-routing escalation
mechanism) rather than adding new infrastructure: CreateRetryTask gains an
optional fire_at; when set, the child is inserted 'deferred' and the existing
PromoteDueDeferredTasksForRuntime sweeper — already run promote-first on every
claim poll — flips it to 'queued' at fire_at. No migration, no claim-query
change, no daemon change.

- retryAttemptCeiling: raise provider_network's ceiling to 3 (other reasons
  keep max_attempts=2); applied in both retryEligible and
  MaybeRetryFailedTask's budget pre-check so the primary and sweeper paths agree.
- retryDelayForAttempt: only provider_network's final attempt is deferred (5s);
  every other retry — including provider_network's first — stays immediate.
- FailTask + MaybeRetryFailedTask pass fire_at and skip the queued
  broadcast/notify for a deferred child (promotion emits them at fire time).

Timing: the deferred child fires on the first claim poll at/after fire_at, so
>= 5s; on an otherwise-idle runtime it can stretch to the poll interval — the
same behaviour deferred escalations already have.

Tests: pure schedule/eligibility coverage (TestProviderNetworkRetrySchedule)
plus a DB test asserting fire_at controls deferred-vs-queued, attempt=3, and
resume-safety.

Co-authored-by: multica-agent <github@multica.ai>

* fix(task): persist reason-aware retry budget; respect max_attempts=1 disable (MUL-4910)

Addresses the pre-merge review must-fix: retryAttemptCeiling raised
provider_network to 3 unconditionally, which (1) overrode the
max_attempts=1 "auto-retry disabled" contract and (2) persisted a
self-contradictory child (attempt=3, max_attempts=2) that leaks to the
task API, so a naive attempt < max_attempts consumer would misjudge the
budget.

- retryAttemptCeiling now returns taskMaxAttempts unchanged when it is <= 1
  (disabled stays disabled) and only ever WIDENS otherwise (max(col, 3) for
  provider_network) — a higher configured budget is kept.
- CreateRetryTask takes an optional max_attempts; FailTask and
  MaybeRetryFailedTask write the reason-aware effective ceiling into the
  child so the whole retry chain self-describes (attempt=3, max_attempts=3).
  NULL inherits the parent column, so non-provider_network reasons are
  unchanged.

Tests:
- pure: ceiling widens to 3, keeps a higher budget, and never revives a
  disabled (max_attempts=1) task; eligibility rejects the disabled case.
- DB (end-to-end FailTask): default budget → deferred final child at
  attempt=3/max_attempts=3; first failure → immediate child; max_attempts=1
  → no child. Plus CreateRetryTask persists the passed budget / inherits on NULL.

Co-authored-by: multica-agent <github@multica.ai>

---------

Co-authored-by: Bohan-J <bohan@devv.ai>
Co-authored-by: multica-agent <github@multica.ai>

2026-07-17 16:47:59 +08:00

..

classify_test.go

fix(task): auto-retry transient "Connection closed mid-response" like chat (MUL-4910) (#5565)

2026-07-17 16:47:59 +08:00

classify.go

fix(task): auto-retry transient "Connection closed mid-response" like chat (MUL-4910) (#5565)

2026-07-17 16:47:59 +08:00

failure_test.go

…

failure.go

…