mirror of
https://github.com/multica-ai/multica.git
synced 2026-08-04 17:18:35 +02:00
fix/codex-patch-apply-payload
7 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Bohan Jiang
|
30318b79bc |
MUL-5426: fix(daemon): retire sessions whose history the provider refuses to replay (#6083)
* fix(daemon): retire sessions whose history the provider refuses to replay A run killed mid-reply (machine shutdown, force-quit, SIGKILL) can leave an empty assistant message in the agent CLI's transcript. Every later resume replays it, the provider rejects the request, and the (agent, issue) pair is bricked with no self-healing and no user-facing recovery. Multica already has the mechanism for this — poisoned-session classification — but its detector paired "400" with "invalid_request_error", which is the Anthropic wire shape. The same defect reported by any other provider carried neither token, so it classified as agent_error.unknown: resume-safe by omission. GetLastTaskSession kept handing back the dead session on every follow-up, manual Rerun resolved it through the same predicate, and the in-turn fresh-session retry never fired because ResumeRejected is false here (nothing rejected the resume — the transcript loaded and the provider refused to replay it). Add taskfailure.UnresumableHistory, which recognises the defect by what the provider says is wrong — some content is empty, and here is which message in the history — rather than by status code or provider name. Both signals are required, so a tool reporting "field must not be empty" does not match. Wire it into the four places that decide whether a session survives: - classifyPoisonedError, so the task is written as api_invalid_request - shouldRetryWithFreshSession, so the turn recovers on all 17 backends instead of the subset whose adapter learned to detect it; the tools == 0 gate is unchanged, so a run that already used a tool is never re-run - ResumeUnsafeFailure, covering the manual-Rerun path - both resume queries, as defense-in-depth for hosts whose daemon predates this (self-host daemons upgrade on their own cadence) Fixes #6066. Also covers the daemon half of #5760. Co-authored-by: multica-agent <github@multica.ai> * fix(session): close the Chat and fresh-retry paths that resurrect a poisoned session Review found the previous commit stopped short in two places, both of which put the dead transcript back in play. Chat never consulted the guarded query. The claim handler reads chat_session.session_id first and only falls back to GetLastChatTaskSession when it is empty, so a poisoned pointer there bypasses every filter that query applies. The fail path merely declined to OVERWRITE the pointer, leaving it in place. It now clears it in the same transaction, matched on session and runtime so a concurrent turn's newer pointer survives. The promote guard moves to ResumeUnsafeFailure as well — the reason-only check passed an un-upgraded daemon's agent_error.unknown row and re-pinned what the clear had just removed. GetLastChatTaskSession also kept the row-level filter the issue query dropped in GH #5975: it discarded the newest poisoned row and fell back to an older completed row carrying the same dead session. It now judges each session by its latest terminal state, matching GetLastTaskSession. A recovered turn could not retire anything. A terminal report carried one session_id, and an empty one meant both "nothing to report" and "forget the old session", so a fresh-session retry that SUCCEEDED left the id it retried away from selectable — through an older completed row on the issue, or through the chat pointer. agent_task_queue.retired_session_id records the abandonment itself, reported on every terminal path including completed, and both resume lookups exclude it. This is the contract gap the previous PR deferred; the fresh-retry path now runs on all backends, so deferring it is not safe. Also narrows what the cross-backend test claims: it pins the shared decision, not that all 17 adapters surface the error into Result.Error (#5760 is the counter-example), and says so. Co-authored-by: multica-agent <github@multica.ai> * test(session): require pgx.ErrNoRows in the resume-exclusion assertions The `if err == nil && prior.SessionID.Valid` form these tests shared is false-green: any real fault — undefined column, syntax error, dead connection — makes err non-nil, so the condition is false and the test passes. Run against a database missing this branch's new column, the exclusion tests reported PASS on a SQLSTATE 42703, meaning they could not have caught a broken query. requireSessionExcluded demands pgx.ErrNoRows specifically and fails loudly on anything else, so a green run now means the filter worked rather than the query never ran. Applied to all nine sites, not just the four this branch added: the other five guard the same GetLastTaskSession exclusion behaviour that this branch changes, so leaving them false-green would leave the change under-tested. All nine pass on a correctly migrated database. Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Bohan-J <bohan@devv.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Bohan Jiang
|
85a14cde37 |
fix(daemon): gate codex session pointer writes on rollout presence (MUL-5305) (#5960)
* fix(daemon): gate codex session pointer writes on rollout presence (MUL-5305) Codex issue follow-ups on local_directory projects intermittently lost their session: the server sent a prior session whose rollout was not in the task CODEX_HOME, so the daemon dropped the resume and started a fresh thread (gateCodexResumeToRolloutPresence), losing the conversation. Root of the bad pointer: the daemon persists a Codex session id as the resumable pointer at two points -- the mid-flight pin and the terminal report -- before the rollout is guaranteed on disk. A task that exits early (crash / runtime offline / timeout) leaves a pinned/reported session id with no rollout; GetLastTaskSession (which accepts failed rows) then hands it to the next follow-up, which drops it. Enforce the invariant at write time: only record a Codex session as the resumable pointer once its rollout is present in the per-issue store, with a short bounded wait for flush. If it never lands, don't overwrite the last good pointer -- a blanked session_id becomes NULL server-side, so GetLastTaskSession falls back to the most recent session whose rollout is real. Non-Codex providers are unaffected; crash recovery is preserved because a present rollout still pins. - codexSessionResumable: shared write-time presence check (bounded wait) - runTask: gate the terminal session_id before reporting - executeAndDrain: gate the mid-flight pin (thread codexHome through) - tests: helper cases + behavioral pin test Co-authored-by: multica-agent <github@multica.ai> * fix(daemon): address review — don't silently downgrade completed sessions (MUL-5305) Follow-up to review feedback on #5960: - Must-fix 1 (silent downgrade): limit the write-time session withholding to NON-completed terminal states. A missing rollout means no resumable conversation was persisted, so a withheld non-completed attempt loses nothing; a completed session is authoritative and, if its rollout is anomalously absent, is still recorded so the next run's resume gate discloses the loss (PriorSessionResumeUnavailable, MUL-4424) instead of silently falling back to an older session. Extracted resumableTerminalSessionID. - Non-blocking risk: pin the mid-flight resume pointer with a per-status presence check instead of one fixed 2s window, and set sessionPinned only once the rollout is confirmed, so a rollout that lands shortly after the first status is still pinned this run. - Must-fix 2 (regression coverage): pin skipped when rollout absent (no /session call); terminal helper (completed keeps / failed withholds); and a DB-backed GetLastTaskSession test proving the next claim falls back to the older recorded session when the latest was blanked. Co-authored-by: multica-agent <github@multica.ai> * fix(daemon): disclose Codex session continuity gaps end-to-end (MUL-5305) Addresses review feedback on #5960. Must-fix 1 — a completed turn whose rollout is missing is exactly the #5934 case (the reporter waits for each turn to finish), so it can no longer be excluded from withholding. Withhold the session for ANY terminal state, and pair the withhold with a persisted continuity-gap signal so the next claim still discloses the loss even while resuming an older good session: - new agent_task_queue.session_rollout_missing column (migration 224) - daemon sends session_rollout_missing on the terminal report; the handler clears the resume pointer (MarkTaskSessionRolloutMissing, overriding FailAgentTask's COALESCE) and flags the row - claim reads GetLatestTaskRolloutMissing and sets a new prior_session_resume_unavailable response field, which the daemon ORs into the brief's PriorSessionResumeUnavailable disclosure Must-fix 2 — Codex reveals the session id on a single task_started status, so a one-shot presence check missed a rollout that flushed later and lost in-flight crash recovery. Pin via a background waiter bounded by the run's context that pins the moment the rollout lands. Tests: - completed + rollout missing -> next claim withholds the bad session AND flags the continuity gap (cross-layer DB test) - session pinned once its rollout appears after the status (mid-run) - pin skipped while the rollout is absent Co-authored-by: multica-agent <github@multica.ai> * fix(server): make continuity-gap write atomic + disclose on all claim paths (MUL-5305) Addresses review round 3 of #5960. Must-fix 1 — the previous handler-level marker ran AFTER the terminal transaction committed, and FailTask creates + wakes the auto-retry inside that same transaction, so a retry could claim the rollout-missing session before the marker cleared it (and a marker failure was swallowed). Move session_rollout_missing INTO the terminal write: CompleteAgentTask and FailAgentTask now force session_id NULL (overriding Fail's COALESCE that would keep a stale mid-flight pin) and set the flag in the SAME UPDATE, so the withhold + gap flag commit atomically with the retry creation. The flag is threaded through TaskService.CompleteTask/FailTask; the swallowed best-effort MarkTaskSessionRolloutMissing query is removed. Must-fix 2 — the daemon withholds for all Codex tasks, but only the issue non-rerun claim consumed the disclosure. Now every fallback path sets prior_session_resume_unavailable: the manual-rerun branch reads the source task's session_rollout_missing, and the chat branch reads a new GetLatestChatTaskRolloutMissing. Tests (cross-layer DB): - completed + rollout missing via the real CompleteAgentTask terminal write -> session withheld AND gap flagged - failed + rollout missing forces session_id NULL over the COALESCE- preserved mid-flight pin in ONE statement Deploy order: migration + server first, daemon second (new fields are omitempty and ignored by an old peer). Co-authored-by: multica-agent <github@multica.ai> * fix(handler): return 5xx on FailTask error + cover claim-response gap paths (MUL-5305) Addresses review round 4 of #5960. Must-fix 1 — the FailTask handler returned 400 on a service/DB error, but the daemon's terminal callback treats 400 as permanent (postJSONWithRetry / isTransientError bails without retrying). Since the fail transaction is now the sole persistence point for the withheld session + continuity-gap flag + auto-retry, a rolled-back fail must be retried, so return 5xx (an invalid request body still returns 400), mirroring CompleteTask. Regression: client.FailTask retries on a transient 5xx and eventually succeeds. Must-fix 2 — add claim-response-level regressions that drive the two new disclosure branches through buildClaimedTaskResponse: - chat: the latest terminal task on the session withheld -> the next chat claim sets prior_session_resume_unavailable - manual rerun: the source task withheld -> the rerun claim discloses These handler DB tests run under CI's fully-migrated database (the local workspace DB cannot set up the handler fixture). Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Bohan-J <bohan@devv.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Multica Eve
|
18d41151eb |
feat(gc): batch issue reconciliation (#5534)
Co-authored-by: Eve <eve@multica-ai.local> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Multica Eve
|
ac62f72c2a |
MUL-4480: make daemon workspace sync event-driven (#5354)
* feat(daemon): make workspace sync event-driven Co-authored-by: multica-agent <github@multica.ai> * fix(daemon): preserve trailing workspace changes Co-authored-by: multica-agent <github@multica.ai> * fix(workspace): reconcile failed creates Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Eve <eve@multica-ai.local> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Multica Eve
|
bf161f2f9c |
fix(tasks): preserve merged comment delivery (#5192)
Track actual claim-time delivery, support legacy daemons, and repair comment batches across claim, retry, edit, and delete races. MUL-4348 Co-authored-by: Eve <eve@multica-ai.local> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Bohan Jiang
|
90a737fc7e |
fix(daemon): retry terminal task callbacks on transient errors (MUL-2780) (#3443)
CompleteTask / FailTask used to be fire-once. A 1-second upstream 502 burst would drop the call, then the immediate fail-fallback also 502'd, leaving the task stuck in `running` forever and showing the agent as "still working" in the UI. Add a bounded retry around the two terminal callbacks: 4s, 8s, 16s, 32s, 64s backoff schedule (5 retries, ~124s ceiling), retrying only on transient errors (5xx, 408, 429, transport-level) and bailing immediately on permanent 4xx. Also fix a latent bug where a transient complete failure would silently downgrade a successful run to a fail: the fallback now triggers only on permanent errors. Server-side CompleteTask / FailTask are already idempotent on "already terminal", so replays from a retry are safe even if the prior 502'd response was actually persisted. Co-authored-by: J <j@multica.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
LinYushen
|
b624cd98ad |
feat: identify clients via X-Client-Platform/Version/OS (#1477)
* feat: identify clients via X-Client-Platform/Version/OS
Adds client identification headers (and matching WS query params) across
all first-party clients so the server can split logs/metrics/gating by
caller without parsing User-Agent.
- HTTP: X-Client-Platform, X-Client-Version, X-Client-OS
- WS: client_platform, client_version, client_os query params
- Platform ∈ {web, desktop, cli, daemon}; OS ∈ {macos, windows, linux}
Wired through the shared TS ApiClient/WSClient via a new identity option
on CoreProvider. Web reads its version from package.json/env; Desktop
captures version + OS synchronously in preload via sendSync IPC. Go CLI
and daemon clients populate the same headers using runtime.GOOS
(normalized darwin → macos).
Server-side adds a ClientMetadata middleware that stashes the headers in
request context; the request logger and logger.RequestAttrs surface them
on every access log and handler-level log. Realtime hub logs the same
fields on websocket connect.
CORS allowlist extended for the new headers.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* test: address client-identity PR nits
- Memoize the CoreProvider identity object on Web and Desktop, and key
WSProvider's effect on identity primitives instead of the object
reference, so unrelated parent re-renders no longer tear down and
reconnect the WebSocket.
- Add direct header-injection tests for the CLI and daemon Go HTTP
clients (X-Client-Platform/Version/OS) and a normalizeGOOS unit test
on both packages.
- Add a TS test for WSClient that asserts client_platform/client_version/
client_os land on the upgrade URL and never leak the auth token.
- Add a hub test that dials the WS endpoint with client_* query params
and asserts the "websocket connected" log entry surfaces them as
structured attributes.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|