mirror of
https://github.com/multica-ai/multica.git
synced 2026-07-09 23:46:20 +02:00
8572a79950
* MUL-2215: fix(daemon): close handleRuntimeGone success/straggler race handleRuntimeGone coalesced concurrent recoveries with a per-workspace `reregisterNextAttempt` slot that was deleted immediately on success. A late-arriving goroutine whose `removeStaleRuntime` was delayed by mutex contention could reach the coalesce gate after the winner cleared the slot, observe no slot, re-claim, and double-register — the source of the intermittent `register endpoint called 2 times under stampede, want 1` failure on PR #2348. The slot delete on success is intentional (a genuinely later distinct deletion in the same workspace must register again, validated by TestHandleRuntimeGone_DistinctDeletionsWithinCoalesceWindowBothRecover), so we can't just extend the slot's lifetime. Add a second per-workspace gate: `reregisterLastCompletedAt`. Every call captures `entryAt` at the top of handleRuntimeGone; at the coalesce gate a caller bails if `lastCompletedAt >= entryAt`, i.e. a peer's register completed AFTER we entered the function. Same-wave stragglers bail deterministically; distinct later events have `entryAt > lastCompletedAt` and proceed. Extracted the gate into `tryClaimRegisterSlot` / `recordRegisterCompletion` so the race can be exercised deterministically with synthetic timestamps instead of relying on `-count=N` to win the scheduling lottery. - TestHandleRuntimeGone_CoalescesConcurrentCallers: -count=500 -race clean (previously intermittent). - New unit tests cover the straggler bail, the distinct-later-event claim, failure backoff suppression, and peer-holds-slot coalescing. Co-authored-by: multica-agent <github@multica.ai> * MUL-2215: narrow completion stamp to success path Second review caught that recordRegisterCompletion stamped lastCompletedAt on both success and failure. A failed register has not covered any workspace state, so a same-wave straggler whose entryAt predates the failure must be allowed to retry once the failure backoff expires — the previous behavior would let the failure-time stamp also hide that straggler. workspaceSyncLoop only retries when a workspace's runtimeIDs fully drain, so partial-deletion recovery has to come from the straggler path. Failure path now only updates reregisterNextAttempt; success path keeps its existing stamp + slot clear. Add a regression test covering the entryAt-before-failed-completion / arrival-past-backoff edge. Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: multica-agent <github@multica.ai>