Files
multica/server/pkg
Bohan Jiang 60bae62622 feat(codex): add per-exec_command watchdog to escape dropped function_call_output (MUL-2337) (#2779)
* feat(codex): add per-exec_command watchdog to escape dropped function_call_output (MUL-2337)

Codex app-server can drop the second function_call_output when two
exec_command calls fan out in the same turn and both async-yield through
the yield_time_ms boundary (observed 2026-05-18, MUL-2334 — Trump Agent
wedged for 6+ min with no semantic activity events to drive any existing
timer). The model then waits forever for the missing output; only the
10-minute semantic inactivity timeout would eventually rescue the run.

Add a per-call watchdog in the codex client that tracks open
exec_command / commandExecution items by call_id and fails the turn
quickly (default 2 min, configurable via ExecOptions.ExecCommandStuckTimeout)
when one stays open without progress. outputDelta events reset the
per-call progress timestamp so long-running streaming commands aren't
flagged.

This is a daemon-side mitigation only — codex itself still has the
upstream race, but the daemon no longer burns the full inactivity budget
before the run is marked failed and a new run can recover.

Co-authored-by: multica-agent <github@multica.ai>

* feat(codex): track legacy exec_command_output_delta in watchdog (MUL-2337)

Mirrors the raw v2 item/commandExecution/outputDelta refresh on the legacy
codex/event protocol so a long-running streaming exec doesn't get falsely
flagged as stuck after begin + 2 min.

Co-authored-by: multica-agent <github@multica.ai>

---------

Co-authored-by: multica-agent <github@multica.ai>
2026-05-18 15:14:45 +08:00
..