* feat(composio): server-side connect flow + connections REST (Notion MVP) (MUL-3720) (#4608)
* feat(composio): server-side connect flow + connections REST (Notion MVP) (MUL-3720)
Compose the merged server/pkg/composio SDK into a user-facing connection
manager: signed-state connect handshake, local user_composio_connection
mirror, idempotent disconnect, and a per-user MCP session helper (not yet
wired into task dispatch).
- migration 127_user_composio_connection (no FK/cascade, per DB rules)
- sqlc queries: upsert (idempotent on user_id+connected_account_id), list
active, owner-scoped get, mark revoked
- internal/integrations/composio: signed HMAC-SHA256 state, BeginConnect,
CompleteCallback (idempotent upsert), ListConnections, Disconnect
(upstream 404 = idempotent success), CreateMCPSession (no-op when empty,
pins connected_accounts per toolkit), CallbackRedirect
- REST handlers under /api/integrations/composio (user-scoped, 503 when
COMPOSIO_API_KEY unset): connect/init, callback (302), connections list,
delete
- router wiring gated by COMPOSIO_API_KEY; COMPOSIO_AUTH_CONFIGS_JSON maps
toolkit->auth_config (MVP: notion); state secret from COMPOSIO_STATE_SECRET
or derived from JWT_SECRET; callback base from COMPOSIO_CALLBACK_BASE_URL
or MULTICA_PUBLIC_URL
- tests: state (expire/tamper/wrong-secret), service (mapping, callback
idempotency, non-success, disconnect owner/404 idempotency, MCP pin),
handlers (httptest), redact regression for Bearer mcp_ tokens
MVP scope: Notion only; no task-dispatch overlay, sharing, or webhook
event handling (later stages).
Co-authored-by: multica-agent <github@multica.ai>
* fix(composio): bind callback account to user + idempotent revoked disconnect (MUL-3720)
Address PR 4608 review (CHANGES_REQUESTED):
- callback: verify connected_account_id with Composio before mirroring it.
The signed state only proved user/toolkit/exp, so a valid state paired with
a tampered connected_account_id would be written verbatim. CompleteCallback
now calls ListConnectedAccounts and fails closed (ErrAccountVerification)
unless the account belongs to the state's user (composio_user_id == multica
user id) and was created under the toolkit's auth config. No row is written
on mismatch / unknown account / upstream error.
- disconnect: short-circuit to a no-op when the local row is already revoked,
before touching upstream. Previously a second DELETE re-hit Composio and a
non-404 upstream error surfaced as a 502, breaking the 204-idempotent
contract.
- CreateMCPSession: document the v1 single-active-connection-per-(user,toolkit)
constraint and make duplicate selection deterministic (newest-wins, rows are
connected_at DESC) instead of order-dependent map overwrite. Stage 3 owns the
real single-account-enforcement vs multi-account-shape decision.
Tests: tampered/wrong-auth-config/unknown-account callback rejection, revoked-row
disconnect no-op (asserts upstream not re-hit). composio pkg 85% coverage; all
green.
Co-authored-by: multica-agent <github@multica.ai>
* feat(composio): list all toolkits + dynamic auth-config resolution (MUL-3720)
Yushen's follow-up to the Notion MVP: surface the full Composio toolkit
catalog, render it in Settings, and drop the static env mapping in favor of
dynamic auth-config discovery.
Config correctness (per Composio docs):
- Remove COMPOSIO_AUTH_CONFIGS_JSON entirely. The toolkit→auth_config mapping
is now resolved at request time from the project's /auth_configs (cached,
5-min TTL), so enabling a toolkit is a dashboard action, not a redeploy.
- Do NOT add COMPOSIO_PROJECT_ID. The project API key (x-api-key) authenticates
to exactly one project; the project is resolved from the key. Only org-level
endpoints use x-org-api-key, which this integration never calls.
Backend:
- SDK: server/pkg/composio/auth_configs.go — ListAuthConfigs (toolkit_slug,
is_composio_managed, show_disabled, limit, cursor).
- service: dynamic resolver (authConfigMap cache; betterAuthConfig prefers a
custom/white-label config over Composio-managed, newest wins); BeginConnect
and CompleteCallback resolve via it; ListToolkits fetches the full catalog
(paginated, capped) annotated with connectable = has an enabled auth config,
connectable-first ordering.
- handler + route: GET /api/integrations/composio/toolkits (user-scoped, 503
when COMPOSIO_API_KEY unset) returning slug/name/logo/category/connectable.
Frontend:
- core: ComposioToolkit/ComposioConnection types, api client methods, and
composio query options (@multica/core/composio).
- views: Settings → Integrations now has a Composio section rendering every
toolkit as a card with search. Connect is gated on `connectable`;
non-connectable toolkits show a muted "not configured" hint instead of a
dead button. Connected toolkits show a badge + Disconnect (with confirm).
- i18n: composio block added to en/zh-Hans/ja/ko settings.
Tests: SDK + service (dynamic resolution, custom-over-managed preference,
connectable flag, resolver-error soft-degrade) and handler toolkits endpoint;
composio pkg 85.7% coverage. go build/vet/gofmt clean; core+views typecheck,
core+views lint, and core tests (691) all green.
Co-authored-by: multica-agent <github@multica.ai>
* fix(composio): close cross-toolkit callback fail-open by signing auth_config_id into state (MUL-3720)
Re-review blocker: CompleteCallback resolved the toolkit's auth config at
callback time and ignored a resolve error/empty result, while
verifyAccountOwnership skipped the auth-config comparison when the expected
value was empty. A user could then pass another toolkit's connected_account_id
into this toolkit's callback — the owner check passed and it was written under
the wrong toolkit_slug/account binding.
Fix: the auth_config_id is already resolved in BeginConnect (before the state
is signed), so sign it into the state and compare it exactly at callback. No
re-resolve, no fail-open. verifyAccountOwnership now fails closed when the
expected auth config is empty (rejects instead of skipping) and requires an
exact match — closing the cross-toolkit binding gap.
Tests: state round-trips auth_config_id; BeginConnect signs it; callback
rejects wrong/cross-toolkit auth config and an empty (no-mapping) auth config
fails closed. composio pkg 85.2% coverage, all green.
Frontend (non-blocking): the Composio settings tab now surfaces an error when
the connections query fails instead of silently rendering everything as
unconnected.
Co-authored-by: multica-agent <github@multica.ai>
* fix(composio): hide Settings section entirely when integration unconfigured (MUL-3720)
Decision (option 2, hide-then-merge): don't show a card that leaks the internal
COMPOSIO_API_KEY env-var name to every end user. IntegrationsTab now gates the
whole Composio section (heading + body) on the toolkits query — a 503 means the
key is unset, so the section is withheld instead of rendering the not-configured
card. Admin-only setup guidance is a later, role-gated affordance.
Removed the notConfigured card (and now-unused ApiError import) from
ComposioTab; it only mounts when configured. views typecheck + lint clean.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: multica-agent <github@multica.ai>
* feat(composio): Stage 2 frontend polish — callback toast, last_used & expired UI, e2e (MUL-3718) (#4688)
* feat(composio): callback toast + refresh, last_used & expired UI, e2e (MUL-3718)
Co-authored-by: multica-agent <github@multica.ai>
* fix(composio): real callback redirect route + StrictMode-safe toast dedup (MUL-3718 review)
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: multica-agent <github@multica.ai>
* fix(composio): callback endpoint should not require Multica auth (MUL-3843) (#4709)
* fix(composio): move OAuth callback out of the Auth group (MUL-3843)
Composio 302-redirects the browser to /api/integrations/composio/callback
at the end of the OAuth flow, but PR #4608 mounted it inside the cookie-auth
middleware group. When the session cookie is absent (expired session,
SameSite=Strict / Safari ITP, private window, self-hosted callback subdomain)
the Auth middleware returned a hard 401 and a JSON blob instead of the
settings redirect, breaking the flow.
Identity never came from the cookie anyway: it is carried by the HMAC-signed
state param that CompleteCallback verifies (signature, expiry, replay) and
cross-checked by verifyAccountOwnership; h.Composio == nil still 503s. So the
callback is registered alongside the other public OAuth/webhook routes; the
other four composio endpoints stay session-gated.
Refs MUL-3843, MUL-3715.
Co-authored-by: multica-agent <github@multica.ai>
* fix(composio): correct stale callback routing comments (MUL-3843)
The package header and ComposioCallback doc comments still described the
callback as sitting under the Auth middleware group. After the route was
moved out (this PR), update both to state it is a public route whose identity
comes from the signed state — addressing review nit from 张大彪.
Refs MUL-3843.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: multica-agent <github@multica.ai>
* feat(composio): inject MCP overlay into agent runtime at task dispatch (MUL-3721) (#4704)
Stage 3 of the Composio epic. Wires the per-user Composio MCP session into
every agent task so the agent process sees the initiator's connected tools
without any prompt-time plumbing.
Server side
- Migration 128 adds agent_task_queue.runtime_mcp_overlay JSONB plus a
BEFORE-UPDATE trigger that wipes the column on any transition into a
terminal status (completed / failed / cancelled). A trigger is the single
source of truth — future queries that flip status cannot bypass it.
- composio.Service.BuildTaskOverlay(userID) reuses CreateMCPSession and
emits the Claude-style { mcpServers: { composio: { type: http, url,
headers } } } shape the daemon's existing sidecar generators consume.
Returns (nil, nil) on zero active connections so we never burn a
Composio session for a user with nothing to call.
- TaskService grows a Composio ComposioOverlayBuilder seam, wired in
router.go after composiointeg.NewService succeeds. Five enqueue paths
(issue / mention / quick-create / chat / auto-retry) attach the overlay
after CreateAgentTask returns and before the daemon is notified — so
every claim reads a settled row, with no second daemon hop. Best-effort:
a builder failure logs and proceeds with no overlay.
- resolveInitiatorFromTriggerComment derives the initiator user from the
trigger comment when it was authored by a member. Agent-authored
triggers are not treated as initiators (their connected-apps view is
empty by construction).
Daemon side
- handler/daemon.go claim path merges task.runtime_mcp_overlay onto
agent.mcp_config via mergeMCPOverlay before populating
TaskAgentData.McpConfig. Overlay wins on server-name collisions
because it carries the live user-scoped session URL. Errors fall back
to the agent config unchanged — a bad overlay must not surprise-disable
saved MCP tools. The existing execenv sidecar generators (cursor /
codex / openclaw / opencode / hermes-kiro) need no changes: they keep
consuming the merged result through TaskAgentData.McpConfig.
Tests
- 9 merge cases (mcp_overlay_test): both-nil short-circuit, agent-only
pass-through, overlay-only canonicalization, two-side merge, name
collision (overlay wins), top-level key preservation, malformed agent
fallback, malformed overlay fallback, non-object server rejection.
- 4 dispatch cases (composio): zero-connections returns nil without
CreateSession, happy-path emits the right shape with the right user
id, empty-URL defensive branch, SDK error surfacing.
- 4 TaskService helper cases: nil Composio is a no-op (Queries-safe),
invalid initiator does not call the builder, nil overlay skips the
UPDATE, builder error swallowed without panic.
- Migration 128 verified to roll up + down + up cleanly against the test
database.
Out of scope (deferred): assignment-triggered enqueue paths with no
trigger comment get no overlay attached today (no initiator UUID flows
through enqueueIssueTask in that case). Retry paths recompute the overlay
fresh from the parent's initiator_user_id instead of inheriting the bearer
from the parent row, so a stale token can never resurface on a retry.
Co-authored-by: Eve <eve@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
* feat(composio): per-agent allowlist + originator-scoped MCP overlay (MUL-3869) (#4736)
* feat(composio): per-agent allowlist + originator-scoped MCP overlay (MUL-3869)
Stage 3.1 of the Composio epic (MUL-3721 parent). PR #4704 wired in the
runtime_mcp_overlay column and a per-task dispatch hook; this change
inverts the default from "all-on" to opt-in and locks the overlay to the
agent owner's own connected apps:
- Agents carry composio_toolkit_allowlist TEXT[]. NULL or [] => no MCP.
Owner-only read/write; non-owner GET/PUT silently redacts/drops the
field (same shape as mcp_config).
- agent_task_queue carries originator_user_id UUID. Set from the
top-of-chain HUMAN at every enqueue path:
* issue/mention comment by member -> author_id
* issue/mention comment by agent -> inherit via comment.source_task_id
-> parent task originator_user_id
* quick-create -> requester_id
* chat -> initiator_user_id
* retry -> SQL-inherited from parent row
* autopilot -> NULL (system-driven)
- BuildTaskOverlay (composio dispatch) now takes (ctx, originatorUserID,
agent) and short-circuits on five gates: invalid originator,
originator != agent.owner_id, empty allowlist, empty intersection of
allowlist ∩ active connections, defensive empty session URL. Composio
CreateSession is called with BOTH `toolkits.slugs` (the intersection)
AND `connected_accounts` (the pinned account ids), narrowing the
tool-router twice.
- The originator-vs-owner gate closes the agent-fanout privacy hole: any
workspace member who can @-mention a public agent used to project the
owner's connected apps into their run. Now the overlay only mounts
when the human at the top of the chain IS the agent owner.
Tests:
- dispatch_test.go covers all 5 gates plus uppercase/whitespace slug
normalisation.
- task_runtime_mcp_overlay_test.go covers the no-op gates of the new
applyRuntimeMCPOverlay signature.
- agent_composio_allowlist_test.go (handler): owner roundtrip
(list/empty/null), workspace-admin silent-drop, owner-only GET
visibility, pure normaliseComposioToolkitAllowlist.
- resolve_originator_test.go (service, DB-backed): member-authored,
agent-authored inherits via comment.source_task_id, invalid id.
Migration 129 up/down/up verified against docker postgres.
Co-authored-by: multica-agent <github@multica.ai>
* chore(composio): gofmt + regenerate sqlc with v1.31.1 (MUL-3869 review nits)
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: Eve <eve@multica-ai.local>
Co-authored-by: multica-agent <github@multica.ai>
* fix(composio): accept nested connected account auth config
* feat(views): creator-only MCP tab for per-agent Composio allowlist (MUL-3870) (#4743)
Stage 3.2 frontend on top of the Stage 3.1 backend (MUL-3869, 4708dba97).
Adds an agent-detail tab that lets the agent owner pick which of their own
active Composio connections this agent may mount as MCP servers, writing the
selection to agent.composio_toolkit_allowlist via the existing PUT /api/agents.
- core/types: composio_toolkit_allowlist (+ _redacted) on Agent; tri-state
composio_toolkit_allowlist on UpdateAgentRequest (omit/no-change, null/clear,
array/replace), matching the backend contract.
- core/agents: useUpdateAgentAllowlist - optimistic mutation hook (patches the
cached workspace agent list, rolls back on error, invalidates on settle).
- views: AgentMcpTab renders the owner's active connections as checkboxes;
empty state links to Settings -> Integrations; defensive redacted state.
- views: wired into AgentOverviewPane as tab "composio_mcp", labeled "MCP Apps"
to disambiguate from the existing raw-JSON "MCP" (mcp_config) tab. The entry
is gated to the creator (currentUserId === agent.owner_id), matching the
backend's owner-only read/write of the allowlist.
- i18n: tabs.composio_mcp + tab_body.composio_mcp.* in en/ja/ko/zh-Hans.
- tests: agent-mcp-tab.test.tsx (gating, toggle->allowlist body, active-only,
empty, redacted); e2e/agent-mcp.spec.ts (creator sees tab + PUT body,
non-creator hidden) with Composio + agent endpoints mocked at the boundary.
Note: the product spec says "creator"; the schema has no creator_id - the
backend gate and redaction are keyed on owner_id, so the tab uses owner_id.
Co-authored-by: multica-agent <github@multica.ai>
* fix(composio): mount remote MCP for codex
* feat(agents): agent invocation permission system (MUL-3963) (#4844)
* feat(agents): agent invocation permission system (permission_mode + invocation targets)
MUL-3963: split who may INVOKE an agent out of the overloaded visibility
column into an explicit, extensible model on feature/composio-integration.
- DB: agent.permission_mode (private|public_to) + agent_invocation_target
table (workspace/member/team targets) + lossless backfill from visibility
(migration 130).
- canInvokeAgent: owner-only for private (NO admin bypass, NO A2A bypass);
public_to honours the allow-list; A2A judged by the top-of-chain originator.
- All trigger paths rewired: issue assign, comment @agent/@squad, chat,
quick-create, autopilot, squad leader, child-done.
- Agent API: permission_mode + invocation_targets on responses and
create/update (owner-only writes); legacy visibility kept as a derived field
so old clients never see a permission widening.
- Composio: BuildTaskOverlay now FOLLOWS invocation permission and uses the
agent OWNER connection (removed the originator==owner gate); front-end warns
when a shared agent enables Composio apps.
- CLI: --permission-mode / --public-to-workspace / --public-to-member (legacy
--visibility still mapped).
- Frontend: AccessPicker (Private / workspace / specific people / team soon),
permission rules mirror canInvokeAgent, Composio warning banner.
- Tests: migration backfill, admin cannot invoke others private, public_to
workspace/member whitelist, A2A by originator, Composio overlay uses owner
connection.
Co-authored-by: multica-agent <github@multica.ai>
* feat(agents): stackable, mixed public_to invocation targets (MUL-3963)
Follow-up on PR #4844: public_to now supports selecting MULTIPLE, MIXED
targets on one agent (e.g. Public to workspace + specific people + team),
with canInvokeAgent admitting on ANY matching target (OR).
- Frontend AccessPicker: reworked from a single exclusive kind into a
stackable multi-select — an "Everyone in workspace" toggle, a member
multi-select checklist, and a (disabled, v1) team placeholder can be
combined freely. Emits the full union of selected targets; empty union
collapses to Private. Existing team targets are preserved across saves.
Added the access.public_group locale string (en/zh-Hans/ja/ko).
- Backend already supported this (agent_invocation_target is multi-row per
agent; create/update take a target ARRAY and batch-replace the whole
allow-list; canInvokeAgent OR-matches). Added tests to lock it in:
mixed member+team targets, overlapping-member batch replace, and
workspace+member stacking then narrowing.
Refs MUL-3963.
Co-authored-by: multica-agent <github@multica.ai>
* fix(agents): address review on invocation permission (MUL-3963)
张大彪 review on PR #4844 — three blockers + product ruling + nits:
1. Migration 130: drop the FK/cascade on agent_invocation_target
(agent_id, created_by) per the Multica no-FK rule; relationships are now
maintained in the app layer (matching MUL-3515 §4). Added
DeleteAgentInvocationTargetsByArchivedRuntimeAgents and call it before
DeleteArchivedAgentsByRuntime in all three runtime-delete paths
(runtime.go x2, runtime_profile.go) so hard-deleting agents can't orphan
target rows.
2. revokeAndRemoveMember: prune the leaving member's member-target grants
(DeleteAgentInvocationTargetsByMember) in the same tx as the member-row
delete, so a re-invited user can't reclaim a stale invocation grant.
3. Empty public_to is a phantom — parsePermissionInput now normalises a
public_to with no resolvable targets to a single workspace target, so
`--permission-mode public_to` alone (and any empty target array) means
"public to workspace" instead of "shared but nobody can run it".
Product ruling: the system/no-human-originator → workspace-target path in
canInvokeAgent is a deliberate, documented exception (webhook/system/
workspace-wide automation); member/team targets still fail closed without a
resolved originator. Documented in code + locked with a test.
Nits: refreshed the stale "originator must be owner" comments — models.go
(via migration 130 COMMENT ON COLUMN + sqlc regen for composio_toolkit_allowlist
and originator_user_id) and agent-mcp-tab.tsx — to the owner-connection +
invocation-permission rules.
Tests: member remove/re-add regression, system workspace exception + member
fail-closed, empty public_to → workspace (plus the earlier mixed/overlap/
batch-replace suite). Migration 130 applied to the test DB; Go handler/service/
composio suites green; views typecheck clean.
Refs MUL-3963.
Co-authored-by: multica-agent <github@multica.ai>
* fix(agents): scope member invocation-target cleanup to one workspace (MUL-3963)
张大彪 3rd review — cross-workspace permission bug + comment nits:
- DeleteAgentInvocationTargetsByMember was a GLOBAL delete by user id, so
removing a user from workspace A also wiped their member-target grants on
agents in workspace B. Scoped it to a single workspace by joining through
agent.workspace_id; revokeAndRemoveMember now passes (workspaceID, userID).
- Regression test TestRevokeMember_InvocationTargetCleanupIsWorkspaceScoped:
same user allow-listed by agents in two workspaces; removal from one leaves
the other workspace's target intact.
- Nits: refreshed the remaining stale "originator == agent.owner_id" /
"owner-vs-originator" comments — CreateRetryTask (agent.sql, regenerated),
and the AgentResponse allowlist doc + ListAgents/UpdateAgent redaction
rationale in agent.go — to the owner-connection + invocation-permission rule.
Migration 130 applied to the test DB; Go handler/service/composio suites green;
go vet clean.
Refs MUL-3963.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: multica-agent <github@multica.ai>
* fix(agents): agent access owner-only editable, read-only for others (MUL-3963) (#4853)
* fix(agents): make agent access owner-only editable, read-only for others (MUL-3963)
Interaction bug: a non-owner (incl. workspace admin) could open the AccessPicker
and set an agent public — the backend silently ignored it and the UI bounced
back to private. Access is owner-only, so non-owners must see a read-only state
and the backend must reject real changes explicitly.
Frontend:
- AccessPicker renders a static, non-interactive read-only state when the
viewer is not the owner: the current access value + a lock affordance + a
tooltip "Only the agent owner can change who can run this agent." No clickable
trigger is rendered, so a non-owner can never open a control the backend would
reject (the GitHub/Notion pattern for permission settings you can see but not
edit). The editable multi-select picker is unchanged for the owner.
- agent-detail-inspector gates the picker on ownership specifically
(currentUserId === agent.owner_id), NOT the general canEdit (which also admits
admins, who may edit other fields but not access).
- New locale key access.owner_only_readonly (en/zh-Hans/ja/ko).
Backend:
- UpdateAgent now returns an explicit 403 when a non-owner submits a REAL
permission change (permissionInputChangesAgent compares requested mode +
target set against the persisted state); a no-op resubmit (admin PATCH-as-PUT
echoing unchanged permission) is still tolerated so admin edits of other
fields keep working. Replaces the previous silent-drop that caused the bounce.
Tests:
- access-picker.test.tsx: non-owner gets a non-interactive read-only display
with the owner-only tooltip; owner gets an interactive picker; owner can pick
a member and stack workspace + member.
- TestUpdateAgent_AccessChangeIsOwnerOnly: admin real change → 403; admin no-op
resubmit → 200; admin editing other fields → 200; owner change → 200.
Incidental: fixed a pre-existing base typecheck break in
slash-command-suggestion.test.tsx (stray `signal` arg not in the suggestion
items type) that otherwise fails the whole @multica/views typecheck.
Refs MUL-3963.
Co-authored-by: multica-agent <github@multica.ai>
* fix(agents): compare legacy visibility, not expanded permission, for no-op detection (MUL-3963)
PR #4853 review: permissionInputChangesAgent expanded a legacy-only
visibility:"private" into a real private permission and compared it against the
agent's actual permission. A member-only public_to agent derives legacy
visibility "private", so an admin PATCH-as-PUT echoing visibility:"private"
while editing another field was misread as a public_to→private downgrade and
rejected with 403 — contradicting the "unchanged permission no-op is allowed"
contract.
Fix (per review): when a request carries ONLY legacy `visibility` (no
permission_mode / invocation_targets), derive the agent's CURRENT legacy
visibility from its real targets and compare the legacy string values. Equal =
no-op (allowed); a real legacy change (e.g. "workspace") still returns 403.
Requests that carry permission_mode / invocation_targets keep the precise
mode+target comparison.
Regression test TestUpdateAgent_LegacyVisibilityNoOpForMemberOnlyPublicTo:
member-only public_to agent — admin submitting visibility:"private" + a
non-permission field → 200 with targets unchanged; admin submitting
visibility:"workspace" → 403.
Go handler/composio suites green; migration 130 applied; go vet clean.
Refs MUL-3963.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: multica-agent <github@multica.ai>
* feat(composio): brief agents on connected apps
* feat(composio): gate MCP apps behind feature flag
* fix(mobile): parse agent invocation permissions
* fix(tests): update agent fixtures for access fields
---------
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: Multica Eve <eve@devv.ai>
Co-authored-by: Eve <eve@multica.ai>
Co-authored-by: Eve <eve@multica-ai.local>
When codex emits a single stdout line larger than the daemon's 10 MB
bufio.Scanner cap (typical trigger: thread/resume on a long-history
session), the reader goroutine returns scanner.Err()="token too long",
markProcessExited fails the in-flight RPC, and the lifecycle goroutine
enters its failure path. That path calls drainAndWait() — stdin.Close()
+ cmd.Wait() — before sending the failed Result. But cmd.Wait() never
returns: codex is alive and blocked writing the rest of the oversized
line into a stdout pipe nobody is reading, so it never reaches its
stdin read syscall and never sees the EOF. The lifecycle goroutine
therefore never sends Result{failed} to its caller, the outer daemon
blocks on the result channel, and the existing PriorSessionID-with-
empty-SessionID fallback never fires — the task is permanently
stalled and codex (Node wrapper + native Rust app-server) leaks until
the OS reaps them.
The cancel() that would have unblocked things via cmd.WaitDelay's
SIGKILL was registered as a defer AFTER drainAndWait, so LIFO defer
order put cancel last — drainAndWait blocks first, cancel never runs.
Fix:
1. drainAndWait now runs the existing graceful-then-cancel pattern
itself, in two bounded phases. Phase 1 waits for readerDone (capped
by codexGracefulShutdownTimeout, so we still give codex its OTEL
flush window on clean exits); on timeout it cancels the runCtx so
cmd.Cancel kills the tree and the reader unblocks. Phase 2 bounds
cmd.Wait() the same way for the scanner-overflow case, where
readerDone closed early but the process is still alive on a full
stdout pipe. The success-path cleanup that previously duplicated the
graceful-cancel pattern around readerDone collapses to a single
drainAndWait() call.
2. cmd.Cancel is set to send SIGKILL to the whole codex process group
(Setpgid via configureProcessGroup, signalProcessGroup on cancel)
instead of just the leader. This addresses YOMXXX's
orphaned-Codex-child concern: the Node wrapper and the native
app-server it spawns now both die when cleanup forces the kill,
rather than the native binary leaking as an orphan reparented to
init. configureProcessGroup is a no-op on Windows.
3. codexGracefulShutdownTimeoutNanos atomic.Int64 mirrors
opencodeTerminateGraceNanos so the regression test can shrink the
grace window from 10 s to 500 ms. Production code is unchanged
(default 10 s).
Outer daemon (daemon.go) already retries with a fresh session when
result.Status == "failed" && PriorSessionID != "" && result.SessionID
== ""; the failed Result now actually reaches it, so the recovery
fires on its own without any daemon-side change.
Tests:
- New regression TestCodexExecuteCleansUpWhenScannerOverflowsOnResume
spawns a fake codex that emits an 11 MB single-line thread/resume
response (trips the scanner cap) and then sleeps without re-reading
stdin. With the original drainAndWait body it blocks at the 10 s
executeFakeCodex deadline ("timeout waiting for result") — verified
by temporarily reverting just the helper body — and with the fix it
completes in ~1.3 s with Result.Status="failed",
Result.SessionID="" so the outer fallback can fire.
- Full codex test suite, full agent package, daemon + execenv +
repocache packages, go build ./..., and go vet on agent/daemon all
pass.
Out of scope (deferred to follow-up per YOMXXX): bumping the 10 MB
bufio.Scanner cap on codex / claude / copilot / cursor / hermes /
kimi / kiro / codebuddy / antigravity / qoder / openclaw / opencode
(pi already sits at 32 MB), and the shared bounded JSON-RPC line
reader that would eliminate the single-line-overflow risk class
entirely. Buffer size alone is not the fix — recovery behaviour is.
Refs: GH#4520
Co-authored-by: Eve <eve@multica-ai.local>
Co-authored-by: multica-agent <github@multica.ai>
The daemon auto-grants Codex item/permissions/requestApproval requests by
echoing back the network / fileSystem profile scoped to the current turn.
Previously a malformed params payload and any permission key outside
network / fileSystem were dropped silently, so a future app-server
protocol that adds a new permission shape would be narrowed away with no
trace in daemon logs.
Log both cases (parse failure and dropped keys) without changing the
granted response. Addresses review nits on #4346 / MUL-3451.
Co-authored-by: J <j@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Codex telemetry was never reaching the OTLP collector for tasks run by the
daemon. The per-task config (including the [otel] block) is copied into
CODEX_HOME correctly, but the lifecycle goroutine closed stdin and then
immediately cancelled the run context, which SIGKILLs the app-server. Codex's
OTEL batch exporters only force-flush on a graceful shutdown, so the buffered
spans/metrics/logs were dropped before they could be exported — short tasks
lost everything, long tasks lost the final batch.
Let codex exit on its own after stdin EOF (running its shutdown + flush path)
and only force-cancel after a bounded grace period if it doesn't, so the reader
goroutine still can't block forever. Also set cmd.WaitDelay, matching the other
long-lived backends (claude, copilot, cursor, …).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Active long-running sessions are no longer killed by a fixed wall-clock deadline. Liveness is delegated to the idle watchdog (MULTICA_AGENT_IDLE_WATCHDOG, default 30m) with a larger in-flight-tool budget (MULTICA_AGENT_TOOL_WATCHDOG, default 2h). MULTICA_AGENT_TIMEOUT is an opt-in absolute cap (default 0 = no cap). The server-side 2.5h sweeper is unchanged as a coarse backstop.
Fixes#3745.
* MUL-2764: feat(agents): add MCP config tab to agent detail page
Backend already stores `mcp_config` and the daemon forwards it to the
runtime CLI via `--mcp-config`; this only adds the UI entry point.
The new tab presents a JSON editor that pretty-prints the existing
config, validates the buffer on every keystroke, and saves through the
existing `PUT /api/agents/{id}` path. Clearing the editor sends
`mcp_config: null`, which the handler reads as "wipe the column" and
the daemon falls back to the CLI's own default.
When the caller can't see secrets (agent actor, or a non-owner
non-admin member), the server already returns `mcp_config: null` with
`mcp_config_redacted: true`; the tab renders a read-only "configured
but hidden" state in that case so a non-privileged member cannot
silently overwrite an admin-owned config by saving an empty editor.
Co-authored-by: multica-agent <github@multica.ai>
* fix(agents): MCP tab — preserve in-flight edits + warn non-Claude runtimes
- Fix stale-editor sync: compare the local draft against the *previous*
original via a ref, so a background agent refetch updates an untouched
editor instead of being silently ignored. Without this, a draft equal to
the OLD original was treated as user-edited after the prop changed, and
the next Save would write the old config back over a concurrent admin
edit.
- Surface a notice inside the tab when the agent's runtime provider is not
Claude — today's daemon only forwards mcp_config via Claude's
--mcp-config, so saving on e.g. a Codex agent was silent but ineffective.
- Tests for both: rerender resyncs an untouched editor, rerender preserves
an in-flight edit, warning renders on non-Claude / hides on Claude.
MUL-2764
Co-authored-by: multica-agent <github@multica.ai>
* MUL-2764: feat(agents): codex MCP support + hide MCP tab on unsupported runtimes
- Backend: codex.go now translates agent.mcp_config (Claude-style
`{"mcpServers": {...}}`) into `-c mcp_servers.<name>=<inline-toml>`
flags for `codex app-server`, so MCP servers configured in the UI
reach Codex's per-task config layer. Bad mcp_config JSON downgrades
to a warn-and-skip so it can't break the agent launch.
- Frontend: AgentOverviewPane hides the MCP tab when the agent's
runtime provider doesn't read mcp_config — only `claude` and `codex`
are supported today, every other provider sees no MCP tab. The
previous in-tab warning is removed (no longer reachable).
- New shared helper `providerSupportsMcpConfig` lives in
`@multica/core/agents` so views and any future caller share one list
of MCP-aware providers.
- Tests: new go-side coverage for stdio + url + multi-server inputs,
TOML string escaping, malformed-input fallback, and arg ordering vs
custom_args; new views-side coverage for which providers surface the
MCP tab. En + zh-Hans copy and parity test refreshed.
Co-authored-by: multica-agent <github@multica.ai>
* MUL-2764: fix(agents): keep codex mcp_config secrets out of argv/logs
Move the agent's mcp_config from a `-c mcp_servers.<id>=<inline-toml>`
argv flag into a daemon-managed `[mcp_servers.*]` block inside the
per-task `$CODEX_HOME/config.toml`. mcp_servers.<id>.env is a documented
Codex config field and the UI already treats mcp_config as redacted for
non-admins; argv would have leaked those values into `ps aux` and the
`agent command` log line. The file is forced to 0600 to keep secrets in
the daemon owner's lane regardless of the seed file's mode.
Also drop user-supplied `-c/--config mcp_servers.*` entries from
custom_args. Codex `-c` is last-wins (verified against codex-cli 0.132.0),
so without filtering, a custom_args entry could silently shadow whatever
the MCP Tab saved.
Strip inherited `[mcp_servers.*]` tables from the per-task config.toml
when the agent has its own mcp_config, mirroring Claude's
`--strict-mcp-config`: avoids TOML "table already exists" errors on
name collisions and matches admin expectations that the MCP Tab is the
authoritative source for that task.
Co-authored-by: multica-agent <github@multica.ai>
* MUL-2764: fix(agents): codex mcp_config three-state semantics + custom_args compat
Address the third review pass:
1. Distinguish nil vs present-but-empty mcp_config. `{}` and
`{"mcpServers":{}}` now count as "admin saved an explicit (empty)
managed set" — strip inherited user `[mcp_servers.*]` and pin an
empty managed marker block. Only SQL NULL / JSON `null` map to
"absent" and fall back to the user's global `~/.codex/config.toml`.
This aligns Codex with the API's three-state contract (omit / null
/ object) and with Claude's `--strict-mcp-config` semantics.
2. Fail closed on `ensureCodexMcpConfig` errors and on managed
mcp_config without CODEX_HOME. Previous warn-and-launch would
silently inherit the user's global MCP servers and look identical
to a successful apply — exactly the surprise the MCP Tab is meant
to remove.
3. Only filter `-c mcp_servers.*` from `custom_args`/`extra_args`
when the agent has a managed mcp_config. Pre-MUL-2764 agents that
configured MCP via custom_args keep working; once an admin opts
in via the MCP Tab the daemon owns the `mcp_servers` namespace
and overrides are dropped (last-wins safety).
4. Update mcp_config locale intro to mention $CODEX_HOME/config.toml
instead of the now-removed `-c mcp_servers.*` argv path.
Tests:
- Split `TestEnsureCodexMcpConfigEmptyInputsAreNoop` into
`TestEnsureCodexMcpConfigAbsentLeavesUserTablesAlone` (nil/null)
and `TestEnsureCodexMcpConfigEmptyManagedSetStripsUserMcp` (`{}`,
`{"mcpServers":{}}`).
- Add `TestEnsureCodexMcpConfigEmptyManagedSetIdempotent` to pin
byte-identical reruns on the empty managed marker block.
- Add `TestHasManagedCodexMcpConfig` covering the eight relevant
inputs.
- Add `TestBuildCodexArgsPreservesCustomMcpOverridesWhenUnmanaged`
and `TestBuildCodexArgsDropsCustomMcpOverridesWhenManaged` to
pin the new gating.
- Add `TestCodexExecuteFailsClosedWhenMcpConfigInvalid` and
`TestCodexExecuteFailsClosedWhenManagedMcpButNoCodexHome` for the
Execute paths.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: J <j@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
* feat(agent): persist thinking_level per agent (MUL-2339)
Adds a nullable `thinking_level` column to the `agent` table so the
backend can route a runtime-native reasoning/effort token (e.g. Claude's
`xhigh`, Codex's `minimal`) through to the agent CLI on every dispatch.
The column is intentionally TEXT rather than an enum — Claude and Codex
publish overlapping but distinct vocabularies and we want the persisted
value to round-trip exactly through whichever CLI receives it. NULL is
the "use runtime default" sentinel that every downstream consumer reads
as "do not inject --effort / reasoning_effort".
This commit is just the storage layer (migration + sqlc); subsequent
commits wire it through the API, daemon, and agent backends.
Co-authored-by: multica-agent <github@multica.ai>
* feat(agent-backend): inject reasoning effort for claude + codex (MUL-2339)
Extends ExecOptions with a runtime-native ThinkingLevel string and wires
it into the Claude and Codex backends. Discovery is driven by the local
CLI so the daemon advertises whatever the host install supports rather
than a hand-maintained list that goes stale.
Per Elon's PR1 review:
- Claude: parses `claude --help` to learn the `--effort` superset and
projects through a per-model allow-list (xhigh is Opus-only; max is
session-only on the smaller models). Falls back to a conservative
static list when the binary is missing or help drift hides the line.
- Codex: drives `codex debug models --output json` so per-model
reasoning subsets and the documented default come directly from the
CLI. The older config-error probe trick is gone — the JSON path is
stable and doesn't pollute stderr with an intentional misconfig.
- Cache key includes (provider, executablePath, cliVersion) so a CLI
upgrade invalidates entries that referenced the older help / catalog.
Per Trump's PR1 constraint, all three Codex injection points
(thread/start.config, thread/resume.config, turn/start.effort) flow
through one helper (`applyCodexReasoningEffort`) so they cannot drift
independently. The shared `codexReasoningCases` fixture in
`thinking_test.go` asserts the same value→{shape, key} contract at
each site for every level the runtimes know about.
Claude's `--effort` is also added to `claudeBlockedArgs` so a user
custom_args entry can't silently outvote the daemon-injected value.
Co-authored-by: multica-agent <github@multica.ai>
* feat(api): wire thinking_level through API + daemon contract (MUL-2339)
End-to-end plumbing for the per-agent reasoning/effort setting:
- AgentResponse / TaskAgentData now carry `thinking_level`; the daemon's
claim response includes it and the daemon's executor passes it through
to agent.ExecOptions, where the Claude and Codex backends already know
what to do with it.
- ModelEntry on the runtime-models wire format gains a `thinking` block
carrying `supported_levels` + `default_level` per model so the UI can
render a runtime-aware picker without the server having to know about
the local CLI install. `handleModelList` projects the agent-package
catalog (including the new Thinking field) into the wire shape.
- CreateAgent / UpdateAgent gate the field with a synchronous provider
enum check (claude / codex only today). UpdateAgent is tri-state:
field omitted = no change, "" = explicit clear (new
`ClearAgentThinkingLevel` query, mirrors the existing mcp_config null
pattern), non-empty = validate then set.
Per Trump's PR1 review, the API NEVER auto-clears on a runtime/model
swap and ALWAYS returns 400 on an unknown literal value — same shape
across CreateAgent, UpdateAgent, and combined patches that move
runtime + level in one request. Per-model combination failures (e.g.
`xhigh` against a model that only supports up to `high`) surface as a
daemon-side task error, not a silent server-side rewrite.
TS types follow the same shape: `Agent.thinking_level`,
`CreateAgentRequest`/`UpdateAgentRequest` add the field, `RuntimeModel`
grows a `thinking` block. Older backends omit the field, which the
front-end treats as "no picker for this model" — installed desktop
builds keep working.
Co-authored-by: multica-agent <github@multica.ai>
* fix(agent): correct codex debug models argv + pin via runner test (MUL-2339)
`codex debug models --output json` is rejected by codex-cli 0.131.0 —
the subcommand emits JSON on stdout by default and has no `--output`
flag. Drop the flag and add `--bundled` to skip the network refresh
discovery doesn't need. Move the argv to a package-level var and add
a test that runs a fake `codex` to assert the binary actually
receives exactly `debug models --bundled`, so the contract can't
silently drift on the next refactor.
Also teach ValidateThinkingLevel to resolve an empty model to the
provider's default model entry. Without this, every default-model
task with a persisted thinking_level would be misjudged "unknown
model" by the daemon guard.
Co-authored-by: multica-agent <github@multica.ai>
* fix(api): reject runtime switch that would leave invalid thinking_level (MUL-2339)
A PATCH that changed `runtime_id` without touching `thinking_level`
used to silently keep the existing value, so a Claude agent storing
`max` could land on a Codex runtime where `max` is not a recognised
token at all, and the daemon would receive a literal-invalid level.
Hold the same "always 400 on literal-invalid, never silent coerce"
rule on this implicit path. When runtime_id changes and the existing
value is not in the new provider's enum, return 400 with the
recovery options (clear via `thinking_level=""` or re-set in the
same PATCH).
Add coverage for both the kept-when-still-valid and the rejected
cases, plus the two recovery paths (clear and replace).
Co-authored-by: multica-agent <github@multica.ai>
* fix(daemon): guard runTask with per-model thinking_level validator (MUL-2339)
ValidateThinkingLevel existed but had no call site — `task.Agent.
ThinkingLevel` flowed straight into ExecOptions, so `xhigh` configured
on a non-Opus Claude model, or API-side stale values that escaped the
provider enum gate, would be injected anyway.
Run the validator before building ExecOptions. Invalid combinations
log a warning and drop the level instead of failing the task: the
agent still runs, just at the runtime's default reasoning effort.
Discovery errors fail open (keep the level, let the CLI surface any
objection) so a transient `claude --help` failure can't strand work.
Empty model is forwarded as-is; the validator resolves it to the
provider's default model internally per the cross-package contract.
Co-authored-by: multica-agent <github@multica.ai>
* chore(agent): drop stale `--output json` comments + unused scanner (MUL-2339)
Codex CLI's `debug models` subcommand emits JSON without an `--output`
flag, and `parseCodexDebugModels` never read from the bufio.Scanner.
Sync the comments with the actual invocation and remove the dead init.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: multica-agent <github@multica.ai>
The watchdog fires on a "no progress" window, so the default mainly
matters for commands that go fully silent (no outputDelta). Bumping
from 2m → 3m leaves more headroom for legitimately slow silent
commands before treating them as a dropped function_call_output, at
a modest cost to recovery latency.
MUL-2337
Co-authored-by: multica-agent <github@multica.ai>
* feat(codex): add per-exec_command watchdog to escape dropped function_call_output (MUL-2337)
Codex app-server can drop the second function_call_output when two
exec_command calls fan out in the same turn and both async-yield through
the yield_time_ms boundary (observed 2026-05-18, MUL-2334 — Trump Agent
wedged for 6+ min with no semantic activity events to drive any existing
timer). The model then waits forever for the missing output; only the
10-minute semantic inactivity timeout would eventually rescue the run.
Add a per-call watchdog in the codex client that tracks open
exec_command / commandExecution items by call_id and fails the turn
quickly (default 2 min, configurable via ExecOptions.ExecCommandStuckTimeout)
when one stays open without progress. outputDelta events reset the
per-call progress timestamp so long-running streaming commands aren't
flagged.
This is a daemon-side mitigation only — codex itself still has the
upstream race, but the daemon no longer burns the full inactivity budget
before the run is marked failed and a new run can recover.
Co-authored-by: multica-agent <github@multica.ai>
* feat(codex): track legacy exec_command_output_delta in watchdog (MUL-2337)
Mirrors the raw v2 item/commandExecution/outputDelta refresh on the legacy
codex/event protocol so a long-running streaming exec doesn't get falsely
flagged as stuck after begin + 2 min.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: multica-agent <github@multica.ai>
* fix(codex): handle MCP elicitation server requests correctly
Fixes#1942.
handleServerRequest responded with {} to unrecognized Codex server
requests including mcpServer/elicitation/request. Codex 0.125+ expects
{action, content, _meta} for elicitation — the empty object causes a
deserialization error and the MCP tool call is reported as user-rejected.
Changes:
- Add mcpServer/elicitation/request case with correct response schema
- Add respondError helper for JSON-RPC error responses
- Return proper JSON-RPC method-not-found error for unknown server
requests instead of silent empty object
- Add tests for MCP elicitation and unknown method handling
* fix: use cfg.Logger instead of global slog in codex handleServerRequest
Switch the unhandled-server-request warning from global slog.Warn to
c.cfg.Logger.Warn for consistency with all other log calls in codex.go.
This ensures the warning appears in daemon run-logs and per-task
pipelines where operators look during triage.
Hoist the existing stderrTail ring-buffer (previously codex-only) into
a shared pkg/agent helper so every Backend that supervises a child CLI
can include the last ~2 KB of that CLI's stderr in Result.Error. Wire
the claude backend through the same path.
Motivation: claude on Windows occasionally exits with a non-zero status
after ~5–8 minutes of a single long-running tool_use, and right now the
daemon only reports "claude exited with error: exit status 3" /
"exit status 0x80000003" — useless for root-causing V8 aborts, Bun
panics, native-module OOMs, or any other CLI-side crash. With the tail
attached, the failure message carries the real signal (panic line, V8
assertion, stderr-printed HTTP error) all the way into the task row's
error field that users see in the API.
Renames withCodexStderr to withAgentStderr(msg, label, tail) so the
helper is self-documenting across providers.
* fix(daemon): suppress agent terminal windows on Windows (#1471)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: add hideAgentWindow to detectCLIVersion and avoid SysProcAttr overwrite
- Add missing hideAgentWindow(cmd) call in detectCLIVersion (claude.go:554)
so --version checks don't flash console windows on Windows.
- Refactor hideAgentWindow to preserve existing SysProcAttr fields
instead of overwriting the entire struct.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat(server): orphan-task recovery + auto-retry + manual rerun (MUL-1128)
When the daemon process crashed mid-task the issue was stuck at
in_progress for up to 2.5h: the in-flight task timeout was the only
mechanism that ever moved the row, and the runtime heartbeat sweeper
only fires after the runtime stays offline for 45s — a quick restart
beats both windows.
This change implements the A+B plan from the issue thread:
A. lifecycle hygiene
- migration 055 adds attempt / max_attempts / parent_task_id /
failure_reason / last_heartbeat_at to agent_task_queue
- new daemon-auth endpoint POST /runtimes/{id}/recover-orphans:
daemon calls it on every register so the server fails any
dispatched/running tasks the previous process left behind
- new daemon-auth endpoint POST /tasks/{id}/session: persists the
agent's session_id + work_dir mid-flight so a crash doesn't
lose the resume pointer (claude+codex emit MessageStatus with
SessionID; daemon forwards on the first one it sees)
- FailAgentTask / FailStaleTasks / FailTasksForOfflineRuntimes
now set failure_reason ('agent_error' / 'timeout' /
'runtime_offline')
B. auto-retry with resume context
- TaskService.MaybeRetryFailedTask spawns a fresh queued attempt
carrying parent's session_id/work_dir when the failure reason
is infrastructure-shaped (timeout, runtime_offline,
runtime_recovery) and attempt < max_attempts; skips autopilot
- wired into the runtime sweeper paths and TaskService.FailTask
so the user transparently sees a new in_progress run instead of
a stuck row
- new user-auth POST /api/issues/{id}/rerun + multica issue rerun
CLI for the manual escape hatch
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(server): address PR review for orphan-task recovery (MUL-1128)
Three review-must-fix items on top of the A+B implementation:
1. recover-orphans now funnels through TaskService.HandleFailedTasks,
the same shared post-failure pipeline used by the runtime sweeper.
This guarantees task:failed events are emitted, agent status is
reconciled, and issues stuck in_progress with no remaining active
task are reset to todo even when no auto-retry is created
(max_attempts exhausted, autopilot, non-retryable reason).
2. RerunIssue now uses CancelAgentTasksByIssueAndAgent, scoped to the
issue's current assignee. The previous implementation called
CancelAgentTasksByIssue, which would collateral-cancel parallel
@-mention agents on the same issue.
3. GetLastTaskSession now considers both completed and failed tasks
(mirroring GetLastChatTaskSession), ordering by the most recent
timestamp. With UpdateAgentTaskSession pinning session_id/work_dir
mid-flight, an auto-retry or manual rerun of a daemon-crash failure
now actually resumes the prior conversation context instead of
starting fresh — matching the stated B-branch behaviour.
go build / go vet pass; the existing service and agent test suites pass.
runtime_sweeper / handler integration tests require a local DB with the
055 migration (and the pre-existing 050 first_executed_at column).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Surface the actual exec path + argv for every agent backend at INFO
so operators can see the exact command without flipping to debug.
Also add the missing log line in pi.go for consistency with the
other nine backends.
* fix(agent/codex): surface stderr tail in initialize / turn startup errors
When codex app-server exits before the JSON-RPC handshake completes —
e.g. because the user put a flag in custom_args that the subcommand
rejects — the Result.Error users see is `codex initialize failed:
codex process exited`, while codex's actual complaint (typically
something like `error: unexpected argument '-m' found`) only lives in
daemon logs.
Wrap the stderr writer with a bounded stderrTail that still forwards
to the slog logWriter but also retains the last 2 KiB of bytes
written. Include that tail on the three startup failure paths
(initialize, startOrResumeThread, turn/start). Runtime cancellation
paths are left untouched — they're our own abort and the stderr
context isn't a clear signal there.
Refs #1308. Complement to #1310 / #1312 — lets "bad custom_args fail
loudly" actually be workable by giving the failure a real message.
* fix(agent/codex): join cmd.Wait() before sampling stderr tail
Addressing review of #1314: reading stderrBuf.Tail() right after
c.request returns "codex process exited" was racy. Nothing in that
path synchronizes with os/exec's internal stderr copy goroutine —
cmd.Wait() is the only documented join point. The original defer ran
cmd.Wait() later, but by then we had already built Result.Error from
a potentially-empty Tail().
Replace the ad-hoc deferred stdin.Close()/cmd.Wait() with a
sync.Once-wrapped drainAndWait closure. Call it explicitly on the
three startup failure paths before sampling the tail; keep it as the
cleanup defer so the success path behaves identically.
Also add TestCodexExecuteSurfacesStderrWhenChildExitsEarly: spawns a
real subprocess that prints to stderr and exits before responding to
initialize, runs it through Execute, and asserts Result.Error
contains the stderr hint. This covers the full timing path the
reviewer flagged, which the helper-level tests in this PR did not.
Follow-up to #1192. Document the v2 protocol contract that the
dispatch-level threadId guard relies on, and lock down the two leakage
paths the guard closes:
- turn/completed from a subagent thread must not call onTurnDone
- item/completed (agentMessage, final_answer) from a subagent thread
must neither leak text into the output builder nor terminate the turn
Without these tests a future refactor that drops or relocates the guard
would not be caught by CI, since existing notification tests omit the
top-level threadId field and pass through unfiltered.
* fix(daemon): filter thread/status/changed by threadId to prevent subagent interference
When Codex CLI has memories enabled, the app-server spawns a memory
consolidation subagent as a separate thread within the same stdio
connection. When that subagent thread finishes and transitions to idle,
the daemon's codex backend mistakenly interprets the idle signal as the
main turn completing, causing it to close stdin and cancel the context
before the real turn produces any output.
Add a threadId check to the thread/status/changed handler so only
status changes from the tracked thread trigger turn completion. Signals
from subagent threads (threadId != c.threadID) are now ignored.
Fixes#1181
* fix(codex): dispatch-level threadId filter for subagent notifications
Codex multiplexes subagent threads (e.g. memory consolidation) on
the same stdio pipe. Previously only thread/status/changed had a
threadId guard, but item/completed (agentMessage + final_answer),
turn/completed, and turn/started from subagent threads could still
trigger onTurnDone or contaminate output.
Move the threadId check to the top of handleRawNotification so all
notification handlers are protected. Remove the now-redundant
per-handler check on thread/status/changed.
Fixesmultica-ai/multica#1181
---------
Co-authored-by: fuxiao <fuxiao@zyql.com>
Every other backend (Claude, Gemini, OpenCode, OpenClaw, Hermes) honors
ExecOptions.ResumeSessionID — only Codex didn't. That's why users on
the Codex runtime saw each new comment on an issue start a fresh Codex
conversation: the daemon persists Result.SessionID per (agent, issue)
and passes it back as PriorSessionID, but codex.go always called
thread/start and never populated SessionID, so the value round-tripped
as empty.
Wire the missing half:
- Extract startOrResumeThread on codexClient. When ResumeSessionID is
set, call thread/resume (per the Codex app-server protocol), passing
only cwd / model / developerInstructions overrides so the thread
keeps its persisted model and reasoning effort. If resume fails
(unknown thread, schema drift, transport error) fall back to
thread/start so the task still runs on a fresh thread.
- Surface the live threadID as Result.SessionID on the final emit so
the daemon stores it and feeds it back into ResumeSessionID on the
next claim.
Tests drive the new helper through the fake stdin harness, covering:
fresh start, successful resume, fallback on resume error, fallback
when resume returns no thread ID, and surfacing of thread/start
failures.
When codex emits `turn/completed` with `status="failed"` or a terminal
top-level `error` notification, the daemon previously treated the turn
as successfully completed, saw no accumulated text, and surfaced the
generic "codex returned empty output" — hiding the real reason (auth,
sandbox, API error, etc.).
Capture `turn.error.message` on failed turns and the `error.message`
from non-retrying top-level error notifications, then propagate them
through `Result.Error` with `finalStatus="failed"` so the daemon's
default branch reports the actual cause.
Add a debug-level log line in every agent backend (claude, codex,
opencode, openclaw, gemini, hermes) that prints the executable path
and full argument list when spawning the agent process. Helps diagnose
custom args, model overrides, and other CLI flag issues.
* feat(agent): add custom CLI arguments support
Allow users to configure custom CLI arguments per agent that get
appended to the agent subprocess command at launch time. This enables
use cases like specifying different models (--model o3), max turns,
or other provider-specific flags without needing separate runtimes.
Changes:
- Add custom_args JSONB column to agent table (migration 041)
- Update API handler to accept/return custom_args in create/update
- Pass custom_args through claim endpoint to daemon
- Append custom_args to CLI commands for all agent backends
- Add ExecOptions.CustomArgs field in agent package
- Add Custom Args tab in agent detail UI
- Add --custom-args flag to CLI agent create/update commands
Closes MUL-802
* fix(agent): filter protocol-critical flags from custom_args
Add per-backend filtering of custom_args to prevent users from
accidentally overriding flags that the daemon hardcodes for its
communication protocol (e.g. --output-format, --input-format,
--permission-mode for Claude).
This follows the same pattern as custom_env's isBlockedEnvKey: we
only block the small, stable set of flags that would break the
daemon↔agent protocol — not every possible dangerous flag. Workspace
members are trusted for everything else.
Each backend defines its own blocked set:
- Claude: -p, --output-format, --input-format, --permission-mode
- Gemini: -p, --yolo, -o
- Codex: --listen
- OpenCode: --format
- OpenClaw: --local, --json, --session-id, --message
- Hermes: none (ACP is positional)
Includes unit tests for the filtering logic.
* fix(agent): address code review nits for custom_args
- Replace module-level `nextArgId` counter with `crypto.randomUUID()`
in custom-args-tab.tsx to avoid SSR ID conflicts
- Add unit tests for custom args passthrough and blocked-arg filtering
in both Claude and Gemini arg builders
Codex tasks running in workspace-write sandbox mode could not resolve
api.multica.ai because the hardcoded sandbox parameter in thread/start
overrode any config.toml settings, and the default sandbox policy blocks
network access.
Changes:
- Remove hardcoded `sandbox: "workspace-write"` from thread/start RPC —
let Codex read sandbox config from its own config.toml instead
- Auto-generate config.toml in per-task CODEX_HOME with
`sandbox_mode = "workspace-write"` and `network_access = true`,
preserving any existing user settings
- Fix Reuse() to restore CodexHome for Codex provider on workdir reuse
Closes#368
Codex doesn't expose token usage through its JSON-RPC app-server
protocol. The turn/completed and task_complete notifications don't
contain usage fields.
Fix: after Codex execution finishes, scan the on-disk session JSONL
files (~/.codex/sessions/YYYY/MM/DD/*.jsonl) for token_count events.
Only files modified after the task's start time are scanned, avoiding
counting unrelated sessions. This matches the same data format the
existing runtime_usage scanner reads.
1. Separate ReportTaskUsage endpoint (POST /api/daemon/tasks/{id}/usage)
so usage is captured independently of complete/fail — fixes usage loss
for failed/blocked tasks.
2. Add usage tracking for all four providers:
- Claude: already done (stream-json message.usage)
- OpenCode: extract from step_finish.part.tokens
- OpenClaw: extract from step_end.data token fields
- Codex: extract from turn/completed and task_complete usage fields
3. Remove usage from CompleteTask payload — all usage goes through the
dedicated endpoint now.
Replace raw fmt/log calls with structured slog logger (Go) and
console-based logger (TypeScript). Add request logging middleware.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove debug log.Printf calls from handler/daemon.go and service/task.go
that used the global log package instead of structured logging
- Remove unused truncate() helper from service/task.go
- Add error handling for EnqueueTaskForIssue in createAgentInitIssue
- Clean up verbose debug logging in daemon/daemon.go handleTask
- Add shutdown sequence comment to codex.go lifecycle goroutine
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The codex backend spawns a long-running app-server process that doesn't
exit after completing a turn. The lifecycle goroutine was waiting on
<-readerDone which blocks on scanner.Scan() until stdout closes — but
stdout never closes because the process stays alive. This caused the
entire poll loop to freeze, preventing any further task processing.
Fix: explicitly close stdin and cancel the context after the turn
completes, which terminates the codex process and unblocks the reader.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix data race on output strings.Builder in codex backend by adding
mutex and waiting for reader goroutine before reading final output
- Fix data race on onTurnDone by initializing it before reader starts
- Fix bug where notificationProtocol zero value "" never matched
"unknown", silently dropping all raw v2 notifications from codex
- Add round-robin polling to prevent runtime starvation in poll loop
- Log errors in claude handleControlRequest instead of silently dropping
- Add 35 tests for pkg/agent covering claude parsing, codex JSON-RPC,
protocol detection, event handling, and helper functions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Replace deprecated strings.Title with manual capitalize
- Fix race: set codexClient.onMessage before starting reader goroutine
- Remove unused msgCh parameter from codexClient.handleLine
- Route agent stderr through logger instead of dumping to os.Stderr
- Use deterministic agent order in ensurePaired (prefer codex)
- Increase message channel buffer from 64 to 256
- Rename test to match function rename (buildPrompt)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add a reusable Go agent package (server/pkg/agent/) that provides a
unified Backend interface for executing prompts via either Claude Code
or Codex. The daemon now auto-detects which CLIs are available at
startup, registers a runtime for each, and routes tasks to the correct
backend based on task.Context.Runtime.Provider.
Key changes:
- server/pkg/agent/agent.go: Backend interface, Message/Result types, factory
- server/pkg/agent/claude.go: Spawns claude CLI with stream-json, parses output
- server/pkg/agent/codex.go: Spawns codex app-server, JSON-RPC 2.0 protocol
- server/cmd/daemon/daemon.go: Multi-runtime registration, round-robin polling,
provider-based backend selection. Removes old runCodexExec/codexResultSchema.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
