Fixes#5862.
The Hermes ACP provider-error sniffer over-matched conversation/tool JSON that
Hermes echoes to stderr as `[INFO] root:` records, flipping already-completed
runs to failed. Error matching is now scoped to real provider-error boundaries:
- Skip INFO/DEBUG root-logger records (single- and multi-line JSON) via a
structural state machine, so echoed payloads never participate in matching.
- Keep genuine bare provider errors (⚠️/❌/📝 Error:) and non-root [ERROR]
records, including after a possibly-truncated INFO record.
- Length only bounds the persisted error summary, never whether a line is
classified as an error (fixes the earlier #1952-style regression).
Reported and initial fix by @YikaJ; remaining boundary fixes added directly by
the maintainers to land this quickly as a production bug.
MUL-5240
cursor-agent's stream-json never populates total_cost_usd or a per-step
cost — the result event's usage object carries token counts only. Both
fields were speculatively copied from Claude Code's schema in the original
Cursor runtime PR (#1057) and were never read. Verified against the real
CLI (2026.07.20, stream-json and json) plus Cursor's CLI docs.
Remove the two dead fields and document that Cursor spend stays estimated
from the static rate table (no authoritative per-turn cost to carry, unlike
Grok). No behavior change.
Co-authored-by: Bohan-J <bohan@devv.ai>
Co-authored-by: multica-agent <github@multica.ai>
Cursor stream-json emits reasoning and tool calls as top-level `thinking` /
`tool_call` events; the parser only looked for them inside assistant messages,
so transcripts showed a single step. Match the top-level events, taking the
tool name from the nested `<name>ToolCall` key and normalizing the packed
call_id. Subtypes are matched explicitly — only `started` opens a tool, only
`completed` closes it, only `delta` carries reasoning — so an unknown/missing
subtype is ignored rather than synthesizing a fake result (which would decrement
the daemon in-flight tool count early and misfire the watchdog) or polluting
reasoning. Covered by a recorded-stream fixture test, an unknown-subtype
regression test, and an opt-in real-CLI smoke test.
kimi-code 0.29 dropped the top-level `models.availableModels` /
`currentModelId` block from its ACP `session/new` response and moved the
same catalog into a `configOptions` entry with `id`/`category` of
"model". parseACPSessionNewModels only understood the old shape, so
discovery silently returned an empty catalog and the model picker showed
"no available models" for an online, correctly-detected kimi runtime.
Parse `configOptions` as a fallback: the `models` block still wins when
present, so no existing ACP provider changes behaviour. `options[].value`
becomes the model id, `options[].name` the label, and `currentValue`
marks the default. Non-model options (thinking level) are deliberately
skipped — they are a separate product surface and would offer values
`session/set_model` cannot honour.
Also log a debug line with the top-level response keys (keys only, never
values) when session/new succeeds but advertises no catalog, so the next
round of upstream schema drift is visible in daemon.log instead of
looking like a PATH or install failure.
Co-authored-by: Bohan-J <bohan@devv.ai>
Co-authored-by: multica-agent <github@multica.ai>
* fix(agent): attribute Grok usage from the turn's own model id
A resumed Grok session with no configured model recorded its entire spend
under the model id "unknown", which matches no pricing row — so the task
reported $0 cost instead of its real spend.
grok.go only learned the model from the session handshake, and ACP's
`session/load` carries no model id (only `session/new` does). When neither
the agent nor MULTICA_GROK_MODEL pins a model, `daemon.go` legitimately
passes an empty model, leaving nothing to attribute the usage to.
Every Grok turn stamps `result._meta.modelId` with what it actually billed
against. Parse it in the shared ACP result parser and use it as the fallback
in grok.go. Other ACP backends are untouched — they keep whatever the
handshake gave them.
Co-authored-by: J <agent@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
* fix(metrics): price the Grok catalog in server-side cost metrics
server/internal/metrics/pricing.go carried no Grok rows at all, so
RecordLLMUsage took the unpriced branch for every Grok turn: llm_cost_usd
reported zero Grok spend while the tokens accumulated in
llm_unpriced_tokens. Internal cost monitoring simply could not see Grok.
Add the six SKUs xAI publishes rates for, mirroring the frontend table in
packages/views/runtimes/utils.ts. Aliases are anchored exact matches like
the gpt-5.6 rows, so `grok-composer-*` (in the catalog, absent from the
price sheet) stays unmapped instead of inheriting a guessed rate.
Short-context tier on purpose: xAI bills a request at 2x once its prompt
reaches 200K tokens, but a usage record aggregates every model call in a
turn and cannot say which tier an individual request hit.
A regression test re-derives the cost of a real grok 0.2.106 turn from the
table and checks it against the costUsdTicks xAI returned for that turn.
Co-authored-by: J <agent@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
* docs(changelog): scope the Grok cost claim to what was actually fixed
The v0.4.9 entry promised "accurate cost" in all four languages, but the
fix corrected catalog pricing and cached-input double-counting — it did not
implement xAI's 2x long-context tier, so a turn whose requests reach 200K
prompt tokens still under-reports by up to 50%. Say what was fixed instead.
Also correct two stale claims in the pricing comment: the daemon tags usage
rows with the runtime provider `grok`, not `xai` (the bare `grok-*` keys are
what make them resolve), and record why thresholding the long-context tier
on an aggregated row would be worse than not pricing it at all.
Co-authored-by: J <agent@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
* feat(usage): carry the provider's own cost through to the usage record
Cost has always been derived client-side as tokens x a static rate, which
cannot express request-level pricing rules. xAI bills a Grok request at 2x
once its prompt reaches 200K tokens, and a task_usage row aggregates every
model call in a turn — so the stored token counts genuinely cannot say which
tier any individual request hit. Thresholding on the aggregate would be worse
than the status quo: it turns a bounded 50% under-estimate into an unbounded
over-estimate for turns made of many short requests.
Grok already reports what it charged, per turn, in `_meta.usage.costUsdTicks`.
Parse it, carry it through agent -> daemon -> API, and store it on task_usage
as a nullable BIGINT of 1e-10 USD ticks (integer, so sub-cent turns stay exact
end to end). NULL means the provider reported no cost — every pre-existing row
and every provider that doesn't return one. No backfill: there is no
authoritative figure to recover for those, and inventing one is the guess this
removes.
A single hourly bucket can mix rows that carry a cost with rows that don't, so
task_usage_hourly gains both halves: `cost_usd_ticks` sums the authoritative
side, and `uncosted_*_tokens` carry exactly the tokens that still need a
rate-table estimate. Consumers report authoritative + estimate(uncosted),
which degrades to today's behaviour when nothing in the bucket is
authoritative. The existing token columns keep covering every row, so token
displays are untouched. The new columns are additive with defaults, so the
unique key, the dirty-queue shape, and migration 102's triggers are unaffected.
Co-authored-by: J <agent@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
* feat(usage): prefer the provider's own cost over the rate table
With the authoritative figure now stored, both cost consumers use it: the
usage dashboard (estimateCost / estimateCostBreakdown) and the server-side
llm_cost_usd metric. Each reports `authoritative + estimate(uncosted tokens)`,
so a row or bucket that mixes priced and unpriced sources stays whole.
The static rate tables remain, but for Grok they are now a fallback — they
still price usage recorded by a daemon too old to report cost, and every
provider that reports none. Custom pricing overrides likewise apply only to
the estimated half: they are a user's guess at a rate, and the authoritative
half is not a guess. A model with no rate-table row but a provider-reported
cost now also drops out of the "unmapped models" banner, since asking the user
to supply a rate for it would invite overriding a real bill.
llm_cost_usd is labelled by token_type and the provider reports one number per
turn, so the charge is distributed across the buckets in the rate table's own
proportions. Only the total is authoritative; the split stays an estimate,
which is why this scales the existing buckets rather than inventing a label.
estimateCostBreakdown does the same, keeping the stacked chart summing to the
headline figure instead of silently under-drawing every Grok row.
Co-authored-by: J <agent@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
* docs(changelog): say Grok cost now follows xAI's actual charge
The earlier wording scoped the claim down to catalog pricing and cached input
because the long-context tier was still unhandled. It is handled now — the
cost comes from what xAI charged for the turn — so the entry can say so.
Co-authored-by: J <agent@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
* fix(usage): keep the provider's cost when the model has no rate row
Both cost consumers bailed out before reading the authoritative figure when
the rate table had no row for the model. A `grok-composer-*` turn — in the
Grok Build catalog, absent from xAI's price sheet — was therefore reported as
$0 spend even though xAI told us exactly what it charged.
Worse on the client: estimateCost returned the real cost while
estimateCostBreakdown returned zeros, so the headline and the stacked chart
disagreed on precisely the rows whose cost is exact — and the unmapped-models
banner was (correctly) hidden, so nothing explained the discrepancy.
Handle the charge before the rate lookup in both places. Without rates there
is nothing to split a total by, so it lands whole in the `input` bucket, the
same fallback distributeAuthoritativeCost already uses when it has no shape to
scale. Tokens with no rate keep going to llm_unpriced_tokens: "unpriced"
describes the rate table, not the money.
Co-authored-by: J <agent@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
* perf(usage): drop the historical rewrite from the cost-split migration
Migration 213 rewrote every existing task_usage_hourly row to seed the
uncosted counters. That is a full-table UPDATE inside a schema migration —
lock time, WAL and bloat all scaling with table size — for rows this issue
explicitly does not care about.
Deleting the UPDATE alone would have zeroed historical cost: with
`NOT NULL DEFAULT 0`, an untouched row asserts "nothing here needs
estimating", so every pre-split bucket would report $0 until the rollup
happened to touch it. Make the uncosted columns nullable with no default
instead. NULL means "never recomputed since the split existed", readers
COALESCE it to the row's own token total ("estimate all of it"), and the
pre-split behaviour is preserved exactly — with nothing to seed, so no
rewrite. A bare ADD COLUMN is metadata-only, so this is now fast DDL.
Rows heal into the split naturally as the rollup recomputes their buckets.
Verified on a fresh database: a legacy-shaped row reads back as its full
tokens to estimate, and a group mixing legacy and post-split buckets sums to
the authoritative cost plus both rows' estimable tokens.
Co-authored-by: J <agent@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: Bohan-J <bohan@devv.ai>
Co-authored-by: J <agent@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Grok Build reports cachedReadTokens inside inputTokens (totalTokens ==
input + output on a real 0.2.106 turn, and that turn's costUsdTicks
matches xAI's rates only when the cached prefix is billed once). The
shared ACP parser persisted both counters raw, so the usage dashboard
charged the cached prefix at the full input rate *and* the cache-read
rate — ~4x the real spend on a cache-heavy turn.
Re-bucket cached reads out of input when totalTokens proves the overlap,
the same normalization codex.go already applies. Backends that report
mutually-exclusive buckets or omit totalTokens are untouched.
Co-authored-by: J <agent@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
* feat(agents): add per-agent runtime skill controls
Co-authored-by: multica-agent <github@multica.ai>
* fix(agents): renumber runtime-skill migration and broadcast agent:status on toggle
Address the MUL-5101 review blockers on PR #5686:
- Rebase onto main and renumber the runtime-skill-disable migration
202 -> 203. main added 202_runtime_profile_add_qwen, so the pair
collided on prefix 202 and migrations_lint_test would reject the
duplicate. 203 is the next free prefix.
- Publish an "agent:status" event after persisting a
disabled_runtime_skills override, mirroring the workspace-skill toggle
in writeUpdatedAgentSkills. The realtime layer keys off this event to
invalidate workspaceKeys.agents, so other open web/desktop/mobile
clients now drop their stale toggle state instead of only the
initiating tab refreshing. Reload junction-table skills before the
broadcast so it doesn't signal cleared skills (#3459).
- Add a handler regression test proving the broadcast fires on both
disable and enable.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: Walt <walt@multica.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(agent): inject --yolo in Qwen headless runs so shell/edit/write tools are available (Fixes#5743)
Qwen Code's non-interactive mode (`-p … --output-format stream-json`) uses a
fail-closed approval policy: it silently drops `run_shell_command`, `edit`,
`write_file`, and `monitor` from the tool registry unless bypass mode is
active. Every other Multica-supported coding adapter already injects its
equivalent permission flag as a daemon-owned argument (e.g. Claude uses
`--permission-mode bypassPermissions`, Grok uses `--always-approve`, Qoder
uses `--yolo --acp`). Qwen was the only exception.
Changes:
- `buildQwenArgs`: append `--yolo` after the protocol flags and before any
custom args so headless daemon runs always receive the full tool set.
- `qwenBlockedArgs`: add `--yolo`, `-y`, `--approval-mode`, and
`--allowed-tools` as daemon-owned flags that are stripped from custom_args.
This prevents users from accidentally or intentionally disabling bypass mode
or narrowing the allowed tool set via per-agent settings. `--exclude-tools`
is intentionally left unblocked so users can still hard-deny specific tools.
- `TestBuildQwenArgsKeepsProtocolManaged`: extend with the new blocked flags
and assert daemon-owned `--yolo` appears exactly once.
- `TestBuildQwenArgsYoloAlwaysPresent`: new test asserting `--yolo` is present
even when `ExecOptions` carries no custom args.
* fix(agent): correct Qwen permission args and docs
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: Eve <eve@multica-ai.local>
Co-authored-by: multica-agent <github@multica.ai>
* fix(agent): parse Grok ACP token usage from session/prompt _meta
Grok Build places per-turn metering under result._meta (and
_meta.usage), not the top-level ACP usage field. Multica's shared
parser only read result.usage, so Grok tasks recorded empty token
usage and cost dashboards stayed at zero.
Fall back to _meta.usage (then flat _meta counters) when top-level
usage is absent. Prefer standard top-level usage when both are
present. Update the Grok fake ACP fixture and add regression tests
against the live 0.2.x payload shape.
* test(agent): cover zero ACP usage meta fallback
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: Eve <eve@multica-ai.local>
Co-authored-by: multica-agent <github@multica.ai>
Retries once when Codex's model catalog refresh blocks the first turn, with a narrow safety gate: first-turn no-progress timeout, zero semantic progress, catalog-refresh evidence in stderr, and a confirmed-reaped process tree. Clears ResumeSessionID on retry so a stalled thread is never resumed, and buffers the leading session pin so a discarded attempt cannot pin the resume pointer.
* fix(daemon): gate fresh-session retry on tools executed, not session id (MUL-4966)
Switching provider accounts leaves the stored session id pointing at a
conversation the new account does not own. The daemon still passes it to
--resume, the provider rejects it, and the task dies before doing any work.
The existing fresh-session fallback was supposed to catch this but was
gated on `result.SessionID == ""`, which is not a lifecycle fact:
- Too narrow: a backend that echoes the requested id back when it rejects
a resume keeps SessionID non-empty, so the fallback never fired — the
reported bug.
- Too broad: a provider 401 before the first stream message also leaves
SessionID empty, so an unrecoverable auth failure burned a second full
run.
Gate on `tools == 0` instead. That states the property that actually makes
a retry safe — the agent executed no tool, so it mutated nothing, so
re-running cannot double-post a comment (comment creation has no
idempotency key and a duplicate re-fires its @mention triggers), reopen a
PR, or re-plan on top of its own half-finished work in the reused workdir.
Auth failures are excluded, mirroring retryableReasons in service/task.go.
The predicate is extracted to shouldRetryWithFreshSession so the tests
exercise production logic; both existing fallback tests re-implemented the
condition inline and would not have caught a regression in it.
Co-authored-by: multica-agent <github@multica.ai>
* fix(daemon): gate fresh-session retry on a positive resume-rejected signal (MUL-4966)
Review of the previous commit was right: `tools == 0` plus "not an auth
error" answers whether re-running is *safe*, not whether a new session can
*fix* the failure. Those are orthogonal, and answering the second by
exclusion inverts the burden of proof — the failures a fresh session cures
are a small enumerable set, while the ones it cannot are open-ended.
Concretely, the previous predicate fresh-retried on provider_network,
429/529, quota, 5xx and unclassified startup failures. provider_network is
the sharpest conflict: internal/service/task.go marks it resume-safe
(MUL-4910) specifically so the platform retry inherits the session and
continues the truncated conversation. Resetting the session first made that
contract unsatisfiable, silently discarding conversation context on a
transient blip — and rate limits got an immediate no-backoff re-run.
Replace the inference with positive evidence: agent.Result gains an explicit
ResumeRejected field, set only when a backend has proof the resume itself
was refused. claude/codebuddy/qwen derive it from resumeWasRejected, which
promotes the predicate resolveSessionID was already computing and encoding
as the side effect of blanking SessionID — using an empty string to carry
that meaning is what made the original bug possible. SessionID keeps being
dropped for a rejected resume (a dead pointer must not be persisted), but it
is no longer the signal the daemon reads to decide *why* a run failed.
The six ACP backends that recover from "session not found" set the flag at
the same points they already clear the id, so their existing recovery is not
caught by the narrower gate. codex needs nothing: thread/resume already falls
back to thread/start in-process, and deliberately does not on transport
errors.
Matching now includes the account-switch guardrail reported in #5704
(Claude Code 2.1.207, zh-CN): "400 此 session 已绑定另外的ai账号，请执行
/new 开启新 session". The en-US wording of the same guardrail has not been
captured yet, so those variants are marked inferred in the source; a miss
degrades to a terminal failure carrying the provider's raw text rather than
a mis-routed run.
Tests: backend-level fixtures drive ResumeRejected from real stream-json for
both the account-binding 400 and a network drop, and the predicate now
covers network/rate-limit/quota/5xx/auth/unclassified as explicit
non-retries.
Co-authored-by: multica-agent <github@multica.ai>
* fix(daemon): restore fresh-session recovery for backends with no rejection signal (MUL-4966)
Final review caught qwen regressing: its verified rejection string
("No saved session found with ID ...", already captured in
testdata/qwen-code-0.20.0-resume-not-found.stderr.txt) was not in the phrase
list, and qwen reports no session id on that path, so the new inclusion gate
turned a working auto-recovery into a terminal failure.
Auditing the other 17 resume-capable backends showed qwen was not alone.
antigravity, copilot, cursor, deveco and opencode all recovered from a
refused resume purely by reporting an empty SessionID, and none of them has
any rejection detection to convert into ResumeRejected — copilot's own
comment documents the hole (session.error before session.start), and
antigravity's helper returns "" when "the CLI exited before dispatching".
Making ResumeRejected the sole gate silently removed recovery from all five.
Fixing that by guessing rejection phrases for five more CLIs is the wrong
trade: no real output has been captured for any of them, and a false
positive discards a recoverable session pointer. So the gate is now two
tiers. Positive evidence (ResumeRejected) decides on its own where a backend
can produce it. Where none is available, an empty SessionID still gates the
retry — it proves no session was established, which is exactly what the
pre-change behaviour relied on — minus the classes a fresh session provably
cannot cure (network, rate limit, quota, provider 5xx, auth). That keeps the
resume-safe contract in internal/service/task.go intact while restoring what
these five backends had.
Also renames claudeResumeRejectedPhrases to resumeRejectedPhrases: it is
matched by claude, codebuddy and qwen, so a qwen-only string living under a
claude-prefixed name would be actively misleading.
Tests: qwen's existing missing-resume fixture now asserts ResumeRejected
(verified failing without the phrase), and the predicate covers the
no-signal tiers — retry when nothing was established, no retry once a
session exists or the failure classifies as uncurable.
Co-authored-by: multica-agent <github@multica.ai>
* fix(daemon): scope the no-signal fallback to backends that cannot detect rejections (MUL-4966)
Final review caught the compatibility path applying to every backend, not
just the five it was justified for. shouldRetryWithFreshSession only saw
(Result, priorSessionID, tools), so a false ResumeRejected could not be told
apart from a backend that has no way to answer — and claude/codebuddy/qwen/ACP
startup failures with no session id still fell through to the exclusion
branch. That contradicted both the stated intent and the function's own doc
comment ("where a backend can produce it, it is the whole answer").
Make the capability explicit. agent.ResumeRejectionUndetectable names the
five backends that scrape SessionID out of stream output and have no
rejection detection at all; the daemon takes provider and consults it, so a
capable backend reporting false is now taken at its word. Membership is
opt-in, so a new backend fails closed instead of silently inheriting a
guess-based retry.
Also completes the exclusion set: missing config, unavailable model, missing
executable, unsupported runtime version and (defensively) agent timeout all
have defined non-session remedies and were reaching `default: true`. What is
left through stays narrow — unknown, process failure, unparseable output,
context overflow — because a real rejection from these five most likely
surfaces as a non-zero exit or unparseable output, none of them reporting one
explicitly.
Tests: one identical result asserted across all five undetectable backends
(retries), twelve capable ones (no retry), and an unregistered provider
(fails closed), plus table cases for each newly excluded reason. Classifier
inputs were verified to map to the intended reasons rather than passing by
accident.
Also updates the ResumeRejected doc comment, which still said the daemon
gates on it alone.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: Bohan-J <bohan@devv.ai>
Co-authored-by: multica-agent <github@multica.ai>
* fix(agent/cursor): send prompt on stdin so CLI-like flags cannot be re-tokenised (MUL-4992)
On Windows a Cursor task whose prompt contains CLI-like flags failed in ~2s
with `error: unknown option '-X'` and no agent output (#5649). A pasted build
log such as
go build -ldflags "-X main.version=foo" -o bin/server ./cmd/server
was enough to kill the run.
buildCursorArgs put the whole prompt in argv as the positional after -p.
The official Windows launcher chain ends in `& node.exe index.js $args`
inside cursor-agent.ps1, where PowerShell re-serialises $args onto node's
command line. Under Windows PowerShell 5.1 and pwsh <= 7.2 (Legacy native
argument passing) an argument holding embedded double quotes is not
re-escaped, so the quoted region closes early, node's argv parser re-splits
at the interior spaces, and `-X` reaches commander.js as a standalone flag.
#1709 removed the cmd.exe `%*` re-tokenisation but stopped at the
Go -> PowerShell boundary, one hop before this. Its Windows tests only
compare the argv slice Go builds and never execute a shim, which is why the
gap stayed invisible.
Fix: keep the prompt off every command line. cursor-agent's -p is a boolean
print-mode switch and the prompt is positional; with no positional prompt and
a non-TTY stdin the CLI reads stdin to EOF and uses that as the prompt. So
drop the prompt from argv on all platforms and write it to stdin, leaving
only fixed, content-free flags in argv. No shell or launcher on any platform
can re-tokenise what is not on a command line.
The write runs in its own goroutine: a prompt larger than the pipe buffer
(~64 KiB) blocks mid-write until the child drains it, and the child cannot
drain while nothing reads its stdout. Closing stdin signals end-of-prompt, so
it is closed on both success and error paths, and on cancellation to release
a blocked write. Write failures surface in the result diagnostic, ranked
below explicit agent errors so an early child exit (bad auth, bad flag) is
not masked by the resulting EPIPE.
Tests: a prompt carrying the exact `-ldflags "-X ..."` shape must arrive
byte-for-byte on stdin and appear nowhere in argv; a 512 KiB prompt must not
deadlock; the prompt is written verbatim (the CLI trims it, we do not). Both
new unix tests fail against the pre-fix code. A Windows-tagged test drives a
real PowerShell host through the same .cmd -> -File rewrite.
Closes#5649
Co-authored-by: multica-agent <github@multica.ai>
* test(agent/cursor): prove the Windows launcher fix on both PowerShell hosts in CI
The stdin fix has to hold on the host that actually exhibits the bug.
powershell.exe (5.1) and pwsh <= 7.2 default to Legacy native argument
passing; pwsh >= 7.3 defaults to Standard. A fix verified only on the newer
host would not be a fix for the reporter.
Run the shim probe against every PowerShell host on PATH rather than only the
one defaultPowerShellLookup would select, and hook the windows-tagged launcher
tests into the existing windows-execenv CI job. These tests are windows-tagged
and the backend job runs on ubuntu, so until now they ran nowhere.
Co-authored-by: multica-agent <github@multica.ai>
* test(ci): run Windows launcher tests verbosely so skips are visible
A skipped or unmatched test still reports "ok", which would make the
Windows job look like coverage it is not providing.
Co-authored-by: multica-agent <github@multica.ai>
* test(agent/cursor): close both coverage gaps in the #5649 regression tests
Two tests claimed guarantees they did not actually establish.
Windows: the fake cursor-agent.ps1 called [Console]::In.ReadToEnd() and wrote
the result itself, so it never launched a native child. The official shim ends
in `& node.exe index.js $args`, and that last hop is precisely where the bug
lives -- PowerShell re-serialises $args onto the child command line, and
whether the child inherits stdin was left unproven. The fake ps1 now
re-executes the test binary as a real native child (helper-process idiom),
which records the argv it actually received and drains stdin. Both PowerShell
hosts still run.
Interlock: the large-prompt fake drained stdin before writing any stdout, so
the child always unblocked the parent immediately and the test passed even
against a synchronous write -- it could not fail for the reason it existed.
The fake now floods stdout past pipe capacity *before* reading stdin, creating
the real mutual block. Verified: with the writer made synchronous the test
deadlocks to its 30s timeout, and passes only with the concurrent writer.
Also adds the missing cancellation case: a child that never reads stdin leaves
the writer blocked forever, so cancelling the context must close stdin,
release the writer and settle the run as aborted.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: Bohan-J <bohan@devv.ai>
Co-authored-by: multica-agent <github@multica.ai>
* fix(daemon): run Codex unsandboxed on Windows to stop reject-by-policy (MUL-4957)
Windows has no Landlock/Seatbelt-equivalent filesystem sandbox that the
daemon configures, so the per-task `sandbox_mode = "workspace-write"` it
wrote was unenforceable. Worse than having no sandbox, it pushed Codex
into rejecting non-safe mutation commands "by policy": `multica issue
create` fails with "was rejected by policy" because Codex can neither
sandbox the command nor (under approval_policy = "never") escalate it to
the daemon's auto-approver, so the request never reaches the approver.
Mirror the existing macOS fallback and give Windows danger-full-access so
those commands run. Also generalize the danger-full-access warn log so it
no longer hardcodes "on macOS" and only surfaces the macOS-specific
upgrade hint on macOS (new codexSandboxPolicy.Hint field).
Co-authored-by: multica-agent <github@multica.ai>
* fix(daemon): correct Windows sandbox rationale and respect user windows.sandbox (MUL-4957)
Addresses two review must-fixes on #5672:
1. Correct a false security fact. The comments and log Reason claimed
Windows has no filesystem sandbox backend. Codex 0.144.5 does ship a
native Windows sandbox (windows.sandbox = "unelevated"/"elevated"); it
is experimental with open upstream reliability bugs, so the daemon
defaults to danger-full-access as a deliberate compatibility choice.
Enabling the native sandbox is tracked as separate follow-up work.
2. Stop silently downgrading users who opted into isolation. The fallback
was unconditional. Add codexSandboxPolicyForConfig: on Windows an
explicit windows.sandbox = unelevated|elevated keeps workspace-write so
Codex enforces task isolation with the user's chosen backend;
danger-full-access applies only when windows.sandbox is absent,
disabled, or unparseable. This is also the branch point for a future
native-sandbox rollout (flip the default; callers unchanged).
Adds fixture tests locking the priority (user opt-in kept vs. unconfigured
fallback) plus predicate coverage for codexSandboxPolicyForConfig.
Co-authored-by: multica-agent <github@multica.ai>
* fix(daemon): fail closed on undecidable Windows sandbox config, honor -c windows.sandbox (MUL-4957)
Second review round on #5672. Two must-fixes.
1. Undecidable config no longer fails open. The old bool detector collapsed
"unparseable / invalid value / failed copy" into "unconfigured" and then
loosened to danger-full-access. Replaced with a tri-state
(absent/native/undecidable): only exact-lowercase unelevated|elevated (the
sole values Codex accepts — verified: any other value makes Codex refuse to
load the config) counts as native; any other present value, unparseable
TOML, a read error, or a missing per-task config when a shared
~/.codex/config.toml exists (i.e. the copy failed) is undecidable and fails
closed to workspace-write — it never loosens — logged at error level.
2. windows.sandbox set via `-c`/`--config` custom args is now honored. Such
args never land in config.toml, so config-only detection silently
downgraded those users' isolation. The effective Codex args (daemon
defaults + profile-fixed + per-agent custom_args) are threaded through
PrepareParams/ReuseParams/CodexHomeOptions into the sandbox decision and
scanned for a windows.sandbox override (inline, two-token, quoted, spaced;
last-wins).
Also drops issue-status-bound source comments (openai/codex#24098 has since
closed). Adds unit coverage for config/args classification, the fold
precedence (undecidable > native > absent), and the copy-failed fail-closed
path.
Co-authored-by: multica-agent <github@multica.ai>
* fix(daemon): fail closed on config-sync errors and honor shell-quoted -c windows.sandbox (MUL-4957)
Round-3 review must-fixes:
1. resolveWindowsSandboxState now takes the config.toml sync error and a
tri-state shared-config presence instead of re-stat-ing inside. A failed
sync (stale/absent per-task copy) or an un-stat-able shared source is
undecidable and keeps workspace-write, closing the fail-open where a failed
sync was read as "unconfigured". Splits IO from the decision so the paths
are unit-testable without faulting the filesystem.
2. The Windows sandbox decision consumes agent.NormalizeCodexLaunchArgs (the
shared helper buildCodexArgs now uses) so a shell-quoted -c windows.sandbox
opt-in is normalized identically to launch, instead of being missed by a
raw-token scan and silently downgraded.
Co-authored-by: multica-agent <github@multica.ai>
* fix(daemon): abort when the Codex sandbox block cannot be written (MUL-4957)
Round-4 review must-fix: ensureCodexSandboxConfig failures were warn-and-continue,
so a computed fail-closed workspace-write policy could stay only in memory while
config.toml kept a stale danger-full-access from a prior run — the decision
failed closed but the effective config failed open.
prepareCodexHomeWithOpts now returns the error, which blocks startup on both
paths: fresh Prepare fails the task, and Reuse leaves env.CodexHome unset, which
configureCodexTaskShellEnvironment already refuses to start.
Regression covers the full reuse scenario (stale danger-full-access + failed
config sync + failed managed-block write); it fails with "got nil" without the
fix.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: Bohan-J <bohan@devv.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: J <j@multica.ai>
When agent.model equals the model the Hermes ACP session already reports
as current, Multica was still replaying session/set_model. Hermes'
set_model re-runs provider auto-detection on the model id, which for a
provider:model id whose parsed provider matches the session's current
provider (e.g. a named custom endpoint exposed as "custom") can mis-route
to a different provider — custom:deepseek-v4-pro resolves into the
OpenRouter catalog and the turn fails with an auth error. Only the retry
via session resume, where the corrupted provider no longer matches the
custom: prefix, succeeds.
Capture the runtime's reported currentModelId from session/new and
session/resume and skip the redundant switch when it already equals the
requested model. An empty/unparsable current model falls through and
still sends set_model, preserving prior behaviour. The explicit
provider:model id is never rewritten. Upstream: NousResearch/hermes-agent#59089.
MUL-5029
Co-authored-by: Bohan-J <bohan@devv.ai>
Co-authored-by: multica-agent <github@multica.ai>
* fix(kiro): require positive proof for -32603 completion rescue (MUL-4860, #5509)
Addresses the #5511 review (Elon):
Must-fix 1 — a missing result is not proof of success. A mid-command
crash / internal session failure produces the same 'tool use, no result,
-32603' shape as a genuine completion, so the previous 'saw the use but
never saw a result' rescue could mark an unfinished task completed. The
guard now rescues only on POSITIVE proof: a terminal ToolResult with
status=='completed' for a finishing tool (goal_complete / comment-add).
Must-fix 2 — the previous sticky booleans only ever wrote true, so a
later failed delivery could not override an earlier success, and call
ordering was ignored. Completion state is now the status of the MOST
RECENT finishing-tool result (keyed per CallID): 'progress completed →
final failed → -32603' stays failed; 'first failed → retry completed →
-32603' is a real completion.
Comment-add is still recognized by its command payload regardless of the
tool's display title (the #5509 core ask), so a completed comment-add
carrying a non-terminal title is preserved through the close handshake.
Tests: result-less running tool stays failed (both goal_complete and
comment-add), completed non-terminal-title comment-add is preserved,
failed-final-overrides-earlier-success, and completed-retry-after-failure.
Co-authored-by: multica-agent <github@multica.ai>
* fix(acp): don't drop completed tool_call_update when rawOutput is an object (MUL-4860, #5509)
Captured a live kiro-cli 2.12.3 + gpt-5.6-sol ACP trace and found the
true root cause: the completed tool_call_update sends rawOutput as an
OBJECT ({"items":[{"Json":{...}}]}), but handleToolCallUpdate typed
rawOutput/output as Go strings. json.Unmarshal then failed with
'cannot unmarshal object into Go struct field .rawOutput of type string'
and the handler returned early — silently dropping the ENTIRE update,
status:"completed" included. So no completion signal ever reached the
kiro completion-preservation guard, and the durable-but-closed task was
marked failed. This also explains the issue's 'shell tool recorded as
running': the tool_call title is literally 'Running: <cmd>' (kind
execute), which normalizes to 'running', not 'terminal'.
Fix: type rawOutput/output as json.RawMessage and render them via a new
acpRawText helper that accepts both a JSON string and a structured
value. This is a shared ACP-layer fix (hermes/kimi/kiro).
Combined with the earlier payload-based comment-add recognition and the
positive-proof completion guard, the completed result now flows through
and the -32603 close handshake correctly preserves completion.
Adds TestKiroBackendPreservesCompletionOnRealGPT56SolFrames (the exact
captured wire shape end-to-end) and TestACPRawText. Verified the
end-to-end test fails (status=failed) when rawOutput is typed as string
and passes after the fix.
Co-authored-by: multica-agent <github@multica.ai>
* fix(kiro): remove duplicate stale-session branch that faked completion (MUL-4860, #5509)
Addresses Elon's 3rd-round review of #5515. The -32603 guard had two
consecutive, identically-conditioned else-if branches for a stale resumed
session (opts.ResumeSessionID != "" && isACPSessionNotFound(err)). The
first wrongly forced finalStatus="completed" and kept the stale
SessionID; the second — the correct handler that clears SessionID so the
daemon's fresh-session retry fires — was unreachable. So a recovery whose
resumed session was gone at session/prompt time was reported as a fake
success AND skipped the retry.
Remove the bogus first branch, keeping the positive-proof completion
branch and the original stale-session (SessionID="") branch.
Adds TestKiroBackendClearsSessionIDWhenPromptSessionNotFound (the prompt
path; existing coverage only exercised session/set_model). Verified it
returns completed+ses_stale on the pre-fix code and failed+empty
SessionID after. Also gofmt on the touched files.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: Eve <eve@multica-ai.local>
Co-authored-by: multica-agent <github@multica.ai>
The -32603 completion-preservation guard coupled to the Claude/Kiro ACP
event shape. GPT-5.6 Sol leaves the finishing tool (goal_complete /
'multica issue comment add') parked at 'running' and titles the shell
tool with a name that does not normalize to 'terminal', so:
- isKiroIssueCommentAddTool's msg.Tool=='terminal' gate dropped the tool
use entirely, and
- with no completed/failed ToolCallUpdate, no ToolResult is emitted, so
the saw*Completed flags never flipped.
Completed tasks were therefore reversed to failed with
agent_error.provider_server_error ~17s after finishing their work.
Fix:
- Recognize comment-add by its command payload, independent of the
normalized tool title.
- Track each finishing tool along use / result / completed axes and
preserve completed when the -32603 close handshake follows a tool use
that never produced a terminal result. A genuinely failed ToolResult
still trips saw*Result and keeps the task failed.
Adds regression fixtures for the running-tool path (comment-add and
goal_complete) plus a failed-result safety case.
Co-authored-by: Eve <eve@multica-ai.local>
Co-authored-by: multica-agent <github@multica.ai>
* fix(agent): fail opencode runs whose stream ends without a terminal signal
* fix(agent): treat step_finish(reason "tool-calls") as non-terminal in the EOF guard
Review follow-up: step bracketing alone left a false-green window. A tool
step closes with step_finish(reason "tool-calls") before its continuation
step_start, so a stream that dies in that gap had no open step and still
reported "completed".
Parse part.reason (live-probed on opencode 1.17.16: tool loop emits
reasons "tool-calls" then "stop") and keep the run non-terminal after a
"tool-calls" finish until the next step_start. Any other reason —
including absent, for older opencode versions that predate the field —
stays terminal so healthy runs on old protocols are not mass-failed.
Tests: tool-calls finish then EOF now fails; the multi-step happy path
uses the real wire shape ending in reason "stop"; a reason-less
step_finish (legacy protocol) still completes.
* fix(agent): keep stop tool steps pending continuation
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: Eve <eve@multica-ai.local>
Co-authored-by: multica-agent <github@multica.ai>
Add xAI Grok Build (`grok`) as a first-class Multica runtime over ACP
(`grok --no-auto-update agent --always-approve stdio`), reusing
hermesClient like traecli and kimi. Includes daemon discovery,
protocol_family migration 174, model discovery, MCP passthrough,
thinking effort, frontend branding, and product docs.
Follows xAI's documented headless ACP flow: after `initialize`, read the
advertised `authMethods` and send `authenticate` (preferring xai.api_key
when XAI_API_KEY is set, else the cached login token) before any session
operation — a real, logged-in CLI rejects session/new and session/load
without it. Model discovery performs the same handshake so it returns the
live catalog instead of the static fallback.
`--no-auto-update` is passed as a global flag (and kept daemon-owned in
the blocked custom-arg set) so a background update check can't stall an
unattended ACP task. Thinking effort uses the current `--effort` flag,
and the minimum grok version is 0.2.89 (ACP + authenticate + session/load
+ session/set_model + MCP + --effort).
Closes#2895
* fix(daemon): isolate Codex sessions per task to unblock initialize (MUL-4424)
Codex 0.143+ backfills a per-home session-state DB by enumerating every
rollout visible under sessions/ during `initialize`. The per-task
CODEX_HOME symlinked the shared ~/.codex/sessions in, so a machine with a
large accumulated history (one reporter: ~2000 rollouts / ~22 GiB) stalled
`initialize` for tens of seconds — the app-server started but the task
produced no output before it was cancelled (github #5273).
Give each task its own local sessions/ directory instead:
- Fresh task: create an empty local sessions/ so backfill is trivial.
- Reused task with a real sessions/ dir: it is authoritative — leave it.
- Reused task still holding a legacy symlink (older build): migrate in
place. Replace the symlink with a real dir; when resuming, symlink only
the single rollout being resumed (never copy — a rollout can be GiB and
this is on initialize's critical path); and drop the stale, rebuildable
session-state DB (state_*.sqlite*, session_index.jsonl) so Codex
re-indexes the task-local sessions. Unrelated per-task DBs (goals_*,
logs_*, memories_*) are left intact.
Also point the token-usage fallback scan at the backend's per-task
CODEX_HOME instead of the daemon-global home, so usage isn't lost now that
sessions are isolated there.
Complements the #5319 handshake watchdog (which turns a silent stall into a
loud, phased timeout); this removes the underlying cause.
Co-authored-by: multica-agent <github@multica.ai>
* fix(daemon): address Codex session isolation review (MUL-4424)
Resolves the three blockers from Elon's review of #5360:
1. local_directory context loss. local_directory tasks get a fresh
codex-home per task ID (the daemon never reuses their workdir), so
task-local isolation stranded every follow-up run with an empty
sessions dir and silently restarted the conversation. Their only
stable, GC-safe cross-task store is the user's own ~/.codex/sessions
(a persistent store under WorkspacesRoot would be orphan-GC'd), so keep
the shared-sessions symlink for them (IsLocalDirectory). Managed tasks
stay isolated.
2. Migration resume robustness.
- Rollout lookup now covers the flat layout and background-compressed
.jsonl.zst rollouts, not just nested YYYY/MM/DD *.jsonl — both are
legitimate Codex 0.144 history that were previously judged "not
found", silently dropping resume.
- Exposure hard-links first, then symlinks, never copies — hard links
need no privilege and work on Windows within a volume, so the
zero-copy path is exercised identically on CI.
- The daemon now verifies the rollout is actually present in the task
CODEX_HOME (execenv.CodexResumeRolloutPresent) before the brief is
generated; if absent it clears the resume from both the backend and
the brief instead of telling the agent it is continuing a lost thread.
3. session_index.jsonl is no longer deleted during migration — Codex uses
it as the authoritative thread-id -> name store (not rebuildable from
rollouts). Only the rebuildable state_*.sqlite* is reset.
Tests: 2-round local_directory resume across task IDs; compressed/flat
lookup; hard-link zero-copy (os.SameFile); session_index preserved;
CodexResumeRolloutPresent + the daemon gate helper (present keeps /
absent drops / non-codex + empty no-op).
Co-authored-by: multica-agent <github@multica.ai>
* fix(daemon): scope Codex sessions to a per-issue store; disclose lost resumes (MUL-4424)
Addresses the three blockers from Elon's second review of #5360.
1. local_directory still enumerated the whole machine history. The prior
fix re-linked the entire ~/.codex/sessions into every fresh local_directory
codex-home, so Codex still backfilled from thousands of unrelated rollouts on
`initialize` (measured ~8.3s with 3450 rollouts; the reporter's 22 GiB could
exceed the 30s watchdog). Point sessions/ at a persistent, per-(agent, issue)
store under the shared Codex home (multica-sessions/<agent>/<issue>) that holds
only this issue's rollouts. It is keyed stably across task IDs and lives
outside the task-scoped envRoot the GC reclaims, so follow-up runs resume it
while `initialize` only ever sees this issue's history.
2. Windows cross-volume resume was lost. Exposing a single rollout by hard-link
(same-volume only) then file symlink (needs Windows privilege) can't cross a
volume boundary. The store now lives on the shared Codex volume, so the resume
rollout is hard-linked there zero-copy, and sessions/ is exposed to the task
home via a directory link — a symlink on Unix, a junction on Windows — which
crosses volumes without privilege and never copies a (possibly GiB) rollout on
initialize's critical path. There is no remaining per-file cross-volume link.
3. An unavailable resume was a silent downgrade. Both resume gates
(gateResumeToReusedWorkdir, gateCodexResumeToRolloutPresence) now set
PriorSessionResumeUnavailable, and the runtime brief renders a Session
Continuity Notice telling the agent to disclose to the user, up front in its
reply, that the previous conversation context could not be restored and this
run starts fresh — turning a silent restart into a user-visible one. The task
is not failed: it can still do useful work without the prior context.
Managed fresh / reused-real-dir tasks keep their task-local, GC-collected
sessions dir unchanged; only the legacy-symlink migration with a resume routes
through the store (cross-volume-safe), and a home already linked to the store is
treated as authoritative on reuse.
Tests: local_directory per-issue store (only this issue's history, no whole-
machine leak); no-key fallback to an empty dir; two-round resume across task IDs
through the store; legacy migration routed through the store with a zero-copy
hard link; reused store link stays authoritative; both gates set the
resume-unavailable flag; brief renders the continuity notice only when a resume
was lost. execenv + daemon + pkg/agent packages, go vet, and gofmt all pass.
Co-authored-by: multica-agent <github@multica.ai>
* fix(daemon): disclose live resume-RPC loss; bound Codex session store lifecycle (MUL-4424)
Addresses the two blockers from Elon's third review of #5360.
1. A real thread/resume failure was still a silent new session. The brief's
Session Continuity Notice only covers losses the daemon detects before launch
(workdir not reused, rollout absent). But when the rollout is present yet
Codex rejects the live thread/resume (corrupt/incompatible rollout, server-side
thread GC, schema drift), startOrResumeThread falls back to thread/start and
the run succeeds on a fresh thread with no user-facing signal. Carry the
original resume intent into the backend as ExecOptions.ResumeExpected (set from
the post-gate PriorSessionID, so a pre-flight drop still routes through the
brief and never double-notifies), and when a resume was expected but the
backend landed on a fresh thread, prepend the same continuity notice to the
first turn/start input. This also covers the daemon's transport-error
fresh-session retry, which clears ResumeSessionID but not ResumeExpected.
2. The persistent per-issue store had no data lifecycle. multica-sessions stores
live outside the task-scoped envRoot the GC reclaims (so resume survives across
task IDs), which meant a done/abandoned issue's prompts and full rollouts (one
reporter: a single 1.5 GiB rollout) accumulated forever and were never freed on
issue/agent/workspace deletion. Add PruneCodexSessionStores: the daemon GC loop
reclaims any store untouched for GCCodexSessionTTL (default 14 days, configurable
via MULTICA_GC_CODEX_SESSION_TTL, 0 disables). A store's newest rollout mtime is
its last activity, so an active or recently-resumed task keeps its store fresh
and is never reclaimed, while a deleted issue's store ages out — an eventual
reclamation guarantee without needing deletion events.
Tests: codexTurnInput discloses on resume fallback and stays silent on success /
fresh start (paired with the existing live-RPC fallback test); store pruning
reclaims aged stores, keeps recent ones, isolates issues, cleans empty agent
dirs, and is disable-able. execenv / daemon / pkg/agent, go vet, gofmt all pass.
Co-authored-by: multica-agent <github@multica.ai>
* fix(daemon): protect a reopened Codex session store from GC mid-mount (MUL-4424)
Addresses Elon's fourth-review blocker: reopening an issue idle past
GCCodexSessionTTL could lose context, because mounting its per-issue session
store (MkdirAll + rollout lookup + task-home link) never refreshed the store's
mtime, so a GC cycle firing before the resumed turn wrote its first rollout saw
a >TTL-old store and reclaimed it — a stat->remove race with no in-use guard.
Two complementary defenses:
- Activity refresh: linkCodexSessionsToStore now os.Chtimes the store to now
after linking, so codexStoreStat (which reads the newest mtime as last
activity) sees a just-used store. This fixes the sequential repro — a mount
immediately followed by a prune keeps the store.
- In-process active-store guard: the daemon marks the per-issue store in-use
(execenv.CodexSessionStorePath) from before Prepare/Reuse mounts it until the
task ends, and PruneCodexSessionStores now takes an isActive predicate and
skips any store a live task holds. Because prepare and prune run in the same
process, this closes the remaining concurrent stat->remove window the mtime
refresh alone cannot. Reference-counted, mirroring the env-root guard.
Tests: a reopened >TTL store survives a GC cycle after remount and stays
resumable; an idle-on-disk store marked active is skipped, then reclaimed once
inactive; the existing idle-reclaim / isolation / disable / empty-agent-dir
cases still pass. execenv + daemon, go vet, gofmt all pass.
Co-authored-by: multica-agent <github@multica.ai>
* fix(daemon): make Codex store delete atomic with mark-active (MUL-4424)
Addresses Elon's fifth-review blocker: the active-store guard's check and delete
were not one atomic step. PruneCodexSessionStores called isActive (which locked,
read, and unlocked) and only then RemoveAll'd, leaving a window where a task
could markActiveCodexStore between the check and the removal and still lose its
store — the exact mark-then-delete interleaving Elon reproduced.
Replace the point-in-time isActive predicate with a reserve-for-deletion
protocol that shares one lock with mark-active:
- reserveCodexStoreForDeletion(store) atomically refuses when a live task holds
the store (or another delete already reserved it) and otherwise marks it
reserved, all under one activeCodexStoresMu acquisition. PruneCodexSessionStores
reserves before RemoveAll and commits after, so confirm-inactive and remove are
effectively atomic against a concurrent mark.
- markActiveCodexStore now waits (on a sync.Cond) while a store is reserved, so a
task never mounts a store mid-removal; committing the removal wakes it and the
store is recreated fresh by Prepare (with the continuity notice).
So mark-before-reserve keeps the store (reserve refused); reserve-before-mark
removes it and blocks the late mark until the removal commits. The genuinely
idle case still reclaims.
Tests (daemon, run under -race): mark-then-reserve is refused; reserve blocks a
concurrent mark until commit then the store reads active; a second reserve is
refused mid-flight. The execenv prune tests move to the reserve seam; the
activity-refresh / reopen-then-prune / isolation / disable cases still pass.
Co-authored-by: multica-agent <github@multica.ai>
* fix(daemon): namespace Codex session stores per profile for cross-daemon safety (MUL-4424)
Addresses Elon's sixth-review blocker: the in-process reservation guard cannot
span processes, but Multica supports multiple profile daemons on one machine
(e.g. production + staging) that share the same ~/.codex. Each daemon's GC
scanned the whole multica-sessions root, so a staging daemon could reclaim a
store a production task was actively resuming — its reservation lived only in the
other process's memory.
Isolate by profile instead of trying to lock across processes:
- Store path is now <shared>/multica-sessions/<namespace>/<agent>/<issue>, where
namespace is the daemon's profile (empty -> "default"). PrepareParams/ReuseParams
carry Profile; codexSessionStoreKey and CodexSessionStorePath fold it in.
- PruneCodexSessionStores takes the profile and scans ONLY that namespace, so a
daemon never even sees another profile's stores, let alone deletes them. The
per-profile trees are disjoint, so the in-process guard is sufficient within a
namespace (profiles get separate daemon state, so no two daemons share one).
Test: a "staging"-owned idle store is untouched by a default-profile prune and
reclaimed only by staging's own prune. Existing prune/guard/reopen tests move
under the namespace. execenv + daemon under -race, go vet, gofmt all pass.
Co-authored-by: multica-agent <github@multica.ai>
* fix(daemon): make the Codex store profile→namespace map injective (MUL-4424)
Addresses Elon's seventh-review blocker: the per-profile namespace was derived by
dropping unsafe characters, which is not injective. The CLI treats the empty
(default) profile and a profile literally named "default" as separate daemons,
yet both mapped to namespace "default"; likewise "staging.prod" and "stagingprod"
both mapped to "stagingprod". Two distinct daemons then shared one store tree, so
one could again reclaim the other's live session — the cross-process blocker
reopened for those profile names.
Make codexSessionStoreNamespace injective: the empty profile gets a reserved
bare literal "default", and every named profile is hex-encoded (bijective,
filesystem-safe) under a "p_" prefix a bare literal can never collide with. So
"" -> "default" while "default" -> "p_64656661756c74", and "staging.prod" /
"stagingprod" get distinct hex segments. sanitizeCodexPathSegment stays for the
UUID agent/issue segments (injective for real UUIDs); only the user-controlled
profile needed the encoding.
Tests: codexSessionStoreNamespace is distinct for "" vs "default", punctuation
variants, case variants, and an encoded-looking name; and end-to-end, pruning one
profile never reclaims the other's store for the "" vs "default" and
"staging.prod" vs "stagingprod" pairs. execenv + daemon under -race, go vet,
gofmt all pass.
Co-authored-by: multica-agent <github@multica.ai>
* fix(daemon): fixed-length Codex store namespace so long profiles fit (MUL-4424)
Addresses Elon's eighth-review blocker: hex-encoding the full profile doubled the
namespace segment length. A profile can be as long as a filesystem segment allows
(~255 bytes) and the CLI persists it as its own config dir, but the store
namespace "p_" + hex(profile) reached 2 + 127*2 = 256 bytes at 127 chars,
overflowing the 255-byte single-segment limit — the profile's own dir created
fine, then the session store failed with "file name too long".
Derive the namespace from a fixed-length hash instead: a named profile is now
"p_" + hex(sha256(profile)) — a constant 64 hex chars (66 with the prefix),
filesystem-safe and collision-resistant. The empty (default) profile keeps its
reserved bare literal "default", which the "p_"-prefixed 66-char segment can
never equal. Still injective across the CLI's distinct-daemon cases; just no
longer length-expanding.
Test: the namespace stays <=255 bytes and creatable for profiles up to the
255-byte segment limit (127- and 255-char cases that overflowed under hex); the
prior injectivity and cross-profile prune-isolation tests still hold. execenv +
daemon under -race, go vet, gofmt all pass.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: J <j@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
* fix(execenv): overlay per-task HERMES_HOME so Hermes discovers bound skills
Hermes has no workspace-relative skill discovery — it scans <HERMES_HOME>/skills
first, then skills.external_dirs from config.yaml (verified against the bundled
agent/skill_utils.py). The daemon wrote assigned skills to the generic
.agent_context/skills/ fallback, which Hermes never reads, so they silently never
took effect (#5242).
When (and only when) an agent has skills bound, redirect HERMES_HOME to a minimal
per-task compatibility overlay; a skill-less Hermes task keeps its real home and
original behavior:
- mirror every top-level entry of the shared home via symlink except the
overlay-owned ones (denylist), reconciling entries deleted from the shared home;
- derive a task-local config.yaml whose skills.external_dirs references the shared
skills dir plus the user's existing external_dirs, expanded against the
sanitized effective child env (unknown vars preserved, blocklisted keys resolved
to the process value) and normalized to absolute paths;
- write only the bound skills into the task-local skills/ dir (home skills scanned
first, so they win); global skills are referenced, not copied;
- keep memories/ overlay-owned (fresh per-task dir) AND disable the external
memory.provider, so neither on-disk memory nor a shared backend crosses tasks;
- keep active_profile/profiles out of the overlay so Hermes can't follow a sticky
profile and redirect past it at startup.
Profile handling mirrors hermes_cli.profiles: the daemon reads -p/--profile with
agent.HermesProfileFromArgs and seeds the overlay from that profile's home via
ResolveHermesSourceHome (default/invalid -> base, valid name -> <base>/profiles/
<name>, validated; a missing named profile fails closed). The profile flags are
stripped from the acp argv ONLY when the overlay is active (hermesLaunchArgs), so
a skill-less task's profile passes through unchanged. HERMES_HOME is no longer
custom_env-blocklisted: no skills -> user value passes through; skills -> overlay
overrides after layering. Fail closed — Prepare errors, Reuse returns nil.
Task home 0700, derived config 0600 via atomic replace. Platform-native default
home (%LOCALAPPDATA%\hermes, incl. the LOCALAPPDATA-missing fallback, on Windows).
Tests span execenv/daemon/agent: no-skill no-op, child-env layering + env
sanitization, profile parse/unquote + conditional strip + final args/env per
scenario, custom/profile/default/invalid/missing/Windows source home, sticky-
profile not mirrored, memory dir isolation + external provider disable, mirror
reconciliation, external_dirs rebasing + sanitized/unknown-var expansion,
local-precedence slug, perms, fail-closed, resume teardown. Docs (en + ja/ko/zh).
Fixes#5242
* fix(hermes): make profile selection one resolver contract matching Hermes
Round 5 review: the profile chain approximated Hermes' semantics in three
separate places (argv parsing, source-home selection, arg filtering), so it
diverged from native Hermes in several merge-blocking cases. Collapse it into
one authoritative resolution:
- agent.ParseHermesProfileArgs replaces HermesProfileFromArgs/
FilterHermesProfileArgs. It reproduces _apply_profile_override step 1/1b
(first occurrence, value-flag skipping, `--` and `mcp add --args` boundaries,
space-form profile-id guard) and returns the exact argv occurrence to consume;
StripHermesProfileArgs removes only that occurrence.
- execenv.ResolveHermesProfile replaces ResolveHermesSourceHome. It derives the
Hermes root exactly like get_default_hermes_root (an already-profile-scoped
HERMES_HOME roots at its grandparent), selects an explicit profile first,
otherwise trusts a profile-scoped home (step 1.5) and only then the sticky
<root>/active_profile (step 2), and validates via normalize/validate_profile_name
(reserved hermes/test/tmp/root/sudo and empty inline `--profile=` are hard
errors). Profiles always resolve under the root, so `-p default` re-roots and
`-p <sibling>` is a sibling, never nested.
- The daemon runs one parse + resolve, fails the task closed on a reserved/
invalid selection (matching Hermes' sys.exit(1)), and exports the selected
source home as the effective env's HERMES_HOME so ${HERMES_HOME} in a profile's
skills.external_dirs expands against the selected profile home (as native
Hermes does before loading config.yaml), not the root or the overlay.
Regressions added: root + sticky named profile selection; already-profile-scoped
home with no flag; that home with -p default and -p <sibling>; reserved and empty
inline profile values; and a selected profile whose external_dirs contains
${HERMES_HOME}.
* fix(hermes): overlay-owned derived .env + symlink-resolved root
Round 6 review, two remaining overlay-bypass paths:
1. A source `.env` could redirect HERMES_HOME after profile resolution. Hermes
runs `_apply_profile_override()` then `load_hermes_dotenv()`, which loads
`<HERMES_HOME>/.env` with override=True — so a mirrored source `.env` carrying
an out-of-band `HERMES_HOME=` overwrote the overlay's home, repointing skill
discovery and memory back at the source. `.env` is now overlay-owned and
DERIVED (writeDerivedHermesEnv): it preserves the source's credentials/settings
but strips any `HERMES_HOME` assignment and pins `HERMES_HOME` to the overlay
last (single-quoted, literal), written 0600 via atomic replace. It is written
even when the source has none, so Hermes' project-`.env` fallback (override=True
only when no user `.env` loaded) can't relocate the home either.
2. Root derivation was lexical-only, diverging from `get_default_hermes_root`,
which compares `env_path.resolve()` with `native_home.resolve()`. A HERMES_HOME
symlinked into `<native>/profiles/<x>` was treated as its own root, so
`-p default`/`-p <sibling>` resolved wrong. `hermesRootFromHomeFor` now resolves
symlinks (Path.resolve(strict=False)-style best effort) for the containment
decision while keeping the returned root unresolved, matching Hermes.
Regressions: source `.env` with HERMES_HOME replayed through the override=True
dotenv order (bound skill + task memory stay on the overlay; creds preserved);
minimal overlay `.env` created when the source has none; and a symlinked profile
home resolving `-p default`/`-p <sibling>` to the native root.
Fixes#5300. The daemon hardcoded optionId="approve_for_session" when auto-approving Hermes' session/request_permission, but Hermes' ACP edit-approval offers only ["allow_once","deny"] and rejects anything else, silently blocking every file write. handleAgentRequest now selects an option the agent actually offered — a safe session/single-use grant, else an offered reject_once to deny just that action, else a JSON-RPC error — never a permanent allow_always or a whole-turn cancelled. Includes regression + branch-coverage tests.
* fix(codex): bound app-server startup RPCs
Co-authored-by: multica-agent <github@multica.ai>
* test(codex): de-flake bounded-handshake test
The single 500ms handshake bound was shared by the successful preamble
RPCs, so a slow fork/exec of the /bin/sh fake app-server could make
initialize spuriously time out under parallel load. Raise the test bound
to 3s (still below the 5s semantic timeout and 10s harness ceiling) and
loosen the elapsed assertion to match.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: Eve <eve@multica-ai.local>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: J <j@multica.ai>
Discover the Codex model list and per-model reasoning efforts from the installed CLI (codex debug models --bundled), with a verified static fallback for old/offline installs. Server gates token syntax; the daemon validates the exact (model, effort) pair.
Closes#5197
MUL-4354
Follow-up to #5188 addressing the second-round review.
- ValidateThinkingLevel now fails an empty codex model closed instead of
borrowing the flagged Default (gpt-5.6-sol). An empty model follows
config.toml, which can resolve to any installed model; Sol alone advertises
`ultra`, so the old borrow green-lit levels Luna / gpt-5.5 don't support and
Codex doesn't reject. Checked before ListModels so a discovery error can't
fail it open. Frontend pickModelEntry mirrors this (no per-model effort
preview for an empty codex model); the persisted-orphan clear path stays.
- parseCodexDebugModels drops efforts without a known label so the picker
never advertises a level the Create/Update enum gate would 400 on save; the
contract test now drives the real parser with an unknown effort instead of
comparing two hand-written maps.
- gpt-5.6 price aliases anchor to a literal dot (not the [.-] class), so
dashed variants like gpt-5-6-luna surface as unmapped on both backend and
frontend rather than silently borrowing a tier.
Co-authored-by: J <j@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
* feat(models): add Codex gpt-5.6 series (sol/terra/luna) to model list & pricing (MUL-4347)
Co-authored-by: multica-agent <github@multica.ai>
* fix(models): official gpt-5.6 pricing, exact aliases, max/ultra effort levels (MUL-4347)
- Replace provisional gpt-5.6 rates with OpenAI's official announcement
values (sol 5/30, terra 2.5/15, luna 1/6); cache read 0.1x input, cache
write 1.25x input (frontend + backend, kept in sync).
- Anchor gpt-5.6 price aliases to exact match so unknown suffixed variants
surface as unmapped instead of borrowing a tier.
- Add Codex 0.144.1 max/ultra effort levels to the label map and server
enum so the daemon-advertised catalog matches what the API can persist;
add a catalog->API contract test.
- Clarify that the codex Default flag is the effort-validation anchor, not a
user-facing badge.
- Note the cache-write measurement limitation (codex usage stream doesn't
report cache-write tokens yet).
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: J <j@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Detect the root/sudo + bypassPermissions launch condition before starting Claude Code and fail fast with an actionable error (run as non-root, or set IS_SANDBOX=1 in a genuine container/sandbox).
Closes#3278
MUL-4095
* fix(agent): pi agent final output excludes intermediate steps
Updated PI agent to only retain the final result in JSON output.
Previously, `text_delta` included both intermediate steps and final
content.
Now, output is reset on each `text_start` to concatenate only the
final text.
* fix(agent) Replace `message_update.text_start` with `turn_start` event. add test
`turn_start` begins a new turn, Reset output on it to exclude
intermediate texts.
a1b336d73e/packages/coding-agent/docs/rpc.md (events)
Trae (traecli) already has a New() backend, launch header (traecli acp
serve) and provider branding, but was missing from every protocol_family
whitelist, so custom runtime profiles based on Trae were rejected and it
never appeared in the family picker.
Add traecli to SupportedTypes (Go), RUNTIME_PROFILE_PROTOCOL_FAMILIES
(TS), the lockstep test's want map, and a new migration 136 widening the
runtime_profile_protocol_family_check constraint.
MUL-4094, #4945
Co-authored-by: J <j@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Qoder CN (`qoderclicn`) users could only reach a working custom runtime by
misrouting through the Kiro backend, which launches `<cmd> acp
--trust-all-tools`. That is incompatible with Qoder's global `--acp` / `--yolo`
argv, so the task failed immediately with `kiro initialize failed` and no run
messages.
Expose `qoder` in the custom-profile protocol_family whitelist across every
lockstep layer:
- server/pkg/agent SupportedTypes (+ whitelist pin test)
- migration 134 runtime_profile protocol_family CHECK (NOT VALID, mirroring 126)
- packages/core RUNTIME_PROFILE_PROTOCOL_FAMILIES
The existing qoderBackend already honors an ExecutablePath override and launches
`<cmd> --yolo --acp`, so a profile with protocol_family=qoder and
command_name=qoderclicn now launches with the correct argv instead of the Kiro
shape. Provider branding/logo for qoder already exists.
MUL-4018
Co-authored-by: J <j@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
