Active long-running sessions are no longer killed by a fixed wall-clock deadline. Liveness is delegated to the idle watchdog (MULTICA_AGENT_IDLE_WATCHDOG, default 30m) with a larger in-flight-tool budget (MULTICA_AGENT_TOOL_WATCHDOG, default 2h). MULTICA_AGENT_TIMEOUT is an opt-in absolute cap (default 0 = no cap). The server-side 2.5h sweeper is unchanged as a coarse backstop.
Fixes#3745.
* fix(openclaw): parse whole buffer instead of line-by-line scanner
Follow-up to c87d7676 (WOR-10). The stdout/stderr swap fixed the dominant
case but `processOutput` still scanned line-by-line and only attempted a
whole-buffer parse from a fragile fallback path. Pretty-printed JSON
(openclaw 2026.5.x emits the result blob indented across many lines) made
every individual line unparseable on its own — `{`, ` "payloads": [`,
` {`, etc. — so the success path hinged entirely on the fallback
joining `rawLines` and re-trying.
Under load (daemon restarts racing the close-on-cancel goroutine, partial
chunked reads when stdout closes mid-flight) the line scanner could see
truncated input that never reassembled into valid JSON, surfacing
"openclaw returned no parseable output" against runs where the agent had
in fact completed the work and posted comments. Roughly 30–40% of recent
runs in v0.2.27 logs hit this path; multica still wrote a `task_failed`
inbox row for each one even though the underlying issue had moved to
`in_review` or `done`.
The fix:
- processOutput now reads the full stdout buffer with `io.ReadAll` first.
- A new `parseWholeBufferOpenclawResult` helper attempts a single
`json.Unmarshal` against the entire buffer (after trimming, and after
optionally stripping leading non-JSON log lines). When it matches, we
build the result and return — the line scanner never runs.
- If the whole-buffer parse fails, we fall through to the existing NDJSON
line-by-line scanner. This preserves streaming-event support (kept for
forward compatibility and other backends) without leaving openclaw's
dominant pretty-printed shape at the mercy of timing.
- The failure path now emits a `(got N bytes; preview: ...)` suffix on
the canonical "no parseable output" error so future debugging isn't
blind. The exact canonical phrase is preserved for empty buffers so
existing dashboards / log-grep tooling keep matching.
Tests:
- TestOpenclawProcessOutputWholeBufferPrettyJSON: feeds a hand-crafted
multi-line indented blob (multiple payloads, nested agentMeta, usage
map) and asserts every field round-trips through the whole-buffer fast
path.
- TestOpenclawProcessOutputDeeplyIndentedFixture: re-runs the recorded
openclaw 2026.5.5 stdout fixture (1070 lines) directly through
parseWholeBufferOpenclawResult, asserting the bug-shape parses cleanly
on the first attempt without falling through to NDJSON scanning.
- TestOpenclawProcessOutputEmptyBufferErrorIncludesByteCount: tightens
the empty-buffer failure path, asserts the canonical phrase survives so
observability tooling keeps working.
All existing tests in the openclaw + buildOpenclawArgs suites stay green
(streaming NDJSON event tests, lifecycle tests, structured-error tests,
usage-field-variant tests). The two pre-existing flaky timeout-tight
codex tests (TestCodexExecuteSemanticInactivityAllowsContinuous*) fail on
both this branch and on c87d7676 baseline; they are unrelated and out of
scope here.
Co-authored-by: multica-agent <github@multica.ai>
* fix(openclaw): drop dead preview branch, document streaming regression
Rebase + review-fix follow-up on top of f27df2d9b.
processOutput's preview branch was unreachable: openclawNoParseableOutputError
was only called from the `!gotEvents && trimmed == ""` path, which by
construction means the entire scanned buffer collapsed to whitespace, so the
`(got N bytes; preview: ...)` formatter could never fire on a non-empty buffer.
Replace the helper with a single canonical-string constant (callsite is now
inline) and update the test name to match what it actually asserts (the
canonical empty-buffer error string is preserved for external log-grep /
dashboard consumers).
Also document on processOutput that the line-scanner path is no longer
truly streaming after the io.ReadAll switch: events accumulate until
stdout closes. OpenClaw 2026.5.x does not emit streaming events so this
regression is invisible today, but flag it for the next backend that
might.
Misc: switch the scanner's input source from
`strings.NewReader(string(buf))` to `bytes.NewReader(buf)` to drop one
unnecessary byte/string round-trip.
MUL-1908
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: J (Multica agent) <j@multica.local>
Follow-up to #2716. Updates two stale comments that still described
openclaw's `name` and `id` as interchangeable. The actual contract:
`id` is the routing key passed to `openclaw agent --agent <id>`;
`name` is a human display label and is not safe to pass to the CLI.
No behavior change.
Co-authored-by: multica-agent <github@multica.ai>
PR #2101 swapped the openclaw runtime adapter from reading --json on
stderr to stdout. That fixed openclaw 2026.5+ but inverted the breakage
for pre-2026.5 builds — those still write JSON to stderr, so the
adapter now sees an empty stdout and falls through to the same
"openclaw returned no parseable output" failure that 2026.5+ users
saw before #2101.
Add a per-task version gate inside openclawBackend.Execute that runs
`openclaw --version`, parses the dotted version, and rejects anything
below 2026.5.5 with a hardcoded upgrade hint:
openclaw <detected> is below the minimum supported version 2026.5.5.
Run `openclaw update` to upgrade and try again.
The check is intentionally per-task and uncached so users who upgrade
do not need to restart the daemon — the next task automatically
re-checks. ~20ms per task is negligible vs. the typical run.
Co-authored-by: multica-agent <github@multica.ai>
Multica's openclaw runtime adapter has been reading agent output from
stderr since the early openclaw integration days. Current openclaw
(2026.5.5, c37871e) writes its --json blob exclusively to stdout:
$ openclaw agent --local --json --agent main --message 'say hi' >stdout 2>stderr
STDOUT bytes: 27401
STDERR bytes: 0
Result: every successful turn was followed by a daemon-generated system
comment 'openclaw returned no parseable output', visible to users,
looked like the agent broke when it didn't. Reproduced live on WOR-2,
turn at 2026-05-05 16:35 UTC; daemon log confirmed the full result JSON
arrived on the [openclaw:stdout] debug channel and was discarded while
the empty stderr pipe hit the no-events fallback.
Changes
- server/pkg/agent/openclaw.go: swap pipes, StdoutPipe() for the JSON
stream, cmd.Stderr = newLogWriter(...) for log overflow. Cleanup
goroutine now closes stdout on cancel. Comments and the read-error
errMsg updated to reflect the new pipe.
- server/pkg/agent/openclaw_test.go: TestOpenclawProcessOutputReadError
asserts on 'read stdout' (was 'read stderr'), string-only fix,
no behavior change. New TestOpenclawProcessOutputStdoutFixture feeds
a recorded openclaw 2026.5.5 --json blob through processOutput and
asserts result + messages parse cleanly.
- server/pkg/agent/testdata/openclaw-2026.5.5-stdout.json: 27401-byte
fixture captured fresh from the openclaw CLI for the regression test.
Side effects (net positive)
- Log lines openclaw writes to stderr (security warnings, tool errors)
now show up under [openclaw:stderr] instead of being silently consumed
by the JSON parser.
- Daemon's success_pattern heuristic (empty-output -> 'blocked')
becomes meaningful again because result.Output actually populates.
Closes WOR-10.
* fix(daemon): suppress agent terminal windows on Windows (#1471)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: add hideAgentWindow to detectCLIVersion and avoid SysProcAttr overwrite
- Add missing hideAgentWindow(cmd) call in detectCLIVersion (claude.go:554)
so --version checks don't flash console windows on Windows.
- Refactor hideAgentWindow to preserve existing SysProcAttr fields
instead of overwriting the entire struct.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
OpenClaw's `--json` result blob carries the actual LLM identifier in
`meta.agentMeta.model` (e.g. `deepseek-chat`, `claude-sonnet-4`),
alongside `provider` and the usage breakdown. The backend was reading
the surrounding `agentMeta.usage` and `agentMeta.sessionId` but skipping
the `model` field entirely, then attributing every run's tokens to
`opts.Model` — which for openclaw is the *agent name* passed via
`--agent`, not a real model identifier — falling all the way through to
"unknown" when no agent.model was configured.
Surface the runtime-reported model:
- `openclawEventResult` gains a `model` string.
- `buildOpenclawEventResult` reads `agentMeta.model` (trimmed; empty
string when absent for forward-compat with older runtimes / partial
outputs).
- `processOutput` propagates it through the result-blob branch.
- `Execute`'s usage map prefers `scanResult.model`, falling back to
`opts.Model` then `"unknown"` — preserving the prior behavior path
for any runtime that doesn't surface its own model yet.
Two unit tests cover both the populated and missing cases.
Refs: #1395
Surface the actual exec path + argv for every agent backend at INFO
so operators can see the exact command without flipping to debug.
Also add the missing log line in pi.go for consistency with the
other nine backends.
Adds a first-class `model` field on agents so users can pick the LLM model from the create / settings UI instead of editing `custom_env` / `custom_args`. Each provider's dropdown is populated from the live CLI when possible (`opencode models`, `pi --list-models`, `openclaw agents list --json`, `cursor-agent --list-models`, hermes ACP `session/new` → `SessionModelState`), with a static catalog for providers that don't enumerate.
Daemon resolves the runtime model as `agent.model → MULTICA_<PROVIDER>_MODEL → ""` — empty passes through so each backend's CLI picks its own default, avoiding static-guess drift.
Per-provider honouring:
- Claude / Codex / OpenCode / Cursor / Gemini / Pi / Copilot — CLI `--model` / thread payload.
- OpenClaw — `opts.Model` is mapped to `--agent <name>` (the CLI rejects `--model`).
- Hermes — `session/set_model` ACP RPC; stderr is sniffed for provider-level errors so HTTP 4xx from the configured LLM surfaces instead of "empty output"; explicit-model failures mark the task `failed`.
Supporting changes: migration 050 adds `agent.model`; daemon ↔ server heartbeat piggyback carries a model-discovery request; new REST endpoints under `/api/runtimes/{id}/models`; `multica agent create --model` / `update --model`; shared `ModelDropdown` in `packages/views/agents` (searchable, creatable, provider-grouped, default-badge, runtime-supported gate).
Fixes#1332.
Two regressions introduced in #910 (2026-04-14, "OpenClaw backend P0+P1
improvements") that together block all openclaw users:
1. `openclaw agent` does not accept `--model` or `--system-prompt`, so
any agent configured with a Model field crashed in ~700ms with
`exit status 1`. Remove both forwards, and add them to
openclawBlockedArgs so custom_args can't reintroduce the crash.
Model is bound at registration time via `openclaw agents
add/update --model`.
2. AgentInstructions were written to `{workDir}/AGENTS.md` by
execenv.InjectRuntimeConfig, but openclaw loads bootstrap files
from its own workspace dir — the file was never read, so every
agent's Instructions field was silently discarded. Populate
opts.SystemPrompt for the openclaw provider in runTask and
prepend it to the `--message` payload in the backend so the
model actually receives the instructions.
Other providers surface instructions through their native runtime
config file (CLAUDE.md / AGENTS.md / GEMINI.md) and are intentionally
left unchanged to avoid double injection.
Extract buildOpenclawArgs so arg construction is directly testable;
add unit tests covering the removed flags, the SystemPrompt prepend,
and custom_args filtering.
Add a debug-level log line in every agent backend (claude, codex,
opencode, openclaw, gemini, hermes) that prints the executable path
and full argument list when spawning the agent process. Helps diagnose
custom args, model overrides, and other CLI flag issues.
* feat(agent): add custom CLI arguments support
Allow users to configure custom CLI arguments per agent that get
appended to the agent subprocess command at launch time. This enables
use cases like specifying different models (--model o3), max turns,
or other provider-specific flags without needing separate runtimes.
Changes:
- Add custom_args JSONB column to agent table (migration 041)
- Update API handler to accept/return custom_args in create/update
- Pass custom_args through claim endpoint to daemon
- Append custom_args to CLI commands for all agent backends
- Add ExecOptions.CustomArgs field in agent package
- Add Custom Args tab in agent detail UI
- Add --custom-args flag to CLI agent create/update commands
Closes MUL-802
* fix(agent): filter protocol-critical flags from custom_args
Add per-backend filtering of custom_args to prevent users from
accidentally overriding flags that the daemon hardcodes for its
communication protocol (e.g. --output-format, --input-format,
--permission-mode for Claude).
This follows the same pattern as custom_env's isBlockedEnvKey: we
only block the small, stable set of flags that would break the
daemon↔agent protocol — not every possible dangerous flag. Workspace
members are trusted for everything else.
Each backend defines its own blocked set:
- Claude: -p, --output-format, --input-format, --permission-mode
- Gemini: -p, --yolo, -o
- Codex: --listen
- OpenCode: --format
- OpenClaw: --local, --json, --session-id, --message
- Hermes: none (ACP is positional)
Includes unit tests for the filtering logic.
* fix(agent): address code review nits for custom_args
- Replace module-level `nextArgId` counter with `crypto.randomUUID()`
in custom-args-tab.tsx to avoid SSR ID conflicts
- Add unit tests for custom args passthrough and blocked-arg filtering
in both Claude and Gemini arg builders
OpenClaw outputs its --json result as pretty-printed multi-line JSON to
stderr. The line-by-line scanner never found a valid JSON object on any
single line, causing the raw JSON to be returned as the chat response.
After exhausting line-by-line parsing, try parsing the accumulated
output as a whole before falling back to raw text.
Closes MUL-725
When an agent CLI process hangs (e.g. a tool call blocks on unreachable
I/O), the daemon's scanner blocks indefinitely on stdout, preventing the
Result from ever being sent. This causes tasks to stay in "running"
state permanently with no further events.
Three-layer fix:
1. Agent backends (claude, opencode, openclaw, gemini): add a watchdog
goroutine that closes the stdout/stderr pipe when the context is
cancelled, forcing the scanner to unblock. Also set cmd.WaitDelay
so Go force-closes pipes after 10s if the process doesn't exit.
2. daemon executeAndDrain: add an independent drain timeout (backend
timeout + 30s buffer) with context-aware select on both the message
channel and the result channel, so the daemon never blocks forever.
3. daemon ping path: add context-aware select so pings don't deadlock
if the agent backend stalls.
Closes#925
Co-authored-by: Devv <devv@Devvs-Mac-mini.local>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Combined P0 and P1 improvements to the OpenClaw agent backend, informed
by PaperClip's adapter architecture:
P0 — User experience:
- Streaming output — emit MessageText as NDJSON events arrive in real
time, instead of waiting for the final result blob
- Tool use support — parse and emit MessageToolUse/MessageToolResult
from streaming events, matching Claude and OpenCode backends
- Model & system prompt — pass --model and --system-prompt to the
OpenClaw CLI when configured
P1 — Robustness:
- Hardened JSON parsing — tryParseOpenclawResult requires lines to
start with '{', eliminating fragile brace-scanning that could
false-match JSON fragments in log lines
- Lifecycle event handling — new "lifecycle" event type with phase
tracking (error/failed/cancelled), plus structured error objects
(error.name, error.data.message) matching PaperClip's pattern
- Usage field name variants — parseOpenclawUsage supports multiple
naming conventions (input/inputTokens/input_tokens, cacheRead/
cachedInputTokens/cache_read_input_tokens, etc.) with incremental
accumulation across step_finish events
Backwards compatible with the legacy single JSON blob format.
31 tests covering all new functionality.
Closes MUL-726
Some OpenClaw JSON outputs contain durationMs but lack payloads field.
The original condition rejected these results, causing the agent to
return "openclaw returned no parseable output" instead of the actual
execution result.
Fix by accepting results that have either payloads OR durationMs > 0.
Fixes#830
Co-authored-by: leaderlemon <leaderlemon@users.noreply.github.com>
processOutput() used strings.Index(raw, "{") to find the JSON start,
but error lines like `raw_params={"command":"..."}` contain braces that
get matched first, causing JSON parsing to fail and the entire raw
stderr (including internal metadata) to be returned as the agent comment.
Now tries each '{' position until one successfully unmarshals as a valid
openclawResult, skipping braces embedded in log/error lines.
* fix(agent): use --message flag for OpenClaw CLI invocation
OpenClaw CLI changed its prompt flag from `-p` to `--message`. The old
flag caused tasks to fail immediately with "required option '-m,
--message <text>' not specified".
Fixes#713, relates to #703.
* fix(agent): rewrite openclaw backend to match actual CLI interface
- Replace unsupported flags (-p, --output-format, --yes) with correct
ones (--message, --json, --local, --session-id)
- Read JSON result from stderr (where openclaw writes it)
- Parse openclaw's actual output format ({payloads, meta})
- Auto-generate session ID for each task execution
- Show "live log not available" hint in agent live card when timeline
is empty (openclaw doesn't support streaming)
1. Separate ReportTaskUsage endpoint (POST /api/daemon/tasks/{id}/usage)
so usage is captured independently of complete/fail — fixes usage loss
for failed/blocked tasks.
2. Add usage tracking for all four providers:
- Claude: already done (stream-json message.usage)
- OpenCode: extract from step_finish.part.tokens
- OpenClaw: extract from step_end.data token fields
- Codex: extract from turn/completed and task_complete usage fields
3. Remove usage from CompleteTask payload — all usage goes through the
dedicated endpoint now.
- Remove duplicate extractOCToolOutput, reuse extractToolOutput from opencode.go
- Rename extractEventText → openclawExtractText to avoid package-level name collisions
- Add clarifying comments for error status stickiness and result event behavior
- Remove redundant extractOCToolOutput tests (already covered by opencode tests)
Add OpenClaw as a fourth supported agent runtime alongside Claude Code,
Codex, and OpenCode. OpenClaw CLI (`openclaw agent -p ... --output-format
stream-json`) is integrated via the same Backend interface pattern.
Changes:
- Add openclawBackend in server/pkg/agent/openclaw.go with NDJSON
event stream parsing (text, thinking, tool_call, error, step, result)
- Register "openclaw" in the agent factory (agent.go)
- Add MULTICA_OPENCLAW_PATH / MULTICA_OPENCLAW_MODEL env var detection
in daemon config
- Include "openclaw" in AGENTS.md config injection alongside codex/opencode
- Add comprehensive unit tests for all event handlers and processEvents
