Commit Graph

14 Commits

Author SHA1 Message Date
YOMXXX
34d4cd3a28 feat(openclaw): support connecting to existing OpenClaw gateway (#3260) [MUL-3158] (#3664)
* feat(openclaw): support connecting to existing OpenClaw gateway (#3260)

When the daemon host is a lightweight dev machine or CI coordinator, the
heavy agent work (LLM inference, code execution, tool use) often belongs
on a more powerful remote server already running an OpenClaw gateway.
Multica historically hard-coded `openclaw agent --local`, forcing every
turn to execute in-process on the daemon host.

This change adds an opt-in gateway routing mode controlled per-agent via
`runtime_config`:

  {
    "mode": "gateway",
    "gateway": { "host": "...", "port": 18789, "token": "...", "tls": false }
  }

- Backend: ExecOptions gains OpenclawMode + OpenclawGateway; buildOpenclawArgs
  drops `--local` when mode == "gateway". Per-task openclaw-config.json
  wrapper pins gateway.{host,port,auth.{mode,token},tls} so users do not
  need to edit the daemon host's `~/.openclaw/openclaw.json` to point at
  a different endpoint.
- Daemon: AgentData carries the raw runtime_config; decoding is fail-soft
  (malformed JSON falls back to local mode rather than blocking dispatch).
- API: gateway.token is masked to "***" on every GET; PATCH replays the
  sentinel back, and the update handler restores the persisted token so
  the round-trip never destroys the secret. Defense-in-depth masking on
  WS broadcasts, plus String/MarshalJSON masking on the in-memory struct
  to block stray `%+v` / json.Marshal leaks.
- UI: openclaw-only "Routing" tab on the agent detail page with mode
  selector + structured endpoint form. Token uses a "saved — submit a
  new value to rotate" UX and matching backend preserve hook.

Empty `runtime_config` keeps the historical embedded behaviour, so
existing agents are unaffected.

* fix(openclaw): address #3664 review — drop dead gateway field, gate pin on mode

Per Bohan-J's review:

- Remove the dead ExecOptions.OpenclawGateway field (+ its String/MarshalJSON and
  the daemon.go construction block). It carried the plaintext bearer token but was
  never read — buildOpenclawArgs only consumes OpenclawMode and the live gateway
  path runs through execenv.OpenclawGatewayPin — so this narrows the secret's
  footprint.
- Gate the gateway pin on mode=="gateway" in decodeOpenclawRuntimeConfig: a
  {"mode":"local","gateway":{...,"token"}} payload no longer writes the token into
  the 0o600 per-task wrapper that --local makes openclaw ignore.
- Warn on an unrecognized non-empty mode (e.g. "gatway") instead of silently
  falling back to local.
- Run preserveMaskedGatewayToken in CreateAgent too, so a literal "***" at create
  time can't persist as a real bearer token.
- Document the gateway host:port trust boundary (SSRF note for shared daemon hosts).

Adds regression tests for the local-mode pin drop and the unknown-mode warning.
2026-06-13 15:33:28 +08:00
Joey Frasier (Boothe)
76cd8275ff fix(openclaw): parse whole buffer instead of line-by-line scanner (MUL-1908) (#2292)
* fix(openclaw): parse whole buffer instead of line-by-line scanner

Follow-up to c87d7676 (WOR-10). The stdout/stderr swap fixed the dominant
case but `processOutput` still scanned line-by-line and only attempted a
whole-buffer parse from a fragile fallback path. Pretty-printed JSON
(openclaw 2026.5.x emits the result blob indented across many lines) made
every individual line unparseable on its own — `{`, `  "payloads": [`,
`    {`, etc. — so the success path hinged entirely on the fallback
joining `rawLines` and re-trying.

Under load (daemon restarts racing the close-on-cancel goroutine, partial
chunked reads when stdout closes mid-flight) the line scanner could see
truncated input that never reassembled into valid JSON, surfacing
"openclaw returned no parseable output" against runs where the agent had
in fact completed the work and posted comments. Roughly 30–40% of recent
runs in v0.2.27 logs hit this path; multica still wrote a `task_failed`
inbox row for each one even though the underlying issue had moved to
`in_review` or `done`.

The fix:

- processOutput now reads the full stdout buffer with `io.ReadAll` first.
- A new `parseWholeBufferOpenclawResult` helper attempts a single
  `json.Unmarshal` against the entire buffer (after trimming, and after
  optionally stripping leading non-JSON log lines). When it matches, we
  build the result and return — the line scanner never runs.
- If the whole-buffer parse fails, we fall through to the existing NDJSON
  line-by-line scanner. This preserves streaming-event support (kept for
  forward compatibility and other backends) without leaving openclaw's
  dominant pretty-printed shape at the mercy of timing.
- The failure path now emits a `(got N bytes; preview: ...)` suffix on
  the canonical "no parseable output" error so future debugging isn't
  blind. The exact canonical phrase is preserved for empty buffers so
  existing dashboards / log-grep tooling keep matching.

Tests:

- TestOpenclawProcessOutputWholeBufferPrettyJSON: feeds a hand-crafted
  multi-line indented blob (multiple payloads, nested agentMeta, usage
  map) and asserts every field round-trips through the whole-buffer fast
  path.
- TestOpenclawProcessOutputDeeplyIndentedFixture: re-runs the recorded
  openclaw 2026.5.5 stdout fixture (1070 lines) directly through
  parseWholeBufferOpenclawResult, asserting the bug-shape parses cleanly
  on the first attempt without falling through to NDJSON scanning.
- TestOpenclawProcessOutputEmptyBufferErrorIncludesByteCount: tightens
  the empty-buffer failure path, asserts the canonical phrase survives so
  observability tooling keeps working.

All existing tests in the openclaw + buildOpenclawArgs suites stay green
(streaming NDJSON event tests, lifecycle tests, structured-error tests,
usage-field-variant tests). The two pre-existing flaky timeout-tight
codex tests (TestCodexExecuteSemanticInactivityAllowsContinuous*) fail on
both this branch and on c87d7676 baseline; they are unrelated and out of
scope here.

Co-authored-by: multica-agent <github@multica.ai>

* fix(openclaw): drop dead preview branch, document streaming regression

Rebase + review-fix follow-up on top of f27df2d9b.

processOutput's preview branch was unreachable: openclawNoParseableOutputError
was only called from the `!gotEvents && trimmed == ""` path, which by
construction means the entire scanned buffer collapsed to whitespace, so the
`(got N bytes; preview: ...)` formatter could never fire on a non-empty buffer.
Replace the helper with a single canonical-string constant (callsite is now
inline) and update the test name to match what it actually asserts (the
canonical empty-buffer error string is preserved for external log-grep /
dashboard consumers).

Also document on processOutput that the line-scanner path is no longer
truly streaming after the io.ReadAll switch: events accumulate until
stdout closes. OpenClaw 2026.5.x does not emit streaming events so this
regression is invisible today, but flag it for the next backend that
might.

Misc: switch the scanner's input source from
`strings.NewReader(string(buf))` to `bytes.NewReader(buf)` to drop one
unnecessary byte/string round-trip.

MUL-1908

Co-authored-by: multica-agent <github@multica.ai>

---------

Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: J (Multica agent) <j@multica.local>
2026-05-19 17:42:41 +08:00
Bohan Jiang
0af67c8159 fix(agent/openclaw): block tasks if openclaw < 2026.5.5 with upgrade hint (#2181)
PR #2101 swapped the openclaw runtime adapter from reading --json on
stderr to stdout. That fixed openclaw 2026.5+ but inverted the breakage
for pre-2026.5 builds — those still write JSON to stderr, so the
adapter now sees an empty stdout and falls through to the same
"openclaw returned no parseable output" failure that 2026.5+ users
saw before #2101.

Add a per-task version gate inside openclawBackend.Execute that runs
`openclaw --version`, parses the dotted version, and rejects anything
below 2026.5.5 with a hardcoded upgrade hint:

    openclaw <detected> is below the minimum supported version 2026.5.5.
    Run `openclaw update` to upgrade and try again.

The check is intentionally per-task and uncached so users who upgrade
do not need to restart the daemon — the next task automatically
re-checks. ~20ms per task is negligible vs. the typical run.

Co-authored-by: multica-agent <github@multica.ai>
2026-05-07 02:11:47 +08:00
Joey Frasier (Boothe)
af971e1e5c fix(agent/openclaw): read --json from stdout, not stderr (#2101)
Multica's openclaw runtime adapter has been reading agent output from
stderr since the early openclaw integration days. Current openclaw
(2026.5.5, c37871e) writes its --json blob exclusively to stdout:

    $ openclaw agent --local --json --agent main --message 'say hi' >stdout 2>stderr
    STDOUT bytes: 27401
    STDERR bytes:     0

Result: every successful turn was followed by a daemon-generated system
comment 'openclaw returned no parseable output', visible to users,
looked like the agent broke when it didn't. Reproduced live on WOR-2,
turn at 2026-05-05 16:35 UTC; daemon log confirmed the full result JSON
arrived on the [openclaw:stdout] debug channel and was discarded while
the empty stderr pipe hit the no-events fallback.

Changes
- server/pkg/agent/openclaw.go: swap pipes, StdoutPipe() for the JSON
  stream, cmd.Stderr = newLogWriter(...) for log overflow. Cleanup
  goroutine now closes stdout on cancel. Comments and the read-error
  errMsg updated to reflect the new pipe.
- server/pkg/agent/openclaw_test.go: TestOpenclawProcessOutputReadError
  asserts on 'read stdout' (was 'read stderr'), string-only fix,
  no behavior change. New TestOpenclawProcessOutputStdoutFixture feeds
  a recorded openclaw 2026.5.5 --json blob through processOutput and
  asserts result + messages parse cleanly.
- server/pkg/agent/testdata/openclaw-2026.5.5-stdout.json: 27401-byte
  fixture captured fresh from the openclaw CLI for the regression test.

Side effects (net positive)
- Log lines openclaw writes to stderr (security warnings, tool errors)
  now show up under [openclaw:stderr] instead of being silently consumed
  by the JSON parser.
- Daemon's success_pattern heuristic (empty-output -> 'blocked')
  becomes meaningful again because result.Output actually populates.

Closes WOR-10.
2026-05-07 01:50:16 +08:00
Bohan Jiang
c5a00d8b8c fix(agent/openclaw): extract real model from meta.agentMeta.model (#1426)
OpenClaw's `--json` result blob carries the actual LLM identifier in
`meta.agentMeta.model` (e.g. `deepseek-chat`, `claude-sonnet-4`),
alongside `provider` and the usage breakdown. The backend was reading
the surrounding `agentMeta.usage` and `agentMeta.sessionId` but skipping
the `model` field entirely, then attributing every run's tokens to
`opts.Model` — which for openclaw is the *agent name* passed via
`--agent`, not a real model identifier — falling all the way through to
"unknown" when no agent.model was configured.

Surface the runtime-reported model:

- `openclawEventResult` gains a `model` string.
- `buildOpenclawEventResult` reads `agentMeta.model` (trimmed; empty
  string when absent for forward-compat with older runtimes / partial
  outputs).
- `processOutput` propagates it through the result-blob branch.
- `Execute`'s usage map prefers `scanResult.model`, falling back to
  `opts.Model` then `"unknown"` — preserving the prior behavior path
  for any runtime that doesn't surface its own model yet.

Two unit tests cover both the populated and missing cases.

Refs: #1395
2026-04-21 14:32:31 +08:00
Jiayuan Zhang
b291db11c2 feat(agents): add per-agent model field with provider-aware dropdown (#1399)
Adds a first-class `model` field on agents so users can pick the LLM model from the create / settings UI instead of editing `custom_env` / `custom_args`. Each provider's dropdown is populated from the live CLI when possible (`opencode models`, `pi --list-models`, `openclaw agents list --json`, `cursor-agent --list-models`, hermes ACP `session/new` → `SessionModelState`), with a static catalog for providers that don't enumerate.

Daemon resolves the runtime model as `agent.model → MULTICA_<PROVIDER>_MODEL → ""` — empty passes through so each backend's CLI picks its own default, avoiding static-guess drift.

Per-provider honouring:
- Claude / Codex / OpenCode / Cursor / Gemini / Pi / Copilot — CLI `--model` / thread payload.
- OpenClaw — `opts.Model` is mapped to `--agent <name>` (the CLI rejects `--model`).
- Hermes — `session/set_model` ACP RPC; stderr is sniffed for provider-level errors so HTTP 4xx from the configured LLM surfaces instead of "empty output"; explicit-model failures mark the task `failed`.

Supporting changes: migration 050 adds `agent.model`; daemon ↔ server heartbeat piggyback carries a model-discovery request; new REST endpoints under `/api/runtimes/{id}/models`; `multica agent create --model` / `update --model`; shared `ModelDropdown` in `packages/views/agents` (searchable, creatable, provider-grouped, default-badge, runtime-supported gate).
2026-04-21 00:06:34 +08:00
Bohan Jiang
bd445782d5 fix(openclaw): stop passing unsupported flags and actually deliver AgentInstructions (#1362)
Fixes #1332.

Two regressions introduced in #910 (2026-04-14, "OpenClaw backend P0+P1
improvements") that together block all openclaw users:

1. `openclaw agent` does not accept `--model` or `--system-prompt`, so
   any agent configured with a Model field crashed in ~700ms with
   `exit status 1`. Remove both forwards, and add them to
   openclawBlockedArgs so custom_args can't reintroduce the crash.
   Model is bound at registration time via `openclaw agents
   add/update --model`.

2. AgentInstructions were written to `{workDir}/AGENTS.md` by
   execenv.InjectRuntimeConfig, but openclaw loads bootstrap files
   from its own workspace dir — the file was never read, so every
   agent's Instructions field was silently discarded. Populate
   opts.SystemPrompt for the openclaw provider in runTask and
   prepend it to the `--message` payload in the backend so the
   model actually receives the instructions.

Other providers surface instructions through their native runtime
config file (CLAUDE.md / AGENTS.md / GEMINI.md) and are intentionally
left unchanged to avoid double injection.

Extract buildOpenclawArgs so arg construction is directly testable;
add unit tests covering the removed flags, the SystemPrompt prepend,
and custom_args filtering.
2026-04-20 14:01:41 +08:00
Jiang Bohan
bb4944bae2 fix(openclaw): handle pretty-printed multi-line JSON output
OpenClaw outputs its --json result as pretty-printed multi-line JSON to
stderr. The line-by-line scanner never found a valid JSON object on any
single line, causing the raw JSON to be returned as the chat response.

After exhausting line-by-line parsing, try parsing the accumulated
output as a whole before falling back to raw text.

Closes MUL-725
2026-04-14 14:39:32 +08:00
Jiang Bohan
a0d43ca31a feat(agent): OpenClaw backend P0+P1 improvements
Combined P0 and P1 improvements to the OpenClaw agent backend, informed
by PaperClip's adapter architecture:

P0 — User experience:
- Streaming output — emit MessageText as NDJSON events arrive in real
  time, instead of waiting for the final result blob
- Tool use support — parse and emit MessageToolUse/MessageToolResult
  from streaming events, matching Claude and OpenCode backends
- Model & system prompt — pass --model and --system-prompt to the
  OpenClaw CLI when configured

P1 — Robustness:
- Hardened JSON parsing — tryParseOpenclawResult requires lines to
  start with '{', eliminating fragile brace-scanning that could
  false-match JSON fragments in log lines
- Lifecycle event handling — new "lifecycle" event type with phase
  tracking (error/failed/cancelled), plus structured error objects
  (error.name, error.data.message) matching PaperClip's pattern
- Usage field name variants — parseOpenclawUsage supports multiple
  naming conventions (input/inputTokens/input_tokens, cacheRead/
  cachedInputTokens/cache_read_input_tokens, etc.) with incremental
  accumulation across step_finish events

Backwards compatible with the legacy single JSON blob format.
31 tests covering all new functionality.

Closes MUL-726
2026-04-14 01:52:03 +08:00
Roshan Warrier
e044c7e84b fix(agent): parse openclaw result incrementally (#836)
Co-authored-by: txhno <198242577+txhno@users.noreply.github.com>
2026-04-13 15:29:34 +08:00
Bohan Jiang
d11824807a fix(agent): handle braces in stderr log lines before openclaw JSON result (#718)
processOutput() used strings.Index(raw, "{") to find the JSON start,
but error lines like `raw_params={"command":"..."}` contain braces that
get matched first, causing JSON parsing to fail and the entire raw
stderr (including internal metadata) to be returned as the agent comment.

Now tries each '{' position until one successfully unmarshals as a valid
openclawResult, skipping braces embedded in log/error lines.
2026-04-11 23:07:58 +08:00
Bohan Jiang
e477d64548 fix(cli): poll health endpoint instead of fixed sleep in daemon start (#716)
* fix(cli): poll health endpoint instead of fixed sleep in daemon start

The daemon start command waited a fixed 2 seconds then checked the
health endpoint once. If the daemon took longer to initialize (auth,
workspace loading), the check failed and printed a misleading error
even though the daemon started successfully.

Replace the single check with a polling loop (500ms interval, 15s
timeout) so the CLI waits for the daemon to actually be ready.

* fix(agent): rewrite openclaw tests to match new backend API

The openclaw backend was rewritten in #715 to parse a single JSON blob
instead of streaming NDJSON events. The tests still referenced the old
types (openclawEvent) and methods (handleOCTextEvent, etc.), causing a
build failure in CI.

Rewrite all tests to exercise the new processOutput method and
openclawInt64 helper.
2026-04-11 22:25:19 +08:00
Jiang Bohan
dd2ce90b1d fix(agent): address openclaw review feedback
- Remove duplicate extractOCToolOutput, reuse extractToolOutput from opencode.go
- Rename extractEventText → openclawExtractText to avoid package-level name collisions
- Add clarifying comments for error status stickiness and result event behavior
- Remove redundant extractOCToolOutput tests (already covered by opencode tests)
2026-04-07 14:52:54 +08:00
Jiang Bohan
5cf4ba803d feat(agent): add OpenClaw runtime support
Add OpenClaw as a fourth supported agent runtime alongside Claude Code,
Codex, and OpenCode. OpenClaw CLI (`openclaw agent -p ... --output-format
stream-json`) is integrated via the same Backend interface pattern.

Changes:
- Add openclawBackend in server/pkg/agent/openclaw.go with NDJSON
  event stream parsing (text, thinking, tool_call, error, step, result)
- Register "openclaw" in the agent factory (agent.go)
- Add MULTICA_OPENCLAW_PATH / MULTICA_OPENCLAW_MODEL env var detection
  in daemon config
- Include "openclaw" in AGENTS.md config injection alongside codex/opencode
- Add comprehensive unit tests for all event handlers and processEvents
2026-04-07 14:40:51 +08:00