highperfocused/multica
1
0
Fork 0
mirror of https://github.com/multica-ai/multica.git synced 2026-07-23 18:17:44 +02:00
Code Issues Packages Projects Releases Wiki Activity
3,639 Commits 1,800 Branches 130 Tags
agent/lambda/7a73e08e
Commit Graph

5 Commits

Author SHA1 Message Date
YOMXXX
34d4cd3a28 feat(openclaw): support connecting to existing OpenClaw gateway (#3260) [MUL-3158] (#3664) 
* feat(openclaw): support connecting to existing OpenClaw gateway (#3260)

When the daemon host is a lightweight dev machine or CI coordinator, the
heavy agent work (LLM inference, code execution, tool use) often belongs
on a more powerful remote server already running an OpenClaw gateway.
Multica historically hard-coded `openclaw agent --local`, forcing every
turn to execute in-process on the daemon host.

This change adds an opt-in gateway routing mode controlled per-agent via
`runtime_config`:

  {
    "mode": "gateway",
    "gateway": { "host": "...", "port": 18789, "token": "...", "tls": false }
  }

- Backend: ExecOptions gains OpenclawMode + OpenclawGateway; buildOpenclawArgs
  drops `--local` when mode == "gateway". Per-task openclaw-config.json
  wrapper pins gateway.{host,port,auth.{mode,token},tls} so users do not
  need to edit the daemon host's `~/.openclaw/openclaw.json` to point at
  a different endpoint.
- Daemon: AgentData carries the raw runtime_config; decoding is fail-soft
  (malformed JSON falls back to local mode rather than blocking dispatch).
- API: gateway.token is masked to "***" on every GET; PATCH replays the
  sentinel back, and the update handler restores the persisted token so
  the round-trip never destroys the secret. Defense-in-depth masking on
  WS broadcasts, plus String/MarshalJSON masking on the in-memory struct
  to block stray `%+v` / json.Marshal leaks.
- UI: openclaw-only "Routing" tab on the agent detail page with mode
  selector + structured endpoint form. Token uses a "saved — submit a
  new value to rotate" UX and matching backend preserve hook.

Empty `runtime_config` keeps the historical embedded behaviour, so
existing agents are unaffected.

* fix(openclaw): address #3664 review — drop dead gateway field, gate pin on mode

Per Bohan-J's review:

- Remove the dead ExecOptions.OpenclawGateway field (+ its String/MarshalJSON and
  the daemon.go construction block). It carried the plaintext bearer token but was
  never read — buildOpenclawArgs only consumes OpenclawMode and the live gateway
  path runs through execenv.OpenclawGatewayPin — so this narrows the secret's
  footprint.
- Gate the gateway pin on mode=="gateway" in decodeOpenclawRuntimeConfig: a
  {"mode":"local","gateway":{...,"token"}} payload no longer writes the token into
  the 0o600 per-task wrapper that --local makes openclaw ignore.
- Warn on an unrecognized non-empty mode (e.g. "gatway") instead of silently
  falling back to local.
- Run preserveMaskedGatewayToken in CreateAgent too, so a literal "***" at create
  time can't persist as a real bearer token.
- Document the gateway host:port trust boundary (SSRF note for shared daemon hosts).

Adds regression tests for the local-mode pin drop and the unknown-mode warning.
 2026-06-13 15:33:28 +08:00
liujianqiang-niu
5be7d1bc17 MUL-3136 fix(openclaw): parse config path from last non-empty line of CLI output 
Fix OpenClaw config discovery when `openclaw config file` prints Doctor warning UI before the actual config path. The daemon now uses the last non-empty stdout line as the path while preserving the existing tilde expansion, absolute-path validation, stat checks, and fail-closed behavior.

Tests: go test ./internal/daemon/execenv
 2026-06-08 17:22:02 +08:00
Bohan Jiang
fa076d38f2 MUL-2778 feat(agent): wire mcp_config through OpenClaw runtime (#3450) 
* MUL-2778 feat(agent): wire mcp_config through OpenClaw runtime

The MCP config tab (#3419) lets admins save mcp_config on an agent, and
recent work (#3439) plumbed it through the three ACP runtimes. OpenClaw
still ignored the field, leaving the Tab silently inert for any
OpenClaw-backed agent.

Translate the agent's Claude-style `{"mcpServers": {...}}` into the
per-task OpenClaw wrapper's `mcp.servers` block — OpenClaw resolves MCP
via its own config schema rather than ExecOptions, so the existing
OPENCLAW_CONFIG_PATH preparer is the right seam. Fail closed on
malformed JSON / entries missing `command` or `url`, matching the
fail-closed posture the preparer already uses for the agents.list step.
Null / absent mcp_config leaves the wrapper free of an `mcp` key so the
user's global mcp.servers flows through untouched; an explicit empty
managed set (`{}` / `{"mcpServers":{}}`) is honoured as "admin saved no
servers" mirroring `hasManagedCodexMcpConfig`.

Strict-mode replacement (drop user-only servers entirely) would require
OpenClaw to do a per-key replace rather than a deep merge at
`mcp.servers`; the comment documents that caveat rather than relying on
undocumented behaviour.

Also adds `openclaw` to `MCP_SUPPORTED_PROVIDERS` so the MCP Tab
actually surfaces in the agent overview pane, and pins the new
visibility case with a renderPane test.

Co-authored-by: multica-agent <github@multica.ai>

* MUL-2778 fix(agent): make openclaw mcp_config strict-replace via sanitized snapshot

Elon flagged on #3450 that the previous wiring let user-only mcp.servers
leak through the wrapper's `$include` of the live user config: deep-merge
at `mcp.servers` keeps user-only names, and the strict-empty case
(`{ "mcpServers": {} }`) silently inherited user globals.

Switch the strict-replace path to write a sanitized snapshot of the
user's fully resolved config (via `openclaw config get --json`) with the
`mcp` block stripped, then have the wrapper `$include` the snapshot
instead of the live user file. With the user's `mcp` gone from the
$include resolution, the wrapper's `mcp.servers` is the only definition
the embedded OpenClaw sees — managed only, including the explicit empty
set.

The snapshot lives in envRoot at 0o600 alongside the wrapper so the GC
reaper sweeps it with the rest of the task scratch, and no extra
OPENCLAW_INCLUDE_ROOTS entry is needed (same-dir $include).

Fail-closed on `config get --json` errors so the daemon never silently
falls back to the leaky $include path. The inherit branch (null
mcp_config) still uses the live user file directly — no extra CLI
roundtrip and no snapshot is written.

New tests pin the contract Elon's review required:

- TestPrepareOpenclawConfigStrictReplacesUserMcpServers: user has
  global_one + shared, managed has shared + managed_only → wrapper has
  exactly {shared (managed value), managed_only}; global_one does NOT
  leak; snapshot file has the user's `mcp` stripped while preserving
  gateway / providers / API keys.
- TestPrepareOpenclawConfigStrictEmptyManagedSetDropsUserMcp: empty
  managed set drops user's global_one (both `{}` and
  `{"mcpServers":{}}` cases).
- TestPrepareOpenclawConfigNullMcpConfigKeepsUserInclude: null path
  inherits the live user config, writes no snapshot, makes no extra CLI
  call.
- TestPrepareOpenclawConfigFailsClosedOnResolvedConfigError: errors
  during `config get --json` surface; no stale wrapper or snapshot.
- TestPrepareOpenclawConfigManagedSetFreshInstall: fresh install with
  managed mcp_config skips the snapshot dance entirely.

Also tightens en + zh-Hans MCP Tab copy to mention OpenClaw goes via the
per-task wrapper, and to use OpenClaw's own `transport` field rather
than Claude's `type` for HTTP/SSE entries.

Co-authored-by: multica-agent <github@multica.ai>

* MUL-2778 fix(agent): narrow openclaw snapshot strip to mcp.servers only

Elon's third-round must-fix: the previous strict-replace snapshot deleted
the entire `mcp` block, which wiped out non-server settings under `mcp`
like `sessionIdleTtlMs`. Those are documented OpenClaw config keys
(https://docs.openclaw.ai/gateway/configuration-reference#mcp) outside
the MCP Tab's scope — the agent's saved mcp_config only manages server
definitions, so other `mcp.*` tuning the user set must survive.

Replace the blanket `delete(resolved, "mcp")` with a stripUserMcpServers
helper that:

- deletes only `mcp.servers` when `mcp` is an object
- drops the parent `mcp` key only when the object is empty after the
  strip (so we don't emit `mcp: {}` placeholders)
- leaves non-object `mcp` values untouched (we only know how to strip
  servers from the documented shape)

Pinned with TestPrepareOpenclawConfigStrictPreservesNonServerMcpKeys:
user resolved has both `mcp.sessionIdleTtlMs: 300000` and
`mcp.servers.global_one`; after the strict path runs the snapshot
keeps the TTL and drops the servers map, and the wrapper's
`mcp.servers` is exactly the managed set with no leak.

Co-authored-by: multica-agent <github@multica.ai>

---------

Co-authored-by: J <j@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
 2026-05-28 18:43:02 +08:00
Bohan Jiang
ee4ec3b76d MUL-2784 fix(daemon): cleanup sidecar tree (.agent_context / .multica / provider skills) after local_directory tasks (#3444) 
* fix(daemon): cleanup .agent_context / .multica / provider skill sidecars after local_directory tasks (MUL-2784)

PR #3438 (MUL-2753) only restored CLAUDE.md / AGENTS.md / GEMINI.md to
their pre-task bytes; the sidecar tree writeContextFiles seeds
(.agent_context/, .multica/, .claude/skills/, .github/skills/,
.opencode/skills/, skills/, .pi/skills/, .cursor/skills/,
.kimi/skills/, .kiro/skills/, .agents/skills/, fallback
.agent_context/skills/) was explicitly deferred to this follow-up. In
local_directory mode the agent's workdir is the user's repo, so each
task accumulates one more layer of those directories in the user's
tree.

Plan A: track every file/dir Prepare creates inside workDir in a
sidecarManifest written to envRoot/.multica_sidecar_manifest.json
(daemon scratch — never in the user's workdir). On local_directory
teardown CleanupSidecars walks the manifest, removes the recorded
files, then rmdir-iterates the recorded directories in reverse.
Pre-existing files and directories are deliberately NOT recorded, so
a user-installed .claude/skills/my-own-skill/ sibling — or any
unrelated file the user keeps under .claude/, .github/, etc. — is
preserved bit-for-bit. Non-empty rmdir fails ENOTEMPTY and is
silently skipped, which is the signal that the user owns the
directory.

Daemon wiring lives next to the existing CleanupRuntimeConfig defer
in runTask: runtime brief first, sidecars second. Cloud-mode runs
still write a manifest for symmetry but never trigger the cleanup
(the GC loop wipes envRoot wholesale).

Tests (sidecar_manifest_test.go) cover the round-trip invariant per
the issue's acceptance criteria:

- empty workdir → Prepare → Cleanup → empty workdir, byte-exact, for
  every file-based provider (claude, codex, copilot, opencode,
  openclaw, hermes, pi, cursor, kimi, kiro, antigravity, gemini),
- user's .claude/skills/my-own-skill/ (and equivalents per
  provider) survives Cleanup intact,
- unrelated user files under .claude/, .github/, etc. survive,
- three repeated cycles do not accumulate any orphan state,
- project_resources branch (.multica/project/resources.json) is
  also reversible,
- recordWriteFile refuses to record pre-existing files,
- recordMkdirAll refuses to record pre-existing dirs,
- Cleanup is a no-op when the manifest file is missing.

Co-authored-by: multica-agent <github@multica.ai>

* fix(daemon): refuse to overwrite pre-existing sidecar paths; pick collision-free skill slugs (MUL-2784 review)

Addresses PR #3444 review (Elon):

**Must-fix #1**: recordWriteFile used to overwrite pre-existing target
files unconditionally and only skip the manifest record. That destroys
user bytes at write time AND leaves the corrupted contents in place at
cleanup time — the byte-exact contract the issue requires is violated
on both halves. Fixed by making recordWriteFile detect any pre-existing
entry (regular file, symlink, directory) via Lstat and return a
sentinel errPathPreExists without touching the path. The user's bytes
are preserved verbatim.

For per-skill collisions (user's .claude/skills/issue-review/ vs
Multica's "Issue Review"), writeSkillFiles now allocates a
collision-free sibling slug via allocateCollisionFreeSkillDir: first
attempt is the natural slug, then `<base>-multica`,
`<base>-multica-2`, …, bounded at 64 attempts. Provider-native
discovery still picks the skill up (every subdir under skillsParent is
a distinct skill) and the user's path stays bit-for-bit intact.

For Multica-only namespace files (.agent_context/issue_context.md,
.multica/project/resources.json), the writer swallows errPathPreExists
and continues — the runtime brief already carries every fact those
files would, so a collision degrades to brief-only mode rather than
destroying user content.

**Must-fix #2**: Added byte-exact collision matrix tests covering
every file-based provider (claude / codex / copilot / opencode /
openclaw / hermes / pi / cursor / kimi / kiro / antigravity / gemini):

- TestPrepareThenCleanupSidecarsSameSlugCollisionPerProvider: seeds
  user's `<provider>/skills/issue-review/SKILL.md` plus a private
  notes.md sibling, runs Prepare → Inject → Cleanup, asserts
  workdir snapshot is byte-identical to seed.
- TestPrepareThenCleanupSidecarsIssueContextCollisionPerProvider:
  seeds user's `.agent_context/issue_context.md`, asserts round-trip
  preserves it.
- TestPrepareThenCleanupSidecarsProjectResourcesCollisionPerProvider:
  same for `.multica/project/resources.json`.
- TestPrepareThenCleanupSidecarsMultiSkillCollisionFreeAllocation:
  end-to-end check that the Multica skill lands at the
  collision-free sibling and Cleanup removes only the Multica side.
- TestAllocateCollisionFreeSkillDir: directed unit test pinning the
  slug-bumping sequence.
- TestRecordWriteFileRefusesToOverwritePreExistingFile (was
  TestRecordWriteFileSkipsPreExistingFile): flipped to assert the
  user's bytes survive and errPathPreExists is returned.
- TestRecordWriteFileRefusesToOverwriteSymlinkOrDir: covers the
  Lstat path for non-file entries.

**Should-fix**: CleanupSidecars used to swallow ANY non-ENOENT rmdir
error as "user content present," silently dropping real I/O failures
(EACCES, EPERM, EBUSY). Now it re-reads the directory after a failed
rmdir via the new dirHasEntries helper — non-empty → silently skip
(ENOTEMPTY, the intended branch); empty → genuine error, captured
into firstErr and surfaced. Plus directed tests:

- TestCleanupSidecarsSurfacesRealRmdirErrors
- TestDirHasEntries

Local verification:
- go test ./internal/daemon/execenv/... — all green
- go test ./internal/daemon/... — all green
- go vet ./... — clean

Co-authored-by: multica-agent <github@multica.ai>

* fix(daemon): surface original rmdir error when post-rmdir ReadDir also fails (MUL-2784 review)

Addresses remaining PR #3444 review blocker (Elon): dirHasEntries used
to return true when ReadDir failed with anything other than ENOENT,
which made CleanupSidecars treat every locked / faulted directory as
ENOTEMPTY and silently drop the original rmdir error. The v1 fix from
the previous round closed the EACCES-on-empty-dir branch but missed
the case where the chmod also blocks ReadDir — exactly the failure
mode the review called out.

Helper change: dirHasEntries now returns (hasEntries, ok bool):

  - (false, true)  — dir exists and is empty (or missing, race-safe)
  - (true,  true)  — dir has user content (the ENOTEMPTY branch)
  - (_,     false) — ReadDir failed (EACCES, ENOTDIR, EIO, …); the
                     caller cannot tell ENOTEMPTY from a real error
                     and MUST surface the original rmdir error

CleanupSidecars switches on (ok, hasEntries):

  - !ok               → surface the ORIGINAL rmdir error (not the
                        ReadDir failure — that's diagnostic plumbing
                        and would distract from the root cause)
  - ok && hasEntries  → swallow silently (intended ENOTEMPTY branch;
                        preserve user content)
  - ok && !hasEntries → surface the rmdir error (empty dir + EACCES /
                        EPERM / EBUSY → genuine cleanup failure)

Tests:

  - TestDirHasEntries: extended with a regular-file sub-case (ReadDir
    returns ENOTDIR) asserting (false, false). The v1 helper returned
    (true) here, hiding the bug.
  - TestCleanupSidecarsSwallowsMissingAndNonEmptyDirs: renamed from
    TestCleanupSidecarsSurfacesRealRmdirErrors. The old name claimed
    to test the surfacing path but never actually exercised it.
  - TestCleanupSidecarsSurfacesEACCESOnEmptyRecordedDir: chmod parent
    to 0o555 so rmdir(recorded) fails EACCES while ReadDir(recorded)
    still succeeds (empty). Asserts firstErr is non-nil and references
    both the recorded path and the rmdir branch. Skipped when running
    as root (chmod is bypassed for uid 0).
  - TestCleanupSidecarsSurfacesEACCESWhenReadDirFailsToo: the must-fix
    case — chmod parent 0o555 AND chmod recorded 0o000 so BOTH rmdir
    and ReadDir fail. The surfaced error must be the ORIGINAL rmdir
    failure, not the ReadDir one. Skipped on uid 0.

Local verification:
- go test ./internal/daemon/execenv/... — all green
- go test ./internal/daemon/... — all green
- go vet ./... — clean

Co-authored-by: multica-agent <github@multica.ai>

---------

Co-authored-by: J <j@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
 2026-05-28 17:22:47 +08:00
Bohan Jiang
464201ba0d feat(execenv): native OpenClaw skill discovery via per-task config (MUL-2219) (#2628) 
* feat(execenv): native OpenClaw skill discovery via per-task config

MUL-2213 stopped lying about native discovery and routed openclaw skills
to .agent_context/skills/ — a path openclaw's scanner never reads.
Multica skills attached to openclaw-backed agents were still invisible to
the runtime; the AGENTS.md fallback was only a documentation patch.

OpenClaw's skill scanner walks <workspaceDir>/skills/ (plus a few other
roots), and workspaceDir is resolved from the openclaw config file —
specifically agents.list[id].workspace → agents.defaults.workspace →
~/.openclaw/workspace. There is no CLI flag or env var override on the
agent runtime; the only knob is the config file.

This change wires a per-task synthesized config:

  1. execenv.prepareOpenclawConfig deep-copies the user's existing
     openclaw.json (priority: $OPENCLAW_CONFIG_PATH, else
     ~/.openclaw/openclaw.json), rewrites agents.defaults.workspace AND
     every agents.list[].workspace to the task workdir, and writes the
     result to {envRoot}/openclaw-config.json. Provider sections,
     registered agents, model providers, gateway settings — everything
     openclaw needs to actually start — are preserved as-is.
  2. resolveSkillsDir for "openclaw" now points at {workDir}/skills/,
     which is the first path openclaw scans under workspaceDir. Skills
     written here are picked up natively.
  3. daemon.go exports OPENCLAW_CONFIG_PATH={env.OpenclawConfigPath} on
     the openclaw subprocess and adds OPENCLAW_CONFIG_PATH to the
     custom_env blocklist so users cannot accidentally override it.
  4. buildMetaSkillContent now lists openclaw alongside the
     "discovered automatically" providers; the .agent_context/skills/
     fallback line stays for gemini/hermes.

The new regression test TestPrepareOpenclawSkillWriteMatchesScanPath is
the one MUL-2219's DoD calls out: it resolves the workspaceDir the way
openclaw does (reading agents.defaults.workspace out of the synthesized
config) and proves {workspaceDir}/skills/<name>/SKILL.md is what Multica
actually wrote. The pre-MUL-2219 fix asserted "we wrote a file" without
checking the scanner would ever see it — which is how the dead drop into
.openclaw/skills/ landed in #2621's first commit.

Verified locally: minimum-viable synthesized config validates via
`openclaw config validate`, and `OPENCLAW_CONFIG_PATH=<path> openclaw
config get agents.defaults.workspace` returns the task workdir as
expected. MUL-2219

Co-authored-by: multica-agent <github@multica.ai>

* fix(execenv): delegate openclaw config parsing to CLI and fail closed

Address Elon's must-fix on PR #2628: the previous implementation parsed
~/.openclaw/openclaw.json with encoding/json, which cannot read JSON5
or follow $include — the OpenClaw spec's actual format. When parsing
failed, prepareOpenclawConfig silently emitted a minimal config, which
could boot OpenClaw without the user's registered agents, model
providers, or API keys.

Two changes:

1. Delegate active-config-path resolution and config reading to the
   openclaw CLI itself. `openclaw config file` locates the active
   config (covering OPENCLAW_CONFIG_PATH / OPENCLAW_STATE_DIR /
   OPENCLAW_HOME / default and the legacy chain), and the wrapper we
   write uses $include to point at it so OpenClaw's own loader handles
   JSON5, $include nesting, env-substitution, and secret refs. We read
   only agents.list via `openclaw config get --json` to rewrite each
   entry's workspace — secrets, comments, and includes in the user
   config are never touched.

2. Remove the silent minimal-config fallback. Any CLI failure,
   malformed output, or write error now surfaces as a hard error from
   Prepare / Reuse. The only "synthesize minimal" path left is a fresh
   install (CLI reports a path but the file doesn't exist), where
   there is no user data to lose.

The per-task override still rewrites every agents.list[].workspace,
not just agents.defaults.workspace — this is intentional task
isolation, documented in prepareOpenclawConfig and the PR body. A
host-scope per-agent workspace would otherwise silently route the
scanner back to the user's shared workspace.

Cleanups Elon flagged in the same review:
- daemon.go inline-system-prompt comment no longer claims openclaw
  ignores the task workdir; it does load it now, and the inline brief
  is a belt-and-suspenders carryover for older releases.
- execenv.go openclaw block no longer references "skill file paths in
  the inline brief" — the brief uses "discovered automatically".

Reuse() switches to a ReuseParams struct so the openclaw binary path
threads through alongside CodexVersion without a 6th positional arg.

MUL-2219

Co-authored-by: multica-agent <github@multica.ai>

* fix(execenv): grant OpenClaw $include cross-dir confinement for per-task wrapper

The per-task wrapper at envRoot/openclaw-config.json $includes the user's
active config (typically ~/.openclaw/openclaw.json), but OpenClaw confines
$include resolution to the wrapper file's directory unless the target's
parent is granted via OPENCLAW_INCLUDE_ROOTS. Without this, OpenClaw refuses
to follow the link at runtime and the wrapper boots with no user-registered
agents.

prepareOpenclawConfig now returns dirname(activePath) as IncludeRoot, and
the daemon prepends it to whatever the user already has in
OPENCLAW_INCLUDE_ROOTS via the new composeOpenclawIncludeRoots helper
(dedupes, drops empty segments, preserves user-configured roots). Fresh
install emits no $include and leaves the env var untouched.

Adds OPENCLAW_INCLUDE_ROOTS to the custom_env blocklist so a per-agent
override cannot strip the granted root.

Regression tests:
- TestPrepareOpenclawConfigWrapperLoadableUnderIncludeConfinement asserts
  every $include target's dirname is covered by the IncludeRoot we surface.
- TestPrepareEnvironmentOpenclawWiresIncludeRoot covers the non-fresh-install
  Environment wiring.
- TestComposeOpenclawIncludeRoots covers the daemon-side env composition
  (preserve, dedupe, drop empties).

Co-authored-by: multica-agent <github@multica.ai>

---------

Co-authored-by: multica-agent <github@multica.ai>
 2026-05-14 22:35:31 +08:00