mirror of
https://github.com/multica-ai/multica.git
synced 2026-07-05 21:39:54 +02:00
main
69 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
LinYushen
|
cb68669c73 |
feat(composio): gate MCP apps behind feature flag (#4876)
* feat(composio): server-side connect flow + connections REST (Notion MVP) (MUL-3720) (#4608)
* feat(composio): server-side connect flow + connections REST (Notion MVP) (MUL-3720)
Compose the merged server/pkg/composio SDK into a user-facing connection
manager: signed-state connect handshake, local user_composio_connection
mirror, idempotent disconnect, and a per-user MCP session helper (not yet
wired into task dispatch).
- migration 127_user_composio_connection (no FK/cascade, per DB rules)
- sqlc queries: upsert (idempotent on user_id+connected_account_id), list
active, owner-scoped get, mark revoked
- internal/integrations/composio: signed HMAC-SHA256 state, BeginConnect,
CompleteCallback (idempotent upsert), ListConnections, Disconnect
(upstream 404 = idempotent success), CreateMCPSession (no-op when empty,
pins connected_accounts per toolkit), CallbackRedirect
- REST handlers under /api/integrations/composio (user-scoped, 503 when
COMPOSIO_API_KEY unset): connect/init, callback (302), connections list,
delete
- router wiring gated by COMPOSIO_API_KEY; COMPOSIO_AUTH_CONFIGS_JSON maps
toolkit->auth_config (MVP: notion); state secret from COMPOSIO_STATE_SECRET
or derived from JWT_SECRET; callback base from COMPOSIO_CALLBACK_BASE_URL
or MULTICA_PUBLIC_URL
- tests: state (expire/tamper/wrong-secret), service (mapping, callback
idempotency, non-success, disconnect owner/404 idempotency, MCP pin),
handlers (httptest), redact regression for Bearer mcp_ tokens
MVP scope: Notion only; no task-dispatch overlay, sharing, or webhook
event handling (later stages).
Co-authored-by: multica-agent <github@multica.ai>
* fix(composio): bind callback account to user + idempotent revoked disconnect (MUL-3720)
Address PR 4608 review (CHANGES_REQUESTED):
- callback: verify connected_account_id with Composio before mirroring it.
The signed state only proved user/toolkit/exp, so a valid state paired with
a tampered connected_account_id would be written verbatim. CompleteCallback
now calls ListConnectedAccounts and fails closed (ErrAccountVerification)
unless the account belongs to the state's user (composio_user_id == multica
user id) and was created under the toolkit's auth config. No row is written
on mismatch / unknown account / upstream error.
- disconnect: short-circuit to a no-op when the local row is already revoked,
before touching upstream. Previously a second DELETE re-hit Composio and a
non-404 upstream error surfaced as a 502, breaking the 204-idempotent
contract.
- CreateMCPSession: document the v1 single-active-connection-per-(user,toolkit)
constraint and make duplicate selection deterministic (newest-wins, rows are
connected_at DESC) instead of order-dependent map overwrite. Stage 3 owns the
real single-account-enforcement vs multi-account-shape decision.
Tests: tampered/wrong-auth-config/unknown-account callback rejection, revoked-row
disconnect no-op (asserts upstream not re-hit). composio pkg 85% coverage; all
green.
Co-authored-by: multica-agent <github@multica.ai>
* feat(composio): list all toolkits + dynamic auth-config resolution (MUL-3720)
Yushen's follow-up to the Notion MVP: surface the full Composio toolkit
catalog, render it in Settings, and drop the static env mapping in favor of
dynamic auth-config discovery.
Config correctness (per Composio docs):
- Remove COMPOSIO_AUTH_CONFIGS_JSON entirely. The toolkit→auth_config mapping
is now resolved at request time from the project's /auth_configs (cached,
5-min TTL), so enabling a toolkit is a dashboard action, not a redeploy.
- Do NOT add COMPOSIO_PROJECT_ID. The project API key (x-api-key) authenticates
to exactly one project; the project is resolved from the key. Only org-level
endpoints use x-org-api-key, which this integration never calls.
Backend:
- SDK: server/pkg/composio/auth_configs.go — ListAuthConfigs (toolkit_slug,
is_composio_managed, show_disabled, limit, cursor).
- service: dynamic resolver (authConfigMap cache; betterAuthConfig prefers a
custom/white-label config over Composio-managed, newest wins); BeginConnect
and CompleteCallback resolve via it; ListToolkits fetches the full catalog
(paginated, capped) annotated with connectable = has an enabled auth config,
connectable-first ordering.
- handler + route: GET /api/integrations/composio/toolkits (user-scoped, 503
when COMPOSIO_API_KEY unset) returning slug/name/logo/category/connectable.
Frontend:
- core: ComposioToolkit/ComposioConnection types, api client methods, and
composio query options (@multica/core/composio).
- views: Settings → Integrations now has a Composio section rendering every
toolkit as a card with search. Connect is gated on `connectable`;
non-connectable toolkits show a muted "not configured" hint instead of a
dead button. Connected toolkits show a badge + Disconnect (with confirm).
- i18n: composio block added to en/zh-Hans/ja/ko settings.
Tests: SDK + service (dynamic resolution, custom-over-managed preference,
connectable flag, resolver-error soft-degrade) and handler toolkits endpoint;
composio pkg 85.7% coverage. go build/vet/gofmt clean; core+views typecheck,
core+views lint, and core tests (691) all green.
Co-authored-by: multica-agent <github@multica.ai>
* fix(composio): close cross-toolkit callback fail-open by signing auth_config_id into state (MUL-3720)
Re-review blocker: CompleteCallback resolved the toolkit's auth config at
callback time and ignored a resolve error/empty result, while
verifyAccountOwnership skipped the auth-config comparison when the expected
value was empty. A user could then pass another toolkit's connected_account_id
into this toolkit's callback — the owner check passed and it was written under
the wrong toolkit_slug/account binding.
Fix: the auth_config_id is already resolved in BeginConnect (before the state
is signed), so sign it into the state and compare it exactly at callback. No
re-resolve, no fail-open. verifyAccountOwnership now fails closed when the
expected auth config is empty (rejects instead of skipping) and requires an
exact match — closing the cross-toolkit binding gap.
Tests: state round-trips auth_config_id; BeginConnect signs it; callback
rejects wrong/cross-toolkit auth config and an empty (no-mapping) auth config
fails closed. composio pkg 85.2% coverage, all green.
Frontend (non-blocking): the Composio settings tab now surfaces an error when
the connections query fails instead of silently rendering everything as
unconnected.
Co-authored-by: multica-agent <github@multica.ai>
* fix(composio): hide Settings section entirely when integration unconfigured (MUL-3720)
Decision (option 2, hide-then-merge): don't show a card that leaks the internal
COMPOSIO_API_KEY env-var name to every end user. IntegrationsTab now gates the
whole Composio section (heading + body) on the toolkits query — a 503 means the
key is unset, so the section is withheld instead of rendering the not-configured
card. Admin-only setup guidance is a later, role-gated affordance.
Removed the notConfigured card (and now-unused ApiError import) from
ComposioTab; it only mounts when configured. views typecheck + lint clean.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: multica-agent <github@multica.ai>
* feat(composio): Stage 2 frontend polish — callback toast, last_used & expired UI, e2e (MUL-3718) (#4688)
* feat(composio): callback toast + refresh, last_used & expired UI, e2e (MUL-3718)
Co-authored-by: multica-agent <github@multica.ai>
* fix(composio): real callback redirect route + StrictMode-safe toast dedup (MUL-3718 review)
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: multica-agent <github@multica.ai>
* fix(composio): callback endpoint should not require Multica auth (MUL-3843) (#4709)
* fix(composio): move OAuth callback out of the Auth group (MUL-3843)
Composio 302-redirects the browser to /api/integrations/composio/callback
at the end of the OAuth flow, but PR #4608 mounted it inside the cookie-auth
middleware group. When the session cookie is absent (expired session,
SameSite=Strict / Safari ITP, private window, self-hosted callback subdomain)
the Auth middleware returned a hard 401 and a JSON blob instead of the
settings redirect, breaking the flow.
Identity never came from the cookie anyway: it is carried by the HMAC-signed
state param that CompleteCallback verifies (signature, expiry, replay) and
cross-checked by verifyAccountOwnership; h.Composio == nil still 503s. So the
callback is registered alongside the other public OAuth/webhook routes; the
other four composio endpoints stay session-gated.
Refs MUL-3843, MUL-3715.
Co-authored-by: multica-agent <github@multica.ai>
* fix(composio): correct stale callback routing comments (MUL-3843)
The package header and ComposioCallback doc comments still described the
callback as sitting under the Auth middleware group. After the route was
moved out (this PR), update both to state it is a public route whose identity
comes from the signed state — addressing review nit from 张大彪.
Refs MUL-3843.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: multica-agent <github@multica.ai>
* feat(composio): inject MCP overlay into agent runtime at task dispatch (MUL-3721) (#4704)
Stage 3 of the Composio epic. Wires the per-user Composio MCP session into
every agent task so the agent process sees the initiator's connected tools
without any prompt-time plumbing.
Server side
- Migration 128 adds agent_task_queue.runtime_mcp_overlay JSONB plus a
BEFORE-UPDATE trigger that wipes the column on any transition into a
terminal status (completed / failed / cancelled). A trigger is the single
source of truth — future queries that flip status cannot bypass it.
- composio.Service.BuildTaskOverlay(userID) reuses CreateMCPSession and
emits the Claude-style { mcpServers: { composio: { type: http, url,
headers } } } shape the daemon's existing sidecar generators consume.
Returns (nil, nil) on zero active connections so we never burn a
Composio session for a user with nothing to call.
- TaskService grows a Composio ComposioOverlayBuilder seam, wired in
router.go after composiointeg.NewService succeeds. Five enqueue paths
(issue / mention / quick-create / chat / auto-retry) attach the overlay
after CreateAgentTask returns and before the daemon is notified — so
every claim reads a settled row, with no second daemon hop. Best-effort:
a builder failure logs and proceeds with no overlay.
- resolveInitiatorFromTriggerComment derives the initiator user from the
trigger comment when it was authored by a member. Agent-authored
triggers are not treated as initiators (their connected-apps view is
empty by construction).
Daemon side
- handler/daemon.go claim path merges task.runtime_mcp_overlay onto
agent.mcp_config via mergeMCPOverlay before populating
TaskAgentData.McpConfig. Overlay wins on server-name collisions
because it carries the live user-scoped session URL. Errors fall back
to the agent config unchanged — a bad overlay must not surprise-disable
saved MCP tools. The existing execenv sidecar generators (cursor /
codex / openclaw / opencode / hermes-kiro) need no changes: they keep
consuming the merged result through TaskAgentData.McpConfig.
Tests
- 9 merge cases (mcp_overlay_test): both-nil short-circuit, agent-only
pass-through, overlay-only canonicalization, two-side merge, name
collision (overlay wins), top-level key preservation, malformed agent
fallback, malformed overlay fallback, non-object server rejection.
- 4 dispatch cases (composio): zero-connections returns nil without
CreateSession, happy-path emits the right shape with the right user
id, empty-URL defensive branch, SDK error surfacing.
- 4 TaskService helper cases: nil Composio is a no-op (Queries-safe),
invalid initiator does not call the builder, nil overlay skips the
UPDATE, builder error swallowed without panic.
- Migration 128 verified to roll up + down + up cleanly against the test
database.
Out of scope (deferred): assignment-triggered enqueue paths with no
trigger comment get no overlay attached today (no initiator UUID flows
through enqueueIssueTask in that case). Retry paths recompute the overlay
fresh from the parent's initiator_user_id instead of inheriting the bearer
from the parent row, so a stale token can never resurface on a retry.
Co-authored-by: Eve <eve@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
* feat(composio): per-agent allowlist + originator-scoped MCP overlay (MUL-3869) (#4736)
* feat(composio): per-agent allowlist + originator-scoped MCP overlay (MUL-3869)
Stage 3.1 of the Composio epic (MUL-3721 parent). PR #4704 wired in the
runtime_mcp_overlay column and a per-task dispatch hook; this change
inverts the default from "all-on" to opt-in and locks the overlay to the
agent owner's own connected apps:
- Agents carry composio_toolkit_allowlist TEXT[]. NULL or [] => no MCP.
Owner-only read/write; non-owner GET/PUT silently redacts/drops the
field (same shape as mcp_config).
- agent_task_queue carries originator_user_id UUID. Set from the
top-of-chain HUMAN at every enqueue path:
* issue/mention comment by member -> author_id
* issue/mention comment by agent -> inherit via comment.source_task_id
-> parent task originator_user_id
* quick-create -> requester_id
* chat -> initiator_user_id
* retry -> SQL-inherited from parent row
* autopilot -> NULL (system-driven)
- BuildTaskOverlay (composio dispatch) now takes (ctx, originatorUserID,
agent) and short-circuits on five gates: invalid originator,
originator != agent.owner_id, empty allowlist, empty intersection of
allowlist ∩ active connections, defensive empty session URL. Composio
CreateSession is called with BOTH `toolkits.slugs` (the intersection)
AND `connected_accounts` (the pinned account ids), narrowing the
tool-router twice.
- The originator-vs-owner gate closes the agent-fanout privacy hole: any
workspace member who can @-mention a public agent used to project the
owner's connected apps into their run. Now the overlay only mounts
when the human at the top of the chain IS the agent owner.
Tests:
- dispatch_test.go covers all 5 gates plus uppercase/whitespace slug
normalisation.
- task_runtime_mcp_overlay_test.go covers the no-op gates of the new
applyRuntimeMCPOverlay signature.
- agent_composio_allowlist_test.go (handler): owner roundtrip
(list/empty/null), workspace-admin silent-drop, owner-only GET
visibility, pure normaliseComposioToolkitAllowlist.
- resolve_originator_test.go (service, DB-backed): member-authored,
agent-authored inherits via comment.source_task_id, invalid id.
Migration 129 up/down/up verified against docker postgres.
Co-authored-by: multica-agent <github@multica.ai>
* chore(composio): gofmt + regenerate sqlc with v1.31.1 (MUL-3869 review nits)
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: Eve <eve@multica-ai.local>
Co-authored-by: multica-agent <github@multica.ai>
* fix(composio): accept nested connected account auth config
* feat(views): creator-only MCP tab for per-agent Composio allowlist (MUL-3870) (#4743)
Stage 3.2 frontend on top of the Stage 3.1 backend (MUL-3869,
|
||
|
Naiyuan Qing
|
7b3cad664b |
revert(self-host): remove source channel reporting (#4799)
* Revert "test(onboarding): cover official source reporting controls (#4782)" This reverts commit |
||
|
Naiyuan Qing
|
26142d74aa |
feat(self-host): collect anonymous source channels mul-3878 (#4741)
* feat(self-host): collect anonymous source channels * feat(self-host): include other source text * feat(self-host): report source channel domains * feat(self-host): add source domain reporting control * fix(self-host): simplify source reporting copy * fix(self-host): simplify source channel reporting gate * fix(self-host): limit source reporting triggers * chore(self-host): point source reporting at staging |
||
|
Bohan Jiang
|
50a48cef1e |
feat(slack): unified multica chat history pull for channel backfill (MUL-3871) (#4747)
* feat(slack): add unified `multica chat history` pull for channel backfill (MUL-3871) Agents @mentioned in a Slack thread/channel only saw the triggering message, never the prior conversation (GitHub #4717). Instead of force-assembling a recent-context block on every inbound (the Feishu approach), expose a single channel-agnostic pull command the agent runs on demand. - channel: normalized HistoryMessage/HistoryPage/HistoryOptions vocab so the agent sees one shape regardless of platform. - slack.History: resolves session -> binding -> installation -> bot token and reads conversations.replies (real thread) or conversations.history (DM / top-level channel, capturing sibling messages). thread_ts is recorded on the binding config at session creation to pick the right call. - handler GET /api/chat/history: authorized purely by the task-scoped token (stamped X-Task-ID -> the task's own chat session), so an agent can only read the conversation it is currently running for. - multica chat history CLI command (no args; same for every channel). - buildChatPrompt nudge so the agent discovers the command. Feishu is intentionally untouched. Adding a platform = implement the reader. Co-authored-by: multica-agent <github@multica.ai> * fix(slack): require task-token actor source on chat history endpoint Niko's review caught a privilege-boundary hole: the endpoint trusted X-Task-ID, but it is mounted under the general Auth group where a normal JWT / mul_ PAT request does NOT strip a client-forged X-Task-ID — only the mat_ task-token branch stamps it. A workspace member who knew a chat task id could forge the header and read that task's Slack channel/DM/thread history. Gate on the server-set X-Actor-Source == "task_token" (the Auth middleware deletes any client-supplied value and re-stamps it only on the mat_ branch), then trust X-Task-ID. Adds a regression test: a forged X-Task-ID without the task-token actor source is rejected with 403 and never reaches the reader. Co-authored-by: multica-agent <github@multica.ai> * fix(slack): thread-first history for follow-ups, channel for first turn (MUL-3871) A Slack conversation has two nested histories: the surrounding channel and the agent's own thread (the bot's first reply opens a thread on the @mention). The first version picked replies-vs-history from a thread_ts fixed at session creation, so a session started by a top-level @mention always read CHANNEL history — even on follow-ups inside the bot's thread, which should read THREAD history first. - Add a HistoryScope (auto|thread|channel). The handler resolves auto: first turn (no prior bot reply) -> channel; follow-up -> thread. The agent can override with --scope channel|thread, and the response reports the scope read. - The thread root is derived from the binding (last_thread_id / composite-key suffix), available for every engaged group session, instead of the creation-time thread_ts (now removed from the binding config). - A DM degrades a thread request to channel history (DMs have no threads). - Prompt guidance + CLI help updated to explain the policy. Tests: scope selection (thread/channel/DM-fallback/no-root), root derivation, and handler auto-resolution (first->channel, follow-up->thread, explicit override). Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: J <j@multica.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Naiyuan Qing
|
b336f07617 |
Revert "feat(analytics): anonymous self-host onboarding source beacon (MUL-37…" (#4712)
This reverts commit
|
||
|
Naiyuan Qing
|
63eb6f73ad |
feat(analytics): anonymous self-host onboarding source beacon (MUL-3708) (#4691)
* feat(analytics): anonymous self-host onboarding source beacon (MUL-3708) Production self-host servers now report the anonymous onboarding "how did you hear about us" channel to Multica's public write-only ingest, so the self-host source distribution becomes visible alongside official cloud. Official cloud keeps its existing PostHog capture unchanged; this is a submit-time beacon, not a background telemetry pipeline. - server/internal/sourcebeacon: ShouldSend gate (production + non-local + non-*.multica.ai app host, fail-closed — judged by the app/frontend host, not the backend URL, which official often leaves unset), per-instance salted hashing, deterministic event uuid, fire-and-forget sender. - POST /api/telemetry/self-host-source: public, write-only, per-IP rate-limited, 4 KiB body cap, channel allowlist, strict unknown-field rejection. Lands in PostHog as self_host_source_channel with a deterministic uuid (best-effort dedup), $process_person_profile=false, and deployment=self_host — a distinct event name so it never pollutes the official onboarding funnel. - Hook in PatchOnboarding fires once when the source is first set; never blocks onboarding. Only channel enum(s) + two per-instance hashes leave the box — never user_id/email/name/workspace/org/domain/role/use_case/the source_other free-text/IP. - migration 128: system_settings singleton holding instance_salt. - frontend: self-host-only anonymous-collection notice on the source step, gated by a new /api/config self_host_source_notice flag (en/zh-Hans/ko/ja). - analytics.Event gains an optional top-level uuid; docs/analytics.md, SELF_HOSTING.md and .env.example document exactly what is/isn't sent and how to disable it (ANALYTICS_DISABLED). Also fixes the long-standing team_size→source drift in docs/analytics.md. Verified locally: go build/vet, go test (sourcebeacon, analytics, handler), pnpm typecheck (all packages), locale parity (157), step-source (6) + core config/schema (69) vitest, lint (0 errors). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: multica-agent <github@multica.ai> * fix(analytics): wire self-host source beacon through metrics, guard nil pool (MUL-3708) Addresses Howard CI blockers on #4691 (no product-direction change): - loadInstanceSalt returns "" on nil pool; salt is only loaded when ShouldSendFromEnv() is true, via a bounded (5s) context — restores the "router constructible without a DB" invariant (nil-pool routing tests). - Add multica_self_host_source_channel_total counter (by source) + an IncForEvent case, so every analytics event is paired with a Prometheus counter. NormalizeSourceChannel reuses sourcebeacon allowlist (no 3rd copy). - Beacon handler now builds the event via the analytics.SelfHostSourceChannel helper and ships it through obsmetrics.RecordEvent (no naked Capture); not IsMetricsOnly, so it still reaches PostHog. - Prime the new family in the registry-families test. Verified: go build/vet, go test ./internal/metrics ./internal/sourcebeacon ./internal/handler ./cmd/server (incl. the 3 named blockers + registry + record-event-helper lints) all green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Bohan Jiang
|
11a3cf206b |
feat(slack): bring-your-own-app install + per-installation Socket Mode (MUL-3666) (#4566)
* feat(slack): single app-level Socket Mode connection routed by team_id (MUL-3666) Reshape the Slack adapter from the stage-3 per-installation Socket Mode model into the multi-tenant B2 connection model: ONE deployment-level Socket Mode connection (app-level xapp- token, env MULTICA_SLACK_APP_TOKEN) receives the Events API stream for every installed workspace and routes each inbound event to its channel_installation by team_id — the existing GetChannelInstallationByAppID routing, unchanged. - AppConnector: the single shared connection (slack/app_connector.go). No leader election — per the design "one (or a few)" connections are fine: each replica opens one, Slack delivers each event to one of them, and the existing (installation, message_id) two-phase dedup guarantees exactly-once processing. Resolves the per-team bot user id (via the same app_id query) to detect/strip @-mentions, since one connection serves many workspaces. - Inbound translation (Events API -> channel.InboundMessage) extracted to slack/inbound.go as free functions parameterized by the per-team bot identity. - channel.go trimmed to the outbound Send-only sender; per-installation config (config.go) no longer carries an app-level token — installs hold only the per-workspace bot token (xoxb-) for outbound, since xapp- can't be OAuth'd. - engine.Supervisor now skips channel types with no registered Factory, so Slack installs (driven by the app-level connector, not per-installation channels) no longer churn the lease/Build loop. - Wiring: router.go builds the connector when MULTICA_SLACK_APP_TOKEN is set; main.go runs it alongside the Supervisor. Feishu untouched; channel_* schema unchanged. Verified: go build ./..., go vet ./..., gofmt, and go test ./internal/integrations/... all pass. Co-authored-by: multica-agent <github@multica.ai> * feat(slack): OAuth self-serve install backend (MUL-3666) Add the in-product OAuth install flow that creates Slack installations, the keystone the B2 connector consumes. - slack.InstallService: Begin (build authorize URL, seal workspace/agent/ initiator into the OAuth state), Complete (verify state, exchange code via oauth.v2.access, upsert channel_type='slack' install with the bot token encrypted at rest, auto-bind the installer's Slack id so their first message is not dropped), plus List/Get/Revoke. State is stateless: sealed with the deployment secretbox + an embedded expiry, no session store. - HTTP handlers (handler/slack.go): member-visible list, admin-only begin + revoke, and the public OAuth callback (recovers context from the sealed state, redirects the browser back to Settings → Integrations with a result flag). - Routes + wiring: workspace-scoped list/begin/revoke mirror the Lark admin/member split; the callback is a public route like GitHub's. Built from MULTICA_SLACK_CLIENT_ID/SECRET (+ redirect derived from MULTICA_PUBLIC_URL, override MULTICA_SLACK_REDIRECT_URL; scopes via MULTICA_SLACK_SCOPES). - Realtime: slack_installation:created / :revoked events. Verified: go build ./..., go vet, gofmt, and go test ./internal/integrations/slack/... all pass (new install_test.go covers state sign/verify/expiry/tamper, authorize URL, code exchange + encrypted upsert + installer bind, and oauth error paths). Co-authored-by: multica-agent <github@multica.ai> * feat(slack): in-product OAuth install UI for web + desktop (MUL-3666) Add the "Connect Slack" self-serve install UI mirroring the Feishu/Lark integration, completing the in-product install half of B2. Slack's OAuth flow is a redirect (not a device-code QR poll), so the UI is simpler than Lark's. - core: SlackInstallation / List / Begin types; api.listSlackInstallations / beginSlackInstall / deleteSlackInstallation; slackKeys + slackInstallationsOptions query; realtime invalidation on slack_installation:* events. - views: slack-tab.tsx (SlackTab settings panel + per-agent SlackAgentBindButton + connected badge + disconnect confirm). Connect calls beginSlackInstall and hands the authorize URL to openExternal (system browser on desktop, new tab on web); Slack bounces to the backend callback which lands the install, and the realtime event refreshes the list. Wired into the Settings → Integrations tab and the agent-detail Integrations tab alongside Lark. - i18n: en + zh-Hans settings.slack.* strings. Verified: pnpm typecheck (full monorepo, 6/6) and pnpm lint (@multica/core, @multica/views — 0 errors) pass. Co-authored-by: multica-agent <github@multica.ai> * feat(slack): outbound Replier + user-binding redeem flow (MUL-3666) Fill the stage-3 Replier=nil tail so non-installer Slack users can onboard and get status feedback — completing B2 end to end. - slack.OutboundReplier (engine.OutboundReplier): on NeedsBinding it mints a single-use binding token and DMs/replies a "link your account" prompt with the redeem URL (wrapped as <url|label> so formatMrkdwn doesn't mangle the base64url token); on AgentOffline/AgentArchived it posts a status notice; on an /issue-created Ingest it confirms the new issue. Plain chat stays silent (the agent's own reply lands via EventChatDone). Reuses the bot-token Send path and reads the installation row from ResolvedInstallation.Platform — no new transport. - slack.BindingTokenService: Mint + transactional RedeemAndBind over the generic channel_binding_token / channel_user_binding queries (channel_type='slack'), mirroring lark.BindingTokenService. 15-min TTL, SHA256-hashed tokens, the three typed failure modes (invalid/expired, already-assigned, not-member). - HTTP: POST /api/slack/binding/redeem (public, session-authed) maps the failures to 410/409/403. NewSlackResolverSet now takes the replier (nil disables it). - Frontend: /slack/bind redeem page (packages/views/slack + apps/web route) + api.redeemSlackBindingToken + en/zh slack_bind copy. Verified: go build ./..., go vet, gofmt, go test ./internal/integrations/... (new replier_test.go covers all outcome branches + the prompt URL), plus full pnpm typecheck (6/6) and pnpm lint (0 errors). Co-authored-by: multica-agent <github@multica.ai> * fix(slack): address review must-fixes — connector leak, team-keyed install, /issue copy (MUL-3666) Three fixes from Niko's review: 1. AppConnector.connectOnce leaked the Socket Mode goroutine/connection on a handler error: it ran sm.RunContext on the long-lived ctx and returned the error without cancelling it, so a transient DB/router error left the old connection alive (consuming events into an unread channel) while Run opened a second one. Each connection now runs under its own cancellable context and a deferred cancel + join tears it down on every exit path before reconnect. 2. Slack re-install collided with the (channel_type, app_id) unique index: connecting the same Slack team to a different agent failed because the upsert conflict key was (workspace_id, agent_id, channel_type). Add a team-keyed UpsertChannelInstallationByAppID (ON CONFLICT on the (channel_type, app_id) index, updating agent_id) and use it for the Slack OAuth install, so re-connecting a workspace moves the bot to the chosen agent instead of erroring. Feishu's per-agent upsert is unchanged. 3. /issue clarified: it is not a registered Slack slash command (no `commands` scope), so Slack never routes one to us. Issue creation runs through the message path — `@bot /issue <title>` in a channel or `/issue <title>` in a DM — which the engine parser handles. Documented in the connector and the user-facing copy (en + zh). Verified: go build ./..., go vet, gofmt, go test ./internal/integrations/..., make sqlc, plus pnpm typecheck (6/6) and pnpm lint (0 errors). Co-authored-by: multica-agent <github@multica.ai> * fix(slack): make OAuth install transactional — agent-move binding consistency + cross-workspace guard (MUL-3666) Address Elon's review: the team-keyed upsert kept the same installation row and only flipped agent_id, but engine session reuse matches purely on (installation_id, channel_chat_id) and each chat_session is permanently tied to the agent it was created under — so after moving a Slack team from Agent A to Agent B, existing DMs/threads kept routing to Agent A; only brand-new channels/threads reached B. Cross-workspace re-install was worse: the SQL also moved workspace_id while the application-layer user/chat-session bindings stayed behind, inheriting the previous workspace's relations. InstallService.Complete now runs one transaction (lookup → upsert → retire → installer-bind), all application-layer per the no-FK rule: - Look up the existing installation by team_id (config->>'app_id'). - Reject a silent cross-workspace ownership change (ErrTeamOwnedByAnotherWorkspace → callback redirects with slack_error=team_in_other_workspace). The owning workspace must disconnect first. - On an agent change within the same workspace, retire the installation's chat-session bindings (new DeleteChannelChatSessionBindingsByInstallation) so the next message creates a fresh session under the new agent. The chat_session rows are preserved for history; user bindings stay valid (same users/workspace). - Installer auto-bind moves into the tx; an already-bound-elsewhere id is a benign skip, a real DB error aborts the whole install. InstallService now takes a TxStarter; the queries seam gains WithTx (dbInstallQueries adapter) so Complete stays unit-testable with a fake tx. Verified: make sqlc, go build ./..., go vet, gofmt, go test ./internal/integrations/... (new tests: agent-move retire, same-agent no-retire, cross-workspace reject, fresh-install no-retire). Co-authored-by: multica-agent <github@multica.ai> * fix(slack): atomic cross-workspace install guard + green up frontend CI (MUL-3666) Two things: address Elon's review and fix the failing frontend CI job. Review (atomic cross-workspace guard): the previous guard was a SELECT before the upsert, which loses the concurrent-OAuth race — two workspaces can both read no rows, one inserts, the other's ON CONFLICT update then silently re-points the team. Move the guard into the upsert itself: ON CONFLICT ... DO UPDATE ... WHERE channel_installation.workspace_id = EXCLUDED.workspace_id, and map the empty RETURNING (pgx.ErrNoRows) to ErrTeamOwnedByAnotherWorkspace. The pre-SELECT now only feeds the agent-change cleanup. Also corrected the error copy: a team stays bound to its first Multica workspace (revoke is soft, keeping the row + unique index), so migration is an operator action, not "disconnect first". CI (frontend vitest, @multica/views#test): - The agent IntegrationsTab now renders the real SlackAgentBindButton, whose connected badge calls useQueryClient — absent from integrations-tab.test.tsx's react-query mock. Hoisted the owner/admin gate above the per-platform sections (one role notice instead of one per platform), made the agents members_note generic (en/zh/ja/ko), and updated the test (mock @multica/core/slack, stub SlackAgentBindButton, assert both platforms). - Added slack-tab.test.tsx covering the real SlackAgentBindButton / SlackTab. - locale parity: added the slack (settings) + slack_bind (common) blocks to ja and ko so every EN key has a translated counterpart. Verified: make sqlc, go build ./..., go vet, gofmt, go test ./internal/integrations/...; pnpm --filter @multica/views test (1478 pass), pnpm typecheck (6/6), pnpm lint (0 errors). Co-authored-by: multica-agent <github@multica.ai> * fix(slack): surface agent-page Slack entry points when Lark is off (MUL-3666) The agent-detail Integrations tab and the inspector's Integrations section only considered Lark, so a Slack-only deployment (Lark disabled) showed neither the Integrations tab nor a Connect-Slack button — the per-agent entry points were unreachable. - agent-overview-pane: gate the Integrations tab on Lark OR Slack configured (new slackInstallationsOptions query), not Lark alone. - agent-detail-inspector: render SlackAgentBindButton alongside LarkAgentBindButton in the Integrations section. - regression test: the Integrations tab appears when only Slack is configured. Verified: pnpm typecheck (6/6), pnpm --filter @multica/views test (1478+ pass), pnpm lint (0 errors). Co-authored-by: multica-agent <github@multica.ai> * feat(slack): BYO-app install backend — paste xoxb+xapp, per-app install keyed by real app id (MUL-3666) Adds the bring-your-own-app install path so multiple agents can each have their own bot identity in the SAME Slack workspace (hosted B2 caps at one agent/workspace). User pastes their app's bot token (xoxb-) + app-level token (xapp-); we validate the bot token via auth.test, parse the real Slack app id from the xapp- token, encrypt both tokens, and persist a per-app installation keyed by that app id (real 'A…' ids never collide with hosted 'T…' team ids in the existing unique index — no schema change). - config.go: add app_token_encrypted (BYO discriminator + per-app socket token) - install.go: extract shared persistInstall (atomic cross-ws guard + agent-move retire) - byo_install.go: RegisterBYO + auth.test + app-id parse - handler + route: POST /api/workspaces/{id}/slack/install/byo (admin-only) - tests: keying, encryption, invalid tokens, auth.test failure, cross-ws, agent move Follow-ups (separate commits): per-app Socket Mode connector that consumes the stored app token; in-product BYO install dialog (video + paste form). Co-authored-by: multica-agent <github@multica.ai> * refactor(slack): drop OAuth, unify on BYO per-installation model (MUL-3666) Per product decision, Slack drops the hosted-app OAuth path entirely and unifies on bring-your-own-app (BYO): every installation carries its OWN app-level token and gets its OWN Socket Mode connection, so multiple agents can each have a distinct bot identity in one Slack workspace. - Remove OAuth install (Begin/Complete/code-exchange/sealed state/OAuthConfig/ default scopes), the OAuth callback + begin handlers + routes, and the MULTICA_SLACK_CLIENT_ID/SECRET/REDIRECT/APP_TOKEN env wiring. - Replace the single deployment-level AppConnector with a per-installation slackChannel (authenticated with its own xapp- token) registered as a channel Factory, so the engine Supervisor drives one Socket Mode connection per installation (exactly like Feishu). inbound/outbound/resolvers reused as-is. - Route inbound by the event's api_app_id (== the installation's real app id), not team_id. - InstallService slims to at-rest encryption + the shared persistInstall + list/get/revoke; install is the BYO paste path only (byo_install.go). - Tests: drop the OAuth tests; slack + handler + engine all green. Follow-up (frontend): replace the OAuth "Connect Slack" button with the BYO paste dialog (the begin endpoint it calls is now gone). Co-authored-by: multica-agent <github@multica.ai> * fix(slack): verify BYO bot + app tokens are from the same app, and the app token is live (MUL-3666) Niko review: RegisterBYO only parsed the app id from the xapp string and auth.test'd the bot token, so pasting app A's bot token with app B's app token would 'connect' but be broken (inbound on B's socket, outbound with A's identity). Now: resolve the bot's owning app id via bots.info (on the bot_id from auth.test) and require it to equal the xapp's app id; and live- validate the app token via apps.connections.open. Reject (no persist) on mismatch or a dead app token. Co-authored-by: multica-agent <github@multica.ai> * feat(slack): in-product BYO install dialog (paste bot + app tokens) (MUL-3666) The OAuth begin endpoint was removed server-side, so the "Connect Slack" button now opens a dialog where the admin pastes the bot token (xoxb-) and app-level token (xapp-) of the Slack app they created, and submits to the BYO install endpoint. Includes an optional setup-video link (URL constant, left empty until the walkthrough is recorded). - core: drop beginSlackInstall / BeginSlackInstallResponse; add registerSlackBYO + RegisterSlackBYORequest. - views: SlackAgentBindButton opens the BYO dialog; refreshed comments and install_supported docs (now means "configured", no OAuth). - i18n: new slack.byo_* keys + refreshed page_description in en/zh-Hans/ja/ko. - tests: dialog submit path; views vitest (1479), typecheck, lint, locale parity all green. Co-authored-by: multica-agent <github@multica.ai> * fix(slack): Elon review — team_id routing guard, per-agent reconnect, users:read hint (MUL-3666) 1. Inbound routing keys on api_app_id (the APP, not the Slack workspace), so additionally require the event's team_id to match the installation's stored team. A distributed BYO app installed into another Slack workspace emits the same app id and would otherwise mis-route to this Multica installation. Extracted installationServesTeam() + unit test. 2. BYO install is now agent-keyed (UpsertChannelInstallation, conflict on workspace_id+agent_id+channel_type): one bot per agent. Disconnect → reconnect a NEW app for the SAME agent now UPDATES that agent's row in place instead of violating the (workspace, agent, channel) unique. A unique violation on the (channel_type, app_id) routing index → ErrTeamOwnedByAnother- Workspace (the app is already connected to another agent/workspace). No chat-session retire is needed: a row's agent_id never changes. 3. UX: bots.info (the same-app check) needs the users:read scope — the connect dialog now lists the required bot scopes including it, and the error text says so. Backend build/vet/gofmt/test + views vitest + typecheck + locale parity green. Co-authored-by: multica-agent <github@multica.ai> * fix(slack): publish slack_installation:created on BYO connect; refresh stale comments (MUL-3666) Niko final review: RegisterSlackBYO wrote the response but never published EventSlackInstallationCreated, so only the installer's own tab refreshed — other open clients (Settings, Agent Integrations, other tabs) did not see the new bot in realtime, inconsistent with the revoke event and Lark. Now publishes it on success via a small publishSlackInstallationCreated helper, with a unit test (Bus.Publish is synchronous). Also refreshed comments that still described the removed hosted-OAuth / single deployment-level AppConnector model (handler SlackInstall field, channel.go / inbound.go / outbound.go / byo_install.go). PR title updated separately to the BYO per-installation Socket Mode model. Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: J <j@multica.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Bohan Jiang
|
35e5455953 |
fix: allow split-origin attachment previews (#4539)
Co-authored-by: J <j@multica.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Bohan Jiang
|
3e21e58df0 |
feat(channel): channel-agnostic engine (Supervisor + Router); Feishu as channel.Channel (MUL-3620) (#4512)
* feat(channel): add channel-agnostic engine Supervisor (MUL-3620) Stage-1 (MUL-3515) shipped the channel abstraction but nothing drove it. Add the generic engine that does: - channel.InboundHandler + Config.Handler: the single shared inbound entry the engine injects into every adapter (Hermes set_message_handler model). - channel.Channel.Connect now blocks for the connection lifetime (doc), so the supervisor can tie lease renewal to connection liveness. - new package channel/engine: Supervisor, generalized out of lark.Hub. It enumerates active installations across ALL channel types (no hard-coded feishu), fences each behind the WS lease CAS, builds the platform Channel via channel.Registry, drives Connect/Disconnect with backoff+jitter, and restarts on credential rotation. Knows nothing about any platform. channel.Channel is now driven by an engine; integrations/channel has an external consumer. Feishu adapter + boot cutover follow next. Tests: supervisor_test.go covers lease CAS, reclaim, reap-on-revoke, rotation restart + token fencing, backoff on build error, lease-loss teardown, bounded release, shutdown timeout. Race-clean. Co-authored-by: multica-agent <github@multica.ai> * feat(lark): drive Feishu through the channel engine; remove lark.Hub (MUL-3620) Refactor Feishu into the first channel.Channel and cut boot over to the channel-agnostic engine.Supervisor, removing the Feishu-only Hub. - feishuChannel implements channel.Channel: Connect runs the existing WS long-conn connector for one installation; Send posts a text reply via the Lark IM API; Capabilities declares Feishu's feature set. RegisterFeishu wires it to channel.TypeFeishu — adding a platform is now 'register a Factory', no engine edit. - FeishuRuntime extracts the former Hub.handleEvent / scheduleReply: runs the Dispatcher and drives the detached typing indicator + OutcomeReplier off the connector ACK path. main.go drains it on shutdown after the supervisor stops delivering events. - channelInstallationStore (engine.InstallationStore) enumerates active installations across ALL channel types via the new de-hardcoded query ListAllActiveChannelInstallations; the Supervisor routes each row to its registered Factory by channel_type. Generic per-row fingerprint replaces the feishu-specific one. - boot: engine.Supervisor replaces lark.Hub.Run; MULTICA_LARK_HUB_DISABLED keeps its name for runbook compatibility. - delete hub.go / hub_pgx.go / hub_test.go; relocate the connector contract (EventConnector/EventEmitter), uuidString, and the reply-path tests (-> feishu_runtime_test.go) so coverage is preserved. No channel_* schema change. Feishu behaviour unchanged; lark + channel + engine tests green under -race; go build/vet ./... clean. Remaining (follow-up): lift the Dispatcher pipeline into a channel- agnostic engine.Router over channel.InboundMessage + resolver interfaces, so the inbound core stops being Lark-shaped and adding a channel needs zero core edits (validated by Slack, MUL-3516). Co-authored-by: multica-agent <github@multica.ai> * feat(channel): add channel-agnostic engine.Router (inbound pipeline) (MUL-3620) Generalize lark.Dispatcher's inbound pipeline into engine.Router: the single shared channel.InboundHandler the Supervisor injects into every Channel. It routes by ChannelType to a registered ResolverSet and runs the same ordered pipeline for every platform (install route -> two-phase dedup -> group @bot filter -> identity+membership -> ensure session -> append+mark -> /issue -> debounced run), then drives the detached OutboundReplier + typing indicator. Platform specifics live behind resolver interfaces (InstallationResolver, IdentityResolver, Deduper, SessionBinder, Auditor, OutboundReplier, TypingNotifier) + shared services (IssueCreator/TaskEnqueuer/SessionReader). Adding a platform is 'register a ResolverSet', not 'edit the Router'. Outcome / DropReason values match the legacy lark ones 1:1. Additive: lark.Dispatcher untouched and still wired; the feishu ResolverSet, the cutover, and the old-path removal land next. channel.InboundMessage gains ForceFresh (the normalized /fresh affordance). Batcher moved into engine. router_test.go covers the pipeline invariants (routing, dedup finalize states, group filter, identity, membership, ensure/append, /issue, debounce, flush offline, force-fresh, drain) with generic fakes; race-clean. Co-authored-by: multica-agent <github@multica.ai> * feat(lark): cut Feishu over to engine.Router; remove lark.Dispatcher; core no longer Lark-shaped (MUL-3620) Wire the channel-agnostic engine.Router (added in the prior commit) as the shared inbound handler and refactor Feishu into a ResolverSet, completing the generic-engine cutover. The inbound core (engine.Router) now contains zero platform specifics. - Feishu ResolverSet (feishu_resolvers.go): InstallationResolver, IdentityResolver, Deduper, SessionBinder, Auditor, OutboundReplier, TypingNotifier — each backed by the existing ChannelStore / ChatSessionService / OutcomeReplier / typing indicator, translating at the channel.InboundMessage boundary (platform fields read from Raw). origin_type stays 'lark_chat'. - feishuChannel now produces a normalized channel.InboundMessage and hands it to the engine handler via channel.Config.Handler; the old Raw round-trip through lark.Dispatcher is gone. - Remove lark.Dispatcher, FeishuRuntime, and lark's pending_batcher (the engine owns the pipeline + batcher now); their behavioural coverage moved to engine.Router tests. Surviving native types (InboundMessage / Outcome / DispatchResult) relocated to feishu_types.go. elon review nits addressed: - The channel engine (Registry + Router + Supervisor) is now built UNCONDITIONALLY, outside the MULTICA_LARK_SECRET_KEY gate, so a non-Lark deployment runs it; Feishu registers its Factory + ResolverSet only when its key is present. - channel.Config.Raw is now genuinely the platform config JSONB (channel_installation.config): the feishu factory builds a credentials-only Installation from it, and the workspace/agent identity is resolved per message by the Router — no full-db-row marshaling. - feishuChannel gains direct unit tests: factory config decode, Send text + reply-target mapping, Capabilities, inbound normalization + Raw round-trip, msg-type + result mapping. No channel_* schema change. go build/vet ./... clean; channel + engine + lark green under -race. Feishu behaviour preserved (pipeline logic lifted verbatim, only generalized). Co-authored-by: multica-agent <github@multica.ai> * docs(channel): fix stale comments on the channel engine boot (MUL-3620) Address Elon's review nit: three comments still described the pre-cutover behavior. - handler.go: ChannelSupervisor is built UNCONDITIONALLY now, not nil when MULTICA_LARK_SECRET_KEY is unset. - main.go: same — the supervisor always exists; only MULTICA_LARK_HUB_DISABLED parks it. - router.go: with no platform registered the store still lists active rows; Registry.Build returns ErrUnknownType and the supervisor backs off (it does not 'find no installations'). Comment-only; no behavior change. Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: J <j@multica.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Multica Eve
|
1ac3a03e5d |
MUL-3618: dispatch daemon feature flag snapshots (#4509)
* MUL-3618: dispatch daemon feature flag snapshots Co-authored-by: multica-agent <github@multica.ai> * MUL-3618: narrow daemon flag snapshots to process scope Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Eve <eve@multica-ai.local> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Bohan Jiang
|
a123dfc2df |
MUL-3508: stage sub-issues so the parent wakes per stage, not per child (#4410)
* feat(issues): stage sub-issues so the parent wakes per stage, not per child Sub-issues under a parent can be grouped into ordered stages (issue.stage). The child-done -> parent notification + assignee wake now fire only when a stage barrier closes: every sub-issue in the lowest unfinished stage has reached a terminal status (done/cancelled). An unstaged sibling set is one implicit stage, so the parent is woken once when the last sub-issue finishes instead of on every child — the default fix for the fire-on-every-child cascade reported in discussion #4320 / MUL-3508. Stage advancement stays agent-driven: the server only detects the closed barrier and wakes the parent assignee, who decides whether to promote the next stage. - DB: nullable issue.stage (CHECK >= 1) + sqlc regen - API: stage on issue create/update/response and batch update - CLI: `issue create`/`issue update` --stage; new `issue children` command that lists sub-issues grouped by stage (table + json) - stageBarrierClosed / stageProgressSummary in issue_child_done.go, with the wake comment now stage-aware, plus unit tests - skill docs (multica-working-on-issues SKILL.md + source map) Web UI (create-form stage picker, sidebar edit, group-by-stage display) is a follow-up; the API already returns stage for it to consume. MUL-3508 Co-authored-by: multica-agent <github@multica.ai> * fix(issues): address review on stage barrier (cancel, batch, unstaged) Resolves the three blockers from the PR review: 1. Cancel can close a stage. The child-done barrier now fires on any non-terminal -> terminal transition (done OR cancelled), not just done. isTerminalChildStatus already treats cancelled as terminal (a cancelled sibling never finishes, so it must not hold a stage open), so a cancelled last-open child now closes its stage and wakes the parent. Keying on the transition also makes a later cancelled -> done edit a no-op, avoiding a lagging duplicate wake. 2. Batch update of stage no longer no-ops. `hasMutation` now includes "stage", so `{"updates":{"stage":N}}` persists instead of returning {"updated": 0}. 3. Unstaged children no longer participate in the staged frontier. In a staged sibling set, NULL-stage children neither hold a stage open nor fire on their own completion, and the wake comment no longer renders "Stage 0". This matches migration 123 ("NULL does not participate in staged grouping") and the CLI's separate unstaged group, removing the footgun where an unstaged backlog child silently blocked Stage 1. Tests: cancellation closes a stage (staged + unstaged), unstaged ignored in a staged set, stage summary skips unstaged, and a stage-only batch update persists. MUL-3508 Co-authored-by: multica-agent <github@multica.ai> * feat(web): stage UI — create picker, sidebar edit, group sub-issues by stage Frontend for the sub-issue stage feature (web + desktop, shared via packages): - core: `stage` on the Issue type + create/update request types; zod IssueSchema parses it (defaults to null for older backends) with schema tests for the numeric and omitted cases. - StagePicker component (mirrors the other property pickers): "No stage" + Stage 1..N, offering one beyond the current/sibling max. - Create-issue modal: a Stage pill, shown only when a parent is selected, threaded into the create payload. - Issue detail sidebar: an editable Stage row + "add property" entry, gated to sub-issues (issues with a parent). - Sub-issue list grouped by stage with per-stage headers (flat when unstaged). - i18n: stage keys across en / zh-Hans / ja / ko (parity test passes). Verified: full typecheck (6/6), core (591) + views (1433) vitest suites, lint clean (no new findings). Backend/CLI shipped earlier in this PR. MUL-3508 Co-authored-by: multica-agent <github@multica.ai> * test(issues): add stage to Issue fixtures merged from main The merge brought in new Issue fixtures that predate the required `stage` field: core issues/batch.test.ts, views batch-action-toolbar.test.tsx, and the mobile EMPTY_ISSUE_FALLBACK sentinel. Add `stage: null` so they satisfy the Issue type (mobile reuses core's IssueSchema for parsing, so only the sentinel needs it). MUL-3508 Co-authored-by: multica-agent <github@multica.ai> * fix(web): feed StagePicker the sibling max stage so higher stages stay selectable The StagePicker accepts maxStage to extend its option list beyond the floored Stage 1-3, but neither call site passed it, so a parent with an existing Stage 4/5 child could not pick that stage when creating a new sub-issue or editing one in the sidebar. - Compute the sibling max stage at both call sites: the create modal now loads the parent's children (childIssuesOptions) and the detail sidebar reuses the already-loaded parentChildIssues. - Extract maxSiblingStage + stageOptions as pure helpers on stage-picker and unit-test them (the regression: a Stage 5 sibling keeps Stage 5 selectable and offers Stage 6). MUL-3508 Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: J <j@multica.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Hzzzzzx
|
745832b536 |
MUL-3433: fix(daemon): restore claim slow-log payload observability without gzip
Squash merge PR #4322 after review approval.\n\nMUL-3433 |
||
|
LinYushen
|
e1c6754304 |
Revert "fix: gzip daemon claim responses (#4259)" (#4307)
This reverts commit
|
||
|
Bohan Jiang
|
2f398c36ad |
fix: gzip daemon claim responses (#4259)
Co-authored-by: J <j@multica.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
LinYushen
|
77ac17ef49 |
Make custom runtimes appear immediately (#4234)
* Make custom runtimes appear immediately * Scope daemon profile refresh by authorized runtimes * Relay runtime profile refresh hints * Localize runtime profile close label |
||
|
Multica Eve
|
ae27058b0a |
fix(attachments): unified download endpoint with mode + presign + proxy (MUL-2976) (#3747)
Fix attachment download for self-hosted deployments using private S3-compatible buckets without CloudFront. Closes #3721. **Server** - New unified `GET /api/attachments/{id}/download` endpoint that picks CloudFront / S3 presign / server proxy at request time. - `ATTACHMENT_DOWNLOAD_MODE=auto|cloudfront|presign|proxy` and `ATTACHMENT_DOWNLOAD_URL_TTL` env knobs; `auto` routes Docker hostnames / localhost / private IPs through the proxy and public S3 endpoints through presign. - `Storage.PresignGet` capability; S3 implementation generates presigned GET URLs. - `attachmentToResponse` returns the unified relative endpoint instead of leaking raw unsigned S3 URLs when CloudFront is not configured. Proxy path streams via `io.Copy` with `Content-Disposition` / `Content-Length` / `Cache-Control: no-store` / `X-Content-Type-Options: nosniff`. **Clients** - CLI / Desktop / Mobile resolve relative `download_url` values against the configured API base. Desktop covers the Electron native download bridge and the media preview modal; Mobile covers `Linking.openURL`, the markdown image RN loader, and the composer's completed non-image file chip. - Mobile gains a minimal Node-environment vitest lane wired into `mobile-verify.yml`. **Docs** - `.env.example`, `docker-compose.selfhost.yml`, `SELF_HOSTING_ADVANCED.md`, and the `environment-variables` doc set updated with the new env keys and the `ATTACHMENT_DOWNLOAD_MODE=proxy` recommendation for Docker / VPC-internal object stores. **Tests** - `internal/storage`, `internal/cli`, `internal/handler` (download endpoint, mode selection, proxy header, `/content` non-regression), `cmd/server` (trusted proxy parser). - `packages/views/editor/use-download-attachment.test.tsx` and `attachment-preview-modal.test.tsx` exercise relative URL resolution + absolute pass-through. - `apps/mobile/lib/attachment-url.test.ts` covers every helper branch plus the composer non-image chip case. |
||
|
Bohan Jiang
|
598a6c51f2 |
refactor(server/lark): collapse HTTP_ENABLED + WS_ENABLED into the SECRET_KEY gate (MUL-2671) (#3717)
MULTICA_LARK_HTTP_ENABLED and MULTICA_LARK_WS_ENABLED were staging knobs from the multi-PR rollout of the Lark MVP — they let the DB schema + inbound dispatcher land before the HTTP wire was real, and before the WS long-conn protocol was wired. Now that the MVP has shipped end-to-end, "I set SECRET_KEY but I don't want to talk to Lark" is not a useful production state: setting the at-rest master key is the operator's opt-in for the integration as a whole. Collapse the gate down to MULTICA_LARK_SECRET_KEY alone. When the key is present, wire the real HTTPAPIClient + the real WSLongConnConnector. CI / integration tests that want stub-style behaviour can point MULTICA_LARK_HTTP_BASE_URL at a mock server (already supported) instead of toggling a separate flag. Host overrides (HTTP_BASE_URL, REGISTRATION_DOMAIN, CALLBACK_BASE_URL) stay — those are real ops needs for international tenants / staging. stubAPIClient + NoopConnectorFactory remain exported because the test suite uses them directly; only the router boot path stops reaching for them. The connector factory keeps its noop fallback for the case where the endpoint fetcher fails to construct, so a malformed MULTICA_LARK_CALLBACK_BASE_URL degrades gracefully (visible as "connector=noop" in the boot log) instead of panicking the server. Lark integration + handler tests still pass; go vet clean. Co-authored-by: J <j@multica.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Bohan Jiang
|
8c98940b79 |
Lark Bot integration MVP: migration + service boundary (MUL-2671) (#3277)
* feat(db): add Lark integration migration (MUL-2671) Introduces seven tables for the 飞书 Bot integration MVP — per-agent PersonalAgent installations, user/chat bindings, inbound dedup + non-content drop audit, outbound card mapping, and short-lived single-use member binding tokens. Schema notes: - chat_session schema unchanged; Lark routes through a separate binding table rather than adding a metadata JSONB column. - Outbound card mapping is task/message scoped so multiple runs on the same session can't stomp each other's cards. - lark_inbound_audit stores routing / identity / drop_reason ONLY, never message body — the audit channel for unbound users and group messages that don't address the Bot. - app_secret stores ciphertext (encryption helper lands in a follow-up commit on this branch); DB never sees plaintext. Co-authored-by: multica-agent <github@multica.ai> * feat(util): add secretbox AES-256-GCM helper for at-rest secrets First consumer is lark_installation.app_secret (MUL-2671 §4.4), but the helper is intentionally generic — future per-tenant secrets that must not appear in a DB dump can reuse it. Construction: AES-256-GCM with a per-message random nonce, providing authenticated encryption. Tampered ciphertext fails Open instead of silently decrypting to garbage. Master key loaded from a base64 env var via LoadKey; key rotation is not in scope yet. Co-authored-by: multica-agent <github@multica.ai> * refactor(issues): extract IssueService.Create as single create entry (MUL-2671) Establishes the service-layer boundary mandated by Elon's 二审 of MUL-2671 §4.8: issue creation no longer lives inside the HTTP handler. Both the HTTP POST /issues handler and the future Lark /issue command call into service.IssueService.Create, so duplicate guard, issue numbering, attachment linking, broadcast, analytics, and agent/squad enqueue stay aligned. Handler responsibilities shrink to parsing the HTTP request, doing actor resolution / validation (transport-specific), and converting service results into the IssueResponse + 201. The transaction-wrapped core, attachment link, event publish, analytics capture, and agent/squad enqueue all move into service.IssueService.Create. A BroadcastPayload callback on the service keeps the WS broadcast shape (the full IssueResponse) without forcing the service to depend on handler-layer response types. Co-authored-by: multica-agent <github@multica.ai> * feat(integrations): add Lark package skeleton (MUL-2671) Establishes the architectural boundaries Elon's 二审 mandated as first-PR blockers without dragging in OAuth, WebSocket, or card-patching code (those land in follow-up PRs): - ChatSessionService interface — channel-aware chat-session entry point for Lark, deliberately separate from the HTTP SendChatMessage handler. The HTTP handler's single-creator guard (creator_id == request user_id) is correct for the browser client but rejects group chat_sessions by construction; Lark needs its own service. - AuditLogger interface — the only path for recording dropped events. Its signature deliberately omits message body, enforcing the drop-audit policy (MUL-2671 §4.7) at the type level: unbound users and non-addressed group messages can't accidentally end up in chat_session. - Typed IDs (OpenID, ChatID) prevent UUIDs from being conflated with Lark-side identifiers at compile time. - DropReason constants align dashboard/audit queries across callers. Co-authored-by: multica-agent <github@multica.ai> * refactor(issues): move parent/project workspace check into IssueService (MUL-2671) Parent existence and project workspace membership now live inside IssueService.Create, inside the same transaction as the duplicate guard and counter increment. The HTTP handler stops re-implementing the lookup; every future create entry (Lark /issue, MCP, API keys) inherits the same boundary without copy-pasting the SQL. Adds two error sentinels (ErrParentIssueNotFound, ErrProjectNotFound) so transports can translate to their own error shapes. Handler-level cross-workspace tests guard the boundary against future regressions. Co-authored-by: multica-agent <github@multica.ai> * fix(db): harden Lark migration safety底座 — TTL cap + workspace FK (MUL-2671) Two storage-layer hardenings that move the must-fix line off "the app layer enforces it" and onto the schema itself, so future write paths or hand-inserted rows cannot regress the invariants. 1) lark_binding_token TTL cap. The DB CHECK was 1 hour as defense-in-depth while the app constant was 15 minutes; the CHECK now matches the product cap (15 minutes). Application constant docstring updated to reflect that storage enforces the same bound. 2) lark_user_binding workspace membership. The table previously only FK'd to workspace / user / installation independently, so a binding could exist for a user no longer in the workspace, or claim a workspace different from its installation's. Two composite FKs close the gap structurally: * (installation_id, workspace_id) → lark_installation(id, workspace_id) — guarantees a binding's workspace_id always matches its installation's workspace_id. A new UNIQUE (id, workspace_id) on lark_installation is added as the FK target. * (workspace_id, multica_user_id) → member(workspace_id, user_id) with ON DELETE CASCADE — when a user is removed from the workspace, the binding cascades away in the same transaction. There is no longer a path where lark_user_binding outlives workspace membership. These two FKs are the schema-level proof for §4.3's "unbound or non-workspace members cannot leak content into chat_session" invariant. Co-authored-by: multica-agent <github@multica.ai> * feat(integrations/lark): inbound services + /issue dispatcher (MUL-2671) Lands the inbound service layer for the Lark Bot MVP, sitting on top of the migration + service-boundary scaffold from the previous commits. What ships: - sqlc queries for all seven lark_* tables (idempotent dedup insert, CAS WS-lease, single-use binding-token consume, etc.) plus GetMostRecentUserChatMessage for the /issue fallback. - AuditLogger backed by lark_inbound_audit; signature deliberately body-free so callers cannot leak content into the drop log. - ChatSessionService: find-or-create chat_session via the binding table (winner-takes-all on the UNIQUE race), append-with-dedup, /issue parser, "previous user message" fallback for bare `/issue` invocation. - Dispatcher orchestrates the inbound pipeline in one place: installation routing → group-mention filter → identity check → ensure session → append+dedup → /issue → enqueue chat task. Group sessions use the installer as creator (stable workspace identity); p2p uses the sender. Agent-offline path falls through with OutcomeAgentOffline so the WS adapter can reply with the offline notice from §4.6. - BindingTokenService: random URL-safe token, SHA-256 stored hash, 15-min TTL pinned at the application AND the DB CHECK; Redeem returns the same opaque error for all rejection cases (no timing oracle on replay). - Unit tests for the parser (13 cases), dispatcher (8 cases via fake Queries/Chat/Audit/IssueCreator/Enqueuer), and binding-token hash/entropy. Real-DB integration tests for OAuth + token redeem land alongside the HTTP handlers in the next commit. Out of scope for this commit (next ones on the same feature branch): OAuth callback, HTTP routes, WebSocket hub, outbound card patcher, frontend. Co-authored-by: multica-agent <github@multica.ai> * feat(integrations/lark): installation HTTP surface + secretbox-gated wiring (MUL-2671) Lands the HTTP boundary on top of the inbound services from the previous commit. What ships: - InstallationService.Upsert: the only path that writes lark_installation. Encrypts app_secret with the secretbox passed in at construction time; refuses to fall back to plaintext storage (returns an error from the constructor if no Box is supplied), so a misconfigured dev environment cannot accidentally land a row with cleartext credentials. Revoke flips status without DELETE so audit trail survives. - HTTP handlers under /api/workspaces/{id}/lark/: * GET /installations — member-visible (Integrations tab renders for non-admins). Soft 200 with empty list + configured:false when MULTICA_LARK_SECRET_KEY is unset, so the tab does not error on self-host that has not opted in. * POST /installations — admin-only; 503 when not configured. Re-validates agent_id ∈ workspace before accepting credentials so a cross-workspace agent UUID is rejected. * DELETE /installations/{id} — admin-only; workspace-scoped lookup so one workspace cannot revoke another's installation by UUID guess. - POST /api/lark/binding/redeem (user-scoped, no workspace context): the only path that mints a lark_user_binding row from user action. Redeemer identity comes from the session, not the token, so a stolen link cannot bind an open_id to an attacker's Multica user. The composite FK on lark_user_binding cascades the binding away if the user is not (or no longer) a workspace member, so a non-member who steals the link gets 403 at the DB layer. - Two new event-bus types in protocol.events: EventLarkInstallationCreated, EventLarkInstallationRevoked. - Router wiring: MULTICA_LARK_SECRET_KEY drives a conditional initialization of h.LarkInstallations + h.LarkBindingTokens. When unset, the integration disables itself with an INFO log and the rest of the server boots normally. - Handler tests cover all four not-configured short-circuits. Happy-path integration tests (real DB, full create→list→revoke cycle and token mint→redeem) ship alongside the WS hub PR. Co-authored-by: multica-agent <github@multica.ai> * fix(integrations/lark): close binding-token rebind & typed task errors (MUL-2671) Two must-fixes from PR review on HEAD |
||
|
LinYushen
|
de900b2ba6 |
feat(server): funnel/community/commercial business metrics + PostHog pairing (MUL-2949) (#3698)
* feat(server): funnel/community/commercial business metrics + PostHog pairing (MUL-2949) PR3 of the Grafana board metrics split (parent MUL-2328). Adds 23 new Prometheus counter/histogram families to the PR2 BusinessMetrics collector covering the activation/community/commercial funnels, and binds every PostHog event emission to a matching metric increment so the two sides cannot drift. Funnel: signup, workspace_created, team_invite_sent/accepted, onboarding_*, cloud_waitlist_joined. Content: issue_created, chat_message_sent, agent_created, squad_created, autopilot_created, issue_executed. Runtime: runtime_registered/ready/failed/offline + ready_seconds histogram, daemon_ws_message_received_total. Autopilot: autopilot_run_started/terminal/skipped. Webhook/GitHub: webhook_delivery_total, github_event_received_total, github_pr_review_total, github_pr_merge_seconds histogram. CloudRuntime: cloudruntime_request_total + duration histogram, wired through a small RequestRecorder interface so the cloudruntime package stays decoupled from metrics. Commercial: feedback_submitted, contact_sales_submitted. The pairing helper metrics.RecordEvent(client, m, ev) emits the PostHog event AND increments the matching counter via IncForEvent dispatch, reading labels from the analytics event Properties. Every existing h.Analytics.Capture(analytics.X(...)) call site has been migrated to the helper across handler/, service/, and cmd/server/runtime_sweeper.go. Lint enforcement (server/internal/metrics/business_pairing_test.go): - TestEveryAnalyticsEventHasPrometheusCounter: every Event* constant in analytics/events.go either dispatches via IncForEvent or is in the taskMetricEvents allow-list (PR2 typed RecordTask* methods). - TestNoNakedAnalyticsCaptureInHandlersOrServices: AST-walks handler/ service/cmd-server for direct Analytics.Capture(...) calls — only service/task.go's captureTaskEvent helper is allow-listed. - TestEveryAnalyticsRecordEventTakesAnalyticsHelper: validates the third arg of every metrics.RecordEvent call is built from analytics.*. Cardinality protection: all new label values pass through fixed allow-lists in labels_pr3.go; unknown values collapse to 'other'/'unknown'/'error'. Refs: - Spec MUL-2328 / MUL-2949. - Builds on PR2 (MUL-2948) — collectors registered through the same BusinessMetrics struct, no separate Registry. - Uses PR1's taskfailure.Reason (MUL-2946) for runtime_failed's failure_reason label via NormalizeFailureReason. Out of scope: Sampler-class metrics (PR4 / MUL-2947), pr_review_total emission point (no review event handler exists yet — counter is defined, TODO to wire up when /api/webhooks/github grows pull_request_review handling). Co-authored-by: multica-agent <github@multica.ai> * fix(server): tighten PR3 review items — signup_source bucket, fill platform/kind/form_source enums, onboarding_started server emission, lint scope (MUL-2949) Addresses 张大彪's review on #3698: 1. signup_source: NormalizeSignupSource added to labels_pr3.go with a fixed allow-list bucket (direct/google/twitter/linkedin/.../other). Parses JSON cookie payload for utm_source/source/referrer fields, strips URL schemes, maps well-known hostnames to channel buckets. PostHog event still ships the raw cookie value for analytics; only the Prometheus label is bucketed. 2. Filled the unknown/other label gaps: - analytics.IssueCreated and analytics.ChatMessageSent now take a platform parameter sourced from middleware.ClientMetadataFromContext (X-Client-Platform header) at the handler. Autopilot-originated issues stamp PlatformServer. - analytics.FeedbackSubmitted now takes a kind parameter; CreateFeedback reads req.Kind (default "general") so the picker selection lights up the metric's kind label instead of long-term "other". - analytics.ContactSalesSubmitted now takes a formSource (page / onboarding / agents_page); CreateContactSales reads req.Source. The metric reads ev.Properties["form_source"] so the analytics CoreProperties.Source ("marketing_contact_sales") stays backward-compat for PostHog dashboards. 3. analytics.OnboardingStarted helper added; server-side emission lives in PatchOnboarding, fired exactly once per user on the first PATCH that carries a non-empty questionnaire payload (firstTouch logic compares prior bytes against {} / null). Frontend onboarding_started keeps firing on page open; the server emission is what guarantees the Prometheus counter exists so Grafana can be cross-checked against the PostHog funnel without depending on the SDK roundtrip. 4. business_pairing_test.go tightened: - TestNoNakedAnalyticsCaptureInHandlersOrServices now allow-lists at function granularity (just captureTaskEvent in service/task.go), not whole-file. Any future naked Capture in the same file fails CI. - TestEveryAnalyticsRecordEventTakesAnalyticsHelper now does def-use tracking inside the enclosing FuncDecl: when RecordEvent's third arg is an *ast.Ident, the test walks the function body for the assignment that defined it and confirms the RHS is an analytics.<Helper>(...) call. Bare local idents that didn't originate from analytics are now caught. 5. gofmt -w applied across the touched files; gofmt -l clean. Tests: go test ./internal/metrics/... ./internal/analytics/... pass. Pre-existing TestClaimTask_/TestWebhook_MergedPR/TestDeleteIssueByIdentifier failures on origin/main are DB-environment-dependent and not regressions from this change. Co-authored-by: multica-agent <github@multica.ai> * fix(server): normalise onboarding_started platform label + regression test (MUL-2949) Addresses 张大彪's last review nit: - IncForEvent's EventOnboardingStarted case now wraps the platform property with NormalizePlatform, matching every other platform-bearing metric. A misbehaving frontend can no longer leak a raw X-Client-Platform header value into the multica_onboarding_started_total{platform=...} series. - New labels_pr3_test.go covers every PR3 normalizer with both a happy-path value and an unknown value, asserting the unknown collapses to the documented fallback bucket. Includes a focused regression for onboarding_started: emits one event with an attacker-shaped platform string and asserts the metric only exposes web + unknown label values (no raw header bleed). - testutil.go gains a small GatherForTest helper so the regression test can pull the typed MetricFamily map without re-implementing the registry-walk dance. Co-authored-by: multica-agent <github@multica.ai> * fix(server): NormalizeTaskSource on workspace_created + document lint limitations (MUL-2949) Final review touch-ups before merge: - IncForEvent's EventWorkspaceCreated case wraps source through NormalizeTaskSource, matching the other source-bearing dispatches (issue_created, agent_created, issue_executed). Closes the last raw property leak in the dispatcher table. - business_pairing_test.go inline docstrings now spell out the two known limitations of the lint gate that 张大彪 / Eve flagged: analyticsBackedIdents matches by ident NAME (not SSA def-use, so a nested-scope shadow could pass) and isMetricsRecordEvent hard-codes the import alias set. PR description carries a Follow-ups section with the same two items so the work is visible after merge. Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: 魏和尚 <agent+wei@multica.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Bohan Jiang
|
5900d8b637 |
fix(issues): make start_date/due_date timezone-stable calendar days (#3618) (#3692)
* fix(issues): store start_date/due_date as DATE, not timestamp (MUL-2925) These fields are calendar days (the pickers offer no time-of-day), but were stored as TIMESTAMPTZ. A client serializing local midnight via toISOString() folded its timezone into the instant, so the day shifted by the local offset (GH #3618). Migrate the columns to DATE and parse/serialize date-only "YYYY-MM-DD". ParseCalendarDate still accepts legacy RFC3339 (truncated to the UTC day) so older clients keep working. Co-authored-by: multica-agent <github@multica.ai> * fix(issues): render start_date/due_date as timezone-stable calendar days (MUL-2925) Pickers now emit date-only "YYYY-MM-DD" (local calendar day) instead of toISOString(), and every read formats via the shared @multica/core/issues/date helpers with timeZone:"UTC" so the day never shifts with the viewer's offset. The Gantt's existing UTC bucketing is now correct. Covers web/desktop pickers, quick-set menu, list/board/detail/activity, and the mobile due-date picker. Co-authored-by: multica-agent <github@multica.ai> * fix(issues): address date-only review — loud-fail ambiguous dates, finish display sweep (MUL-2925) Review follow-ups on #3692: - ParseCalendarDate no longer silently truncates a legacy non-midnight RFC3339 to the wrong UTC day; it accepts only YYYY-MM-DD or an exact UTC-midnight instant and rejects ambiguous ones loudly. Adds util unit tests. - migration 112 pins the TIMESTAMPTZ->DATE conversion to UTC explicitly via AT TIME ZONE 'UTC' (was session-timezone dependent); down migration too. - Convert remaining date-change display sites to formatDateOnly: inbox detail label (web) and mobile activity + inbox labels (were new Date()+local format). - CLI --start-date/--due-date help now says YYYY-MM-DD, not RFC3339. Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: J <j@multica.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Bohan Jiang
|
90ddfb04e2 |
feat(self-host): DISABLE_WORKSPACE_CREATION env var (MUL-2777) (#3441)
* feat(self-host): DISABLE_WORKSPACE_CREATION env var (MUL-2777, #3433) When self-hosters set DISABLE_WORKSPACE_CREATION=true, POST /api/workspaces returns 403 for every caller and the UI hides every "Create workspace" affordance (sidebar, modal, /workspaces/new page, onboarding Step 2). This closes the gap where ALLOW_SIGNUP=false still let any signed-in user open an isolated workspace the platform admin couldn't see. - server: new Config.DisableWorkspaceCreation, gate in CreateWorkspace, workspace_creation_disabled in /api/config, Go tests. - frontend: new workspaceCreationDisabled in configStore, hide sidebar entry, swap NewWorkspacePage / CreateWorkspaceModal / onboarding StepWorkspace to a "creation disabled, ask for invite" state when the flag is on, EN + zh-Hans locale strings. - ops: .env.example, docker-compose.selfhost, helm values + configmap, SELF_HOSTING.md, SELF_HOSTING_ADVANCED.md, environment-variables docs (EN + zh). Co-authored-by: multica-agent <github@multica.ai> * fix(onboarding): drive create path off workspaceCreationAllowed (#3433) PR #3441 review: when DISABLE_WORKSPACE_CREATION=true and the user already has a workspace, StepWorkspace still walked the resume copy (`headline_resume` / `lede_resume` mentioning "or start another") and `creatingActive` ignored the flag, leaving a stale clickable create CTA possible if /api/config arrived late. Refactor StepWorkspace to derive a single `workspaceCreationAllowed` boolean from the config store. It now drives: - Initial `mode` state (defaults to "existing" when disabled + reusing so the CTA is pre-armed for the only valid action). - `creatingActive` so the footer CTA cannot fall back into the create branch even mid-render. - Eyebrow / headline / lede strings — adds `creation_disabled_{eyebrow,headline,lede}_resume` (EN + zh-Hans) for the disabled + reusing variant. Tests: cover the three reachable shapes — flag off + no existing, flag on + no existing, flag on + existing. Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: J <j@multica.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Bohan Jiang
|
13f74e651a |
feat(agents): remove custom_env from agent resources, add audited env endpoint (MUL-2600) (#3209)
* feat(agents): remove custom_env from agent resources, add audited env endpoint (MUL-2600)
The agent resource shape (list / get / create / update / archive /
restore responses + WebSocket events) no longer carries `custom_env`
values. Reads/writes of env now flow exclusively through a dedicated
`/api/agents/{id}/env` endpoint that is owner/admin-only, rejects
agent-actor sessions, applies a "****" sentinel preserve guard on
PUT, and writes a persistent audit row per reveal/update.
Why
- `multica agent list --output json` historically returned plaintext
`custom_env` for owner/admin callers (the redaction gate gave only
members the masked map). Any agent token running on the workspace
inherits its owner's role and could read every other agent's
secrets just by listing.
- Patching list/get redaction alone (PR #3175 direction) left
symmetric leaks via mutation responses, WS events, the "reveal"
path itself (no actor-aware auth), and a `****` overwrite footgun
on UpdateAgent.
What changed
- Backend: drop `custom_env` from AgentResponse; add coarse
`has_custom_env` + `custom_env_key_count`. Strip env handling from
UpdateAgent (silently ignored if sent). Keep CreateAgent's
custom_env acceptance.
- Backend: new GET/PUT `/api/agents/{id}/env` handlers in
`internal/handler/agent_env.go`:
- resolveActor → 403 for agent actors (closes the lateral-movement
path).
- Owner/admin role gate via existing helper.
- PUT honours value == "****" as "preserve existing value".
- Both write to `activity_log` with `agent_env_revealed` /
`agent_env_updated` actions. Audit details record key names only,
never values.
- Daemon claim path (`ClaimAgentTask`) unchanged — `TaskAgentData`
still carries plaintext env for runtime injection.
- SQL: new `UpdateAgentCustomEnv` query; sqlc regenerated (v1.31.1).
- CLI: new `multica agent env get|set` subcommands. `--custom-env*`
flags removed from `multica agent update`; the no-fields error
now points to the new path.
- Frontend: drop env fields from `Agent` + `UpdateAgentRequest`; add
`getAgentEnv` / `updateAgentEnv` client methods; rewrite env-tab
to show "N variables configured" + explicit "Reveal & edit"
button, fetching values only on intentional reveal.
- Locales: parity-safe additions to en + zh-Hans.
- Docs: agents-create.{mdx,zh.mdx} reflect the new threat model and
endpoint.
- Mobile: schema drops `custom_env` / `custom_env_redacted`, adds
metadata fields.
Tests
- Handler tests pinned the new invariants: no env in list/get
responses, owner reveal happy-path + audit row, agent-actor 403,
`****` sentinel preserves real values, UpdateAgent silently
ignores `custom_env`, pure `mergeAgentEnv` cases.
- CLI tests pivot to the new flag surface: `agent update` MUST NOT
expose the env flags; `agent env set` MUST expose
--custom-env-stdin/--custom-env-file.
- Frontend test fixtures updated; pnpm typecheck / test / lint
pass cleanly.
This is a breaking API change. Scripts that read `custom_env` from
`/api/agents` must migrate to `GET /api/agents/{id}/env`.
Co-authored-by: multica-agent <github@multica.ai>
* fix(agents): close actor-spoofing + audit fail-closed in env endpoints (MUL-2600)
Addresses Elon's review of #3209:
* Mint a task-scoped `mat_` token per claim, bound to (agent, task,
workspace, owner). Daemon injects it into the agent process in place
of its own credential. Auth middleware authoritatively rebuilds
X-User-ID / X-Agent-ID / X-Task-ID from the token row and sets
X-Actor-Source=task_token; that header is server-set only — incoming
values are stripped before any auth branch runs. resolveActor honors
the header so an agent that strips X-Agent-ID / X-Task-ID still
resolves as actor=agent.
* GetAgentEnv / UpdateAgentEnv are now fail-closed on audit-log
failures: GET refuses to return plaintext, PUT persists inside the
same tx as the audit row so they commit/roll back together.
* PUT /api/agents/{id} returns 400 when the body carries custom_env
instead of silently dropping it — directs callers to the audited env
endpoint.
* Agent actors never see mcp_config, even when the underlying member
is owner/admin; mutation broadcasts go through a redaction shim so
WS subscribers don't pick it up either.
* Fix backend test that asserted dense JSON (jsonb::text renders
whitespace) and frontend test that assumed a unique "Test User"
match.
Co-authored-by: multica-agent <github@multica.ai>
* fix(agents): close residual MUL-2600 gaps from review (MUL-2600)
Migration 108 FK now correctly references agent_task_queue(id) instead
of the non-existent agent_task table; the previous name blocked CI
backend migrations.
Task-token-authenticated requests can no longer be re-routed at a
different workspace by passing workspace_slug / workspace_id /
?workspace_id / a URL workspace param. ResolveWorkspaceIDFromRequest
and resolveWorkspaceUUID both short-circuit on X-Actor-Source=task_token
and return only the token-bound X-Workspace-ID; buildMiddleware adds a
defence-in-depth 403 if any URL-resolved workspace disagrees with the
token binding.
mcp_config no longer leaks back to agent actors through UpdateAgent /
CreateAgent / ArchiveAgent / RestoreAgent HTTP responses — the same
redactAgentResponseForActor helper that GetAgent/ListAgents use is now
applied to mutation responses too. WS broadcasts were already redacted
via broadcastAgentResponse.
FailTask and every TaskService cancel path (CancelTask /
CancelTasksForIssue / CancelTasksForAgent / CancelTasksByTriggerComment
/ BroadcastCancelledTasks) now eagerly DeleteTaskTokensByTask so the
mat_ token's 24h window doesn't outlive a terminated task. Failure is
non-fatal — the FK cascade and expiry remain durable guards.
Doc-only: clarify that PUT /api/agents/{id} now hard-rejects bodies
that carry custom_env (was previously "silently ignores").
Tests:
- middleware: TestResolveWorkspaceIDFromRequest gains a task_token
case asserting client-supplied slug/id/query cannot override the
bound workspace.
- handler: TestUpdateAgent_RedactsMcpConfigForAgentActor and
TestUpdateAgent_KeepsMcpConfigForMemberActor pin the mutation-
response redaction contract per actor type.
Co-authored-by: multica-agent <github@multica.ai>
* fix(agents): match redacted mcp_config as JSON null, not Go nil (MUL-2600)
`AgentResponse.McpConfig` is `json.RawMessage` without `omitempty`, so
the redacted response serialises as `"mcp_config": null`. On decode,
`json.RawMessage` keeps the literal bytes `null` rather than collapsing
to Go nil, which made the assertion fire on a non-leak.
The product contract (field always present, distinguished from "no
config" via `mcp_config_redacted`) is intentional, so adjust the test
to check for "no secret-bearing content" instead of weakening the
contract via `omitempty`.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: multica-agent <github@multica.ai>
|
||
|
Bohan Jiang
|
0c767c0052 |
feat(issues): per-issue metadata KV (MUL-2017) (#2845)
* feat(issues): per-issue metadata KV (MUL-2017)
Adds a small JSONB KV map to every issue for agent pipeline state (attempts,
PR number, pipeline status, ...). Keys match a narrow regex, values are
primitives (string / number / bool), capped at 50 keys per issue and 8KB
per blob. Defense-in-depth via two CHECK constraints (object shape + size).
All mutations are single-key atomic (jsonb_set / `- key`). `UpdateIssue`
intentionally does NOT touch metadata: a whole-blob overwrite would race
with concurrent agent writes.
GET /api/issues/:id/metadata
PUT /api/issues/:id/metadata/:key body: { "value": <primitive> }
DELETE /api/issues/:id/metadata/:key
Containment filter on list: GET /api/issues?metadata=<json-object> uses
PG `@>` against a `jsonb_path_ops` GIN index. Mirrored across ListIssues,
CountIssues, ListOpenIssues, and the hand-rolled ListGroupedIssues SQL so
CLI/API and UI grouped views stay consistent.
CLI: multica issue metadata {list,get,set,delete}
multica issue list --metadata key=value (repeatable, AND)
set has --type to override the default value-sniffing
Co-authored-by: multica-agent <github@multica.ai>
* fix(issues): metadata test bugs + wire realtime + read-only display (MUL-2017)
- Fix two failing handler tests blocking backend CI:
- reset decode target after delete so map merge does not mask removal
- url.PathEscape the key segment so spaces no longer panic NewRequest
- Wire issue_metadata:changed end to end so the detail / list / my-issues
caches stay in sync with set/delete events (other tabs, CLI writes).
- Add a read-only Metadata strip to the issue detail sidebar; hidden when
the issue has no keys so it stays quiet in the common case.
Co-authored-by: multica-agent <github@multica.ai>
* feat(runtime): teach agents to read/write issue metadata (MUL-2017)
Add an `## Issue Metadata` section to the runtime brief plus a
`metadata list` step on entry and a `metadata set`/`delete` step on
exit. Section only emits when the task carries an issue id (comment- or
assignment-triggered); chat / quick-create / run-only autopilot stay
clean so they don't fire failing CLI calls.
Co-authored-by: multica-agent <github@multica.ai>
* fix(issues): bump metadata migration to 105 and drop attempts as example (MUL-2017)
main is now at 104_drop_runtime_timezone; the migrator picks
LatestVersion() by sorted filename, so a slot before the tail would
let DBs that have already run 099–104 think they're up-to-date while
the issue.metadata column is missing — runtime would then fail with
column does not exist. Renumbering to 105 puts the migration at the
tail and forces it to run.
Also drop attempts as a positive example across docs/code comments and
test fixtures — the runtime instruction prompt already lists it under
"What NOT to pin" (runtime bookkeeping). Replace with pr_number, which
is in the recommended-keys set, so docs/tests speak the same language
as the prompt.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: multica-agent <github@multica.ai>
|
||
|
YYClaw
|
614dfae884 |
MUL-2488 feat(timezone): Scheduling / Viewing two-layer timezone architecture (#2968)
* docs(timezone): add scheduling/viewing timezone architecture RFC * feat(db): replace daily rollups with task_usage_hourly, add user.timezone Migrations 100-104: add "user".timezone (Viewing tz), build the UTC hourly task_usage_hourly rollup with its pipeline, drop the legacy task_usage_daily / task_usage_dashboard_daily pipelines, and drop the agent_runtime.timezone column. Report queries now slice day boundaries at read time by the caller-supplied @tz instead of materialising in a fixed tz. Regenerate sqlc. * feat(server): add task_usage_hourly backfill command Replace the two legacy backfill commands (daily / dashboard_daily) with a single backfill_task_usage_hourly that loads historical task_usage into the new UTC hourly rollup, sliced per workspace. * refactor(server): resolve viewing timezone in report handlers Report handlers resolve the Viewing tz per request (?tz query param, then user.timezone, then UTC) and pass it to the hourly-rollup queries. Drop the UseDailyRollup feature flags and the old raw-scan/daily-rollup dual paths, remove the /api/usage endpoints, and stop the daemon from reporting and the runtime handler from accepting host timezone. * refactor(core): switch report queries to viewing timezone API client and dashboard/runtime queries send ?tz with each report request, the user schema/types carry the new timezone field, and the runtime timezone field/mutation is removed. * feat(views): add viewing timezone preference and UI Add the useViewingTimezone hook and a Timezone setting in Preferences; report charts and the dashboard week boundary follow the viewer tz. Remove the runtime detail timezone editor and its locale strings. * fix(test): update fixtures and stabilize tests for timezone refactor The timezone architecture refactor changed several types without updating dependent test code: - RuntimeDevice no longer has a timezone field — drop it from the create-agent-dialog runtime fixture. - User now requires a timezone field — add it to the apps/web mockUser fixture. - The PreferencesTab timezone tests asserted on the async save handler (PATCH then store update) with a bare expect, racing the mutation's settle callback, and timed out querying the Select's ~600-option IANA list on a loaded CI runner. Wrap the assertions in waitFor and extend the timeout for those three tests. * docs(timezone): document self-host migration order and trigger invariant Add a SELF-HOST UPGRADE ORDER runbook to the backfill command's package comment: applying migrations 100-104 in a single migrate-up drops the legacy daily rollups before the hourly backfill runs, leaving dashboards empty until cron catches up. Add an INVARIANT comment on trg_atq_dirty_hourly noting that agent_id must be added to the trigger's OF list if it ever becomes mutable, otherwise dirty buckets for the old agent_id are silently missed. * style(runtimes): drop trailing blank line in runtime-detail |
||
|
Multica Eve
|
41cb91abd9 |
feat: add cloud runtime fleet proxy API (MUL-2453) (#2986)
* feat: add cloud runtime fleet proxy API Co-authored-by: multica-agent <github@multica.ai> * test: cover cloud runtime handler nits Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Eve <eve@multica-ai.local> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Zohar Babin
|
15152c6ccd |
feat(auth): cache workspace membership for daemon heartbeat path (MUL-2247) (#2638)
* feat(auth): cache workspace membership for daemon heartbeat path Cache workspace membership existence (not role) in Redis to eliminate a DB round-trip on every PAT-authenticated daemon heartbeat. Follows the existing PATCache nil-safe pattern. Key design decisions per reviewer feedback: - Cache existence only (sentinel "1"), not role string. Authorization decisions that depend on role always hit the DB directly. This eliminates the cache-aside race where a stale elevated role could persist after a downgrade. - Proactive invalidation on UpdateMember, DeleteMember, LeaveWorkspace, and DeleteWorkspace (iterates members before cascade delete). - 5 min TTL. Combined with PATCache (10 min), worst-case revocation delay is max(10m, 5m) = 10 min — consistent with original PATCache design decision. Limitations: - Non-members still hit DB on every request (negative caching not implemented — the scenario is rare for daemon endpoints which require valid workspace-scoped tokens). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: multica-agent <github@multica.ai> * test(auth): drive membership cache invalidation through real handlers - TestRequireDaemonWorkspaceAccess_CacheHit now uses a ghost user with no member row, so the only path to a granted access is the cache short-circuit. Without priming the cache the access check must fail; with priming it must succeed. A future change that bypasses the cache would fail the second assertion. - Replaces the cache-only InvalidatedOnMemberRemoval test (which only re-exercised the auth-package primitive) with four handler-driven tests that exercise DeleteMember, UpdateMember, LeaveWorkspace and DeleteWorkspace via their real HTTP handlers. Each test prepares a real member, primes the cache, calls the handler, and asserts the cache entry is gone — so a refactor that drops one of the Invalidate(...) calls in workspace.go will fail CI. Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: multica-agent <github@multica.ai> Co-authored-by: Jiang Bohan <bhjiang@outlook.com> |
||
|
Kerim Incedayi
|
9418d2a2c1 |
feat(autopilots): webhook triggers (server + CLI + UI + docs) MUL-2049 (#2348)
* feat(server): add webhook trigger DB migration + sqlc queries
Lays the foundation for webhook autopilot triggers:
- partial unique index on autopilot_trigger.webhook_token (kind=webhook only)
so the public ingress route can resolve a trigger in O(1)
- GetWebhookTriggerByToken / TouchAutopilotTriggerFiredAt /
RotateAutopilotTriggerWebhookToken / SetAutopilotTriggerWebhookToken
queries, regenerated with sqlc
* feat(server): webhook token generator + payload normalizer
Two pure helpers for the webhook autopilot work:
- generateWebhookToken: 32 random bytes -> base64-url, "awt_" prefix.
256 bits of entropy keeps brute-force off the table; the prefix makes
leaked tokens recognisable in logs.
- normalizeWebhookPayload: turns arbitrary JSON into the WebhookEnvelope
shape (event/eventPayload/request) used by trigger_payload. Header- and
body-based event inference covers GitHub, GitLab, X-Event-Type, and
caller-provided envelopes; scalar/empty/invalid bodies are rejected so
the handler can answer 400.
* feat(server): generate webhook tokens and expose rotate endpoint
- New handler.Config.PublicURL fed by MULTICA_PUBLIC_URL env so
/api/autopilots/.../triggers responses can include an absolute
webhook_url alongside the always-present webhook_path.
- CreateAutopilotTrigger now mints a webhook_token via crypto/rand
for kind=webhook and ignores cron/timezone for non-schedule kinds.
api triggers stay accepted-but-inert per PLAN.md.
- New POST /api/autopilots/{id}/triggers/{triggerId}/rotate-webhook-token
protected by the existing workspace auth group; old tokens stop
working immediately because the unique-index lookup keys on the
current row value.
* feat(server): public webhook ingress route + per-token rate limiter
- New POST /api/webhooks/autopilots/{token} route, mounted outside the
authenticated group: the path token is the credential. Workspace
context is derived from the joined autopilot row, never headers.
- Body capped at 256 KiB via http.MaxBytesReader; oversized payloads
return 413 mid-read instead of being fully buffered.
- Disabled triggers / paused / archived autopilots return
200 {"status":"ignored"} so providers stop retrying.
- Skipped-runtime dispatches surface 200 {"status":"skipped"} with the
reason from the autopilot service's pre-flight admission check.
- WebhookRateLimiter interface with sliding-window in-memory + Redis
Lua-script implementations. Default 60 req/min per token. Test
coverage on the in-memory path; Redis variant fails open on cache
errors so a Redis hiccup never blocks ingress.
- Integration tests exercise token generation, dispatch, payload
envelope persistence, GitHub-header inference, paused/disabled
short-circuits, oversized rejection, and rotate-then-old-token-404.
* feat(server): include webhook payload in create_issue description
When an autopilot run is triggered by a webhook and execution_mode is
create_issue, the agent only sees the issue body — never the run's
trigger_payload. Append a 'Webhook event:' line and a fenced JSON block
with the normalized eventPayload so the agent has the inbound context
inline. Schedule / manual runs are unchanged.
Tests cover:
- schedule path keeps existing italic note, no webhook block
- webhook path emits event line + payload block, italic before block
- non-envelope JSON falls back to raw body (defensive)
- non-webhook source with payload still gets no webhook block
* feat(core): types, API client and mutations for webhook triggers
- AutopilotRunStatus gains 'skipped' so the run-list UI handles the
admission-skipped state explicitly instead of falling through to a
generic case (the backend already emits it via MUL-1899).
- AutopilotTrigger picks up optional webhook_path / webhook_url. Both
are optional so older self-hosted servers that pre-date this change
still parse cleanly.
- buildAutopilotWebhookUrl helper composes a usable absolute URL with
the priority webhook_url > apiBaseUrl + path > origin + path > path.
Tested with seven cases covering each branch.
- ApiClient.rotateAutopilotTriggerWebhookToken posts to
/api/autopilots/{id}/triggers/{triggerId}/rotate-webhook-token; the
HTTP-contract test pins URL + method.
- useRotateAutopilotTriggerWebhookToken mutation invalidates
autopilotKeys.detail on settle, mirroring the existing trigger-mutation
pattern.
* feat(views): webhook trigger UI in Add Trigger dialog and trigger row
Add Trigger dialog gains a Schedule/Webhook segmented toggle:
- Schedule reuses TriggerConfigSection unchanged.
- Webhook hides the cron config and shows a help line; the trigger is
created with kind=webhook and the URL is generated server-side.
- Toast text differentiates schedule vs webhook on success.
TriggerRow grows a webhook branch:
- Webhook icon, kind translated via trigger_kind.
- URL shown in a truncating monospace pill, with copy + rotate
buttons. Copy uses navigator.clipboard with toast feedback; rotate
uses an AlertDialog confirm because the old URL stops working
immediately.
- api triggers render a Deprecated badge and skip URL/copy/rotate
affordances.
RunRow gains a 'skipped' RUN_VISUAL entry (muted dash) so admission-
skipped runs don't fall through to a generic case. Source label uses the
new run_source i18n key instead of capitalize.
Locales: en + zh-Hans gain run_status.skipped, run_source.*,
trigger_kind.*, trigger_row.{copy_url,rotate_url,*_confirm_*,toast_*},
add_trigger_dialog.{type_*,webhook_help,toast_added_{schedule,webhook}}.
* feat(cli): support webhook trigger creation and URL rotation
- multica autopilot trigger-add now takes --kind schedule|webhook
(default schedule for backward compatibility). For webhook it skips
--cron / --timezone validation and prints the resulting webhook URL,
preferring the server-provided webhook_url and falling back to
client.BaseURL + webhook_path.
- New multica autopilot trigger-rotate-url <autopilot-id> <trigger-id>
command for rotating the bearer URL of a webhook trigger.
* docs(autopilots): add webhook trigger guide (en + zh)
Replaces the 'Webhook and API triggers are not available yet' section
with end-to-end webhook documentation: how the URL is generated, what
payload shapes are accepted, the inferred-event rules, the bearer-secret
warning + rotate flow, status-code semantics for accepted/skipped/
ignored/4xx/5xx outcomes, and the MULTICA_PUBLIC_URL self-host
configuration.
Run history list now mentions skipped status. The 'unavailable
features' section narrows to api-kind triggers, HMAC signing, IP
allowlists, and provider presets.
* feat(views): add Schedule/Webhook toggle to the create autopilot dialog
Closes the gap where a brand-new autopilot could only be created with a
schedule trigger. The right-column config now has a Trigger section
with a segmented Schedule/Webhook control:
- Schedule keeps the existing cron/timezone UI.
- Webhook hides the cron UI and shows a help line; on submit, a
kind=webhook trigger is created right after the autopilot.
In edit mode the toggle is intentionally hidden (PLAN.md treats trigger-
type changes as delete-old + create-new, not in-place updates), but the
panel still picks the right kind based on props.triggers[0].kind so a
webhook autopilot doesn't render an irrelevant cron form.
Locales: section_trigger_kind, trigger_kind_{schedule,webhook},
section_webhook, webhook_help_{create,edit} added in en + zh-Hans.
* feat(views): show webhook URL inline after creating a webhook autopilot
After a successful create with kind=webhook, the dialog stays open and
swaps to a confirmation panel showing the freshly minted URL with a
copy button + 'Treat this URL like a password' warning + Done button.
Avoids the friction of "create the autopilot, then go find it in the
list, click in, scroll to triggers, copy URL."
Locales: dialog.webhook_created_{title,description,warning,done} added
in en + zh-Hans.
Schedule create flow is unchanged (toast + close). The success panel is
gated on the trigger returned from the create mutation, so a partial
failure (autopilot created, trigger creation errored) still falls
through to the toast_create_partial path.
* feat(views): show webhook payload in run detail dialog
The agent transcript dialog now accepts an optional headerSlot that
sits above the event list. The autopilot RunRow drops a
WebhookPayloadPreview into that slot when the run came from a webhook
and trigger_payload is non-empty.
The preview is collapsed by default (the transcript itself is the main
event), shows the inferred event name + receivedAt in the header, and
reveals the eventPayload as pretty-printed JSON with a copy button on
expand. Falls back gracefully if the row's trigger_payload doesn't
match the WebhookEnvelope shape — the whole value is shown instead so
nothing is hidden.
Closes the "agent didn't echo the payload, now I can't see what
triggered the run" gap. PLAN.md tracked this as
"Payload preview in run history" under follow-ups.
Locales: webhook_payload.{label, unknown_event, payload, content_type,
copy, copied, copied_short, copy_failed} added in en + zh-Hans.
* chore(server): wire MULTICA_PUBLIC_URL through self-host compose
Two small follow-ups split out of the webhook trigger PR:
- docker-compose.selfhost.yml passes MULTICA_PUBLIC_URL into the
backend container so a self-hosted deployment behind a real domain
gets absolute webhook URLs in the trigger response. Documented in
.env.example with the rationale for not deriving the public host
from request headers.
- Drop a duplicated 'invalid json:' prefix in the webhook ingress
400 error path. normalizeWebhookPayload already prefixes its
errors, so the handler doesn't need to re-prefix.
* fix(migrations): renumber webhook trigger migration 081 → 089 to avoid collision
The branch's 081_autopilot_webhook_triggers.{up,down}.sql collided
numerically with 081_runtime_timezone.{up,down}.sql that landed on
main, making migration apply order undefined. Renumber to 089 so the
file slots after the latest main migration (088_squad_instructions).
The SQL itself doesn't conflict — it only creates a partial unique
index on autopilot_trigger.webhook_token — but the duplicate prefix
is what the migration runner sees, so the filename must move.
* fix(autopilot-webhook): address PR review blocking issues
- Redact bearer tokens from request logs: paths matching
/api/webhooks/autopilots/<token> now log "[redacted]" instead of the
token. The resolved trigger ID is plumbed via context so audit lines
stay useful for debugging. (Review item Blocking #1.)
- Distinguish pgx.ErrNoRows from transient DB errors in token lookup:
no-row stays 404 (so providers don't retry on a deleted webhook),
other errors return 500 (which providers DO retry, avoiding silent
drops on DB blips). (Review item Blocking #2.)
- Add per-IP sliding-window rate limiter that runs BEFORE the token
lookup, so spraying random tokens can no longer probe the
autopilot_trigger index unboundedly. Reuses the existing Lua script
with a separate Redis key namespace; falls open on Redis errors.
Default budget 30 req/min/IP. (Review item Blocking #3.)
The webhook handler now applies the gates in the order: per-IP rate
limit → token lookup → per-token rate limit → handler logic.
* fix(autopilot): atomic webhook trigger creation + strict kind/timezone validation
- Mint the webhook bearer token BEFORE the INSERT and pass it via
CreateAutopilotTriggerParams so the row never exists in a half-written
kind=webhook + webhook_token=NULL state. On the (vanishingly rare)
unique-index collision the whole INSERT is retried with a fresh token
— no UPDATE second step. Removes the now-dead attachFreshWebhookToken
helper. (Review item Recommended #4.)
- Add new GET /api/autopilots/{id}/runs/{runId} endpoint that returns a
single run including the full trigger_payload. The list response is
now slim (omits trigger_payload) so worst-case payload size drops
from ~5 MB to ~5 KB. (Review item Recommended #5, server side.)
- Reject kind=api with 400 ("kind=api is deprecated; use schedule or
webhook") and reject kind=webhook with --timezone with 400 — both
surfaces stragglers loudly instead of silently dropping fields.
CLI mirrors the check so --timezone with --kind webhook errors
client-side. (Review nits.)
- Add --yes (-y) flag and an interactive y/N confirmation prompt to
`multica autopilot trigger-rotate-url` so the destructive rotate
matches the UI's AlertDialog safety. (Review item Recommended #6.)
* fix(views): fetch webhook payload on-demand and truncate at 4 KiB
- Add useAutopilotRun query hook + getAutopilotRun API client method
paired with the new server endpoint. The run-detail dialog now mounts
a WebhookPayloadSlot that fetches the full run (incl. trigger_payload)
lazily — list responses no longer carry up to 256 KiB × N runs of
envelope data.
- WebhookPayloadPreview truncates its in-DOM <pre> at 4 KiB with a
localized marker so jank-y machines aren't asked to render a 256 KiB
JSON blob. The Copy button still yields the full string.
- Adds the truncated_marker i18n string to en + zh-Hans.
Review items Recommended #5 (frontend) and a nit on the preview's
unbounded <pre>.
* test(autopilot-webhook): close coverage gaps flagged in PR review
- request_logger: redactWebhookPath unit tests + integration test
proving the bearer token never lands in slog output, plus the
webhook_trigger_id context plumbing.
- autopilot_webhook_handler: empty body → 400, archived autopilot →
200 ignored, per-IP rate limiter trips before DB lookup, kind=api
and webhook+timezone are rejected at 400, slim list + full detail
endpoint round-trip.
- webhook_rate_limiter: Lua script structure guard (catches reordering
even without a live Redis), plus live-Redis tests for both per-token
and per-IP limiters (REDIS_TEST_URL gated, matching the existing
Redis test pattern in the package).
- WebhookPayloadPreview: envelope rendering, fallback shape, and the
>4 KiB truncation path with full-payload-on-Copy guarantee.
Two branches are documented as code-review-protected rather than
covered by tests: the 500-on-DB-error path requires injecting a stub
Queries (no interface here), and the cross-workspace defense-in-depth
check is unreachable from valid SQL state.
* fix(middleware): SetWebhookTriggerID must mutate request in place
The round-1 helper returned a fresh *http.Request from WithContext, and
the webhook handler did `r = SetWebhookTriggerID(r, ...)`. That swaps
the handler's local pointer but doesn't propagate the new context back
to RequestLogger, which is still holding the original *http.Request —
so the audit line never actually included webhook_trigger_id in
production. The round-1 test happened to pass because it pre-stashed
the value on the request before calling ServeHTTP, bypassing the bug
it was meant to verify.
Switch to in-place mutation via `*r = *r.WithContext(...)` so the
wrapping middleware sees the new context after next.ServeHTTP returns,
and update the test to exercise the real call pattern (set the context
from inside the handler, assert the surrounding logger reads it).
Verified live: an accepted webhook now logs
path=/api/webhooks/autopilots/[redacted] webhook_trigger_id=<uuid>
* fix(autopilot-webhook): symmetric ErrNoRows split + trusted-proxy gate
Round-2 review (Bohan-J, PR #2348 follow-up):
- Must-fix #1: the second lookup at autopilot_webhook.go:258
(GetAutopilot after the token resolves) was folding every error into
404. A transient DB blip would tell a webhook sender "not found" and
it would never retry. Apply the same errors.Is(err, pgx.ErrNoRows)
→ 404 / else → 500 split as the first lookup got in round 1.
- Must-fix #2: clientIPForRateLimit was honoring X-Forwarded-For /
X-Real-IP from any caller. An attacker spraying random tokens could
just rotate the XFF header and the per-IP bucket became per-request,
so the limiter that's specifically supposed to gate spraying before
it hits the DB unique index was bypassed.
New shape — matches Bohan's suggestion exactly:
* Default: r.RemoteAddr only, headers ignored.
* Operator opt-in via MULTICA_TRUSTED_PROXIES (comma-separated
CIDRs). XFF/X-Real-IP are honored only when r.RemoteAddr is
inside one of the listed prefixes; otherwise they're dropped.
Wired through .env.example and docker-compose.selfhost.yml so
self-host operators can configure their reverse-proxy's CIDR.
Invalid CIDRs in the env var are dropped with a single slog.Warn at
startup rather than crashing the server. Uses net/netip (stdlib,
value-typed) for parsing and containment checks.
Verified live on the rebuilt self-host backend: a 35-request spray
from one source with rotating XFF gets the expected 30× 404 + 5× 429,
proving the per-IP bucket is keyed on the real connection IP.
* fix(autopilot): reject cron/timezone PATCH on non-schedule triggers
Round-2 review should-fix. CreateAutopilotTrigger already 400s on
kind=webhook + timezone/cron_expression, but UpdateAutopilotTrigger
silently wrote those fields regardless of prev.Kind. The values then
sat in the DB visible to nobody and read by nothing — a back door that
left the API contract fuzzy across create vs update.
Mirror the create-path discipline: after loading prev, if prev.Kind
!= "schedule" and the PATCH body sets cron_expression or timezone,
return 400 with a clear message. enabled and label remain accepted on
every kind.
The existing prev.Kind == "schedule" guard on next_run_at recompute
stays as belt-and-braces, but with this gate in place the recompute
branch is now reachable only for the kind it was meant for.
* test(autopilot-webhook): close round-2 coverage gaps
- IPRateLimitNotBypassedByXFFSpoof: drives the must-fix #2 invariant
by rotating XFF across three calls from the same RemoteAddr and
asserting the third gets 429. Pre-round-2 this test would have
passed for the wrong reason (limiter trusted XFF, so per-bucket
collision was incidental); now it pins the bypass-closed property.
- IPRateLimitReturns429BeforeDBLookup: updated to set RemoteAddr
explicitly and drop the XFF header it was leaning on. With
TrustedProxies empty (test default) the limiter keys on the real
connection IP, which is what the test wants to assert anyway.
- UpdateAutopilotTrigger_RejectsCronExpressionOnWebhookKind +
UpdateAutopilotTrigger_RejectsTimezoneOnWebhookKind: drive the
round-2 should-fix from the handler boundary.
- UpdateAutopilotTrigger_AcceptsEnabledAndLabelOnWebhookKind: counter
test so a regression to a blanket reject is caught.
* fix(migrations): bump webhook trigger migration 089 → 091
origin/main added 089_squad_no_action_activity_index (and 090_task_is_leader)
since our last rebase, re-colliding with our 089_autopilot_webhook_triggers.
Bump to 091 so the filename ordering is unambiguous again. The SQL is
unchanged — same partial unique index on autopilot_trigger.webhook_token —
only the filename moves.
* fix(views): dedupe skipped icon in autopilot RUN_VISUAL after rebase
The rebase against origin/main merged main's add of `Ban` for the
skipped status next to our round-1 `MinusCircle` entry, leaving the
RUN_VISUAL map with two `skipped` keys (only the last would have been
read at runtime, and MinusCircle had been dropped from the imports
during conflict resolution — so the file would not compile).
Keep main's `Ban` icon (latest design) and a single `skipped` entry.
Carry over the round-1 comment about why the muted styling matters
for failure-ratio readability.
---------
Co-authored-by: Kerim Incedayi <kerim.incedayi@digitalchargingsolutions.com>
|
||
|
Bohan Jiang
|
96695a79c5 |
feat(dashboard): workspace/project token + run-time dashboard MUL-1882 (#2462)
* feat(dashboard): workspace/project token + run-time dashboard
Add a `/{slug}/dashboard` page showing per-agent token spend and execution
time across the whole workspace, with an optional project filter.
Backend:
- Three new sqlc queries against task_usage + agent_task_queue: daily
usage, per-agent usage, per-agent total run-time. All optionally
scoped to a project via sqlc.narg('project_id'), reaching project
through the issue join.
- Handlers under /api/dashboard return the same wire shape the runtime
page already consumes (model preserved for client-side cost math).
Frontend: - Shared DashboardPage in packages/views/dashboard reusing KpiCard,
DailyCostChart, ActorAvatar, and estimateCost from the runtime page
so the visual style and pricing math stay in lock-step.
- Period selector (7/30/90d), project dropdown, four KPI tiles
(cost, tokens, run time, tasks), daily cost chart, and a combined
"cost + run time by agent" list.
- Routed in both web (app/[slug]/(dashboard)/dashboard) and desktop
(memory router); sidebar nav entry added under Workspace group.
Co-authored-by: multica-agent <github@multica.ai>
* fix(dashboard): drop stale project filter and stop double-counting tasks
Two issues caught in PR #2462 review:
1. Project filter held the previous selection's UUID across workspace
switches and project deletions: the dropdown gracefully showed
"All projects" (because the title lookup missed) while the three
dashboard queries kept forwarding the dead UUID, leaving the UI
looking like a full-workspace view but populated with empty
project-scoped data. Validate the picked UUID against the current
projects list before passing it to the queries.
2. The "by agent" table read its task count from the token rollup,
which is grouped per (agent, model). A single task that spans two
models lands twice and the agent's row reads e.g. "2 tasks" when
the real count is 1. Prefer `ListDashboardAgentRunTime`'s per-agent
distinct count when available; fall back to the token aggregate
only for agents with no terminal run yet (in-flight tasks).
Extract the merge into `mergeAgentDashboardRows` so the precedence
rules are unit-tested directly.
Co-authored-by: multica-agent <github@multica.ai>
* test(dashboard): allocate per-workspace issue.number explicitly
TestDashboardEndpoints creates two issues in the shared fixture
workspace. issue.number defaults to 0 (migration 020), and the table
carries UNIQUE (workspace_id, number), so the second insert raced the
first on the same default and failed in CI.
Allocate MAX(number) + 1 per insert so each row gets a fresh number
without stepping on rows other tests left behind in the same workspace.
Co-authored-by: multica-agent <github@multica.ai>
* feat(dashboard): rollup table + cron-driven aggregation for dashboard
Mirror the per-runtime rollup in `task_usage_daily` (migrations 073/077/082)
to remove the per-request raw aggregation the dashboard was doing.
Migration 084 adds:
- `task_usage_dashboard_daily` keyed on
(bucket_date, workspace_id, agent_id, project_id, model) — the
dimensions the dashboard actually queries, with project_id nullable
via UNIQUE NULLS NOT DISTINCT (PG15+) so "no-project" buckets
upsert cleanly.
- `task_usage_dashboard_rollup_state` watermark table.
- `task_usage_dashboard_dirty` invalidation queue.
- Triggers on agent_task_queue DELETE, task_usage DELETE, and
issue.project_id UPDATE — the cases the updated_at watermark can't
see. The project_id trigger re-attributes existing rollup rows when
a user moves an issue across projects.
- `rollup_task_usage_dashboard_daily_window(from, to)` —
idempotent recompute primitive (same shape as 077).
- `rollup_task_usage_dashboard_daily()` cron entry — own advisory
lock (4244) so it serialises independently of the runtime rollup.
- `task_usage_dashboard_rollup_lag_seconds()` health helper.
Sqlc queries `ListDashboardUsageDailyRollup` /
`ListDashboardUsageByAgentRollup` read from the new table; the handler
dispatches between rollup and raw on a separate
`UseDailyRollupForDashboard` config flag
(`USAGE_DASHBOARD_ROLLUP_ENABLED` env). Same fail-safe default (false →
raw) so operators can roll out independently of the per-runtime flag.
Bucket date is UTC (the dashboard aggregates across runtimes that may
sit in different tzs; there's no single correct local boundary).
Adds `cmd/backfill_task_usage_dashboard_daily` mirroring the existing
per-runtime backfill — operator runs it once before flipping the flag.
Tests: - TestDashboardEndpoints now also exercises the rollup read path
(raw vs. rollup, same project-scoped totals).
- TestDashboardRollupReattributesOnProjectChange verifies the
issue.project_id trigger enqueues both old + new buckets and the
next rollup tick zeroes the old project + populates the new one.
Co-authored-by: multica-agent <github@multica.ai>
* fix(dashboard-rollup): close two invalidation gaps
Two leak paths missed by migration 084 review:
1. Issue cascade DELETE — the atq BEFORE DELETE trigger runs AFTER the
issue row is gone, so `LEFT JOIN issue` returns NULL project_id and
the original-project bucket never gets cleared (issue 077 calls this
out for the runtime rollup but didn't need to act on it). Adds an
`issue BEFORE DELETE` trigger that enqueues using OLD.project_id
while the issue row is still readable.
2. `LinkTaskToIssue` (quick-create task attaching to a real issue post-
completion) UPDATEs `agent_task_queue.issue_id` from NULL to a real
id. Migration 084 only watched DELETE on atq, so usage already
rolled up under the no-project bucket stayed attributed to NULL
forever. Extends the atq trigger to fire on UPDATE OF issue_id too,
enqueueing both OLD (NULL project) and NEW (linked issue's project).
Tests: - TestDashboardRollupClearsOnIssueDelete asserts rollup row drops to
zero after issue delete + rollup tick.
- TestDashboardRollupReattributesOnLinkTaskToIssue verifies tokens
move from the NULL bucket to the project bucket after the UPDATE.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: multica-agent <github@multica.ai>
|
||
|
Bohan Jiang
|
b26f850d4e |
feat(agents): gate private-agent surfaces with allowed_principals predicate (#2359)
* feat(agents): gate private-agent surfaces with allowed_principals predicate
Tighten chat/@-mention, history, edit, and delete entry points so private
agents are only reachable by their owner or workspace owner/admin. Agent-to-
agent traffic still bypasses the gate so A2A collaboration keeps working.
- New canAccessPrivateAgent predicate in handler/agent_access.go; used by
comment.enqueueMentionedAgentTasks (replacing the inline check), GetAgent,
ListAgents (filter), ListAgentTasks, GetWorkspaceAgentRunCounts /
Activity30d / TaskSnapshot (workspace-wide aggregations no longer leak
private-agent existence + counts), chat.CreateChatSession,
chat.SendChatMessage (re-checks on every send so role changes can't leave
a stale session as a back-door), and autopilot.shouldSkipDispatch
(caller = autopilot creator).
- allowed_principals is computed inline as {agent.owner_id} ∪ workspace
owner/admin members. No new table — manual config is intentionally not
exposed in v1; the predicate is the extension seam.
- Front-end agent detail page distinguishes 403 (private agent the caller
can't access) from 404 (deleted/missing) and renders a "no access"
placeholder with a back-to-agents button.
- Go tests cover the pure predicate matrix + the four protected surfaces;
vitest passes for the affected views.
Co-authored-by: multica-agent <github@multica.ai>
* feat(agents): gate issue assignment with the private-agent predicate
Refactor validateAssigneePair to call the shared canAccessPrivateAgent
helper. This closes the back door where a plain member could assign a
private agent to an issue and let normal task dispatch run it, side-
stepping the chat / @-mention gate. Agent callers (X-Agent-ID) bypass
so A2A delegation onto a private assignee still works.
Add an integration test covering all three callers (workspace owner,
agent owner, plain member).
Co-authored-by: multica-agent <github@multica.ai>
* fix(agents): close three private-agent gate bypasses found in PR review
1. X-Agent-ID forgery (resolveActor): require X-Task-ID alongside
X-Agent-ID before trusting the agent identity. Without this a plain
workspace member could set X-Agent-ID to any visible agent UUID and
short-circuit the gate to "actor=agent, allow". Daemons already
pair the two headers, so legitimate A2A traffic is unaffected.
2. Chat history read path (chat.go): GetChatSession / ListChatMessages /
GetPendingChatTask / MarkChatSessionRead now go through a new
gateChatSessionForUser helper that re-applies canAccessPrivateAgent
after the ownership check, so a session creator whose role was later
downgraded loses transcript access. ListChatSessions and
ListPendingChatTasks filter their result sets by the same predicate.
3. Cross-workspace @mention (comment.enqueueMentionedAgentTasks):
resolve the mentioned agent via GetAgentInWorkspace scoped to the
issue's workspace so a UUID belonging to a different workspace's
private agent can't slip past the gate (the gate was being applied
against the current workspace's role table, which is the wrong
one).
Regression tests cover each bypass, plus an update to the resolveActor
unit test to reflect the new "X-Agent-ID without X-Task-ID falls back
to member" contract.
Co-authored-by: multica-agent <github@multica.ai>
* test(handler): seed X-Task-ID alongside X-Agent-ID in existing agent-caller tests
After tightening resolveActor to require both headers (X-Agent-ID +
X-Task-ID) for the "agent" actor identity, three existing tests that
set only X-Agent-ID started failing because their requests now resolve
to "member" instead of "agent". Add createHandlerTestTaskForAgent
helper and seed a task per agent-caller assertion. Also patch
TestAgentExplicitMentionStillTriggers — it still passed only because
the @mention path doesn't care about author type for member callers,
but the test claims to exercise the agent path, so make it faithful.
Co-authored-by: multica-agent <github@multica.ai>
* test(handler): finish X-Task-ID seeding + fix cross-workspace mention test schema
The previous CI run still failed in two places:
1. server/cmd/server integration tests — postCommentAsAgent → authRequestWithAgent
only set X-Agent-ID, so resolveActor downgraded the request to "member"
and the on_comment chain produced the wrong task counts. Fix:
authRequestWithAgent now also sets X-Task-ID, fetched or seeded by a new
ensureAgentTask(agentID) helper.
2. TestMentionAgent_RejectsCrossWorkspaceAgentUUID's hand-crafted comment
INSERT was missing comment.workspace_id, which migration 025 made
NOT NULL. Pass testWorkspaceID into the seed row.
Build + vet clean locally; both packages compile.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: multica-agent <github@multica.ai>
|
||
|
Multica Eve
|
ce00e05169 |
Add canonical PostHog core metrics events (#2302)
* Add canonical PostHog core metrics events Co-authored-by: multica-agent <github@multica.ai> * Address analytics review feedback Co-authored-by: multica-agent <github@multica.ai> * Tighten analytics review follow-ups Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Devv <devv@Devvs-Mac-mini.local> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Multica Eve
|
eb067ff077 |
fix(server): aggregate task_usage into daily rollup table to cut DB load (#2256)
* fix(server): aggregate task_usage into daily rollup table to cut DB load ListRuntimeUsage previously did a SUM(...) GROUP BY DATE(created_at), provider, model over the raw task_usage stream once per runtime row on the runtimes list and once per detail page load, scaling O(events) per call. This is the hot read path responsible for sustained load on Postgres. Switch the read path to a materialized daily rollup table maintained by a pg_cron job: - 072_task_usage_daily_rollup: schema for task_usage_daily + task_usage_rollup_state, plus rollup_task_usage_daily_window(p_from, p_to) (window primitive used by both cron and offline backfill, idempotent via ON CONFLICT DO UPDATE adding deltas) and rollup_task_usage_daily() (cron entry point — pg_try_advisory_lock(4242) for serialization, watermark advancement, 5-minute safety lag for late-visible inserts). Also adds idx_task_usage_created_at to help the two lazy endpoints (ListRuntimeUsageByAgent / GetRuntimeUsageByHour) that still hit the raw table. - 073_task_usage_daily_pgcron: CREATE EXTENSION IF NOT EXISTS pg_cron in a DO/EXCEPTION block (mirrors the migration 032 pg_bigm pattern so envs without shared_preload_libraries=pg_cron skip gracefully) and schedules rollup_task_usage_daily() every 5 minutes when the extension is present. - queries/runtime_usage.sql ListRuntimeUsage rewritten to read from task_usage_daily; sqlc regenerated. Other usage queries unchanged. - cmd/backfill_task_usage_daily: one-shot Go command that walks task_usage in monthly slices through rollup_task_usage_daily_window, then stamps the watermark to now()-5m so the cron resumes cleanly. Run once after migrations have applied, before relying on the rollup. - runtime_test.go: TestGetRuntimeUsage_BucketsByUsageTime now invokes rollup_task_usage_daily_window after fixture inserts so the handler sees the rolled-up rows. Synthetic daily rows cleaned up after each test. - runtime_rollup_test.go: new tests covering aggregation correctness, idempotency contract of ON CONFLICT DO UPDATE, and the watermark advancing exactly to now()-5m via the cron entry point. Deployment order: apply migrations → run backfill_task_usage_daily once → pg_cron picks up subsequent windows automatically. Today bucket may be up to ~10 minutes stale (5 min cron + 5 min lag) by design. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: multica-agent <github@multica.ai> * fix(server): make task_usage_daily rollup safe to overlap, replay, and correct Addresses 4 review blockers on the original PR: 1. Cron/backfill double-count race: the rollup function is now idempotent. Window calls find DIRTY KEYS via task_usage.updated_at, then RECOMPUTE each bucket from ground truth and REPLACE the daily row (no more additive ON CONFLICT). Cron and backfill can now overlap safely. 2. Silent pg_cron absence: the read path is gated behind a new USAGE_DAILY_ROLLUP_ENABLED feature flag (default off). The raw task_usage scan is preserved as the fallback. Operators flip the flag per-environment after backfill + cron are confirmed healthy (task_usage_rollup_lag_seconds() helper added for monitoring). 3. UpsertTaskUsage corrections invisible to rollup: added task_usage.updated_at column (default now(), backfilled from created_at), and bumped it on conflict. Corrections now mark the bucket dirty and the next window call recomputes it correctly. 4. CREATE INDEX blocking writes on hot table: split into separate single-statement migrations using CREATE INDEX CONCURRENTLY (074, 075), matching the 035/067 pattern. Also: cron.schedule() removed from migrations entirely. Migration 076 only enables the extension (gracefully on unsupported envs); the actual schedule is a documented operator runbook step that runs AFTER backfill. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: multica-agent <github@multica.ai> * fix(server): trigger-driven invalidation + online-safe migration for task_usage_daily Round-2 review feedback on PR #2256: 1. Add explicit dirty-bucket queue (task_usage_daily_dirty) populated by triggers on agent_task_queue (UPDATE OF runtime_id, DELETE) and task_usage (DELETE). The rollup window function drains both this queue and the updated_at-based discovery, so runtime reassignment and issue-cascade deletes no longer leave the rollup divergent from the raw query. Triggers join via agent (not issue) to look up workspace_id, because when the cascade comes from issue, the issue row is already gone by the time atq's BEFORE DELETE fires; agent stays alive. 2. Make migration 072 online-safe: only ADD COLUMN updated_at TIMESTAMPTZ (nullable, no default → metadata-only ALTER, no row rewrite) and a separate ALTER for SET DEFAULT now() (also metadata-only). No bulk UPDATE on the hot task_usage table. The rollup window function's dirty_keys CTE handles legacy NULL rows via an OR branch, supported by partial index idx_task_usage_created_at_legacy. 3. Refresh stale documentation in cmd/backfill_task_usage_daily/main.go header to describe the current recompute/replace semantics, idempotent re-runnability, and the actual migration numbering (072..077). Tests: - TestRollupTaskUsageDaily_InvalidationOnReassign: verifies usage moves between runtime buckets after ReassignTasksToRuntime-style update. - TestRollupTaskUsageDaily_InvalidationOnIssueDelete: verifies daily bucket is cleared after issue delete cascades through atq → task_usage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: multica-agent <github@multica.ai> * fix(server): close dirty-queue race + move legacy partial index to its own concurrent migration Round-3 review feedback on PR #2256: 1. Blocker: dirty-queue invalidations could be silently lost under concurrency. ON CONFLICT DO NOTHING let a late trigger see the row already enqueued, no-op, and then the rollup drain (WHERE enqueued_at < p_to) would delete the original row — losing the late invalidation. Switched all three trigger enqueue paths to ON CONFLICT DO UPDATE SET enqueued_at = GREATEST(existing, EXCLUDED.enqueued_at), so any invalidation arriving during a rollup tick keeps enqueued_at > p_to (p_to = now() - 5min) and survives the post-tick drain. 2. High: idx_task_usage_created_at_legacy (partial index on hot task_usage table) was being created in the regular 077 migration without CONCURRENTLY. Moved to new migration 078 with CREATE INDEX CONCURRENTLY, matching the pattern of 074/075. 077's down migration leaves the index alone (it is owned by 078). 3. Minor: gofmt -w on runtime_rollup_test.go and backfill_task_usage_daily/main.go (tabs were lost in the original heredoc append). PR description rewritten to describe the current recompute/replace + dirty queue + feature flag design and the 072..078 migration ordering. Tests still green: TestRollupTaskUsageDaily_* (including both new invalidation regressions), TestGetRuntimeUsage_*, TestWorkspaceUsage_*. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: multica-agent <github@multica.ai> * fix(server): unify workspace_id source via agent in rollup window function Round-4 review feedback (J) on PR #2256: M1 (must-fix): The dirty queue triggers resolved workspace_id via `agent.workspace_id`, but the window function's `dirty_from_updates` discovery and `recomputed` recompute join used `issue.workspace_id`. There is no schema-level FK guaranteeing `agent.workspace_id == issue.workspace_id`. Any divergence (future cross-workspace task scenarios, data repairs, migration bugs) would cause: - dirty queue rows with workspace_id from agent - recompute join filtering by workspace_id from issue - 0 matches in recompute → bucket erroneously hits the deleted_empty branch and the daily row is silently dropped - dirty_from_updates path attributing usage to the wrong workspace Replaced both CTEs to JOIN agent (not issue) so trigger / discovery / recompute share one workspace_id source. Comment in 077 explains the constraint. N1: Refreshed two stale references in cmd/backfill_task_usage_daily/main.go (header now says "072..078"; stampWatermark warning now mentions migration 073, where the rollup state table is actually introduced). Test: New TestRollupTaskUsageDaily_WorkspaceMismatch constructs an atq with agent.workspace_id != issue.workspace_id, asserts the bucket lands under agent's workspace (not issue's), and re-asserts after a runtime reassign in the foreign workspace. Acts as a canary if the schema invariant changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Eve <eve@multica.ai> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: multica-agent <github@multica.ai> Co-authored-by: Devv <devv@Devvs-Mac-mini.local> |
||
|
LinYushen
|
cc527c34be |
perf(heartbeat): batch runtime last_seen_at writes (#2213)
Batches runtime heartbeat last_seen_at updates while preserving the 60s flush / 150s sweeper stale-window invariant. Also drains pending heartbeat writes during graceful shutdown. |
||
|
Bohan Jiang
|
09f04847d3 | feat(server): redis-backed runtime liveness with DB fallback (#2121) | ||
|
LinYushen
|
44a0ced558 |
fix(runtime): persist CLI update requests in Redis (#2113)
Co-authored-by: multica-agent <github@multica.ai> |
||
|
Bohan Jiang
|
ba5b7db78e |
fix(server): persist ModelListStore across replicas via Redis (#2022)
* fix(server): persist ModelListStore across replicas via Redis The model picker uses a pending-request pattern: the frontend POSTs to create a request, the daemon pops it on its next heartbeat, runs agent.ListModels locally, and reports back. Until now the store was a plain in-memory map per Handler instance. That works for self-hosted single-instance deploys but fails in any multi-replica environment (Multica Cloud). Each replica has its own map, so: POST /runtimes/:id/models → request stored in replica A GET /runtimes/:id/models/<requestId> → polls land on B/C → 404 daemon heartbeat → only A sees PendingModelList POST .../<requestId>/result → daemon's report has to land on A Success probability ~1/N². The visible symptom is "No models available" in the picker for every provider, even those (Claude/Codex) whose catalog is statically populated end-to-end. Same shape of bug, same Redis-backed fix as multica-ai/multica#1557 did for LocalSkillListStore / LocalSkillImportStore. Reuse the operational playbook (namespaced keys, ZSET-backed pending queue, atomic ZREM+SET-running via the shared Lua script) so we don't introduce a second concurrency model for the same primitive. Changes: - Convert ModelListStore from struct to interface with context-aware methods. Add HasPending for cheap heartbeat-side probing. - InMemoryModelListStore — single-node fallback, used when REDIS_URL is unset (self-hosted dev / tests). - RedisModelListStore — multi-node implementation using the same key layout and Lua atomic claim as RedisLocalSkillListStore. - Use RunStartedAt (not UpdatedAt) as the running-timeout reference point, matching the local-skill stores so subsequent UpdatedAt bumps don't reset the running clock. - Heartbeat now uses the probe-then-pop pattern for the model queue (matching local-skills) so a slow Redis can't stall every connected daemon. Extends heartbeatMetrics + slow-log with probe_model_ms / pop_model_ms / probe_model_timed_out for parity. - Wire the Redis backend in NewRouterWithOptions when rdb != nil. - Tests for both backends. Redis tests gate on REDIS_TEST_URL so laptop runs without Redis still pass; CI provides it. Co-authored-by: multica-agent <github@multica.ai> * fix(server): persist RunStartedAt + retry model report on transient failures Two follow-ups from PR #2022 review: 1. RedisModelListStore was dropping ModelListRequest.RunStartedAt on persistence — the field is tagged json:"-" so it doesn't leak into the HTTP response, which made plain json.Marshal(req) silently discard it. Across-node readers saw RunStartedAt=nil and applyModelListTimeout's running branch became a no-op, so the 60s running-timeout escape hatch never fired. CI's TestRedisModelListStore_RunningTimeout was failing on this exact case. Fix mirrors RedisLocalSkillImportStore's envelope pattern — wrap in an internal struct that re-promotes the field. HTTP shape stays clean. Adds a no-Redis unit test that pins the round trip. 2. Daemon's handleModelList called d.client.ReportModelListResult directly and swallowed any 5xx, leaving the pending request stranded in "running" until its 60s server-side timeout — exactly the failure mode the multi-node store fix was meant to eliminate. Generalize the existing local-skill retry helper into reportRuntimeResultWithRetry (kind: model_list / local_skill_list / local_skill_import) and wire handleModelList through a new reportModelListResult helper. Renames the test-overridable var localSkillReportBackoffs → runtimeReportBackoffs to match. Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: multica-agent <github@multica.ai> |
||
|
Naiyuan Qing
|
4ad0a0b847 |
feat(chat): presence v4 — status pill, failure bubble, elapsed timing (#1856)
A complete UX upgrade for chat sending → receiving → recovering.
* StatusPill replaces the orphan spinner — stage-aware copy
("Reading files · 12s", "Searching the web · 14s", "Typing · 24s"),
shimmer text, monotonic timer, derived effective status, > 60s
warning tone, > 5min cancel button.
* WS writethrough on task:queued / task:dispatch / task:cancelled so
pendingTask cache stays in sync with the daemon state machine without
invalidate-refetch latency. broadcastTaskDispatch now includes
chat_session_id when the task is for a chat session — the existing
payload only carried it on the generic task: events, leaving the pill
stuck at "Queued" until completion.
* Failure fallback — FailTask writes a chat_message tagged with
failure_reason (mirrors the issue path's system comment, gated on
retried==nil). Front-end renders an inline note ("Connection failed",
with a Show details collapsible) instead of the previous black hole.
* Elapsed timing — chat_message.elapsed_ms persists task.completed_at -
task.created_at on success/failure rows. UI shows "Replied in 38s" /
"Failed after 12s" beneath assistant bubbles. Format helper shared
between StatusPill and the persisted caption so the live timer and
final reading never disagree.
* Optimistic burst rebalanced — pendingTask seed + created_at moved
before the HTTP roundtrip so the pill appears the instant the user
hits send; handleStop is fire-and-forget so cancel feels immediate
(server confirmation arrives via task:cancelled WS).
* Presence integration — chat avatars use ActorAvatar (status dot +
hover card); OfflineBanner above the input on offline/unstable;
SessionDropdown shows per-row in-flight/unread pip plus a
cross-session aggregate pip on the closed trigger.
* Editor blur on send so the caret stops competing with the StatusPill
/ streaming reply for the user's attention.
* Chat panel isOpen now persists globally; defaults to OPEN for new
users (storage key absence) so the feature is discoverable. Existing
users' prior choice is respected.
* DB: migrations 062 (failure_reason) + 063 (elapsed_ms), both
ADD COLUMN NULL — fast, non-blocking, backwards compatible.
* WS: task:failed chat path now invalidates chatKeys.messages — fixes
a pre-existing bug where the failure bubble required a page refresh
to appear.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Bohan Jiang
|
86e7de3e41 |
feat(server/auth): cache auth token lookups in Redis with 10m TTL
* feat(server/auth): cache PAT lookups in Redis with 60s TTL
Personal access tokens used to hit Postgres on every request: a SELECT
to resolve token_hash → user_id, plus a fire-and-forget UPDATE of
last_used_at. For a CLI / daemon making many requests per second this
is wasted DB load — the token is the same and the answer hasn't changed.
Add a Redis-backed cache (auth.PATCache) keyed by token hash, TTL 60s:
- On cache hit, the auth middleware skips both the SELECT and the
last_used_at UPDATE. last_used_at is now refreshed at most once per
TTL window per token, not per request.
- On cache miss the middleware falls back to today's behavior: query
Postgres, populate the cache, async-update last_used_at.
- On revoke, the handler invalidates the cache entry so revocation
takes effect immediately rather than waiting for the TTL to expire.
This required changing RevokePersonalAccessToken from :exec to :one
RETURNING token_hash.
The cache is nil-safe: when REDIS_URL isn't configured, NewPATCache
returns nil and the middleware degrades to today's always-hit-DB
behavior. JWT validation is untouched (already DB-free).
Tested with REDIS_TEST_URL — same gating pattern the rest of the
suite uses for Redis-backed tests. New tests cover nil-safety, set/
get/invalidate, TTL, and the middleware short-circuit on cache hit.
* fix(server/auth): clamp PAT cache TTL to token's remaining lifetime
GPT-Boy review caught: a PAT expiring in <60s would still be cached
for the full PATCacheTTL window, so the token could continue passing
auth on cache hit for up to ~60s after its expires_at. The DB query
filters expired tokens (revoked = FALSE AND expires_at > now()), but
that filter never ran on a cache hit.
Make Set take an explicit ttl, and add TTLForExpiry to compute it:
- no expires_at → full PATCacheTTL
- expires_at far → full PATCacheTTL
- expires_at <60s → time until expiry
- already expired → 0, Set skips caching (TOCTOU defense between
the SELECT and the Set, since the SELECT
already filters expired rows)
Regression test pins the clamp behavior end-to-end against Redis.
* feat(server/auth): cache daemon-token + PAT lookups in DaemonAuth, bump TTL to 10m
Daemon /api/daemon/* requests (heartbeat, claim task) hit DaemonAuth
which previously did its own GetDaemonTokenByHash on every request and
*also* duplicated the PAT lookup on the mul_ fallback — bypassing the
cache added in
|
||
|
Bohan Jiang
|
abd69890a8 |
Revert "feat(issues): server-side filters incl. label, fixing pagination drop…" (#1779)
This reverts commit
|
||
|
Bohan Jiang
|
246fcd4ce4 |
feat(issues): server-side filters incl. label, fixing pagination drops (#1776)
* feat(issues): server-side label + filter querying for issue list Extends GET /api/issues with label_ids, priorities, creator_ids, project_ids, include_no_assignee, and include_no_project params, and moves the existing single-value filters onto array-form. Each filter becomes part of the SQL WHERE clause so paginated buckets reflect the user's selection — fixes the bug where client-side filtering hid matches sitting past the first page (#1491). CLI gains a repeatable --label flag; legacy --priority/--assignee/ --project keep working via the single-value compatibility paths. * feat(issues): drive workspace + my-issues filters from the server issueListOptions and myIssueListOptions now key the React Query cache on a normalized filter object, so each filter combination has its own cache entry and a filter change re-fetches with the wire-shape filter applied server-side. Drops the client-side filterIssues step on the issues page, my-issues page, and project detail — that step silently hid matches that lived past the first paginated page (#1491). Adds a Label submenu to the workspace issues filter dropdown, plus labelFilters in the view store. Mutations and ws-updaters fan their optimistic patches across every filter-keyed list cache via qc.setQueriesData on issueKeys.listPrefix(wsId), and the editor's mention-suggestion reads from any matching list cache for instant first paint regardless of which filter is active. * fix(issues): route Members/Agents scope through server-side filter The Members/Agents scope tabs on the workspace issues page were still narrowing client-side via `assignee_type === 'member'`. That hits the exact pagination-blind bug this PR is meant to fix: if the first 50 issues per status don't include the right assignee type, the tab shows "No issues" while later pages have matches. Adds an `assignee_types text[]` filter to ListIssues / ListOpenIssues / CountIssues, threads it through the API client, normalizer and view filter, and maps the scope tab to it. Each scope now keys its own list cache and refetches with the correct first page. Also disables the My Issues "My Agents" query when the user owns no agents — `assignee_ids: []` was getting dropped by both the API client and the query-key normalizer, so the request went out unfiltered and surfaced unrelated issues under "My Agents". |
||
|
devv-eve
|
9db91e89f5 |
feat: add daemon websocket task wakeups (#1772)
* feat: add daemon websocket task wakeups * feat: fan out daemon wakeups across nodes * fix: dedupe daemon wakeup loopback events * fix: lengthen daemon polling fallback interval --------- Co-authored-by: Eve <eve@multica.ai> |
||
|
Bohan Jiang
|
f628e48775 |
refactor(server): error-returning ParseUUID to prevent silent data loss
* refactor(server): make ParseUUID error-returning to prevent silent data loss (MUL-1410) util.ParseUUID previously swallowed errors and returned a zero pgtype.UUID on invalid input. When this zero UUID reached a write query (DELETE/UPDATE), the SQL matched zero rows and the handler returned 2xx success — producing silent data corruption. #1661 (DeleteIssue with identifier-style ID) was the visible symptom; PR #1680 patched that one site, this commit closes the class of bug. Changes: - util.ParseUUID now returns (pgtype.UUID, error). Add util.MustParseUUID for trusted round-trips that should panic on invalid input. - handler/handler.go: parseUUID wrapper now calls MustParseUUID — any unguarded user-input string reaching it surfaces as a recovered panic (chi middleware.Recoverer → 500) instead of silently corrupting data. Add parseUUIDOrBadRequest(w, s, fieldName) for handler entry points. - Convert every Queries.Delete*/Update* call site reachable from raw user input (autopilot, comment, project, skill, skill_file, label, pin, attachment, feedback, issue assignee, daemon runtime, workspace) to validate UUIDs explicitly with parseUUIDOrBadRequest, returning 400 on invalid input. Where a resolved entity.ID is already in scope, write queries now use it directly instead of re-parsing the URL string. - Update getWorkspaceMember + loadIssueForUser to handle invalid UUIDs gracefully (404/400 instead of panic). - Update util/middleware/cmd-level callers (subscriber_listeners, notification_listeners, activity_listeners, scope_authorizer, middleware/workspace) to use the error-returning API. - Add server/internal/util/pgx_test.go covering valid/invalid input and the MustParseUUID panic contract. - Add TestDeleteIssueByIdentifier + TestDeleteIssueRejectsInvalidUUID regression tests in handler_test.go (the original #1661 bug + the invalid-input case). - Document the handler UUID parsing convention in CLAUDE.md so the rule is enforceable in future PR review. * fix(server): address GPT-Boy review of #1748 P1 fixes from PR #1748 review: 1. Migrate remaining request-boundary UUIDs to parseUUIDOrBadRequest so malformed input returns 400 instead of panic/500. Was missing on: - issue.go: workspace_id in CreateIssue/ChildIssueProgress/ListIssues/ SearchIssues/BatchUpdateIssues/BatchDeleteIssues; project_id / parent_issue_id / lead_id / assignee_id / assignee_ids / creator_id filters; batch issue_ids and assignee/parent/project fields in BatchUpdateIssues (skip on bad input via util.ParseUUID, matching the existing per-row continue semantics). - project.go: project id + workspace_id in GetProject/UpdateProject/ DeleteProject; lead_id in CreateProject/UpdateProject; workspace_id in ListProjects + SearchProjects. - handler.go: resolveActor now uses util.ParseUUID for X-Agent-ID / X-Task-ID headers; invalid UUID falls back to "member" (matches pre-existing semantics) instead of panicking. - issue.go: validateAssigneePair returns 400 on invalid workspace_id instead of panicking. 2. Fix issue:deleted WS event payloads to emit uuidToString(issue.ID) instead of the raw URL string. After an identifier-path delete ("MUL-7"), the previous payload would have leaked the identifier to subscribers, leaving stale entries in frontend caches that key by UUID. Updated DeleteIssue (issue.go:1341) and BatchDeleteIssues (issue.go:1641). The slog "issue deleted" log line also now records the resolved UUID so logs match the WS payload. 3. Extend TestDeleteIssueByIdentifier to subscribe to the bus and assert issue:deleted.payload.issue_id is the resolved UUID, not the identifier. * fix(server): validate remaining reviewed UUID inputs * fix(server): validate remaining handler UUID inputs * fix(server): finish request boundary UUID audit * fix(server): validate remaining request body UUIDs * fix(server): validate runtime path UUIDs * fix(server): validate remaining audit UUID inputs --------- Co-authored-by: Eve <eve@multica.ai> |
||
|
Bohan Jiang
|
ad803b86ec |
fix(skills): shared-state runtime local-skill stores (MUL-1288) (#1557)
* fix(skills): shared-state runtime local-skill stores (MUL-1288)
Fixes the bug Bohan surfaced on MUL-1288: behind prod's multi-node API the
runtime-local-skill list/import flow would intermittently time out or 404.
Root cause: LocalSkillListStore and LocalSkillImportStore were per-process
sync.Mutex+map, so when the frontend POST, the daemon heartbeat and the
frontend GET landed on different API instances, each saw a different
pending set. Confirmed against production daemon logs — the failed
request_id never showed up in the daemon's "runtime local skills
requested" log, even though other requests around the same window worked.
Per Yushen's guidance (server must stay stateless; state lives in
storage), migrate both stores to Redis so every node agrees on the same
pending set.
What changed
- LocalSkillListStore / LocalSkillImportStore are now interfaces. Methods
take context.Context and return error.
- InMemoryLocalSkill{List,Import}Store — renamed from the existing types,
kept as the default for single-node dev and the in-process test suite.
- RedisLocalSkill{List,Import}Store — new. Keyed on
mul:local_skill:{list,import}:<id> (JSON record, TTL = retention), with
a per-runtime ZSET mul:local_skill:{list,import}:pending:<runtime_id>
(score = created_at UnixNano) providing cross-node ordering. PopPending
wins the claim via ZREM == 1, so concurrent pops from different nodes
never return the same request twice.
- NewRouter gets an optional *redis.Client; when non-nil it swaps in the
Redis-backed stores. main.go hoists the existing Redis client (already
used by the realtime relay) so both subsystems share one client.
- Handler fields flip to interface types; handler.New still constructs
in-memory stores by default.
- Daemon heartbeat's PopPending call sites thread r.Context() through so
Redis operations inherit request cancellation. Errors warn instead of
poisoning the heartbeat response.
Tests
- Existing in-memory tests updated for the new signatures (ctx + error).
- New runtime_local_skills_redis_store_test.go covers:
- Create/Get/Complete round trip preserves skills payload
- PopPending across two *store instances sharing one rdb (the exact
regression: node A creates, node B pops)
- N concurrent PopPending on one record => exactly one winner
- Pending-timeout threshold transitions the record and removes the zset
member so a later PopPending doesn't return a timed-out request
- Import store round-trips CreatorID (which is json:"-" on the public
struct — needs a Redis envelope so ReportLocalSkillImportResult can
still attribute the created Skill)
- Per-runtime isolation — a PopPending for runtime B does not disturb
A's pending zset
- Tests skip gracefully if REDIS_TEST_URL is unset; CI now spins up a
redis:7-alpine service and exports the URL so the suite actually runs
there.
Out of scope
PingStore / UpdateStore / ModelListStore have the same shape and the
same latent bug (they just fire rarely enough to have gone unnoticed).
Migrating them to Redis is a follow-up — MUL-1288 is specifically the
local-skills break Bohan is blocked on.
* fix(skills): atomic Redis claim + surface store write failures (PR #1557 review)
Two real gaps GPT-Boy flagged:
1. RedisLocalSkill{List,Import}Store.PopPending was doing ZREM then SET as
two separate round-trips. If the SET failed for any reason — transient
Redis error, context cancellation, pod getting SIGKILL'd mid-call — the
request was already gone from the pending zset but the stored record
still said "pending", and no subsequent PopPending would re-dispatch
it. Exactly the "request disappears" class of bug this PR is supposed
to kill.
Fix: push the claim into a Lua script so Redis runs ZREM + SET as one
atomic unit. If ZREM returns 0 (another node won the race), SET is
skipped and the caller retries.
2. ReportLocalSkill{List,Import}Result handlers were logging Complete/Fail
store failures at Warn and still returning 200 OK. That made the
daemon think the report landed when it hadn't, leaving the request
stuck in "running" until the server-side timeout and — worse for the
import flow — leaving the just-created Skill row orphaned in Postgres
so every retry collided with the unique-name constraint.
Fix: escalate to Error + return 500 so the daemon (and monitoring) can
see the write failed. For the import flow, Complete failure after the
Skill row is already committed also triggers a best-effort DeleteSkill
so a daemon retry lands on a clean slate instead of hitting
"a skill with this name already exists" forever.
Tests
- New TestRedisLocalSkillListStore_PopPendingAtomicClaim asserts the
happy-path invariant: after one PopPending the record is "running"
AND a second PopPending returns nothing. Deliberately does NOT poke
Redis internals directly so the test survives any future key-layout
refactor.
- Existing cross-instance / concurrent / timeout / per-runtime tests
continue to pass against the Lua-based claim path (verified locally
against a scratch redis-server; 8/8 Redis tests green).
|
||
|
Bohan Jiang
|
6fd1255873 |
feat(runtimes): remove Test Connection / runtime ping feature (#1554)
* feat(runtimes): remove Test Connection / runtime ping feature The Test Connection action invoked a real single-turn agent run to verify runtime connectivity. In practice it was expensive (reuses none of the normal task exec env, so it also gave misleading results) and low value — daemon heartbeat + Online status already covers the "is the runtime alive" question. Dropping the whole end-to-end probe path: - deletes server handler and in-memory PingStore - drops pending_ping from the heartbeat response and daemon poll loop - removes daemon.handlePing, PendingPing, ReportPingResult - removes the CLI `multica runtime ping` command - removes the PingSection UI block and RuntimePing types / api methods * docs: fix runtime CLI subcommand list in product-overview |
||
|
LinYushen
|
91424752ac |
feat(realtime): phase 0 — extract Broadcaster interface + add metrics (MUL-1138) (#1429)
* feat(realtime): phase 0 — extract Broadcaster interface + add metrics Phase 0 of the WebSocket horizontal-scaling plan tracked in MUL-1138. This change is intentionally behavior-preserving: it sets up the seams needed for later phases (subscribe/unsubscribe protocol, scope-level fanout, Redis Streams relay) without altering any wire protocol or producer call sites. What changed - New realtime.Broadcaster interface covering the three fanout methods producers already use on *Hub (BroadcastToWorkspace, SendToUser, Broadcast). *Hub continues to satisfy it; a future Redis-backed implementation can be dropped in without touching listeners. - registerListeners now depends on realtime.Broadcaster instead of *realtime.Hub, isolating the bus → realtime fanout layer behind an interface. - New realtime.Metrics singleton with atomic counters: connects, disconnects, active connections, slow-client evictions, total messages sent/dropped, and per-event-type send counters. Wired into Hub register/unregister/broadcast paths and into every listener. - New GET /health/realtime endpoint returning a JSON snapshot of the metrics so we can observe baseline fanout pressure before phase 1. Why phase 0 first GPT-Boy's only-Redis plan and CC-Girl's review both call out the same prerequisite: get a Broadcaster seam and visibility in place before introducing scope-level subscriptions or a Redis relay. Doing this as a standalone step keeps each later PR focused and trivially revertable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(realtime): only-Redis fanout — scopes, subscribe protocol, Redis Streams relay (MUL-1138) Implements the final-version plan agreed in MUL-1138 on top of phase 0: * Hub: 4 scope types (workspace/user/task/chat), per-client subscription set, subscribe/unsubscribe WS frames, ScopeAuthorizer hook for task/chat scope auth, first/last-subscriber callbacks for the relay, workspace+user auto-subscribe on connect. * RedisRelay: Broadcaster impl that XADDs every event into ws:scope:{type}:{id}:stream and XREADGROUPs only the scopes for which this node has live subscribers. Per-node consumer group, heartbeat, stale-consumer sweeper, MAXLEN cap, lag/disconnect metrics. * Listeners: route task:* events to ScopeTask, chat:* events to ScopeChat; workspace remains the default for everything else. * events.Event: optional TaskID / ChatSessionID hints so the listener layer can pick the right scope without re-parsing payloads. * Handler: publishTask / publishChat helpers; chat + task message publishers updated to use them. * main.go: when REDIS_URL is set, wrap the hub with NewRedisRelay and pass the relay (instead of the hub) to registerListeners. A db-backed ScopeAuthorizer enforces that task/chat subscribes belong to the caller's workspace. * Metrics: per-scope subscribe/deny counters, redis connect state, node id, lag/dropped counters surfaced via /health/realtime. Behavior in single-node mode (REDIS_URL unset) is unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(realtime): address PR #1429 review must-fix items (MUL-1138) - listeners: keep task/chat events on workspace fanout until the WS client supports scope-subscribe + reconnect-replay. Routing them through BroadcastToScope today (without any client subscriber) would silently drop every chat / task message and break the live timeline, chat unread badges, and pending-task UI. The server-side scope infra (Hub subscribe/unsubscribe, ScopeAuthorizer, Redis Streams relay) stays in place so flipping the switch in the client follow-up PR is a one-line change. - scope_authorizer: ScopeChat now enforces CreatorID == userID, mirroring the HTTP layer (handler/chat.go: GetChatSession / SendChatMessage / MarkChatSessionRead). Without this, any workspace member who learned a session_id could subscribe to chat:message / chat:done / chat:session_read for a peer's private chat. The same creator-only check is applied to ScopeTask when the task is a chat task (task.ChatSessionID set). Issue tasks remain workspace-scoped. - Refactor scope authorizer to depend on a narrow scopeAuthQuerier interface so its decisions can be unit-tested without a live DB. - Add tests: * listeners_scope_test.go pins the workspace-fanout fallback for task:message / task:progress / chat:message / chat:done / chat:session_read. * scope_authorizer_test.go covers chat creator-only access, chat-task creator-only access, and issue-task workspace-only access (creator allowed, peer denied, cross-workspace denied, missing session denied, empty userID denied). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: CC-Girl <cc-girl@multica.ai> |
||
|
gezilinll
|
f247a4f544 |
feat(skills): import runtime local skills into workspace (#1431)
* feat(skills): import runtime local skills into workspace * fix(skills): address runtime local skill review feedback * docs(skills): annotate local provider skill paths --------- Co-authored-by: zhangliang <zhangliang@gaoding.com> |
||
|
devv-eve
|
637bdc8eb3 |
feat(analytics): full PostHog pipeline + 6 funnel events (MUL-1122) (#1367)
* feat(analytics): add PostHog client with async batch shipping Introduces server/internal/analytics, the shipping layer for the product funnel defined in docs/analytics.md. Capture is non-blocking — events are enqueued into a bounded channel and a background worker batches them to PostHog's /batch/ endpoint. A broken backend drops events rather than blocking request handlers. Local dev and self-hosted instances run a noop client until the operator sets POSTHOG_API_KEY. This is PR 1 of MUL-1122; signup and workspace_created emission land in the follow-up commit so this change is independently reviewable. * feat(server): emit signup and workspace_created analytics events Wires analytics.Client through handler.New and main, then emits the first two funnel events: - signup fires from findOrCreateUser (which now reports isNew), covering both the verification-code and Google OAuth entry points — a single emission site guarantees Google signups aren't missed. - workspace_created fires after the CreateWorkspace transaction commits, with is_first_workspace computed from a post-commit ListWorkspaces count so we can distinguish fresh-user activation from returning-user expansion. Tests use analytics.NoopClient so nothing ships from test runs. PR 1 of MUL-1122; runtime_registered and issue_executed follow in later PRs per the plan. * refactor(analytics): drop is_first_workspace from workspace_created Stamping "is this the user's first workspace?" at emit time races under concurrent CreateWorkspace requests: two transactions committing close together can both read a post-commit count greater than one and both emit false. Fixing it at the SQL layer requires a schema change we don't want in PR 1. PostHog answers the same question exactly from the event stream (funnel on "first time user does X" / cohort on $initial_event), so removing the property loses no information and makes the emit side race-free. * docs(analytics): document self-host safety defaults Spell out why self-hosted instances never ship events upstream by default (empty POSTHOG_API_KEY → noop client) and explain how operators can point at their own PostHog project without any code change. * feat(analytics): emit runtime_registered, issue_executed, team_invite_* Three server-side funnel events, all gated on first-time state transitions so retries and re-runs don't inflate the WAW buckets: - runtime_registered fires from DaemonRegister when UpsertAgentRuntime reports (xmax = 0) — i.e. the row was inserted, not updated. Heartbeats and re-registrations stay silent. - issue_executed fires from CompleteTask after an atomic UPDATE issue SET first_executed_at = now() WHERE id = $1 AND first_executed_at IS NULL flips the column for the first time. Retries, re-assignments, and comment-triggered follow-up tasks hit the WHERE clause and no-op. Carries nth_issue_for_workspace so the ≥1/≥2/≥5/≥10 buckets filter without extra queries. - team_invite_sent fires from CreateInvitation and team_invite_accepted from AcceptInvitation, closing the expansion funnel. Adds a 050 migration for issue.first_executed_at plus a partial index so the workspace-scoped executed-count query doesn't scan the never-executed tail. * feat(config): surface PostHog key via /api/config Extends AppConfig with posthog_key / posthog_host sourced from env on every request (so operators can rotate the key via secret refresh without a restart). Reading the key off the server — rather than baking it into the frontend bundle via NEXT_PUBLIC_* — means self-hosted instances inherit the blank key automatically and never ship events upstream. * feat(analytics): wire posthog-js identify + UTM capture on the client Adds @multica/core/analytics — a thin wrapper around posthog-js that owns attribution capture and identity merge. Posthog-js config comes from /api/config (not NEXT_PUBLIC_*), so self-hosted instances whose server returns an empty key automatically run the SDK inert. captureSignupSource stamps a multica_signup_source cookie with UTM params and the referrer's origin (never the full referrer — that can leak OAuth code/state in the callback URL). The backend signup event reads this cookie on new-user creation. Identity flows: - auth-initializer fires identify() right after getMe() resolves, on both cookie and token paths. A getConfig/getMe race is handled by buffering a pending identify inside the analytics module and flushing it once initAnalytics finishes. - auth store calls identify() on verifyCode / loginWithGoogle / loginWithToken and resetAnalytics() on logout so the next login merges cleanly without bleeding events. * docs(analytics): describe runtime_registered, issue_executed, invite events Fills in the schema for the remaining funnel events. Captures the design commentary that belongs next to the contract rather than in a PR description — in particular why issue_executed uses the atomic first_executed_at flip instead of counting task-terminal events, and why runtime_registered relies on xmax = 0 rather than a query-then-write. * fix(analytics): drop non-atomic nth_issue_for_workspace from issue_executed Computing the workspace's Nth-issue ordinal at emit time is not atomic under concurrent first-completions — two transactions can both run MarkIssueFirstExecuted, then both run CountExecutedIssuesInWorkspace, and both observe count=1 before either has committed, so both events go out stamped as n=1. Serialising it would mean a per-workspace advisory lock or a SERIALIZABLE-isolated tx; PostHog answers the same question exactly at query time via row_number() partitioned by workspace_id, so the emit-time property adds risk without adding information. Removes the property from analytics.IssueExecuted, deletes the unused CountExecutedIssuesInWorkspace query, and regenerates sqlc. The partial index stays — any future workspace-scoped executed-issue query will want it. * fix(analytics): wire $pageview and harden signup_source cookie payload Two frontend fixes from the PR review: - PageviewTracker, mounted under WebProviders, fires capturePageview on every Next.js App Router path / query-string change. Without this the capturePageview helper in @multica/core/analytics was never called and the acquisition funnel's / → signup step was empty. - captureSignupSource now caps each UTM / referrer value at 96 chars *before* JSON.stringify, and drops the whole cookie when the serialised payload still exceeds 512 chars. Previously the overall slice(0, 256) could leave a half-JSON string on the wire that neither the backend nor PostHog could parse. Both capturePageview and identify now buffer a single pending call when fired before initAnalytics resolves — otherwise the initial "/" pageview and same-turn login identify race the /api/config fetch and get dropped. resetAnalytics clears both buffers so a logout→login cycle stays clean. * fix(analytics): URL-decode signup_source cookie on read Go does not URL-decode Cookie.Value automatically, so the frontend's JSON-then-encodeURIComponent payload was landing in PostHog as percent-encoded garbage (%7B%22utm_source...). Unescape on read so the backend receives the original JSON string the frontend intended, and drop values that fail to decode or exceed the server-side cap — sending truncated garbage is worse than sending nothing. Oversized-cookie guard matches the frontend's SIGNUP_SOURCE_MAX_LEN. * docs(analytics): reflect nth-issue drop, $pageview wiring, cookie encoding Pulls the schema doc back in line with the code: issue_executed no longer advertises nth_issue_for_workspace (with a note about why PostHog derives it at query time instead), the frontend $pageview section names the actual PageviewTracker component that fires it, and the signup_source section documents the per-value cap / overall drop rule and the encode-on-write / decode-on-read contract. --------- Co-authored-by: Jiang Bohan <bhjiang@outlook.com> |
||
|
Jiayuan Zhang
|
b291db11c2 |
feat(agents): add per-agent model field with provider-aware dropdown (#1399)
Adds a first-class `model` field on agents so users can pick the LLM model from the create / settings UI instead of editing `custom_env` / `custom_args`. Each provider's dropdown is populated from the live CLI when possible (`opencode models`, `pi --list-models`, `openclaw agents list --json`, `cursor-agent --list-models`, hermes ACP `session/new` → `SessionModelState`), with a static catalog for providers that don't enumerate.
Daemon resolves the runtime model as `agent.model → MULTICA_<PROVIDER>_MODEL → ""` — empty passes through so each backend's CLI picks its own default, avoiding static-guess drift.
Per-provider honouring:
- Claude / Codex / OpenCode / Cursor / Gemini / Pi / Copilot — CLI `--model` / thread payload.
- OpenClaw — `opts.Model` is mapped to `--agent <name>` (the CLI rejects `--model`).
- Hermes — `session/set_model` ACP RPC; stderr is sniffed for provider-level errors so HTTP 4xx from the configured LLM surfaces instead of "empty output"; explicit-model failures mark the task `failed`.
Supporting changes: migration 050 adds `agent.model`; daemon ↔ server heartbeat piggyback carries a model-discovery request; new REST endpoints under `/api/runtimes/{id}/models`; `multica agent create --model` / `update --model`; shared `ModelDropdown` in `packages/views/agents` (searchable, creatable, provider-grouped, default-badge, runtime-supported gate).
|
||
|
devv-eve
|
5fa1da448f |
fix(chat): preserve chat session resume pointer across failures (#1360)
* fix(chat): preserve chat session resume pointer across failures The chat 'forgets earlier messages' bug came from PriorSessionID being silently lost in several edge cases: - UpdateChatSessionSession unconditionally overwrote chat_session.session_id, so any task that completed without a session_id (early agent crash, missing result) wiped the resume pointer to NULL. - CompleteAgentTask + UpdateChatSessionSession ran in separate calls. A follow-up chat message claimed in between resumed against a stale (or NULL) session and started over. - FailAgentTask never wrote session_id back, so a task that established a real session before failing lost its resume pointer. - ClaimTaskByRuntime only trusted chat_session.session_id and never fell back to the existing GetLastChatTaskSession query, so a single bad turn could permanently drop the conversation memory. This change: - Use COALESCE in UpdateChatSessionSession so empty inputs preserve the existing pointer; surface DB errors instead of swallowing them. - Run CompleteAgentTask/FailAgentTask + UpdateChatSessionSession inside the same transaction (TaskService now takes a TxStarter). - Extend FailAgentTask + the daemon FailTask path (client, handler, service) to forward session_id/work_dir, so failed/blocked tasks that built a real session still record it. - Fall back to GetLastChatTaskSession in ClaimTaskByRuntime when the chat_session pointer is missing, and include failed tasks in that lookup so a single failure can't lose the conversation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(daemon): forward session_id/work_dir on blocked + timeout paths runTask previously dropped result.SessionID and env.WorkDir on the non-completed return paths: - timeout returned a naked error, so handleTask called FailTask with empty session info and the chat resume pointer was either left stale or eventually overwritten with NULL. - blocked / failed (default branch) returned a TaskResult without SessionID / WorkDir, so even though FailTask now COALESCEs into chat_session, there was no value to write through. - the empty-output completion path was the same: it raised an error even when a real session_id had been built. All three paths now return a TaskResult that carries the SessionID / WorkDir the backend produced. Combined with the COALESCE-based update in UpdateChatSessionSession and the FailTask plumbing introduced in PR #1360, the next chat turn can always resume from the latest agent session — even when the previous turn timed out, was rate-limited, or returned an empty completion — instead of starting over with no memory of the conversation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(copilot): capture session id from session.start as fallback The Copilot backend only read sessionId from the synthetic 'result' event, ignoring the one already present on session.start. When the CLI was killed before result arrived (timeout, cancel, crash, or a session.error mid-turn), the daemon reported SessionID="" and the chat-session resume pointer could not advance — causing the chat to silently drop conversation memory on the next turn. Capture session.start.sessionId into state up front, and only let 'result' overwrite it when it actually carries one. result still wins when present (it is the authoritative end-of-turn record). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(copilot): parse premiumRequests as float to preserve session id Copilot CLI v1.0.32 serializes premiumRequests as a float (e.g. 7.5), not an integer. Our copilotResultUsage struct typed it as int, which made the entire 'result' line fail json.Unmarshal — silently dropping sessionId on every turn. This was the real cause of chat memory loss: the daemon reported SessionID="" to the server, chat_session.session_id stayed NULL, and the next chat turn never received --resume <id>, so each turn started a fresh Copilot session with no prior context. Add a regression test using the real JSON line from CLI v1.0.32 that asserts sessionId is preserved when premiumRequests is fractional. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Devv <devv@Devvs-Mac-mini.local> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Eve <eve@multica.ai> Co-authored-by: yushen <ldnvnbl@gmail.com> |
||
|
Azaan Ali Raza
|
b428f36ca6 |
feat: add ALLOW_SIGNUP + ALLOWED_EMAIL_* for self-hosted instances (#1098)
Closes #930 - Added environment variables to control signups - Updated frontend to hide signup text when disabled - Added backend check to block new user creation via magic link - Updated .env.example |
||
|
Naiyuan Qing
|
f0f3cb5c3a |
fix(server): resolve X-Workspace-Slug in middleware-less handlers (#1165)
Problem ------- The v2 workspace URL refactor (#1141) switched the frontend from sending X-Workspace-ID (UUID) to X-Workspace-Slug. The workspace middleware was updated to accept the slug and translate it via GetWorkspaceBySlug. But the handler package maintained a PARALLEL resolver (`resolveWorkspaceID` in handler.go) used by endpoints that sit outside the workspace middleware — and that resolver was never updated. It only checked context / ?workspace_id / X-Workspace-ID, never the slug. /api/upload-file is the one production route that hit the broken path: it's user-scoped (not behind workspace middleware) because it also serves avatar uploads (no workspace). Post-refactor requests from the frontend arrived with only X-Workspace-Slug; the handler resolver returned "", the code fell into the "no workspace context" branch, and every file upload since v2 landed in S3 with no corresponding DB attachment row — files orphaned, invisible to the UI. Root cause is structural: two resolvers doing the same job, written independently, diverged silently when one was updated. Fix --- Collapse to a single shared helper. middleware.ResolveWorkspaceIDFromRequest is the new canonical resolver; both the middleware's internal `resolveWorkspaceUUID` (for middleware gating) and the handler-side `(h *Handler).resolveWorkspaceID` (promoted from a package function) now delegate to it. Priority order matches what the middleware has had since v2: context > X-Workspace-Slug header > ?workspace_slug query > X-Workspace-ID header > ?workspace_id query. Impact analysis --------------- 47 call sites of the old `resolveWorkspaceID(r)` are renamed to `h.resolveWorkspaceID(r)`. 46 of them sit behind workspace middleware, so they hit the context fast path and see zero behavior change. The one caller that actually gains capability is UploadFile — which now correctly recognizes slug requests and creates DB attachment rows. Tests ----- - New table-driven unit test for ResolveWorkspaceIDFromRequest covers all priority levels and the unknown-slug fallback. - Regression tests for UploadFile: once with X-Workspace-Slug only (the broken path), once with X-Workspace-ID only (legacy CLI/daemon compat path). Both assert that a DB attachment row is created. - Full Go test suite passes; typecheck + pnpm test unaffected. Plan ---- See docs/plans/2026-04-16-unify-workspace-identity-resolver.md for the full first-principles writeup. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |