mirror of
https://github.com/multica-ai/multica.git
synced 2026-07-20 08:41:28 +02:00
agent/lambda/768b92e0
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Bohan Jiang
|
4fac8d772f |
feat(attribution): Human Attribution Phase 1 (MUL-4302) (#5150)
* feat(attribution): Phase 1 foundation — provenance schema + resolver (MUL-4302)
Human Attribution, Phase 1 (地基) first increment. Every agent run must be
traceable to exactly one accountable human AND record at which waterfall level
that human was resolved, so a NULL originator can be told apart from a genuine
'no human in the chain'.
- migration 149: add originator_source (waterfall label) + delegation/retry/
rerun/rule-version lineage + kind-tagged trigger evidence to agent_task_queue.
No FK, no cascade, no CHECK on the source enum (MUL-4302 §7); nullable ADD
COLUMNs = fast metadata-only change on the hot queue table.
- internal/attribution: the accountable-human vocabulary (Source, EvidenceKind,
TriggerKind) + pure, unit-tested classification rules (ClassifyComment/
ClassifyDirect). No DB, no authorization — provenance labeling only.
- service: attributionFor{IssueTask,TriggerComment} gather facts and delegate to
the pure classifier; the legacy originator resolvers now delegate here so
there is one source of truth. originator_user_id's VALUE is unchanged, so the
Composio-overlay and canInvokeAgent A2A authorization boundaries are
byte-for-byte preserved (MUL-4302 §1.3).
- enqueueIssueTask / enqueueMentionTask stamp originator_source + evidence;
CreateRetryTask carries the parent attribution forward and records
retry_of_task_id so retry and manual rerun stay separable (MUL-4302 §5).
Verified: go build ./..., go vet, gofmt clean; new attribution unit tests +
enqueue stamping integration test green; existing resolve_originator tests
unchanged.
Co-authored-by: multica-agent <github@multica.ai>
* feat(attribution): split accountable_user_id from originator, close enqueue bypasses (MUL-4302)
Phase 1, per Bohan's decision on the MUL-4302 thread: audit and authorization
answer different questions and get different columns.
- Migration 151 adds agent_task_queue.accountable_user_id (no FK, no cascade).
Authorization keeps reading ONLY originator_user_id (canInvokeAgent A2A gate,
Composio overlay); audit/UI/usage read accountable_user_id + source + evidence.
- Invariant (finalizeAttribution, single chokepoint + §11 tests): originator
non-null ⟹ accountable equals it. The two diverge only when originator is null
(autopilot / degraded fallback), which is the deferred rule_owner/owner_fallback
increment; this lands the column + mirror-write so that split has a home.
- Close the NULL-source enqueue bypasses Elon flagged: chat, quick-create,
deferred-fallback and run_only-autopilot now stamp originator_source + evidence
(+ accountable where a human exists). Autopilot stays unattributed until the
rule-version snapshot table lands, but is no longer a silent NULL-source row.
Retry inherits accountable_user_id like the rest of the attribution lineage.
- Fix assign/promote attribution (§4): a member who assigns/promotes an existing
issue is now the accountable human (and, by the invariant, originator) ahead of
the issue creator. Threaded as an OPTIONAL actor override, so comment/rerun/
autopilot paths keep today's resolution and create-with-assignee (creator ==
actor) is unchanged. The squad leader gate already judged the same member.
Also merges origin/main: renumbers the attribution migration 149→150 (main took
149 for issue_origin_agent_create) and folds agent_create into ClassifyDirect's
origin inheritance.
go build/vet/gofmt clean; attribution unit tests + service stamp/actor tests +
handler suite pass on a fresh DB migrated through 151.
Co-authored-by: multica-agent <github@multica.ai>
* docs(attribution): fix accountable NULL semantics + close chat/quick-create evidence boundary (MUL-4302)
Addresses Elon's 2nd-round review on PR #5150 (pre-merge doc/evidence items):
- Migration 151 no longer overclaims NULL. accountable_user_id is NULL not only
on pre-migration rows but on NEW rows whose audit source resolved no human yet
(run_only autopilot writes originator_source='unattributed' with NULL
accountable until rule_owner lands). Header + COMMENT ON COLUMN reworded so a
schema reader does not misjudge the invariant.
- Chat now uses the UNIFORM evidence pair (kind=chat, ref=chat_session_id), like
autopilot_run/issue_assignment, instead of relying only on the dedicated
chat_session_id column — new EvidenceChat kind. Added a service test asserting
chat stamps direct_human + chat evidence.
- Quick-create is documented as the ONE intentional no-antecedent-row path: no
comment/issue/session/run exists at enqueue time (the run creates the issue), so
trigger_evidence_kind/ref stay NULL while the human rides originator/accountable
and source is direct_human — not a NULL-source bypass.
No authorization behavior change. attribution + service + handler suites pass on a
DB migrated through 151.
Co-authored-by: multica-agent <github@multica.ai>
* chore(attribution): renumber migrations 150/151 → 157/158 after merging main (MUL-4302)
main's #5162 ("unblock release migrations") renumbered the chat migrations and
took 150 (agent_task_coalesced_comments) and 151 (chat_read_cursor), colliding
with this branch's attribution migrations. Renumber them above main's new highest
(156) so TestMigrationNumericPrefixesStayUniqueAfterLegacySet passes:
- 150_agent_task_attribution → 157_agent_task_attribution
- 151_agent_task_accountable_user → 158_agent_task_accountable_user
Fixed the internal "migration 150" references in 158's header to 157. Migrations
apply cleanly through 158 on a fresh DB; migration lint green.
Co-authored-by: multica-agent <github@multica.ai>
* feat(attribution): autopilot rule_owner — accountable = rule version publisher (MUL-4302)
Implements rule_owner (MUL-4302 §3.4), the first attribution source where the
accountable human diverges from the (NULL) authorization originator.
- Migration 159 adds the append-only autopilot_rule_version snapshot table (no FK,
no cascade); migration 160 adds its CONCURRENTLY lookup index.
- Write-on-publish: CreateAutopilot appends v1 (publisher = creator); UpdateAutopilot
appends a new version when a SUBSTANTIVE autopilot-row field changes (assignee /
status / execution_mode) — cosmetic edits (title/description/template) write none.
Both run inside the existing handler tx (atomic with the autopilot write).
- Dispatch resolution: both autopilot execution modes now resolve the active rule
version and stamp originator_source='rule_owner', accountable_user_id=publisher,
rule_version_id=<snapshot>, with originator_user_id left NULL (authorization
unchanged). run_only stamps CreateAutopilotTask directly; create_issue resolves in
attributionForIssueTask so both modes attribute identically. A missing version /
non-member publisher degrades to unattributed — never fabricates a human.
- finalizeAttribution now enforces the invariant ONE-WAY: it mirrors originator onto
accountable only when originator is valid, leaving an explicitly-set accountable
(rule_owner / future owner_fallback) intact when originator is NULL. Added
rule_version_id to CreateAgentTask so the create_issue path persists it too.
Also merges origin/main and renumbers this branch's attribution migrations
150/151 → 157/158 (main's #5162 took 150/151); rule_version table is 159/160.
Tests: attribution unit RuleOwner + one-way invariant table; service integration
tests proving an autopilot-origin issue stamps rule_owner + rule_version_id (and
degrades to unattributed with no version). Full service/attribution/handler/
migration suites pass on a DB migrated through 160; build/vet/gofmt clean.
Deferred (same PR): trigger-table republish (cron/webhook/event_filters) and
system-pause/archive versioning; owner_fallback + fail-closed; manual rerun.
Co-authored-by: multica-agent <github@multica.ai>
* fix(attribution): manual autopilot trigger → direct_human to the triggering member (MUL-4302)
Elon's blocking finding: a member manually triggering an autopilot was attributed
rule_owner (accountable = rule publisher, originator NULL) like a schedule/webhook
run, so member B triggering member A's autopilot landed accountable=A and carried
no originator authorization context for the run. Per MUL-4302 §4 a manual "run now"
is a direct human action and must attribute direct_human to the triggering member.
- Thread the triggering member from TriggerAutopilot into dispatch: new
DispatchAutopilotManual carries actorUserID (resolved via resolveActor +
memberActorUserID, so only a member actor is a human; an A2A agent actor falls
back to rule_owner). DispatchAutopilot / DispatchAutopilotForPlan keep their
public signatures (pass an invalid actor); only the internal dispatchAutopilot /
dispatchCreateIssue / dispatchRunOnly gained the param, so the many existing
callers are untouched.
- run_only: dispatchRunOnly stamps direct_human (originator == accountable ==
actor, no rule_version) for a manual actor, else rule_owner. CreateAutopilotTask
gains an originator_user_id param for the manual case.
- create_issue: dispatchCreateIssue enqueues a manual trigger via the actor-carrying
*WithHandoff entry points; attributionForIssueTask's autopilot-origin rule_owner
branch is now guarded on !actorUserID.Valid, so a valid actor falls through to the
direct_human override. Both execution modes attribute identically.
- schedule / webhook keep rule_owner (no actor). Trigger-table + system-pause/archive
versioning remain the pre-merge follow-ups.
Tests: the run_only row assertion Elon asked for (schedule → rule_owner row on
CreateAutopilotTask), plus manual direct_human on BOTH modes (run_only and
create_issue), including a manual actor distinct from the rule publisher. Full
service/attribution/handler/migration/scheduler/cmd suites pass on a DB migrated
through 160; build/vet/gofmt clean.
Co-authored-by: multica-agent <github@multica.ai>
* feat(attribution): owner_fallback + fail-closed policy, and manual-rerun direct_human (MUL-4302)
Two of the three remaining Phase 1 items (trigger-table / system-pause versioning
is deferred — see PR description).
owner_fallback + fail-closed (§1/§3.5) — the never-null accountable guarantee:
- attribution.OwnerFallback degrades an UNATTRIBUTED result to owner_fallback:
accountable = agent owner, originator stays NULL (audit-only, authz untouched),
Source.Precise()==false. finalizeAttribution's one-way invariant already allows
accountable-set / originator-NULL divergence, so nothing else changes.
- Migration 161 adds workspace.attribution_fail_closed (default FALSE) + a lean
GetWorkspaceAttributionFailClosed read. (Also added the column to ListWorkspaces'
explicit column list so its row type stays db.Workspace.)
- applyAttributionFallback is applied at every enqueue boundary (issue, mention,
chat, quick-create, deferred-fallback, autopilot run_only): unattributed →
owner_fallback (agent owner) by default, or ErrAttributionFailClosed when the
workspace is fail-closed, which the caller surfaces to refuse the enqueue (the
run does not start). So no run is left without an accountable human, and a
compliance workspace can block unattributable runs instead.
manual rerun (§5) — a rerun is a NEW direct_human trigger to the rerunning member:
- RerunIssue threads the acting member (resolved in the handler via resolveActor)
down to enqueueRerunTask, and attributionForIssueTask is now actor-first so the
actor wins over an INHERITED trigger comment (a rerun keeps the comment for the
daemon's prompt context but must attribute to whoever clicked rerun, not the
original comment's human).
- rerun_of_task_id lineage is recorded via a targeted SetAgentTaskRerunOf update on
the rerun path only (keeping the shared CreateAgentTask insert untouched), so
system retry (retry_of_task_id) and human rerun stay separable in reporting.
Tests: OwnerFallback unit test; owner_fallback + fail-closed-refusal + manual-rerun
(direct_human + rerun_of_task_id) service tests; the prior "degrades to unattributed"
test updated to owner_fallback. Full service/attribution/handler/migration/scheduler/
cmd suites pass on a DB migrated through 161; build/vet/gofmt clean.
Co-authored-by: multica-agent <github@multica.ai>
* fix(attribution): close fail-open holes + move rerun_of_task_id into creation snapshot (MUL-4302)
Addresses Elon's two must-fixes on PR #5150.
1. accountable-never-null / fail-closed had fail-open holes. applyAttributionFallback
now, for an UNATTRIBUTED run, refuses the enqueue (ErrAttributionFailClosed) in
THREE cases instead of silently degrading to a runnable NULL-accountable task:
- workspace policy read fails (or no workspace) → fail closed; we cannot confirm
fallback is permitted, so we don't run an unattributable task on a DB hiccup.
(Only the rare unattributed path pays this; precise runs never read the policy.)
- workspace is fail-closed → refuse (unchanged).
- owner_fallback has no valid agent owner → refuse rather than enqueue a task with
a NULL accountable_user_id.
ErrAttributionFailClosed's doc now covers all three "cannot guarantee an
accountable human" refusals. Added missing-owner / policy-read-failure /
precise-passthrough tests.
2. manual rerun rerun_of_task_id was a post-notify UPDATE (race: the queued event /
daemon claim could see rerun_of_task_id = NULL, and a failed update degraded the
run to a plain direct_human). It now rides the CreateAgentTask insert — threaded
through enqueueIssueTask / enqueueMentionTask as a creation param (like
retry_of_task_id) so it is written in the same statement before the daemon is
notified. Removed the SetAgentTaskRerunOf follow-up query.
Also merges origin/main (unrelated CLI fix #5167, no conflict). Full service /
attribution / handler / migration / scheduler / cmd suites pass on a DB migrated
through 161; build / vet / gofmt clean.
Co-authored-by: multica-agent <github@multica.ai>
* feat(attribution): rule_owner versioning on trigger edits + system-pause/archive (MUL-4302)
The final remaining Phase 1 item: substantive publishes beyond the autopilot row now
republish the rule version, so a run's rule_owner accountable follows whoever last
changed what the rule does.
- Extracted the config-summary + insert into service.RecordAutopilotRuleVersion so
the handler and the (different-package) failure monitor share one writer; the
handler's recordAutopilotRuleVersion is now a thin wrapper.
- Trigger edits: UpdateAutopilotTrigger and DeleteAutopilotTrigger republish the rule
version with the acting member as publisher, ATOMICALLY (tx-wrapped mutation +
version write, mirroring CreateAutopilot/UpdateAutopilot). CreateAutopilotTrigger
republishes best-effort — the webhook path mints its token with a retry loop that
cannot share one tx, and a create is usually initial setup already covered by v1;
a failed write there is benign (active version stays the current publisher, the new
trigger fires under it, no immediate daemon claim rides it).
- Archive (DeleteAutopilot) republishes (member, status=archived), tx-wrapped.
- System auto-pause (failure monitor) republishes with a 'system' publisher,
best-effort — a background sweep to a non-dispatching state (a paused autopilot
never dispatches; a later member resume supersedes).
- RotateWebhookToken / SetSigningSecret deliberately do NOT version: they rotate
credentials, not the rule's behavior (not §3.4 substantive).
Semantics: a system-published (no-member) active version degrades dispatch to
unattributed → owner_fallback, never fabricating a human.
Tests: republish-reattributes (member A → member B supersedes → dispatch resolves to
B; system publisher → unattributed). Full service/attribution/handler/migration/
scheduler/cmd suites pass on a DB migrated through 161; build/vet/gofmt clean.
Also merges origin/main (unrelated frontend feature #5074).
Co-authored-by: multica-agent <github@multica.ai>
* fix(attribution): make trigger-create rule-version republish atomic (MUL-4302)
Addresses Elon's final Phase 1 blocking finding: CreateAutopilotTrigger recorded the
rule-version republish best-effort AFTER the trigger insert. If member B added a
schedule/webhook trigger to member A's autopilot and the version write failed, future
schedule/webhook dispatches would keep attributing to A — violating the rule_owner
invariant that the last member to substantively change the rule owns future runs
("no immediate daemon claim" doesn't save it, since the miss surfaces at the LATER
trigger firing).
Both create paths now write the version in the SAME tx as the trigger INSERT:
- schedule create: wrap CreateAutopilotTrigger + recordAutopilotRuleVersion in one tx.
- webhook create: each mint-with-retry attempt runs in its own tx (insert + version
commit together; a token collision rolls that attempt back and retries with a fresh
token; a version-write failure rolls the trigger back). Passes ap + the acting
member id into the helper.
- removed the best-effort recordTriggerRuleVersionBestEffort helper (and the now-unused
slog import).
Test: TestCreateTrigger_RepublishesRuleVersionAtomically drives both create paths
through the handler and asserts a rule version is published by the acting member.
Existing webhook/trigger/archive handler tests still pass. Also merges origin/main
(unrelated avatar feature #5074). Full service/attribution/handler/migration/
scheduler/cmd suites pass on a DB migrated through 161; build/vet/gofmt clean.
Co-authored-by: multica-agent <github@multica.ai>
* feat(attribution): Phase 2.1 — surface run attribution on the task API (MUL-4302 §9)
First Phase 2 (visibility) increment: the agent-task API now returns the resolved
accountable-human provenance so the UI can render an "on behalf of" badge.
- AgentTaskResponse gains an `attribution` object: source label (never blank —
pre-migration NULL renders "unattributed") + `precise` flag (false for the degraded
owner_fallback / backfill / unattributed sources), the initiator (accountable) and
originator (authorization) user refs, the evidence {kind, ref_id} pointer, and the
rule_version / delegated / retry / rerun lineage ids.
- The label + evidence + raw ids are built in the PURE taskToResponse (no DB), so
every task response carries them. Names are hydrated separately, only on the
user-facing surfaces (ListAgentTasks, ListWorkspaceAgentTaskSnapshot, RerunIssue,
CancelTaskByUser) — daemon-claim paths stay lean.
- Hydration resolves initiator/originator from the GLOBAL user table (departed-member
safe) via a new batch GetUsersByIDs query (no N+1); best-effort, so a lookup hiccup
leaves the raw ids intact.
Tests: pure taskAttributionBase (direct_human / rule_owner NULL-originator /
owner_fallback degraded / pre-migration→unattributed) + DB hydration (fills known
ref, leaves unknown id un-filled, skips nil). Full handler/service/attribution/
migration/scheduler/cmd suites pass on a DB migrated through 161; build/vet/gofmt
clean. The field is additive — the frontend's parseWithFallback ignores unknown keys,
so nothing breaks until the UI increment consumes it.
Also merges origin/main (unrelated editor feature #5090).
Remaining Phase 2 (next increments, same PR): frontend zod schema + "on behalf of"
badge + evidence-chain jump; append-only correction events (write + display).
Co-authored-by: multica-agent <github@multica.ai>
* feat(attribution): Phase 2.2 — on-behalf-of badge in the execution log (MUL-4302 §9)
Surface the accountable human on every agent run row:
- AttributionBadge composes Badge + ActorAvatar, shows "on behalf of <member>"
with the resolution source as a tooltip; degraded (non-precise) attribution
gets a warning tone, and an unresolved initiator renders an explicit
"no responsible member" chip.
- Wire the badge into both active and past rows of the execution log.
- Mirror the attribution shape into AgentTaskResponseSchema (defensive, .loose())
so the cancel-task path carries it through zod; add parse tests.
- Export TaskAttribution/AttributionUser/TaskEvidence from @multica/core/types
and add the attribution block to all four issues.json locales.
Co-authored-by: multica-agent <github@multica.ai>
* fix(attribution): hydrate initiator names on issue-facing task endpoints + bound the badge (MUL-4302 §9)
Address Elon's PR #5150 review:
- ListTasksByIssue (the execution-log data source), GetActiveTaskForIssue and the
issue-scoped CancelTask now call hydrateTaskAttributions, so the "on behalf of
<member>" badge shows the real member name on issue detail instead of falling
back to "someone". Mirrors the existing ListAgentTasks / snapshot behavior.
- AttributionBadge: cap width (max-w-40, min-w-0) and truncate the name span so a
long name / narrow right column can't squeeze out trigger/status/actions; keep
the avatar shrink-0.
- Add a handler test asserting the issue task list returns a hydrated
attribution.initiator.name.
Co-authored-by: multica-agent <github@multica.ai>
* fix(attribution): use semantic AvatarSize 'xs' for the badge avatar
main refactored ActorAvatar.size from a raw pixel number to the semantic
AvatarSize union (packages/ui/lib/avatar-size). Switch the on-behalf-of badge
avatar from size={14} to size="xs" (16px) after merging main.
Co-authored-by: multica-agent <github@multica.ai>
* fix(attribution): stage-cascade falls back to parent-issue provenance, not agent owner (MUL-4302)
When closing the last sub-issue in a Stage wakes the parent's assignee agent, the
run was enqueued via a system-authored child-done comment with no actor, which the
resolver classified as unattributed and then degraded to owner_fallback (the agent's
own owner). That is the wrong accountable human: the woken run should be accountable
to whoever caused the parent issue to exist.
attributionForIssueTask now detects a system-authored trigger comment and falls
through to the parent issue's own provenance — the same creator / agent_create-origin
/ autopilot-origin chain a direct enqueue resolves (so an agent-decomposed parent
attributes via delegation to the human who drove it; a member-created parent to that
member; an autopilot parent to the rule publisher). owner_fallback is now only the
last resort when the parent provenance itself has no human.
- Extract attributionFromComment so attributionForIssueTask can inspect author_type
without a second GetComment; authorization resolution stays byte-identical.
- Add a DB-backed test asserting a system child-done comment resolves to the parent
issue's origin human (delegation), not owner_fallback.
Co-authored-by: multica-agent <github@multica.ai>
* feat(attribution): autopilot runs attribute to the firing trigger's creator (MUL-4302)
Per Bohan: an autopilot schedule/webhook run should be accountable to the human
who created the SPECIFIC trigger that fired it, not the rule publisher. (Manual
triggers already attribute to the invoking member via direct_human — unchanged.)
- Migration 162: add autopilot_trigger.created_by_type/created_by_id (nullable, no
FK/cascade). Capture the creating member at both trigger-create sites (schedule +
webhook).
- New precise source trigger_owner: originator stays NULL (an autonomous fire
carries no human authorization — same authz-safe divergence as rule_owner),
accountable = the trigger's member creator.
- triggerOwnerAttribution resolves run.trigger_id → creator; wired into run_only
dispatch and the create_issue path (bridging issue → active run → trigger_id).
Legacy triggers with no recorded creator, and agent-created triggers, degrade to
rule_owner then owner_fallback — nothing regresses.
- Frontend: trigger_owner source label in all four locales + badge switch case.
- Tests: attribution TriggerOwner unit + Precise/invariant; DB-backed resolver
tests (member creator → trigger_owner; creatorless → rule_owner fallback).
Co-authored-by: multica-agent <github@multica.ai>
* chore(attribution): re-trigger CI (dropped synchronize event on
|
||
|
Multica Eve
|
6b4d690664 |
MUL-4744: restore Autopilot webhook response contract (#5397)
* fix(autopilots): restore webhook response contract Co-authored-by: multica-agent <github@multica.ai> * fix(autopilots): preserve run id on webhook retries Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Eve <eve@multica-ai.local> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Multica Eve
|
e07b5403ab |
MUL-4502: make autopilot webhook admission durable (#5386)
* fix(autopilots): make webhook admission durable Co-authored-by: multica-agent <github@multica.ai> * fix(autopilots): address webhook delivery review Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Eve <eve@multica-ai.local> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Raúl Anatol
|
2b5696703f |
MUL-2703: feat(autopilots): webhook event filters per trigger (MUL-2334 follow-up) (#3231)
* feat(autopilots): webhook event filters per trigger (MUL-2334 follow-up) Adds schema-backed event/action filtering to webhook triggers so operators can declare exactly which GitHub (or generic) events should spawn autopilot runs. Events outside the declared scope are recorded as ignored with reason 'event_filtered' — visible in the delivery log but without expensive run/task creation. Closes #3093 (supersedes the description-parsing approach from that PR). Backend: - Migration 108 adds event_filters JSONB to autopilot_trigger - sqlc queries updated for CREATE / UPDATE / LIST / GET - HandleAutopilotWebhook filters against trigger.event_filters before dispatch - Create/Update trigger handlers accept event_filters in the request body - Response shape includes event_filters so the UI can render it Frontend: - New WebhookEventFilterSection component in the autopilot dialog - Inputs for event name + comma-separated actions - i18n strings added (en + zh-Hans) Tests: - Unit tests for splitWebhookEvent and webhookEventAllowedByTriggerScope - Handler-level integration tests for filtered / allowed / no-filter paths co-authored-by: ZephaniaCN <agent/autopilot-webhook-filter> * fix: recognize gitlab/bitbucket/gitea as providers in splitWebhookEvent TestSplitWebhookEvent failed because only 'github' was recognized as a provider prefix. Extract isKnownProvider() to handle gitlab, bitbucket, and gitea as well. * fix(autopilots): address PR #3231 review for webhook event filters Must-fix from PR #3231 review: 1. event_filters now uses typed []WebhookEventFilter at the HTTP boundary instead of []byte. encoding/json was base64-encoding the field on the way out, so the UI could not .map() the response, and a real JSON array on the way in failed to decode. Response field also decodes the stored JSONB into a typed slice before serialising back. 2. UpdateAutopilotTriggerRequest.EventFilters is *[]WebhookEventFilter with tri-state PATCH semantics: nil pointer = leave alone, [] = clear, [...] = replace. The handler marshals an explicit empty slice to the JSONB literal `[]` so COALESCE overwrites instead of preserves. AutopilotDialog now PATCHes the webhook trigger when event_filters change in edit mode (previously the toast said "updated" while the backend was unchanged). 3. webhookEventAllowedByTriggerScope no longer short-circuits to false on the first event-name match whose actions don't line up. Earlier code silently shadowed any later filter that shared the same event name with disjoint actions. Robustness: validateWebhookEventFilters rejects empty event names / actions at write time, and the matcher fails closed on malformed stored bytes instead of widening the allowlist. Tests: handler tests now post real JSON arrays (the prior []byte path masked the contract bug). Adds round-trip / clear-with-[] / preserve- when-omitted / replace / invalid-filter / filters-on-schedule coverage, plus matcher tests for same-event multi-filter and malformed-deny. Migration renamed 108 → 110 to avoid colliding with main's 108_task_token (came in via the merge from main). |
||
|
Kerim Incedayi
|
9418d2a2c1 |
feat(autopilots): webhook triggers (server + CLI + UI + docs) MUL-2049 (#2348)
* feat(server): add webhook trigger DB migration + sqlc queries
Lays the foundation for webhook autopilot triggers:
- partial unique index on autopilot_trigger.webhook_token (kind=webhook only)
so the public ingress route can resolve a trigger in O(1)
- GetWebhookTriggerByToken / TouchAutopilotTriggerFiredAt /
RotateAutopilotTriggerWebhookToken / SetAutopilotTriggerWebhookToken
queries, regenerated with sqlc
* feat(server): webhook token generator + payload normalizer
Two pure helpers for the webhook autopilot work:
- generateWebhookToken: 32 random bytes -> base64-url, "awt_" prefix.
256 bits of entropy keeps brute-force off the table; the prefix makes
leaked tokens recognisable in logs.
- normalizeWebhookPayload: turns arbitrary JSON into the WebhookEnvelope
shape (event/eventPayload/request) used by trigger_payload. Header- and
body-based event inference covers GitHub, GitLab, X-Event-Type, and
caller-provided envelopes; scalar/empty/invalid bodies are rejected so
the handler can answer 400.
* feat(server): generate webhook tokens and expose rotate endpoint
- New handler.Config.PublicURL fed by MULTICA_PUBLIC_URL env so
/api/autopilots/.../triggers responses can include an absolute
webhook_url alongside the always-present webhook_path.
- CreateAutopilotTrigger now mints a webhook_token via crypto/rand
for kind=webhook and ignores cron/timezone for non-schedule kinds.
api triggers stay accepted-but-inert per PLAN.md.
- New POST /api/autopilots/{id}/triggers/{triggerId}/rotate-webhook-token
protected by the existing workspace auth group; old tokens stop
working immediately because the unique-index lookup keys on the
current row value.
* feat(server): public webhook ingress route + per-token rate limiter
- New POST /api/webhooks/autopilots/{token} route, mounted outside the
authenticated group: the path token is the credential. Workspace
context is derived from the joined autopilot row, never headers.
- Body capped at 256 KiB via http.MaxBytesReader; oversized payloads
return 413 mid-read instead of being fully buffered.
- Disabled triggers / paused / archived autopilots return
200 {"status":"ignored"} so providers stop retrying.
- Skipped-runtime dispatches surface 200 {"status":"skipped"} with the
reason from the autopilot service's pre-flight admission check.
- WebhookRateLimiter interface with sliding-window in-memory + Redis
Lua-script implementations. Default 60 req/min per token. Test
coverage on the in-memory path; Redis variant fails open on cache
errors so a Redis hiccup never blocks ingress.
- Integration tests exercise token generation, dispatch, payload
envelope persistence, GitHub-header inference, paused/disabled
short-circuits, oversized rejection, and rotate-then-old-token-404.
* feat(server): include webhook payload in create_issue description
When an autopilot run is triggered by a webhook and execution_mode is
create_issue, the agent only sees the issue body — never the run's
trigger_payload. Append a 'Webhook event:' line and a fenced JSON block
with the normalized eventPayload so the agent has the inbound context
inline. Schedule / manual runs are unchanged.
Tests cover:
- schedule path keeps existing italic note, no webhook block
- webhook path emits event line + payload block, italic before block
- non-envelope JSON falls back to raw body (defensive)
- non-webhook source with payload still gets no webhook block
* feat(core): types, API client and mutations for webhook triggers
- AutopilotRunStatus gains 'skipped' so the run-list UI handles the
admission-skipped state explicitly instead of falling through to a
generic case (the backend already emits it via MUL-1899).
- AutopilotTrigger picks up optional webhook_path / webhook_url. Both
are optional so older self-hosted servers that pre-date this change
still parse cleanly.
- buildAutopilotWebhookUrl helper composes a usable absolute URL with
the priority webhook_url > apiBaseUrl + path > origin + path > path.
Tested with seven cases covering each branch.
- ApiClient.rotateAutopilotTriggerWebhookToken posts to
/api/autopilots/{id}/triggers/{triggerId}/rotate-webhook-token; the
HTTP-contract test pins URL + method.
- useRotateAutopilotTriggerWebhookToken mutation invalidates
autopilotKeys.detail on settle, mirroring the existing trigger-mutation
pattern.
* feat(views): webhook trigger UI in Add Trigger dialog and trigger row
Add Trigger dialog gains a Schedule/Webhook segmented toggle:
- Schedule reuses TriggerConfigSection unchanged.
- Webhook hides the cron config and shows a help line; the trigger is
created with kind=webhook and the URL is generated server-side.
- Toast text differentiates schedule vs webhook on success.
TriggerRow grows a webhook branch:
- Webhook icon, kind translated via trigger_kind.
- URL shown in a truncating monospace pill, with copy + rotate
buttons. Copy uses navigator.clipboard with toast feedback; rotate
uses an AlertDialog confirm because the old URL stops working
immediately.
- api triggers render a Deprecated badge and skip URL/copy/rotate
affordances.
RunRow gains a 'skipped' RUN_VISUAL entry (muted dash) so admission-
skipped runs don't fall through to a generic case. Source label uses the
new run_source i18n key instead of capitalize.
Locales: en + zh-Hans gain run_status.skipped, run_source.*,
trigger_kind.*, trigger_row.{copy_url,rotate_url,*_confirm_*,toast_*},
add_trigger_dialog.{type_*,webhook_help,toast_added_{schedule,webhook}}.
* feat(cli): support webhook trigger creation and URL rotation
- multica autopilot trigger-add now takes --kind schedule|webhook
(default schedule for backward compatibility). For webhook it skips
--cron / --timezone validation and prints the resulting webhook URL,
preferring the server-provided webhook_url and falling back to
client.BaseURL + webhook_path.
- New multica autopilot trigger-rotate-url <autopilot-id> <trigger-id>
command for rotating the bearer URL of a webhook trigger.
* docs(autopilots): add webhook trigger guide (en + zh)
Replaces the 'Webhook and API triggers are not available yet' section
with end-to-end webhook documentation: how the URL is generated, what
payload shapes are accepted, the inferred-event rules, the bearer-secret
warning + rotate flow, status-code semantics for accepted/skipped/
ignored/4xx/5xx outcomes, and the MULTICA_PUBLIC_URL self-host
configuration.
Run history list now mentions skipped status. The 'unavailable
features' section narrows to api-kind triggers, HMAC signing, IP
allowlists, and provider presets.
* feat(views): add Schedule/Webhook toggle to the create autopilot dialog
Closes the gap where a brand-new autopilot could only be created with a
schedule trigger. The right-column config now has a Trigger section
with a segmented Schedule/Webhook control:
- Schedule keeps the existing cron/timezone UI.
- Webhook hides the cron UI and shows a help line; on submit, a
kind=webhook trigger is created right after the autopilot.
In edit mode the toggle is intentionally hidden (PLAN.md treats trigger-
type changes as delete-old + create-new, not in-place updates), but the
panel still picks the right kind based on props.triggers[0].kind so a
webhook autopilot doesn't render an irrelevant cron form.
Locales: section_trigger_kind, trigger_kind_{schedule,webhook},
section_webhook, webhook_help_{create,edit} added in en + zh-Hans.
* feat(views): show webhook URL inline after creating a webhook autopilot
After a successful create with kind=webhook, the dialog stays open and
swaps to a confirmation panel showing the freshly minted URL with a
copy button + 'Treat this URL like a password' warning + Done button.
Avoids the friction of "create the autopilot, then go find it in the
list, click in, scroll to triggers, copy URL."
Locales: dialog.webhook_created_{title,description,warning,done} added
in en + zh-Hans.
Schedule create flow is unchanged (toast + close). The success panel is
gated on the trigger returned from the create mutation, so a partial
failure (autopilot created, trigger creation errored) still falls
through to the toast_create_partial path.
* feat(views): show webhook payload in run detail dialog
The agent transcript dialog now accepts an optional headerSlot that
sits above the event list. The autopilot RunRow drops a
WebhookPayloadPreview into that slot when the run came from a webhook
and trigger_payload is non-empty.
The preview is collapsed by default (the transcript itself is the main
event), shows the inferred event name + receivedAt in the header, and
reveals the eventPayload as pretty-printed JSON with a copy button on
expand. Falls back gracefully if the row's trigger_payload doesn't
match the WebhookEnvelope shape — the whole value is shown instead so
nothing is hidden.
Closes the "agent didn't echo the payload, now I can't see what
triggered the run" gap. PLAN.md tracked this as
"Payload preview in run history" under follow-ups.
Locales: webhook_payload.{label, unknown_event, payload, content_type,
copy, copied, copied_short, copy_failed} added in en + zh-Hans.
* chore(server): wire MULTICA_PUBLIC_URL through self-host compose
Two small follow-ups split out of the webhook trigger PR:
- docker-compose.selfhost.yml passes MULTICA_PUBLIC_URL into the
backend container so a self-hosted deployment behind a real domain
gets absolute webhook URLs in the trigger response. Documented in
.env.example with the rationale for not deriving the public host
from request headers.
- Drop a duplicated 'invalid json:' prefix in the webhook ingress
400 error path. normalizeWebhookPayload already prefixes its
errors, so the handler doesn't need to re-prefix.
* fix(migrations): renumber webhook trigger migration 081 → 089 to avoid collision
The branch's 081_autopilot_webhook_triggers.{up,down}.sql collided
numerically with 081_runtime_timezone.{up,down}.sql that landed on
main, making migration apply order undefined. Renumber to 089 so the
file slots after the latest main migration (088_squad_instructions).
The SQL itself doesn't conflict — it only creates a partial unique
index on autopilot_trigger.webhook_token — but the duplicate prefix
is what the migration runner sees, so the filename must move.
* fix(autopilot-webhook): address PR review blocking issues
- Redact bearer tokens from request logs: paths matching
/api/webhooks/autopilots/<token> now log "[redacted]" instead of the
token. The resolved trigger ID is plumbed via context so audit lines
stay useful for debugging. (Review item Blocking #1.)
- Distinguish pgx.ErrNoRows from transient DB errors in token lookup:
no-row stays 404 (so providers don't retry on a deleted webhook),
other errors return 500 (which providers DO retry, avoiding silent
drops on DB blips). (Review item Blocking #2.)
- Add per-IP sliding-window rate limiter that runs BEFORE the token
lookup, so spraying random tokens can no longer probe the
autopilot_trigger index unboundedly. Reuses the existing Lua script
with a separate Redis key namespace; falls open on Redis errors.
Default budget 30 req/min/IP. (Review item Blocking #3.)
The webhook handler now applies the gates in the order: per-IP rate
limit → token lookup → per-token rate limit → handler logic.
* fix(autopilot): atomic webhook trigger creation + strict kind/timezone validation
- Mint the webhook bearer token BEFORE the INSERT and pass it via
CreateAutopilotTriggerParams so the row never exists in a half-written
kind=webhook + webhook_token=NULL state. On the (vanishingly rare)
unique-index collision the whole INSERT is retried with a fresh token
— no UPDATE second step. Removes the now-dead attachFreshWebhookToken
helper. (Review item Recommended #4.)
- Add new GET /api/autopilots/{id}/runs/{runId} endpoint that returns a
single run including the full trigger_payload. The list response is
now slim (omits trigger_payload) so worst-case payload size drops
from ~5 MB to ~5 KB. (Review item Recommended #5, server side.)
- Reject kind=api with 400 ("kind=api is deprecated; use schedule or
webhook") and reject kind=webhook with --timezone with 400 — both
surfaces stragglers loudly instead of silently dropping fields.
CLI mirrors the check so --timezone with --kind webhook errors
client-side. (Review nits.)
- Add --yes (-y) flag and an interactive y/N confirmation prompt to
`multica autopilot trigger-rotate-url` so the destructive rotate
matches the UI's AlertDialog safety. (Review item Recommended #6.)
* fix(views): fetch webhook payload on-demand and truncate at 4 KiB
- Add useAutopilotRun query hook + getAutopilotRun API client method
paired with the new server endpoint. The run-detail dialog now mounts
a WebhookPayloadSlot that fetches the full run (incl. trigger_payload)
lazily — list responses no longer carry up to 256 KiB × N runs of
envelope data.
- WebhookPayloadPreview truncates its in-DOM <pre> at 4 KiB with a
localized marker so jank-y machines aren't asked to render a 256 KiB
JSON blob. The Copy button still yields the full string.
- Adds the truncated_marker i18n string to en + zh-Hans.
Review items Recommended #5 (frontend) and a nit on the preview's
unbounded <pre>.
* test(autopilot-webhook): close coverage gaps flagged in PR review
- request_logger: redactWebhookPath unit tests + integration test
proving the bearer token never lands in slog output, plus the
webhook_trigger_id context plumbing.
- autopilot_webhook_handler: empty body → 400, archived autopilot →
200 ignored, per-IP rate limiter trips before DB lookup, kind=api
and webhook+timezone are rejected at 400, slim list + full detail
endpoint round-trip.
- webhook_rate_limiter: Lua script structure guard (catches reordering
even without a live Redis), plus live-Redis tests for both per-token
and per-IP limiters (REDIS_TEST_URL gated, matching the existing
Redis test pattern in the package).
- WebhookPayloadPreview: envelope rendering, fallback shape, and the
>4 KiB truncation path with full-payload-on-Copy guarantee.
Two branches are documented as code-review-protected rather than
covered by tests: the 500-on-DB-error path requires injecting a stub
Queries (no interface here), and the cross-workspace defense-in-depth
check is unreachable from valid SQL state.
* fix(middleware): SetWebhookTriggerID must mutate request in place
The round-1 helper returned a fresh *http.Request from WithContext, and
the webhook handler did `r = SetWebhookTriggerID(r, ...)`. That swaps
the handler's local pointer but doesn't propagate the new context back
to RequestLogger, which is still holding the original *http.Request —
so the audit line never actually included webhook_trigger_id in
production. The round-1 test happened to pass because it pre-stashed
the value on the request before calling ServeHTTP, bypassing the bug
it was meant to verify.
Switch to in-place mutation via `*r = *r.WithContext(...)` so the
wrapping middleware sees the new context after next.ServeHTTP returns,
and update the test to exercise the real call pattern (set the context
from inside the handler, assert the surrounding logger reads it).
Verified live: an accepted webhook now logs
path=/api/webhooks/autopilots/[redacted] webhook_trigger_id=<uuid>
* fix(autopilot-webhook): symmetric ErrNoRows split + trusted-proxy gate
Round-2 review (Bohan-J, PR #2348 follow-up):
- Must-fix #1: the second lookup at autopilot_webhook.go:258
(GetAutopilot after the token resolves) was folding every error into
404. A transient DB blip would tell a webhook sender "not found" and
it would never retry. Apply the same errors.Is(err, pgx.ErrNoRows)
→ 404 / else → 500 split as the first lookup got in round 1.
- Must-fix #2: clientIPForRateLimit was honoring X-Forwarded-For /
X-Real-IP from any caller. An attacker spraying random tokens could
just rotate the XFF header and the per-IP bucket became per-request,
so the limiter that's specifically supposed to gate spraying before
it hits the DB unique index was bypassed.
New shape — matches Bohan's suggestion exactly:
* Default: r.RemoteAddr only, headers ignored.
* Operator opt-in via MULTICA_TRUSTED_PROXIES (comma-separated
CIDRs). XFF/X-Real-IP are honored only when r.RemoteAddr is
inside one of the listed prefixes; otherwise they're dropped.
Wired through .env.example and docker-compose.selfhost.yml so
self-host operators can configure their reverse-proxy's CIDR.
Invalid CIDRs in the env var are dropped with a single slog.Warn at
startup rather than crashing the server. Uses net/netip (stdlib,
value-typed) for parsing and containment checks.
Verified live on the rebuilt self-host backend: a 35-request spray
from one source with rotating XFF gets the expected 30× 404 + 5× 429,
proving the per-IP bucket is keyed on the real connection IP.
* fix(autopilot): reject cron/timezone PATCH on non-schedule triggers
Round-2 review should-fix. CreateAutopilotTrigger already 400s on
kind=webhook + timezone/cron_expression, but UpdateAutopilotTrigger
silently wrote those fields regardless of prev.Kind. The values then
sat in the DB visible to nobody and read by nothing — a back door that
left the API contract fuzzy across create vs update.
Mirror the create-path discipline: after loading prev, if prev.Kind
!= "schedule" and the PATCH body sets cron_expression or timezone,
return 400 with a clear message. enabled and label remain accepted on
every kind.
The existing prev.Kind == "schedule" guard on next_run_at recompute
stays as belt-and-braces, but with this gate in place the recompute
branch is now reachable only for the kind it was meant for.
* test(autopilot-webhook): close round-2 coverage gaps
- IPRateLimitNotBypassedByXFFSpoof: drives the must-fix #2 invariant
by rotating XFF across three calls from the same RemoteAddr and
asserting the third gets 429. Pre-round-2 this test would have
passed for the wrong reason (limiter trusted XFF, so per-bucket
collision was incidental); now it pins the bypass-closed property.
- IPRateLimitReturns429BeforeDBLookup: updated to set RemoteAddr
explicitly and drop the XFF header it was leaning on. With
TrustedProxies empty (test default) the limiter keys on the real
connection IP, which is what the test wants to assert anyway.
- UpdateAutopilotTrigger_RejectsCronExpressionOnWebhookKind +
UpdateAutopilotTrigger_RejectsTimezoneOnWebhookKind: drive the
round-2 should-fix from the handler boundary.
- UpdateAutopilotTrigger_AcceptsEnabledAndLabelOnWebhookKind: counter
test so a regression to a blanket reject is caught.
* fix(migrations): bump webhook trigger migration 089 → 091
origin/main added 089_squad_no_action_activity_index (and 090_task_is_leader)
since our last rebase, re-colliding with our 089_autopilot_webhook_triggers.
Bump to 091 so the filename ordering is unambiguous again. The SQL is
unchanged — same partial unique index on autopilot_trigger.webhook_token —
only the filename moves.
* fix(views): dedupe skipped icon in autopilot RUN_VISUAL after rebase
The rebase against origin/main merged main's add of `Ban` for the
skipped status next to our round-1 `MinusCircle` entry, leaving the
RUN_VISUAL map with two `skipped` keys (only the last would have been
read at runtime, and MinusCircle had been dropped from the imports
during conflict resolution — so the file would not compile).
Keep main's `Ban` icon (latest design) and a single `skipped` entry.
Carry over the round-1 comment about why the muted styling matters
for failure-ratio readability.
---------
Co-authored-by: Kerim Incedayi <kerim.incedayi@digitalchargingsolutions.com>
|