mirror of
https://github.com/multica-ai/multica.git
synced 2026-07-26 20:45:37 +02:00
fix/transcript-virtual-scroll
23 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Bohan Jiang
|
ffa8e16369 |
MUL-5228 fix(usage): bill Grok at xAI's reported cost, fix $0 resumed sessions (#5841)
* fix(agent): attribute Grok usage from the turn's own model id A resumed Grok session with no configured model recorded its entire spend under the model id "unknown", which matches no pricing row — so the task reported $0 cost instead of its real spend. grok.go only learned the model from the session handshake, and ACP's `session/load` carries no model id (only `session/new` does). When neither the agent nor MULTICA_GROK_MODEL pins a model, `daemon.go` legitimately passes an empty model, leaving nothing to attribute the usage to. Every Grok turn stamps `result._meta.modelId` with what it actually billed against. Parse it in the shared ACP result parser and use it as the fallback in grok.go. Other ACP backends are untouched — they keep whatever the handshake gave them. Co-authored-by: J <agent@multica.ai> Co-authored-by: multica-agent <github@multica.ai> * fix(metrics): price the Grok catalog in server-side cost metrics server/internal/metrics/pricing.go carried no Grok rows at all, so RecordLLMUsage took the unpriced branch for every Grok turn: llm_cost_usd reported zero Grok spend while the tokens accumulated in llm_unpriced_tokens. Internal cost monitoring simply could not see Grok. Add the six SKUs xAI publishes rates for, mirroring the frontend table in packages/views/runtimes/utils.ts. Aliases are anchored exact matches like the gpt-5.6 rows, so `grok-composer-*` (in the catalog, absent from the price sheet) stays unmapped instead of inheriting a guessed rate. Short-context tier on purpose: xAI bills a request at 2x once its prompt reaches 200K tokens, but a usage record aggregates every model call in a turn and cannot say which tier an individual request hit. A regression test re-derives the cost of a real grok 0.2.106 turn from the table and checks it against the costUsdTicks xAI returned for that turn. Co-authored-by: J <agent@multica.ai> Co-authored-by: multica-agent <github@multica.ai> * docs(changelog): scope the Grok cost claim to what was actually fixed The v0.4.9 entry promised "accurate cost" in all four languages, but the fix corrected catalog pricing and cached-input double-counting — it did not implement xAI's 2x long-context tier, so a turn whose requests reach 200K prompt tokens still under-reports by up to 50%. Say what was fixed instead. Also correct two stale claims in the pricing comment: the daemon tags usage rows with the runtime provider `grok`, not `xai` (the bare `grok-*` keys are what make them resolve), and record why thresholding the long-context tier on an aggregated row would be worse than not pricing it at all. Co-authored-by: J <agent@multica.ai> Co-authored-by: multica-agent <github@multica.ai> * feat(usage): carry the provider's own cost through to the usage record Cost has always been derived client-side as tokens x a static rate, which cannot express request-level pricing rules. xAI bills a Grok request at 2x once its prompt reaches 200K tokens, and a task_usage row aggregates every model call in a turn — so the stored token counts genuinely cannot say which tier any individual request hit. Thresholding on the aggregate would be worse than the status quo: it turns a bounded 50% under-estimate into an unbounded over-estimate for turns made of many short requests. Grok already reports what it charged, per turn, in `_meta.usage.costUsdTicks`. Parse it, carry it through agent -> daemon -> API, and store it on task_usage as a nullable BIGINT of 1e-10 USD ticks (integer, so sub-cent turns stay exact end to end). NULL means the provider reported no cost — every pre-existing row and every provider that doesn't return one. No backfill: there is no authoritative figure to recover for those, and inventing one is the guess this removes. A single hourly bucket can mix rows that carry a cost with rows that don't, so task_usage_hourly gains both halves: `cost_usd_ticks` sums the authoritative side, and `uncosted_*_tokens` carry exactly the tokens that still need a rate-table estimate. Consumers report authoritative + estimate(uncosted), which degrades to today's behaviour when nothing in the bucket is authoritative. The existing token columns keep covering every row, so token displays are untouched. The new columns are additive with defaults, so the unique key, the dirty-queue shape, and migration 102's triggers are unaffected. Co-authored-by: J <agent@multica.ai> Co-authored-by: multica-agent <github@multica.ai> * feat(usage): prefer the provider's own cost over the rate table With the authoritative figure now stored, both cost consumers use it: the usage dashboard (estimateCost / estimateCostBreakdown) and the server-side llm_cost_usd metric. Each reports `authoritative + estimate(uncosted tokens)`, so a row or bucket that mixes priced and unpriced sources stays whole. The static rate tables remain, but for Grok they are now a fallback — they still price usage recorded by a daemon too old to report cost, and every provider that reports none. Custom pricing overrides likewise apply only to the estimated half: they are a user's guess at a rate, and the authoritative half is not a guess. A model with no rate-table row but a provider-reported cost now also drops out of the "unmapped models" banner, since asking the user to supply a rate for it would invite overriding a real bill. llm_cost_usd is labelled by token_type and the provider reports one number per turn, so the charge is distributed across the buckets in the rate table's own proportions. Only the total is authoritative; the split stays an estimate, which is why this scales the existing buckets rather than inventing a label. estimateCostBreakdown does the same, keeping the stacked chart summing to the headline figure instead of silently under-drawing every Grok row. Co-authored-by: J <agent@multica.ai> Co-authored-by: multica-agent <github@multica.ai> * docs(changelog): say Grok cost now follows xAI's actual charge The earlier wording scoped the claim down to catalog pricing and cached input because the long-context tier was still unhandled. It is handled now — the cost comes from what xAI charged for the turn — so the entry can say so. Co-authored-by: J <agent@multica.ai> Co-authored-by: multica-agent <github@multica.ai> * fix(usage): keep the provider's cost when the model has no rate row Both cost consumers bailed out before reading the authoritative figure when the rate table had no row for the model. A `grok-composer-*` turn — in the Grok Build catalog, absent from xAI's price sheet — was therefore reported as $0 spend even though xAI told us exactly what it charged. Worse on the client: estimateCost returned the real cost while estimateCostBreakdown returned zeros, so the headline and the stacked chart disagreed on precisely the rows whose cost is exact — and the unmapped-models banner was (correctly) hidden, so nothing explained the discrepancy. Handle the charge before the rate lookup in both places. Without rates there is nothing to split a total by, so it lands whole in the `input` bucket, the same fallback distributeAuthoritativeCost already uses when it has no shape to scale. Tokens with no rate keep going to llm_unpriced_tokens: "unpriced" describes the rate table, not the money. Co-authored-by: J <agent@multica.ai> Co-authored-by: multica-agent <github@multica.ai> * perf(usage): drop the historical rewrite from the cost-split migration Migration 213 rewrote every existing task_usage_hourly row to seed the uncosted counters. That is a full-table UPDATE inside a schema migration — lock time, WAL and bloat all scaling with table size — for rows this issue explicitly does not care about. Deleting the UPDATE alone would have zeroed historical cost: with `NOT NULL DEFAULT 0`, an untouched row asserts "nothing here needs estimating", so every pre-split bucket would report $0 until the rollup happened to touch it. Make the uncosted columns nullable with no default instead. NULL means "never recomputed since the split existed", readers COALESCE it to the row's own token total ("estimate all of it"), and the pre-split behaviour is preserved exactly — with nothing to seed, so no rewrite. A bare ADD COLUMN is metadata-only, so this is now fast DDL. Rows heal into the split naturally as the rollup recomputes their buckets. Verified on a fresh database: a legacy-shaped row reads back as its full tokens to estimate, and a group mixing legacy and post-split buckets sums to the authoritative cost plus both rows' estimable tokens. Co-authored-by: J <agent@multica.ai> Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Bohan-J <bohan@devv.ai> Co-authored-by: J <agent@multica.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Jiayuan Zhang
|
a61a8ecfed |
feat(onboarding): merge About-you step, collect source after agents deliver value (#5786)
Flow drops from five steps to three: role + use_case merge into a single About-you screen (one Skip covers both; Continue stamps skip markers on whichever group was left unanswered), and the source question leaves onboarding entirely. Source is now collected only by the workspace source-backfill prompt, which additionally waits until agents/squads have completed at least SOURCE_BACKFILL_MIN_AGENT_DONE_ISSUES (3) issues in the workspace — attribution is asked after Multica has visibly delivered value, not before. The count rides a limit:1 issues query keyed under issueKeys.all so realtime invalidations keep it fresh, enabled only for users who still owe an answer. Server: questionnaire complete() narrows to role + use_case so the funnel step doesn't stall on the now-deferred source; a new metrics-only onboarding_source_submitted event (+ Prometheus counter) tracks the backfill prompt's answer/decline transition once per user. Co-authored-by: Lambda <lambda@multica.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Bowser
|
f8bf6cd8b9 |
feat(runtime): add Qwen Code runtime (MUL-5015)
Merge approved PR #5666. |
||
|
Naiyuan Qing
|
1507997272 |
fix(agent): stop agents shipping local-path links, make Desktop 404 recoverable (MUL-4899) (#5557)
Agents were writing runtime-local paths into deliverables as clickable links (`[screenshot](/Users/agent/work/shot.png)`). Two root causes, both fixed here. A. The brief never stated the delivery contract. Add an always-on delivery invariant (outside writeOutput's kind switch, so no task kind can inherit none) plus a per-surface file-delivery line for each of the five surfaces. Chat splits into two: `attachment upload` works only on web/mobile chat, never on an IM channel, so ChatChannelType is now threaded into TaskContextForEnv. The claim path only ever looked up Slack bindings, so a Feishu session reported as a web chat and got upload guidance for a channel that cannot carry attachments. Probe every channel type. The chat policy is two independent layers and stays that way: delivery keys off "is there a channel at all"; the `chat history` / `chat thread` commands stay Slack-only because both endpoints are hardwired to h.SlackHistory and there is no Feishu reader — ChatInThread only selects between those two commands, so it stays Slack-only too. Add a CLI hard-fail lint on `issue comment add` / `issue create` / `issue update` as the enforcement backstop. Scoped narrowly, since a false positive blocks a real deliverable: agent task context only (a human's PAT run is untouched), real CommonMark link/image/autolink destinations only via goldmark (a path in a code span or fence — how an agent quotes a path it is discussing — is structurally invisible), and three high-confidence signals only (`file://`, inside the workdir, or an existing local file). A bare `/foo` is a valid origin-relative URI and is deliberately allowed. `issue update` has no --attachment flag, so its hint redirects to `comment add` rather than naming an argument it rejects. B. Desktop presented the resulting router 404 as an unknown crash. 8 of 18 desktop_route_error reports were users clicking such a link and being told the app broke and to file a bug. Split the 404 into a first-class Not Found view: no crash framing, no Report error. Its recovery entry comes from the tab store's active workspace, never from the failed pathname — deriving a slug from `/Users/me/shot.png` yields "Users" and a button to `/Users/issues`, a second 404. Also add a will-navigate trusted-origin guard via the shared loadRenderer (main + issue windows). This is origin hardening only, NOT the mechanism for in-app links: client-side routing never fires will-navigate, so app paths never reach it. Issue windows need no 404 work — their router only accepts paths validated by parseIssueWindowPath and they do not listen for multica:navigate, so a bad path cannot reach them. Server-side completion observation is metric/log only and never blocks: it is lexical (`file://` + task work_dir prefix) because the server cannot stat the daemon's filesystem, and the metric label is a closed enum so no path or reply text reaches Prometheus. Verified: pnpm typecheck/lint/test (3582 tests), go vet, full Go suite including new claim-path integration tests. cmd/multica was verified outside the daemon workdir — inside one, 93 of its tests fail identically on origin/main because the suite walks up and finds the runtime's own task marker. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Sanjay Ramadugu
|
06d79c1750 |
feat(agent): add Grok Build CLI as an ACP runtime (#5285)
Add xAI Grok Build (`grok`) as a first-class Multica runtime over ACP (`grok --no-auto-update agent --always-approve stdio`), reusing hermesClient like traecli and kimi. Includes daemon discovery, protocol_family migration 174, model discovery, MCP passthrough, thinking effort, frontend branding, and product docs. Follows xAI's documented headless ACP flow: after `initialize`, read the advertised `authMethods` and send `authenticate` (preferring xai.api_key when XAI_API_KEY is set, else the cached login token) before any session operation — a real, logged-in CLI rejects session/new and session/load without it. Model discovery performs the same handshake so it returns the live catalog instead of the static fallback. `--no-auto-update` is passed as a global flag (and kept daemon-owned in the blocked custom-arg set) so a background update check can't stall an unattended ACP task. Thinking effort uses the current `--effort` flag, and the minimum grok version is 0.2.89 (ACP + authenticate + session/load + session/set_model + MCP + --effort). Closes #2895 |
||
|
Multica Eve
|
e07b5403ab |
MUL-4502: make autopilot webhook admission durable (#5386)
* fix(autopilots): make webhook admission durable Co-authored-by: multica-agent <github@multica.ai> * fix(autopilots): address webhook delivery review Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Eve <eve@multica-ai.local> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Jordan
|
c27919a4d0 |
feat(agents): add DevEco Code (deveco) runtime agent (MUL-4050) (#4916)
Adds DevEco Code (Huawei's HarmonyOS coding agent, built on the OpenCode engine) as a first-class runtime provider: backend/model parser, daemon discovery (probe + login-shell list + Windows .cmd native resolver), runtime profile migration (protocol_family whitelist), provider UI, metrics, and four-language docs. MCP injection is deferred (UI gates it off). Migration numbered 175. |
||
|
Multica Eve
|
ac62f72c2a |
MUL-4480: make daemon workspace sync event-driven (#5354)
* feat(daemon): make workspace sync event-driven Co-authored-by: multica-agent <github@multica.ai> * fix(daemon): preserve trailing workspace changes Co-authored-by: multica-agent <github@multica.ai> * fix(workspace): reconcile failed creates Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Eve <eve@multica-ai.local> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Bohan Jiang
|
8e0fbecab5 |
fix(models): codex empty-model effort validation + exact 5.6 aliases (MUL-4347) (#5196)
Follow-up to #5188 addressing the second-round review. - ValidateThinkingLevel now fails an empty codex model closed instead of borrowing the flagged Default (gpt-5.6-sol). An empty model follows config.toml, which can resolve to any installed model; Sol alone advertises `ultra`, so the old borrow green-lit levels Luna / gpt-5.5 don't support and Codex doesn't reject. Checked before ListModels so a discovery error can't fail it open. Frontend pickModelEntry mirrors this (no per-model effort preview for an empty codex model); the persisted-orphan clear path stays. - parseCodexDebugModels drops efforts without a known label so the picker never advertises a level the Create/Update enum gate would 400 on save; the contract test now drives the real parser with an unknown effort instead of comparing two hand-written maps. - gpt-5.6 price aliases anchor to a literal dot (not the [.-] class), so dashed variants like gpt-5-6-luna surface as unmapped on both backend and frontend rather than silently borrowing a tier. Co-authored-by: J <j@multica.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Bohan Jiang
|
6b980a8e71 |
feat(models): add Codex gpt-5.6 series (sol/terra/luna) to model list & pricing (MUL-4347) (#5188)
* feat(models): add Codex gpt-5.6 series (sol/terra/luna) to model list & pricing (MUL-4347) Co-authored-by: multica-agent <github@multica.ai> * fix(models): official gpt-5.6 pricing, exact aliases, max/ultra effort levels (MUL-4347) - Replace provisional gpt-5.6 rates with OpenAI's official announcement values (sol 5/30, terra 2.5/15, luna 1/6); cache read 0.1x input, cache write 1.25x input (frontend + backend, kept in sync). - Anchor gpt-5.6 price aliases to exact match so unknown suffixed variants surface as unmapped instead of borrowing a tier. - Add Codex 0.144.1 max/ultra effort levels to the label map and server enum so the daemon-advertised catalog matches what the API can persist; add a catalog->API contract test. - Clarify that the codex Default flag is the effort-validation anchor, not a user-facing badge. - Note the cache-write measurement limitation (codex usage stream doesn't report cache-write tokens yet). Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: J <j@multica.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Bohan Jiang
|
3cb5dc3ad6 |
chore(analytics): retire redundant PostHog tracking (MUL-4127) (#4996)
* chore(analytics): retire redundant PostHog tracking (MUL-4127) PostHog had become a chaotic, largely-unused second copy of data we already query from the DB and Grafana. Remove the redundant instrumentation. Server: every product event (signup, workspace_created, issue_created, issue_executed, chat_message_sent, team_invite_*, onboarding_*, agent_created, cloud_waitlist_joined, feedback_submitted, contact_sales_submitted, squad_created, autopilot_created) is now in metricsOnlyEvents, so metrics.RecordEvent still increments the Prometheus/Grafana counter but no longer ships to PostHog. DB rows remain the source of truth. Runtime/autopilot/ agent_task lifecycle were already Prometheus-only. Frontend: delete the PostHog-only funnel instrumentation — $pageview (+ web and desktop trackers), download_intent_expressed/page_viewed/initiated, the onboarding_started mirror, onboarding_runtime_path_selected/detected, feedback_opened, and source_backfill_*. The source-backfill modal itself stays (it PATCHes the questionnaire to the DB). Kept on PostHog (frontend only): $exception autocapture and the client_crash / client_unresponsive stability telemetry (no DB equivalent), plus $identify/$set. captureSignupSource (attribution cookie) stays — it still feeds the signup_source Prometheus label. Verified: pnpm typecheck, pnpm lint (0 errors), vitest (core/views/web/desktop), go test ./internal/analytics/... ./internal/metrics/... Co-authored-by: multica-agent <github@multica.ai> * docs(analytics): fix stale PostHog references after MUL-4127 (review follow-up) Addresses review of #4996 — three spots still described server events as active PostHog signals after they became metrics-only: - docs/analytics.md: issue_executed is no longer a PostHog success signal; it is Prometheus-only (multica_issue_executed_total) + issue.first_executed_at, in both the event contract and the Reconciliation section. - docs/analytics.md: the signup $set_once person properties (email, signup_source) are no longer emitted — signup is Prometheus-only; only the bucketed signup_source survives as the multica_signup_total label. - server/internal/metrics/business_events.go: RecordEvent doc comment no longer claims it ships product events to PostHog / "PostHog is reserved for user/product-behaviour events" — every server event is now metrics-only. Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: J <j@multica.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
bhirstmedia
|
170750242b |
BHI-12314: add Claude Sonnet 5 catalog and pricing support (MUL-3910) (#4783)
Co-authored-by: Ember <ember@Embers-iMac.localdomain> |
||
|
Naiyuan Qing
|
b336f07617 |
Revert "feat(analytics): anonymous self-host onboarding source beacon (MUL-37…" (#4712)
This reverts commit
|
||
|
Naiyuan Qing
|
63eb6f73ad |
feat(analytics): anonymous self-host onboarding source beacon (MUL-3708) (#4691)
* feat(analytics): anonymous self-host onboarding source beacon (MUL-3708) Production self-host servers now report the anonymous onboarding "how did you hear about us" channel to Multica's public write-only ingest, so the self-host source distribution becomes visible alongside official cloud. Official cloud keeps its existing PostHog capture unchanged; this is a submit-time beacon, not a background telemetry pipeline. - server/internal/sourcebeacon: ShouldSend gate (production + non-local + non-*.multica.ai app host, fail-closed — judged by the app/frontend host, not the backend URL, which official often leaves unset), per-instance salted hashing, deterministic event uuid, fire-and-forget sender. - POST /api/telemetry/self-host-source: public, write-only, per-IP rate-limited, 4 KiB body cap, channel allowlist, strict unknown-field rejection. Lands in PostHog as self_host_source_channel with a deterministic uuid (best-effort dedup), $process_person_profile=false, and deployment=self_host — a distinct event name so it never pollutes the official onboarding funnel. - Hook in PatchOnboarding fires once when the source is first set; never blocks onboarding. Only channel enum(s) + two per-instance hashes leave the box — never user_id/email/name/workspace/org/domain/role/use_case/the source_other free-text/IP. - migration 128: system_settings singleton holding instance_salt. - frontend: self-host-only anonymous-collection notice on the source step, gated by a new /api/config self_host_source_notice flag (en/zh-Hans/ko/ja). - analytics.Event gains an optional top-level uuid; docs/analytics.md, SELF_HOSTING.md and .env.example document exactly what is/isn't sent and how to disable it (ANALYTICS_DISABLED). Also fixes the long-standing team_size→source drift in docs/analytics.md. Verified locally: go build/vet, go test (sourcebeacon, analytics, handler), pnpm typecheck (all packages), locale parity (157), step-source (6) + core config/schema (69) vitest, lint (0 errors). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: multica-agent <github@multica.ai> * fix(analytics): wire self-host source beacon through metrics, guard nil pool (MUL-3708) Addresses Howard CI blockers on #4691 (no product-direction change): - loadInstanceSalt returns "" on nil pool; salt is only loaded when ShouldSendFromEnv() is true, via a bounded (5s) context — restores the "router constructible without a DB" invariant (nil-pool routing tests). - Add multica_self_host_source_channel_total counter (by source) + an IncForEvent case, so every analytics event is paired with a Prometheus counter. NormalizeSourceChannel reuses sourcebeacon allowlist (no 3rd copy). - Beacon handler now builds the event via the analytics.SelfHostSourceChannel helper and ships it through obsmetrics.RecordEvent (no naked Capture); not IsMetricsOnly, so it still reaches PostHog. - Prime the new family in the registry-families test. Verified: go build/vet, go test ./internal/metrics ./internal/sourcebeacon ./internal/handler ./cmd/server (incl. the 3 named blockers + registry + record-event-helper lints) all green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Liu Guanzhong
|
4594c776e1 |
feat(agent): add CodeBuddy as first-class CLI backend (#3186)
* feat(agent): add codebuddyBackend struct and buildCodebuddyArgs Introduces the codebuddy agent backend skeleton with args builder that mirrors claudeBackend's protocol flags (stream-json, bypass permissions, blocked args filtering) for the codebuddy CLI fork. * feat(agent): implement codebuddyBackend.Execute with stream-json parsing * feat(agent): wire codebuddy into New() factory and launchHeaders * feat(agent): add codebuddy dynamic model discovery from --help * feat(agent): add codebuddy thinking/effort discovery and providerThinkingEnums * feat(daemon): add codebuddy CLI probe, env vars, and args support * fix(agent): use len(models)==0 for default model instead of loop index * fix(agent): increase codebuddy --help timeout to 35s for slow CLI startup * fix(agent): address codebuddy PR review feedback - Wire codebuddy into execenv: reuse claude's CLAUDE.md, .claude/skills, and ~/.claude/skills paths since CodeBuddy is a Claude Code fork - Replace hardcoded 20-min timeout with runContext for zero-timeout = no-deadline semantics matching all other backends - Restore runContext regression tests lost in rebase merge - Mirror claude.go execution model: concurrent stdin write to prevent pipe deadlock, sync.Once for stdin closure, keep stdin open for control_request auto-approval mid-run - Add control_request handling with auto-approve behavior - Add RequestID/Request fields to codebuddySDKMessage - Add codebuddy to metrics knownRuntimeProviders - Add codebuddy to provider-logo.tsx (reuses ClaudeLogo) - Consolidate --help discovery: shared codebuddyHelpOutput cache eliminates duplicate cold-start invocations --------- Co-authored-by: krislliu <krislliu@tencent.com> |
||
|
Bohan Jiang
|
b1c8eb5f11 |
feat: support Claude Fable 5 pricing (#3982)
Co-authored-by: J <j@multica.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Multica Eve
|
945d684afd |
MUL-2965 fix(metrics): harden business sampler quality (#3738)
* fix(metrics): harden business sampler quality (MUL-2965) Co-authored-by: multica-agent <github@multica.ai> * fix(metrics): alert on sampler acquire failures Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Eve <eve@multica-ai.local> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Bohan Jiang
|
b92c3fbc93 |
chore(analytics): stop shipping operational events to PostHog (MUL-2967) (#3720)
* chore(analytics): stop shipping operational events to PostHog (MUL-2967) Operational / execution-lifecycle telemetry dominated PostHog event volume and drove the bill: runtime_offline alone was ~54% of ~22.6M events/mo, and ~99% of events were billed at the higher identified-event rate. These signals already have Prometheus counters (Grafana), so the PostHog copies were redundant cost. - Add analytics.IsMetricsOnly; metrics.RecordEvent now skips the PostHog Capture for runtime_* and autopilot_run_* while still incrementing their Prometheus counter (their analytics.Event constructors are retained to feed the metric label set via IncForEvent). - Remove the agent_task_* PostHog path entirely: drop captureTaskEvent and the AgentTask* constructors/constants. Their Prometheus side is unchanged via the typed BusinessMetrics.RecordTask* methods. Also remove the now-dead taskDurationMS / willRetryTask helpers. - Update the pairing lint test (no agent_task allow-list, no naked-Capture exception), add a RecordEvent skip test + IsMetricsOnly test, and update docs/analytics.md (taxonomy, per-event banners, reconciliation). Product/funnel events (signup, onboarding, issue_created, issue_executed, chat_message_sent, agent_created, autopilot_created, etc.) are unchanged and still ship to PostHog. Co-authored-by: multica-agent <github@multica.ai> * docs(analytics): correct agent_task Prometheus metric contract (MUL-2967) Address PR review: the agent_task_* "Prometheus-only" banner claimed the old PostHog event properties (task_id, agent_id, duration_ms, error_type, will_retry, ...) were the metric label set. They are not — the real labels are only source/runtime_mode/provider/terminal_status/failure_reason. - Replace the agent_task_* sections with the actual metric names and labels (multica_agent_task_*; see business.go / labels.go), and explain that completed/failed/cancelled are terminal_status values on multica_agent_task_terminal_total, with wall-clock in the *_seconds histograms. - Tighten the runtime_*/autopilot_run_* banners so id properties aren't mistaken for labels. - Drop the stale AgentTask allow-list reference from the pairing lint test header comment. Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: J <j@multica.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
LinYushen
|
9c9afd4a66 |
feat(metrics): BusinessSamplerCollector for active users / queued / runtime gauges (MUL-2947) (#3706)
* feat(metrics): scrape-time BusinessSamplerCollector for active users / queued / runtime gauges (MUL-2947)
Adds an opt-in prometheus.Collector that runs a fixed set of read-only
SQL queries on every /metrics scrape and exposes the results as gauges:
- multica_active_users{window=5m|1h|24h}
- multica_active_workspaces{window=...}
- multica_agent_task_queued{source}
- multica_agent_task_running{source,runtime_mode}
- multica_agent_task_stuck_total{source}
- multica_runtime_online{runtime_mode,provider}
- multica_runtime_heartbeat_age_seconds{runtime_mode} (histogram)
- multica_workspace_total
Plus a self-introspection histogram
multica_business_sampler_query_seconds{name=...} and a counter
multica_business_sampler_query_errors_total{name=...} so the sampler's
own behaviour is observable on /metrics.
Production-safety contract per the PR4 brief:
- every query runs in its own BEGIN READ ONLY tx with
SET LOCAL statement_timeout = '500ms' (configurable)
- the sampler takes a dedicated *pgxpool.Pool option so operators
can isolate it from business traffic
- successful results are cached for 5–10s (default 8s) to absorb
concurrent scrapes from multiple Prometheus replicas
- every SQL has a hard LIMIT 100 fallback
- all label values flow through the existing BusinessMetrics
NormalizeTaskSource / NormalizeRuntimeMode / NormalizeRuntimeProvider
whitelists, so a misbehaving runtime cannot inflate cardinality
- sampler is OPT-IN via RegistryOptions.BusinessSampler — existing
callers that only pass Pool keep their current behaviour and never
start hitting the DB on /metrics
Tests cover: emit shape, TTL cache (one DB call per N scrapes),
bounded cardinality under malicious labels, opt-out (no leakage), and
DB-hang isolation (unreachable host -> /metrics returns within 5s,
query_errors_total advances).
Refs MUL-2947 (depends on PR2 / MUL-2948, merged in #3695).
Co-authored-by: multica-agent <github@multica.ai>
* fix(metrics): address PR4 review — wire sampler in main.go, fix LIMIT bug, add live-DB statement_timeout test
Three fixes from 大彪's review on #3706:
1. main.go was building NewRegistry without the BusinessSampler option,
so the collector was effectively dead code in prod. Now constructs a
dedicated 2-conn pgxpool (newSamplerDBPool) from the same DATABASE_URL
when METRICS_ADDR is set, plumbs it into RegistryOptions.BusinessSampler,
and defers Close() at shutdown. A pool-build failure logs and disables
the sampler instead of taking down the server.
2. queryActiveUsers / queryActiveWorkspaces previously wrapped the
distinct-user/workspace subquery in a 'LIMIT 100', then COUNT(*)'d
the result — capping the active-user gauge at 100 regardless of
reality. Removed the inner LIMIT; the COUNT scalar is one row anyway,
and metric cardinality is bounded by the fixed samplerWindows
allow-list, not by the SQL shape.
3. The previous DB-hang test only exercised the acquire-fails path. Added
business_sampler_pgsleep_test.go which connects to a live Postgres
(skips cleanly when DATABASE_URL is not set), runs SELECT pg_sleep(2)
inside a sampler-style tx with SET LOCAL statement_timeout = '500ms',
and asserts:
- the call returns in well under 1.5 s (proving the server-side
cancellation, not just our caller-side context)
- query_errors_total{name=pg_sleep_canary} advances
- the duration histogram records the cancellation
Verified locally: 550 ms, SQLSTATE 57014 'canceling statement due to
statement timeout' — exactly the safety net the PR claims.
Refs MUL-2947 / PR #3706.
Co-authored-by: multica-agent <github@multica.ai>
* test(metrics): assert SQLSTATE 57014 on pg_sleep cancellation
The previous assertion only checked that the query was cut off in well
under the sleep duration, which a caller-side context cancellation
would also satisfy. Capturing the inner pgconn.PgError and asserting
Code == "57014" ("query_canceled") nails down that Postgres itself
cancelled the statement because of the SET LOCAL statement_timeout —
so a regression that drops the SET LOCAL line fails this test loudly
instead of silently passing on context cancellation.
Refs MUL-2947 / PR #3706 review nit.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: multica-agent <github@multica.ai>
|
||
|
LinYushen
|
de900b2ba6 |
feat(server): funnel/community/commercial business metrics + PostHog pairing (MUL-2949) (#3698)
* feat(server): funnel/community/commercial business metrics + PostHog pairing (MUL-2949) PR3 of the Grafana board metrics split (parent MUL-2328). Adds 23 new Prometheus counter/histogram families to the PR2 BusinessMetrics collector covering the activation/community/commercial funnels, and binds every PostHog event emission to a matching metric increment so the two sides cannot drift. Funnel: signup, workspace_created, team_invite_sent/accepted, onboarding_*, cloud_waitlist_joined. Content: issue_created, chat_message_sent, agent_created, squad_created, autopilot_created, issue_executed. Runtime: runtime_registered/ready/failed/offline + ready_seconds histogram, daemon_ws_message_received_total. Autopilot: autopilot_run_started/terminal/skipped. Webhook/GitHub: webhook_delivery_total, github_event_received_total, github_pr_review_total, github_pr_merge_seconds histogram. CloudRuntime: cloudruntime_request_total + duration histogram, wired through a small RequestRecorder interface so the cloudruntime package stays decoupled from metrics. Commercial: feedback_submitted, contact_sales_submitted. The pairing helper metrics.RecordEvent(client, m, ev) emits the PostHog event AND increments the matching counter via IncForEvent dispatch, reading labels from the analytics event Properties. Every existing h.Analytics.Capture(analytics.X(...)) call site has been migrated to the helper across handler/, service/, and cmd/server/runtime_sweeper.go. Lint enforcement (server/internal/metrics/business_pairing_test.go): - TestEveryAnalyticsEventHasPrometheusCounter: every Event* constant in analytics/events.go either dispatches via IncForEvent or is in the taskMetricEvents allow-list (PR2 typed RecordTask* methods). - TestNoNakedAnalyticsCaptureInHandlersOrServices: AST-walks handler/ service/cmd-server for direct Analytics.Capture(...) calls — only service/task.go's captureTaskEvent helper is allow-listed. - TestEveryAnalyticsRecordEventTakesAnalyticsHelper: validates the third arg of every metrics.RecordEvent call is built from analytics.*. Cardinality protection: all new label values pass through fixed allow-lists in labels_pr3.go; unknown values collapse to 'other'/'unknown'/'error'. Refs: - Spec MUL-2328 / MUL-2949. - Builds on PR2 (MUL-2948) — collectors registered through the same BusinessMetrics struct, no separate Registry. - Uses PR1's taskfailure.Reason (MUL-2946) for runtime_failed's failure_reason label via NormalizeFailureReason. Out of scope: Sampler-class metrics (PR4 / MUL-2947), pr_review_total emission point (no review event handler exists yet — counter is defined, TODO to wire up when /api/webhooks/github grows pull_request_review handling). Co-authored-by: multica-agent <github@multica.ai> * fix(server): tighten PR3 review items — signup_source bucket, fill platform/kind/form_source enums, onboarding_started server emission, lint scope (MUL-2949) Addresses 张大彪's review on #3698: 1. signup_source: NormalizeSignupSource added to labels_pr3.go with a fixed allow-list bucket (direct/google/twitter/linkedin/.../other). Parses JSON cookie payload for utm_source/source/referrer fields, strips URL schemes, maps well-known hostnames to channel buckets. PostHog event still ships the raw cookie value for analytics; only the Prometheus label is bucketed. 2. Filled the unknown/other label gaps: - analytics.IssueCreated and analytics.ChatMessageSent now take a platform parameter sourced from middleware.ClientMetadataFromContext (X-Client-Platform header) at the handler. Autopilot-originated issues stamp PlatformServer. - analytics.FeedbackSubmitted now takes a kind parameter; CreateFeedback reads req.Kind (default "general") so the picker selection lights up the metric's kind label instead of long-term "other". - analytics.ContactSalesSubmitted now takes a formSource (page / onboarding / agents_page); CreateContactSales reads req.Source. The metric reads ev.Properties["form_source"] so the analytics CoreProperties.Source ("marketing_contact_sales") stays backward-compat for PostHog dashboards. 3. analytics.OnboardingStarted helper added; server-side emission lives in PatchOnboarding, fired exactly once per user on the first PATCH that carries a non-empty questionnaire payload (firstTouch logic compares prior bytes against {} / null). Frontend onboarding_started keeps firing on page open; the server emission is what guarantees the Prometheus counter exists so Grafana can be cross-checked against the PostHog funnel without depending on the SDK roundtrip. 4. business_pairing_test.go tightened: - TestNoNakedAnalyticsCaptureInHandlersOrServices now allow-lists at function granularity (just captureTaskEvent in service/task.go), not whole-file. Any future naked Capture in the same file fails CI. - TestEveryAnalyticsRecordEventTakesAnalyticsHelper now does def-use tracking inside the enclosing FuncDecl: when RecordEvent's third arg is an *ast.Ident, the test walks the function body for the assignment that defined it and confirms the RHS is an analytics.<Helper>(...) call. Bare local idents that didn't originate from analytics are now caught. 5. gofmt -w applied across the touched files; gofmt -l clean. Tests: go test ./internal/metrics/... ./internal/analytics/... pass. Pre-existing TestClaimTask_/TestWebhook_MergedPR/TestDeleteIssueByIdentifier failures on origin/main are DB-environment-dependent and not regressions from this change. Co-authored-by: multica-agent <github@multica.ai> * fix(server): normalise onboarding_started platform label + regression test (MUL-2949) Addresses 张大彪's last review nit: - IncForEvent's EventOnboardingStarted case now wraps the platform property with NormalizePlatform, matching every other platform-bearing metric. A misbehaving frontend can no longer leak a raw X-Client-Platform header value into the multica_onboarding_started_total{platform=...} series. - New labels_pr3_test.go covers every PR3 normalizer with both a happy-path value and an unknown value, asserting the unknown collapses to the documented fallback bucket. Includes a focused regression for onboarding_started: emits one event with an attacker-shaped platform string and asserts the metric only exposes web + unknown label values (no raw header bleed). - testutil.go gains a small GatherForTest helper so the regression test can pull the typed MetricFamily map without re-implementing the registry-walk dance. Co-authored-by: multica-agent <github@multica.ai> * fix(server): NormalizeTaskSource on workspace_created + document lint limitations (MUL-2949) Final review touch-ups before merge: - IncForEvent's EventWorkspaceCreated case wraps source through NormalizeTaskSource, matching the other source-bearing dispatches (issue_created, agent_created, issue_executed). Closes the last raw property leak in the dispatcher table. - business_pairing_test.go inline docstrings now spell out the two known limitations of the lint gate that 张大彪 / Eve flagged: analyticsBackedIdents matches by ident NAME (not SSA def-use, so a nested-scope shadow could pass) and isMetricsRecordEvent hard-codes the import alias set. PR description carries a Follow-ups section with the same two items so the work is visible after merge. Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: 魏和尚 <agent+wei@multica.ai> Co-authored-by: multica-agent <github@multica.ai> |
||
|
Multica Eve
|
a72fb020de |
Add business metrics collectors (#3695)
Co-authored-by: Eve <eve@multica-ai.local> Co-authored-by: multica-agent <github@multica.ai> |
||
|
devv-eve
|
9db91e89f5 |
feat: add daemon websocket task wakeups (#1772)
* feat: add daemon websocket task wakeups * feat: fan out daemon wakeups across nodes * fix: dedupe daemon wakeup loopback events * fix: lengthen daemon polling fallback interval --------- Co-authored-by: Eve <eve@multica.ai> |
||
|
devv-eve
|
f864a07bd5 |
feat: add server Prometheus metrics endpoint
Add Prometheus metrics endpoint with local-bind listener support and baseline metrics collectors. |