highperfocused/multica
1
0
Fork 0
mirror of https://github.com/multica-ai/multica.git synced 2026-07-26 12:35:35 +02:00
Code Issues Packages Projects Releases Wiki Activity
4,319 Commits 1,826 Branches 132 Tags
fix/transcript-reading-hierarchy
Commit Graph

5 Commits

Author SHA1 Message Date
Bohan Jiang
ffa8e16369 MUL-5228 fix(usage): bill Grok at xAI's reported cost, fix $0 resumed sessions (#5841) 
* fix(agent): attribute Grok usage from the turn's own model id

A resumed Grok session with no configured model recorded its entire spend
under the model id "unknown", which matches no pricing row — so the task
reported $0 cost instead of its real spend.

grok.go only learned the model from the session handshake, and ACP's
`session/load` carries no model id (only `session/new` does). When neither
the agent nor MULTICA_GROK_MODEL pins a model, `daemon.go` legitimately
passes an empty model, leaving nothing to attribute the usage to.

Every Grok turn stamps `result._meta.modelId` with what it actually billed
against. Parse it in the shared ACP result parser and use it as the fallback
in grok.go. Other ACP backends are untouched — they keep whatever the
handshake gave them.

Co-authored-by: J <agent@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>

* fix(metrics): price the Grok catalog in server-side cost metrics

server/internal/metrics/pricing.go carried no Grok rows at all, so
RecordLLMUsage took the unpriced branch for every Grok turn: llm_cost_usd
reported zero Grok spend while the tokens accumulated in
llm_unpriced_tokens. Internal cost monitoring simply could not see Grok.

Add the six SKUs xAI publishes rates for, mirroring the frontend table in
packages/views/runtimes/utils.ts. Aliases are anchored exact matches like
the gpt-5.6 rows, so `grok-composer-*` (in the catalog, absent from the
price sheet) stays unmapped instead of inheriting a guessed rate.

Short-context tier on purpose: xAI bills a request at 2x once its prompt
reaches 200K tokens, but a usage record aggregates every model call in a
turn and cannot say which tier an individual request hit.

A regression test re-derives the cost of a real grok 0.2.106 turn from the
table and checks it against the costUsdTicks xAI returned for that turn.

Co-authored-by: J <agent@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>

* docs(changelog): scope the Grok cost claim to what was actually fixed

The v0.4.9 entry promised "accurate cost" in all four languages, but the
fix corrected catalog pricing and cached-input double-counting — it did not
implement xAI's 2x long-context tier, so a turn whose requests reach 200K
prompt tokens still under-reports by up to 50%. Say what was fixed instead.

Also correct two stale claims in the pricing comment: the daemon tags usage
rows with the runtime provider `grok`, not `xai` (the bare `grok-*` keys are
what make them resolve), and record why thresholding the long-context tier
on an aggregated row would be worse than not pricing it at all.

Co-authored-by: J <agent@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>

* feat(usage): carry the provider's own cost through to the usage record

Cost has always been derived client-side as tokens x a static rate, which
cannot express request-level pricing rules. xAI bills a Grok request at 2x
once its prompt reaches 200K tokens, and a task_usage row aggregates every
model call in a turn — so the stored token counts genuinely cannot say which
tier any individual request hit. Thresholding on the aggregate would be worse
than the status quo: it turns a bounded 50% under-estimate into an unbounded
over-estimate for turns made of many short requests.

Grok already reports what it charged, per turn, in `_meta.usage.costUsdTicks`.
Parse it, carry it through agent -> daemon -> API, and store it on task_usage
as a nullable BIGINT of 1e-10 USD ticks (integer, so sub-cent turns stay exact
end to end). NULL means the provider reported no cost — every pre-existing row
and every provider that doesn't return one. No backfill: there is no
authoritative figure to recover for those, and inventing one is the guess this
removes.

A single hourly bucket can mix rows that carry a cost with rows that don't, so
task_usage_hourly gains both halves: `cost_usd_ticks` sums the authoritative
side, and `uncosted_*_tokens` carry exactly the tokens that still need a
rate-table estimate. Consumers report authoritative + estimate(uncosted),
which degrades to today's behaviour when nothing in the bucket is
authoritative. The existing token columns keep covering every row, so token
displays are untouched. The new columns are additive with defaults, so the
unique key, the dirty-queue shape, and migration 102's triggers are unaffected.

Co-authored-by: J <agent@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>

* feat(usage): prefer the provider's own cost over the rate table

With the authoritative figure now stored, both cost consumers use it: the
usage dashboard (estimateCost / estimateCostBreakdown) and the server-side
llm_cost_usd metric. Each reports `authoritative + estimate(uncosted tokens)`,
so a row or bucket that mixes priced and unpriced sources stays whole.

The static rate tables remain, but for Grok they are now a fallback — they
still price usage recorded by a daemon too old to report cost, and every
provider that reports none. Custom pricing overrides likewise apply only to
the estimated half: they are a user's guess at a rate, and the authoritative
half is not a guess. A model with no rate-table row but a provider-reported
cost now also drops out of the "unmapped models" banner, since asking the user
to supply a rate for it would invite overriding a real bill.

llm_cost_usd is labelled by token_type and the provider reports one number per
turn, so the charge is distributed across the buckets in the rate table's own
proportions. Only the total is authoritative; the split stays an estimate,
which is why this scales the existing buckets rather than inventing a label.
estimateCostBreakdown does the same, keeping the stacked chart summing to the
headline figure instead of silently under-drawing every Grok row.

Co-authored-by: J <agent@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>

* docs(changelog): say Grok cost now follows xAI's actual charge

The earlier wording scoped the claim down to catalog pricing and cached input
because the long-context tier was still unhandled. It is handled now — the
cost comes from what xAI charged for the turn — so the entry can say so.

Co-authored-by: J <agent@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>

* fix(usage): keep the provider's cost when the model has no rate row

Both cost consumers bailed out before reading the authoritative figure when
the rate table had no row for the model. A `grok-composer-*` turn — in the
Grok Build catalog, absent from xAI's price sheet — was therefore reported as
$0 spend even though xAI told us exactly what it charged.

Worse on the client: estimateCost returned the real cost while
estimateCostBreakdown returned zeros, so the headline and the stacked chart
disagreed on precisely the rows whose cost is exact — and the unmapped-models
banner was (correctly) hidden, so nothing explained the discrepancy.

Handle the charge before the rate lookup in both places. Without rates there
is nothing to split a total by, so it lands whole in the `input` bucket, the
same fallback distributeAuthoritativeCost already uses when it has no shape to
scale. Tokens with no rate keep going to llm_unpriced_tokens: "unpriced"
describes the rate table, not the money.

Co-authored-by: J <agent@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>

* perf(usage): drop the historical rewrite from the cost-split migration

Migration 213 rewrote every existing task_usage_hourly row to seed the
uncosted counters. That is a full-table UPDATE inside a schema migration —
lock time, WAL and bloat all scaling with table size — for rows this issue
explicitly does not care about.

Deleting the UPDATE alone would have zeroed historical cost: with
`NOT NULL DEFAULT 0`, an untouched row asserts "nothing here needs
estimating", so every pre-split bucket would report $0 until the rollup
happened to touch it. Make the uncosted columns nullable with no default
instead. NULL means "never recomputed since the split existed", readers
COALESCE it to the row's own token total ("estimate all of it"), and the
pre-split behaviour is preserved exactly — with nothing to seed, so no
rewrite. A bare ADD COLUMN is metadata-only, so this is now fast DDL.

Rows heal into the split naturally as the rollup recomputes their buckets.

Verified on a fresh database: a legacy-shaped row reads back as its full
tokens to estimate, and a group mixing legacy and post-split buckets sums to
the authoritative cost plus both rows' estimable tokens.

Co-authored-by: J <agent@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>

---------

Co-authored-by: Bohan-J <bohan@devv.ai>
Co-authored-by: J <agent@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
 2026-07-24 01:42:08 +08:00
YYClaw
da4f278330 fix(usage): disambiguate model pricing by provider (MUL-3346) 
Disambiguate client-side model pricing by provider: generic ids (e.g. `auto`) resolve ${provider}/${model} first, so they only price under their real provider instead of borrowing Cursor's rate. Provider is LOWER()-normalized on read and write so mixed-case historical rows merge.

Closes #4199. MUL-3346
 2026-06-18 11:10:06 +08:00
YYClaw
614dfae884 MUL-2488 feat(timezone): Scheduling / Viewing two-layer timezone architecture (#2968) 
* docs(timezone): add scheduling/viewing timezone architecture RFC

* feat(db): replace daily rollups with task_usage_hourly, add user.timezone

Migrations 100-104: add "user".timezone (Viewing tz), build the UTC
hourly task_usage_hourly rollup with its pipeline, drop the legacy
task_usage_daily / task_usage_dashboard_daily pipelines, and drop the
agent_runtime.timezone column. Report queries now slice day boundaries
at read time by the caller-supplied @tz instead of materialising in a
fixed tz. Regenerate sqlc.

* feat(server): add task_usage_hourly backfill command

Replace the two legacy backfill commands (daily / dashboard_daily) with
a single backfill_task_usage_hourly that loads historical task_usage
into the new UTC hourly rollup, sliced per workspace.

* refactor(server): resolve viewing timezone in report handlers

Report handlers resolve the Viewing tz per request (?tz query param,
then user.timezone, then UTC) and pass it to the hourly-rollup queries.
Drop the UseDailyRollup feature flags and the old raw-scan/daily-rollup
dual paths, remove the /api/usage endpoints, and stop the daemon from
reporting and the runtime handler from accepting host timezone.

* refactor(core): switch report queries to viewing timezone

API client and dashboard/runtime queries send ?tz with each report
request, the user schema/types carry the new timezone field, and the
runtime timezone field/mutation is removed.

* feat(views): add viewing timezone preference and UI

Add the useViewingTimezone hook and a Timezone setting in Preferences;
report charts and the dashboard week boundary follow the viewer tz.
Remove the runtime detail timezone editor and its locale strings.

* fix(test): update fixtures and stabilize tests for timezone refactor

The timezone architecture refactor changed several types without
updating dependent test code:

- RuntimeDevice no longer has a timezone field — drop it from the
  create-agent-dialog runtime fixture.
- User now requires a timezone field — add it to the apps/web mockUser
  fixture.
- The PreferencesTab timezone tests asserted on the async save handler
  (PATCH then store update) with a bare expect, racing the mutation's
  settle callback, and timed out querying the Select's ~600-option IANA
  list on a loaded CI runner. Wrap the assertions in waitFor and extend
  the timeout for those three tests.

* docs(timezone): document self-host migration order and trigger invariant

Add a SELF-HOST UPGRADE ORDER runbook to the backfill command's package
comment: applying migrations 100-104 in a single migrate-up drops the
legacy daily rollups before the hourly backfill runs, leaving dashboards
empty until cron catches up.

Add an INVARIANT comment on trg_atq_dirty_hourly noting that agent_id
must be added to the trigger's OF list if it ever becomes mutable,
otherwise dirty buckets for the old agent_id are silently missed.

* style(runtimes): drop trailing blank line in runtime-detail
 2026-05-21 15:33:47 +08:00
Jiayuan Zhang
380c6b5122 feat(usage): add Time and Tasks to daily-trend toggle (MUL-2283) (#2709) 
Extends the workspace /usage page Daily tokens chart toggle from
Tokens | Cost to Tokens | Cost | Time | Tasks, so users see daily
run-time and task-count trends alongside spend without leaving the page.

- New SQL `ListDashboardRunTimeDaily`: per-date totals from
  agent_task_queue (terminal tasks only), scoped to workspace and
  optionally project. Same time anchor as ListDashboardAgentRunTime
  so day boundaries line up.
- New handler GET /api/dashboard/runtime/daily + TanStack Query option.
- New DailyTimeChart (single-series, smart h/m/s unit) and
  DailyTasksChart (completed + failed stacked).
- Empty-state is per-metric so a workspace with tokens but no terminal
  runs (or vice-versa) doesn't get a false "no data".
- i18n: en + zh-Hans daily.metric_time / metric_tasks + titles.

Co-authored-by: multica-agent <github@multica.ai>
 2026-05-15 18:51:02 +02:00
Bohan Jiang
96695a79c5 feat(dashboard): workspace/project token + run-time dashboard MUL-1882 (#2462) 
* feat(dashboard): workspace/project token + run-time dashboard

Add a `/{slug}/dashboard` page showing per-agent token spend and execution
time across the whole workspace, with an optional project filter.

Backend:
  - Three new sqlc queries against task_usage + agent_task_queue: daily
    usage, per-agent usage, per-agent total run-time. All optionally
    scoped to a project via sqlc.narg('project_id'), reaching project
    through the issue join.
  - Handlers under /api/dashboard return the same wire shape the runtime
    page already consumes (model preserved for client-side cost math).

Frontend: - Shared DashboardPage in packages/views/dashboard reusing KpiCard,
    DailyCostChart, ActorAvatar, and estimateCost from the runtime page
    so the visual style and pricing math stay in lock-step.
  - Period selector (7/30/90d), project dropdown, four KPI tiles
    (cost, tokens, run time, tasks), daily cost chart, and a combined
    "cost + run time by agent" list.
  - Routed in both web (app/[slug]/(dashboard)/dashboard) and desktop
    (memory router); sidebar nav entry added under Workspace group.
Co-authored-by: multica-agent <github@multica.ai>

* fix(dashboard): drop stale project filter and stop double-counting tasks

Two issues caught in PR #2462 review:

1. Project filter held the previous selection's UUID across workspace
   switches and project deletions: the dropdown gracefully showed
   "All projects" (because the title lookup missed) while the three
   dashboard queries kept forwarding the dead UUID, leaving the UI
   looking like a full-workspace view but populated with empty
   project-scoped data. Validate the picked UUID against the current
   projects list before passing it to the queries.

2. The "by agent" table read its task count from the token rollup,
   which is grouped per (agent, model). A single task that spans two
   models lands twice and the agent's row reads e.g. "2 tasks" when
   the real count is 1. Prefer `ListDashboardAgentRunTime`'s per-agent
   distinct count when available; fall back to the token aggregate
   only for agents with no terminal run yet (in-flight tasks).

Extract the merge into `mergeAgentDashboardRows` so the precedence
rules are unit-tested directly.

Co-authored-by: multica-agent <github@multica.ai>

* test(dashboard): allocate per-workspace issue.number explicitly

TestDashboardEndpoints creates two issues in the shared fixture
workspace. issue.number defaults to 0 (migration 020), and the table
carries UNIQUE (workspace_id, number), so the second insert raced the
first on the same default and failed in CI.

Allocate MAX(number) + 1 per insert so each row gets a fresh number
without stepping on rows other tests left behind in the same workspace.

Co-authored-by: multica-agent <github@multica.ai>

* feat(dashboard): rollup table + cron-driven aggregation for dashboard

Mirror the per-runtime rollup in `task_usage_daily` (migrations 073/077/082)
to remove the per-request raw aggregation the dashboard was doing.

Migration 084 adds:
  - `task_usage_dashboard_daily` keyed on
    (bucket_date, workspace_id, agent_id, project_id, model) — the
    dimensions the dashboard actually queries, with project_id nullable
    via UNIQUE NULLS NOT DISTINCT (PG15+) so "no-project" buckets
    upsert cleanly.
  - `task_usage_dashboard_rollup_state` watermark table.
  - `task_usage_dashboard_dirty` invalidation queue.
  - Triggers on agent_task_queue DELETE, task_usage DELETE, and
    issue.project_id UPDATE — the cases the updated_at watermark can't
    see. The project_id trigger re-attributes existing rollup rows when
    a user moves an issue across projects.
  - `rollup_task_usage_dashboard_daily_window(from, to)` —
    idempotent recompute primitive (same shape as 077).
  - `rollup_task_usage_dashboard_daily()` cron entry — own advisory
    lock (4244) so it serialises independently of the runtime rollup.
  - `task_usage_dashboard_rollup_lag_seconds()` health helper.

Sqlc queries `ListDashboardUsageDailyRollup` /
`ListDashboardUsageByAgentRollup` read from the new table; the handler
dispatches between rollup and raw on a separate
`UseDailyRollupForDashboard` config flag
(`USAGE_DASHBOARD_ROLLUP_ENABLED` env). Same fail-safe default (false →
raw) so operators can roll out independently of the per-runtime flag.

Bucket date is UTC (the dashboard aggregates across runtimes that may
sit in different tzs; there's no single correct local boundary).

Adds `cmd/backfill_task_usage_dashboard_daily` mirroring the existing
per-runtime backfill — operator runs it once before flipping the flag.

Tests: - TestDashboardEndpoints now also exercises the rollup read path
    (raw vs. rollup, same project-scoped totals).
  - TestDashboardRollupReattributesOnProjectChange verifies the
    issue.project_id trigger enqueues both old + new buckets and the
    next rollup tick zeroes the old project + populates the new one.
Co-authored-by: multica-agent <github@multica.ai>

* fix(dashboard-rollup): close two invalidation gaps

Two leak paths missed by migration 084 review:

1. Issue cascade DELETE — the atq BEFORE DELETE trigger runs AFTER the
   issue row is gone, so `LEFT JOIN issue` returns NULL project_id and
   the original-project bucket never gets cleared (issue 077 calls this
   out for the runtime rollup but didn't need to act on it). Adds an
   `issue BEFORE DELETE` trigger that enqueues using OLD.project_id
   while the issue row is still readable.

2. `LinkTaskToIssue` (quick-create task attaching to a real issue post-
   completion) UPDATEs `agent_task_queue.issue_id` from NULL to a real
   id. Migration 084 only watched DELETE on atq, so usage already
   rolled up under the no-project bucket stayed attributed to NULL
   forever. Extends the atq trigger to fire on UPDATE OF issue_id too,
   enqueueing both OLD (NULL project) and NEW (linked issue's project).

Tests: - TestDashboardRollupClearsOnIssueDelete asserts rollup row drops to
    zero after issue delete + rollup tick.
  - TestDashboardRollupReattributesOnLinkTaskToIssue verifies tokens
    move from the NULL bucket to the project bucket after the UPDATE.
Co-authored-by: multica-agent <github@multica.ai>

---------

Co-authored-by: multica-agent <github@multica.ai>
 2026-05-13 12:51:16 +08:00