mirror of
https://github.com/multica-ai/multica.git
synced 2026-07-30 16:20:35 +02:00
* feat(dashboard): workspace/project token + run-time dashboard
Add a `/{slug}/dashboard` page showing per-agent token spend and execution
time across the whole workspace, with an optional project filter.
Backend:
- Three new sqlc queries against task_usage + agent_task_queue: daily
usage, per-agent usage, per-agent total run-time. All optionally
scoped to a project via sqlc.narg('project_id'), reaching project
through the issue join.
- Handlers under /api/dashboard return the same wire shape the runtime
page already consumes (model preserved for client-side cost math).
Frontend: - Shared DashboardPage in packages/views/dashboard reusing KpiCard,
DailyCostChart, ActorAvatar, and estimateCost from the runtime page
so the visual style and pricing math stay in lock-step.
- Period selector (7/30/90d), project dropdown, four KPI tiles
(cost, tokens, run time, tasks), daily cost chart, and a combined
"cost + run time by agent" list.
- Routed in both web (app/[slug]/(dashboard)/dashboard) and desktop
(memory router); sidebar nav entry added under Workspace group.
Co-authored-by: multica-agent <github@multica.ai>
* fix(dashboard): drop stale project filter and stop double-counting tasks
Two issues caught in PR #2462 review:
1. Project filter held the previous selection's UUID across workspace
switches and project deletions: the dropdown gracefully showed
"All projects" (because the title lookup missed) while the three
dashboard queries kept forwarding the dead UUID, leaving the UI
looking like a full-workspace view but populated with empty
project-scoped data. Validate the picked UUID against the current
projects list before passing it to the queries.
2. The "by agent" table read its task count from the token rollup,
which is grouped per (agent, model). A single task that spans two
models lands twice and the agent's row reads e.g. "2 tasks" when
the real count is 1. Prefer `ListDashboardAgentRunTime`'s per-agent
distinct count when available; fall back to the token aggregate
only for agents with no terminal run yet (in-flight tasks).
Extract the merge into `mergeAgentDashboardRows` so the precedence
rules are unit-tested directly.
Co-authored-by: multica-agent <github@multica.ai>
* test(dashboard): allocate per-workspace issue.number explicitly
TestDashboardEndpoints creates two issues in the shared fixture
workspace. issue.number defaults to 0 (migration 020), and the table
carries UNIQUE (workspace_id, number), so the second insert raced the
first on the same default and failed in CI.
Allocate MAX(number) + 1 per insert so each row gets a fresh number
without stepping on rows other tests left behind in the same workspace.
Co-authored-by: multica-agent <github@multica.ai>
* feat(dashboard): rollup table + cron-driven aggregation for dashboard
Mirror the per-runtime rollup in `task_usage_daily` (migrations 073/077/082)
to remove the per-request raw aggregation the dashboard was doing.
Migration 084 adds:
- `task_usage_dashboard_daily` keyed on
(bucket_date, workspace_id, agent_id, project_id, model) — the
dimensions the dashboard actually queries, with project_id nullable
via UNIQUE NULLS NOT DISTINCT (PG15+) so "no-project" buckets
upsert cleanly.
- `task_usage_dashboard_rollup_state` watermark table.
- `task_usage_dashboard_dirty` invalidation queue.
- Triggers on agent_task_queue DELETE, task_usage DELETE, and
issue.project_id UPDATE — the cases the updated_at watermark can't
see. The project_id trigger re-attributes existing rollup rows when
a user moves an issue across projects.
- `rollup_task_usage_dashboard_daily_window(from, to)` —
idempotent recompute primitive (same shape as 077).
- `rollup_task_usage_dashboard_daily()` cron entry — own advisory
lock (4244) so it serialises independently of the runtime rollup.
- `task_usage_dashboard_rollup_lag_seconds()` health helper.
Sqlc queries `ListDashboardUsageDailyRollup` /
`ListDashboardUsageByAgentRollup` read from the new table; the handler
dispatches between rollup and raw on a separate
`UseDailyRollupForDashboard` config flag
(`USAGE_DASHBOARD_ROLLUP_ENABLED` env). Same fail-safe default (false →
raw) so operators can roll out independently of the per-runtime flag.
Bucket date is UTC (the dashboard aggregates across runtimes that may
sit in different tzs; there's no single correct local boundary).
Adds `cmd/backfill_task_usage_dashboard_daily` mirroring the existing
per-runtime backfill — operator runs it once before flipping the flag.
Tests: - TestDashboardEndpoints now also exercises the rollup read path
(raw vs. rollup, same project-scoped totals).
- TestDashboardRollupReattributesOnProjectChange verifies the
issue.project_id trigger enqueues both old + new buckets and the
next rollup tick zeroes the old project + populates the new one.
Co-authored-by: multica-agent <github@multica.ai>
* fix(dashboard-rollup): close two invalidation gaps
Two leak paths missed by migration 084 review:
1. Issue cascade DELETE — the atq BEFORE DELETE trigger runs AFTER the
issue row is gone, so `LEFT JOIN issue` returns NULL project_id and
the original-project bucket never gets cleared (issue 077 calls this
out for the runtime rollup but didn't need to act on it). Adds an
`issue BEFORE DELETE` trigger that enqueues using OLD.project_id
while the issue row is still readable.
2. `LinkTaskToIssue` (quick-create task attaching to a real issue post-
completion) UPDATEs `agent_task_queue.issue_id` from NULL to a real
id. Migration 084 only watched DELETE on atq, so usage already
rolled up under the no-project bucket stayed attributed to NULL
forever. Extends the atq trigger to fire on UPDATE OF issue_id too,
enqueueing both OLD (NULL project) and NEW (linked issue's project).
Tests: - TestDashboardRollupClearsOnIssueDelete asserts rollup row drops to
zero after issue delete + rollup tick.
- TestDashboardRollupReattributesOnLinkTaskToIssue verifies tokens
move from the NULL bucket to the project bucket after the UPDATE.
Co-authored-by: multica-agent <github@multica.ai>
---------
Co-authored-by: multica-agent <github@multica.ai>
214 lines
6.8 KiB
TypeScript
214 lines
6.8 KiB
TypeScript
import { describe, expect, it } from "vitest";
|
||
import {
|
||
aggregateAgentTokens,
|
||
aggregateDailyCost,
|
||
computeDailyTotals,
|
||
formatDuration,
|
||
mergeAgentDashboardRows,
|
||
} from "./utils";
|
||
|
||
describe("aggregateDailyCost", () => {
|
||
it("collapses multiple rows per day into one stack and sorts by date asc", () => {
|
||
const result = aggregateDailyCost([
|
||
{
|
||
date: "2026-05-10",
|
||
model: "claude-sonnet-4-6",
|
||
input_tokens: 1_000_000,
|
||
output_tokens: 500_000,
|
||
cache_read_tokens: 0,
|
||
cache_write_tokens: 0,
|
||
task_count: 3,
|
||
},
|
||
{
|
||
date: "2026-05-09",
|
||
model: "claude-sonnet-4-6",
|
||
input_tokens: 1_000_000,
|
||
output_tokens: 0,
|
||
cache_read_tokens: 0,
|
||
cache_write_tokens: 0,
|
||
task_count: 1,
|
||
},
|
||
]);
|
||
|
||
// Sort: oldest day first.
|
||
expect(result.map((r) => r.date)).toEqual(["2026-05-09", "2026-05-10"]);
|
||
// claude-sonnet-4-6: input $3/M, output $15/M.
|
||
// 2026-05-09 → 1M input × $3 = $3 input, $0 output, $0 cache.
|
||
expect(result[0]).toMatchObject({ input: 3, output: 0, cacheWrite: 0, total: 3 });
|
||
// 2026-05-10 → $3 input + (0.5M × $15) = $7.5 output. Total $10.5.
|
||
expect(result[1]).toMatchObject({ input: 3, output: 7.5, cacheWrite: 0, total: 10.5 });
|
||
});
|
||
|
||
it("treats unmapped models as zero-cost", () => {
|
||
const result = aggregateDailyCost([
|
||
{
|
||
date: "2026-05-10",
|
||
model: "made-up-model",
|
||
input_tokens: 999_999_999,
|
||
output_tokens: 0,
|
||
cache_read_tokens: 0,
|
||
cache_write_tokens: 0,
|
||
task_count: 0,
|
||
},
|
||
]);
|
||
expect(result[0]?.total).toBe(0);
|
||
});
|
||
});
|
||
|
||
describe("aggregateAgentTokens", () => {
|
||
it("folds per-(agent, model) rows into per-agent totals and sorts by cost desc", () => {
|
||
const rows = aggregateAgentTokens([
|
||
{
|
||
agent_id: "small-spender",
|
||
model: "claude-sonnet-4-6",
|
||
input_tokens: 100_000,
|
||
output_tokens: 0,
|
||
cache_read_tokens: 0,
|
||
cache_write_tokens: 0,
|
||
task_count: 1,
|
||
},
|
||
{
|
||
agent_id: "big-spender",
|
||
model: "claude-sonnet-4-6",
|
||
input_tokens: 5_000_000,
|
||
output_tokens: 0,
|
||
cache_read_tokens: 0,
|
||
cache_write_tokens: 0,
|
||
task_count: 3,
|
||
},
|
||
{
|
||
agent_id: "big-spender",
|
||
model: "claude-haiku-4-5",
|
||
input_tokens: 1_000_000,
|
||
output_tokens: 0,
|
||
cache_read_tokens: 0,
|
||
cache_write_tokens: 0,
|
||
task_count: 2,
|
||
},
|
||
]);
|
||
|
||
expect(rows.map((r) => r.agentId)).toEqual(["big-spender", "small-spender"]);
|
||
expect(rows[0]?.taskCount).toBe(5);
|
||
// big-spender across two models — verify cost > small-spender's.
|
||
expect(rows[0]!.cost).toBeGreaterThan(rows[1]!.cost);
|
||
});
|
||
});
|
||
|
||
describe("computeDailyTotals", () => {
|
||
it("sums tokens across rows and adds estimated cost", () => {
|
||
const totals = computeDailyTotals([
|
||
{
|
||
date: "2026-05-10",
|
||
model: "claude-sonnet-4-6",
|
||
input_tokens: 1_000_000,
|
||
output_tokens: 0,
|
||
cache_read_tokens: 0,
|
||
cache_write_tokens: 0,
|
||
task_count: 2,
|
||
},
|
||
{
|
||
date: "2026-05-09",
|
||
model: "claude-sonnet-4-6",
|
||
input_tokens: 2_000_000,
|
||
output_tokens: 0,
|
||
cache_read_tokens: 0,
|
||
cache_write_tokens: 0,
|
||
task_count: 3,
|
||
},
|
||
]);
|
||
expect(totals.input).toBe(3_000_000);
|
||
expect(totals.cost).toBe(9); // 3M × $3/M
|
||
expect(totals.taskCount).toBe(5);
|
||
});
|
||
});
|
||
|
||
describe("mergeAgentDashboardRows", () => {
|
||
it("uses run-time rollup's per-agent task count, not the token sum", () => {
|
||
// Token rollup returns two (agent, model) rows for the same task
|
||
// (the agent ran one task that touched two models). The token-side
|
||
// aggregator sums per-row task_count and lands at 2; the run-time
|
||
// rollup correctly reports the underlying distinct count of 1.
|
||
const tokenRows = [
|
||
{
|
||
agentId: "agent-a",
|
||
tokens: 3_000_000,
|
||
cost: 12,
|
||
taskCount: 2, // overcounted because (model-1: 1) + (model-2: 1)
|
||
},
|
||
];
|
||
const runTimeRows = [
|
||
{
|
||
agent_id: "agent-a",
|
||
total_seconds: 600,
|
||
task_count: 1, // truth: one task touched both models
|
||
failed_count: 0,
|
||
},
|
||
];
|
||
const merged = mergeAgentDashboardRows(tokenRows, runTimeRows);
|
||
expect(merged).toHaveLength(1);
|
||
expect(merged[0]!.taskCount).toBe(1);
|
||
expect(merged[0]!.seconds).toBe(600);
|
||
});
|
||
|
||
it("falls back to token count when no run-time row exists (in-flight task)", () => {
|
||
// Tokens reported mid-run; task hasn't terminated yet so the run-time
|
||
// rollup is silent on this agent. Keep the token-side estimate
|
||
// instead of dropping the agent from the table entirely.
|
||
const merged = mergeAgentDashboardRows(
|
||
[{ agentId: "agent-b", tokens: 100, cost: 0.5, taskCount: 1 }],
|
||
[],
|
||
);
|
||
expect(merged[0]!.taskCount).toBe(1);
|
||
expect(merged[0]!.seconds).toBe(0);
|
||
});
|
||
|
||
it("includes agents that have run-time but no tokens", () => {
|
||
// Task errored before reporting any usage — run-time row exists but
|
||
// there's no corresponding token row. Agent must still appear on the
|
||
// list with zeroed-out token columns.
|
||
const merged = mergeAgentDashboardRows(
|
||
[],
|
||
[{ agent_id: "agent-c", total_seconds: 30, task_count: 1, failed_count: 1 }],
|
||
);
|
||
expect(merged).toHaveLength(1);
|
||
expect(merged[0]!.tokens).toBe(0);
|
||
expect(merged[0]!.cost).toBe(0);
|
||
expect(merged[0]!.taskCount).toBe(1);
|
||
});
|
||
|
||
it("sorts by cost desc with run-time as a tiebreaker", () => {
|
||
const merged = mergeAgentDashboardRows(
|
||
[
|
||
{ agentId: "low", tokens: 100, cost: 1, taskCount: 1 },
|
||
{ agentId: "high", tokens: 100, cost: 9, taskCount: 1 },
|
||
{ agentId: "zero-cost-long", tokens: 0, cost: 0, taskCount: 0 },
|
||
],
|
||
[
|
||
{ agent_id: "zero-cost-long", total_seconds: 1000, task_count: 5, failed_count: 0 },
|
||
],
|
||
);
|
||
expect(merged.map((r) => r.agentId)).toEqual(["high", "low", "zero-cost-long"]);
|
||
});
|
||
});
|
||
|
||
describe("formatDuration", () => {
|
||
it("formats seconds-only durations", () => {
|
||
expect(formatDuration(45, "<1m")).toBe("45s");
|
||
});
|
||
it("formats minutes and seconds when under one hour", () => {
|
||
expect(formatDuration(150, "<1m")).toBe("2m 30s");
|
||
expect(formatDuration(60, "<1m")).toBe("1m");
|
||
});
|
||
it("formats hours and minutes when under one day", () => {
|
||
expect(formatDuration(3 * 3600 + 17 * 60, "<1m")).toBe("3h 17m");
|
||
expect(formatDuration(3600, "<1m")).toBe("1h");
|
||
});
|
||
it("formats days and hours when more than 24 hours", () => {
|
||
expect(formatDuration(2 * 86400 + 5 * 3600, "<1m")).toBe("2d 5h");
|
||
});
|
||
it("falls back to the supplied label for sub-second durations", () => {
|
||
expect(formatDuration(0, "<1m")).toBe("<1m");
|
||
expect(formatDuration(0.4, "<1m")).toBe("<1m");
|
||
});
|
||
});
|