Files
multica/packages/views/dashboard/utils.test.ts
Bohan Jiang 96695a79c5 feat(dashboard): workspace/project token + run-time dashboard MUL-1882 (#2462)
* feat(dashboard): workspace/project token + run-time dashboard

Add a `/{slug}/dashboard` page showing per-agent token spend and execution
time across the whole workspace, with an optional project filter.

Backend:
  - Three new sqlc queries against task_usage + agent_task_queue: daily
    usage, per-agent usage, per-agent total run-time. All optionally
    scoped to a project via sqlc.narg('project_id'), reaching project
    through the issue join.
  - Handlers under /api/dashboard return the same wire shape the runtime
    page already consumes (model preserved for client-side cost math).

Frontend: - Shared DashboardPage in packages/views/dashboard reusing KpiCard,
    DailyCostChart, ActorAvatar, and estimateCost from the runtime page
    so the visual style and pricing math stay in lock-step.
  - Period selector (7/30/90d), project dropdown, four KPI tiles
    (cost, tokens, run time, tasks), daily cost chart, and a combined
    "cost + run time by agent" list.
  - Routed in both web (app/[slug]/(dashboard)/dashboard) and desktop
    (memory router); sidebar nav entry added under Workspace group.
Co-authored-by: multica-agent <github@multica.ai>

* fix(dashboard): drop stale project filter and stop double-counting tasks

Two issues caught in PR #2462 review:

1. Project filter held the previous selection's UUID across workspace
   switches and project deletions: the dropdown gracefully showed
   "All projects" (because the title lookup missed) while the three
   dashboard queries kept forwarding the dead UUID, leaving the UI
   looking like a full-workspace view but populated with empty
   project-scoped data. Validate the picked UUID against the current
   projects list before passing it to the queries.

2. The "by agent" table read its task count from the token rollup,
   which is grouped per (agent, model). A single task that spans two
   models lands twice and the agent's row reads e.g. "2 tasks" when
   the real count is 1. Prefer `ListDashboardAgentRunTime`'s per-agent
   distinct count when available; fall back to the token aggregate
   only for agents with no terminal run yet (in-flight tasks).

Extract the merge into `mergeAgentDashboardRows` so the precedence
rules are unit-tested directly.

Co-authored-by: multica-agent <github@multica.ai>

* test(dashboard): allocate per-workspace issue.number explicitly

TestDashboardEndpoints creates two issues in the shared fixture
workspace. issue.number defaults to 0 (migration 020), and the table
carries UNIQUE (workspace_id, number), so the second insert raced the
first on the same default and failed in CI.

Allocate MAX(number) + 1 per insert so each row gets a fresh number
without stepping on rows other tests left behind in the same workspace.

Co-authored-by: multica-agent <github@multica.ai>

* feat(dashboard): rollup table + cron-driven aggregation for dashboard

Mirror the per-runtime rollup in `task_usage_daily` (migrations 073/077/082)
to remove the per-request raw aggregation the dashboard was doing.

Migration 084 adds:
  - `task_usage_dashboard_daily` keyed on
    (bucket_date, workspace_id, agent_id, project_id, model) — the
    dimensions the dashboard actually queries, with project_id nullable
    via UNIQUE NULLS NOT DISTINCT (PG15+) so "no-project" buckets
    upsert cleanly.
  - `task_usage_dashboard_rollup_state` watermark table.
  - `task_usage_dashboard_dirty` invalidation queue.
  - Triggers on agent_task_queue DELETE, task_usage DELETE, and
    issue.project_id UPDATE — the cases the updated_at watermark can't
    see. The project_id trigger re-attributes existing rollup rows when
    a user moves an issue across projects.
  - `rollup_task_usage_dashboard_daily_window(from, to)` —
    idempotent recompute primitive (same shape as 077).
  - `rollup_task_usage_dashboard_daily()` cron entry — own advisory
    lock (4244) so it serialises independently of the runtime rollup.
  - `task_usage_dashboard_rollup_lag_seconds()` health helper.

Sqlc queries `ListDashboardUsageDailyRollup` /
`ListDashboardUsageByAgentRollup` read from the new table; the handler
dispatches between rollup and raw on a separate
`UseDailyRollupForDashboard` config flag
(`USAGE_DASHBOARD_ROLLUP_ENABLED` env). Same fail-safe default (false →
raw) so operators can roll out independently of the per-runtime flag.

Bucket date is UTC (the dashboard aggregates across runtimes that may
sit in different tzs; there's no single correct local boundary).

Adds `cmd/backfill_task_usage_dashboard_daily` mirroring the existing
per-runtime backfill — operator runs it once before flipping the flag.

Tests: - TestDashboardEndpoints now also exercises the rollup read path
    (raw vs. rollup, same project-scoped totals).
  - TestDashboardRollupReattributesOnProjectChange verifies the
    issue.project_id trigger enqueues both old + new buckets and the
    next rollup tick zeroes the old project + populates the new one.
Co-authored-by: multica-agent <github@multica.ai>

* fix(dashboard-rollup): close two invalidation gaps

Two leak paths missed by migration 084 review:

1. Issue cascade DELETE — the atq BEFORE DELETE trigger runs AFTER the
   issue row is gone, so `LEFT JOIN issue` returns NULL project_id and
   the original-project bucket never gets cleared (issue 077 calls this
   out for the runtime rollup but didn't need to act on it). Adds an
   `issue BEFORE DELETE` trigger that enqueues using OLD.project_id
   while the issue row is still readable.

2. `LinkTaskToIssue` (quick-create task attaching to a real issue post-
   completion) UPDATEs `agent_task_queue.issue_id` from NULL to a real
   id. Migration 084 only watched DELETE on atq, so usage already
   rolled up under the no-project bucket stayed attributed to NULL
   forever. Extends the atq trigger to fire on UPDATE OF issue_id too,
   enqueueing both OLD (NULL project) and NEW (linked issue's project).

Tests: - TestDashboardRollupClearsOnIssueDelete asserts rollup row drops to
    zero after issue delete + rollup tick.
  - TestDashboardRollupReattributesOnLinkTaskToIssue verifies tokens
    move from the NULL bucket to the project bucket after the UPDATE.
Co-authored-by: multica-agent <github@multica.ai>

---------

Co-authored-by: multica-agent <github@multica.ai>
2026-05-13 12:51:16 +08:00

214 lines
6.8 KiB
TypeScript
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import { describe, expect, it } from "vitest";
import {
aggregateAgentTokens,
aggregateDailyCost,
computeDailyTotals,
formatDuration,
mergeAgentDashboardRows,
} from "./utils";
describe("aggregateDailyCost", () => {
it("collapses multiple rows per day into one stack and sorts by date asc", () => {
const result = aggregateDailyCost([
{
date: "2026-05-10",
model: "claude-sonnet-4-6",
input_tokens: 1_000_000,
output_tokens: 500_000,
cache_read_tokens: 0,
cache_write_tokens: 0,
task_count: 3,
},
{
date: "2026-05-09",
model: "claude-sonnet-4-6",
input_tokens: 1_000_000,
output_tokens: 0,
cache_read_tokens: 0,
cache_write_tokens: 0,
task_count: 1,
},
]);
// Sort: oldest day first.
expect(result.map((r) => r.date)).toEqual(["2026-05-09", "2026-05-10"]);
// claude-sonnet-4-6: input $3/M, output $15/M.
// 2026-05-09 → 1M input × $3 = $3 input, $0 output, $0 cache.
expect(result[0]).toMatchObject({ input: 3, output: 0, cacheWrite: 0, total: 3 });
// 2026-05-10 → $3 input + (0.5M × $15) = $7.5 output. Total $10.5.
expect(result[1]).toMatchObject({ input: 3, output: 7.5, cacheWrite: 0, total: 10.5 });
});
it("treats unmapped models as zero-cost", () => {
const result = aggregateDailyCost([
{
date: "2026-05-10",
model: "made-up-model",
input_tokens: 999_999_999,
output_tokens: 0,
cache_read_tokens: 0,
cache_write_tokens: 0,
task_count: 0,
},
]);
expect(result[0]?.total).toBe(0);
});
});
describe("aggregateAgentTokens", () => {
it("folds per-(agent, model) rows into per-agent totals and sorts by cost desc", () => {
const rows = aggregateAgentTokens([
{
agent_id: "small-spender",
model: "claude-sonnet-4-6",
input_tokens: 100_000,
output_tokens: 0,
cache_read_tokens: 0,
cache_write_tokens: 0,
task_count: 1,
},
{
agent_id: "big-spender",
model: "claude-sonnet-4-6",
input_tokens: 5_000_000,
output_tokens: 0,
cache_read_tokens: 0,
cache_write_tokens: 0,
task_count: 3,
},
{
agent_id: "big-spender",
model: "claude-haiku-4-5",
input_tokens: 1_000_000,
output_tokens: 0,
cache_read_tokens: 0,
cache_write_tokens: 0,
task_count: 2,
},
]);
expect(rows.map((r) => r.agentId)).toEqual(["big-spender", "small-spender"]);
expect(rows[0]?.taskCount).toBe(5);
// big-spender across two models — verify cost > small-spender's.
expect(rows[0]!.cost).toBeGreaterThan(rows[1]!.cost);
});
});
describe("computeDailyTotals", () => {
it("sums tokens across rows and adds estimated cost", () => {
const totals = computeDailyTotals([
{
date: "2026-05-10",
model: "claude-sonnet-4-6",
input_tokens: 1_000_000,
output_tokens: 0,
cache_read_tokens: 0,
cache_write_tokens: 0,
task_count: 2,
},
{
date: "2026-05-09",
model: "claude-sonnet-4-6",
input_tokens: 2_000_000,
output_tokens: 0,
cache_read_tokens: 0,
cache_write_tokens: 0,
task_count: 3,
},
]);
expect(totals.input).toBe(3_000_000);
expect(totals.cost).toBe(9); // 3M × $3/M
expect(totals.taskCount).toBe(5);
});
});
describe("mergeAgentDashboardRows", () => {
it("uses run-time rollup's per-agent task count, not the token sum", () => {
// Token rollup returns two (agent, model) rows for the same task
// (the agent ran one task that touched two models). The token-side
// aggregator sums per-row task_count and lands at 2; the run-time
// rollup correctly reports the underlying distinct count of 1.
const tokenRows = [
{
agentId: "agent-a",
tokens: 3_000_000,
cost: 12,
taskCount: 2, // overcounted because (model-1: 1) + (model-2: 1)
},
];
const runTimeRows = [
{
agent_id: "agent-a",
total_seconds: 600,
task_count: 1, // truth: one task touched both models
failed_count: 0,
},
];
const merged = mergeAgentDashboardRows(tokenRows, runTimeRows);
expect(merged).toHaveLength(1);
expect(merged[0]!.taskCount).toBe(1);
expect(merged[0]!.seconds).toBe(600);
});
it("falls back to token count when no run-time row exists (in-flight task)", () => {
// Tokens reported mid-run; task hasn't terminated yet so the run-time
// rollup is silent on this agent. Keep the token-side estimate
// instead of dropping the agent from the table entirely.
const merged = mergeAgentDashboardRows(
[{ agentId: "agent-b", tokens: 100, cost: 0.5, taskCount: 1 }],
[],
);
expect(merged[0]!.taskCount).toBe(1);
expect(merged[0]!.seconds).toBe(0);
});
it("includes agents that have run-time but no tokens", () => {
// Task errored before reporting any usage — run-time row exists but
// there's no corresponding token row. Agent must still appear on the
// list with zeroed-out token columns.
const merged = mergeAgentDashboardRows(
[],
[{ agent_id: "agent-c", total_seconds: 30, task_count: 1, failed_count: 1 }],
);
expect(merged).toHaveLength(1);
expect(merged[0]!.tokens).toBe(0);
expect(merged[0]!.cost).toBe(0);
expect(merged[0]!.taskCount).toBe(1);
});
it("sorts by cost desc with run-time as a tiebreaker", () => {
const merged = mergeAgentDashboardRows(
[
{ agentId: "low", tokens: 100, cost: 1, taskCount: 1 },
{ agentId: "high", tokens: 100, cost: 9, taskCount: 1 },
{ agentId: "zero-cost-long", tokens: 0, cost: 0, taskCount: 0 },
],
[
{ agent_id: "zero-cost-long", total_seconds: 1000, task_count: 5, failed_count: 0 },
],
);
expect(merged.map((r) => r.agentId)).toEqual(["high", "low", "zero-cost-long"]);
});
});
describe("formatDuration", () => {
it("formats seconds-only durations", () => {
expect(formatDuration(45, "<1m")).toBe("45s");
});
it("formats minutes and seconds when under one hour", () => {
expect(formatDuration(150, "<1m")).toBe("2m 30s");
expect(formatDuration(60, "<1m")).toBe("1m");
});
it("formats hours and minutes when under one day", () => {
expect(formatDuration(3 * 3600 + 17 * 60, "<1m")).toBe("3h 17m");
expect(formatDuration(3600, "<1m")).toBe("1h");
});
it("formats days and hours when more than 24 hours", () => {
expect(formatDuration(2 * 86400 + 5 * 3600, "<1m")).toBe("2d 5h");
});
it("falls back to the supplied label for sub-second durations", () => {
expect(formatDuration(0, "<1m")).toBe("<1m");
expect(formatDuration(0.4, "<1m")).toBe("<1m");
});
});