mirror of
https://github.com/multica-ai/multica.git
synced 2026-07-26 12:35:35 +02:00
* fix(agent): attribute Grok usage from the turn's own model id A resumed Grok session with no configured model recorded its entire spend under the model id "unknown", which matches no pricing row — so the task reported $0 cost instead of its real spend. grok.go only learned the model from the session handshake, and ACP's `session/load` carries no model id (only `session/new` does). When neither the agent nor MULTICA_GROK_MODEL pins a model, `daemon.go` legitimately passes an empty model, leaving nothing to attribute the usage to. Every Grok turn stamps `result._meta.modelId` with what it actually billed against. Parse it in the shared ACP result parser and use it as the fallback in grok.go. Other ACP backends are untouched — they keep whatever the handshake gave them. Co-authored-by: J <agent@multica.ai> Co-authored-by: multica-agent <github@multica.ai> * fix(metrics): price the Grok catalog in server-side cost metrics server/internal/metrics/pricing.go carried no Grok rows at all, so RecordLLMUsage took the unpriced branch for every Grok turn: llm_cost_usd reported zero Grok spend while the tokens accumulated in llm_unpriced_tokens. Internal cost monitoring simply could not see Grok. Add the six SKUs xAI publishes rates for, mirroring the frontend table in packages/views/runtimes/utils.ts. Aliases are anchored exact matches like the gpt-5.6 rows, so `grok-composer-*` (in the catalog, absent from the price sheet) stays unmapped instead of inheriting a guessed rate. Short-context tier on purpose: xAI bills a request at 2x once its prompt reaches 200K tokens, but a usage record aggregates every model call in a turn and cannot say which tier an individual request hit. A regression test re-derives the cost of a real grok 0.2.106 turn from the table and checks it against the costUsdTicks xAI returned for that turn. Co-authored-by: J <agent@multica.ai> Co-authored-by: multica-agent <github@multica.ai> * docs(changelog): scope the Grok cost claim to what was actually fixed The v0.4.9 entry promised "accurate cost" in all four languages, but the fix corrected catalog pricing and cached-input double-counting — it did not implement xAI's 2x long-context tier, so a turn whose requests reach 200K prompt tokens still under-reports by up to 50%. Say what was fixed instead. Also correct two stale claims in the pricing comment: the daemon tags usage rows with the runtime provider `grok`, not `xai` (the bare `grok-*` keys are what make them resolve), and record why thresholding the long-context tier on an aggregated row would be worse than not pricing it at all. Co-authored-by: J <agent@multica.ai> Co-authored-by: multica-agent <github@multica.ai> * feat(usage): carry the provider's own cost through to the usage record Cost has always been derived client-side as tokens x a static rate, which cannot express request-level pricing rules. xAI bills a Grok request at 2x once its prompt reaches 200K tokens, and a task_usage row aggregates every model call in a turn — so the stored token counts genuinely cannot say which tier any individual request hit. Thresholding on the aggregate would be worse than the status quo: it turns a bounded 50% under-estimate into an unbounded over-estimate for turns made of many short requests. Grok already reports what it charged, per turn, in `_meta.usage.costUsdTicks`. Parse it, carry it through agent -> daemon -> API, and store it on task_usage as a nullable BIGINT of 1e-10 USD ticks (integer, so sub-cent turns stay exact end to end). NULL means the provider reported no cost — every pre-existing row and every provider that doesn't return one. No backfill: there is no authoritative figure to recover for those, and inventing one is the guess this removes. A single hourly bucket can mix rows that carry a cost with rows that don't, so task_usage_hourly gains both halves: `cost_usd_ticks` sums the authoritative side, and `uncosted_*_tokens` carry exactly the tokens that still need a rate-table estimate. Consumers report authoritative + estimate(uncosted), which degrades to today's behaviour when nothing in the bucket is authoritative. The existing token columns keep covering every row, so token displays are untouched. The new columns are additive with defaults, so the unique key, the dirty-queue shape, and migration 102's triggers are unaffected. Co-authored-by: J <agent@multica.ai> Co-authored-by: multica-agent <github@multica.ai> * feat(usage): prefer the provider's own cost over the rate table With the authoritative figure now stored, both cost consumers use it: the usage dashboard (estimateCost / estimateCostBreakdown) and the server-side llm_cost_usd metric. Each reports `authoritative + estimate(uncosted tokens)`, so a row or bucket that mixes priced and unpriced sources stays whole. The static rate tables remain, but for Grok they are now a fallback — they still price usage recorded by a daemon too old to report cost, and every provider that reports none. Custom pricing overrides likewise apply only to the estimated half: they are a user's guess at a rate, and the authoritative half is not a guess. A model with no rate-table row but a provider-reported cost now also drops out of the "unmapped models" banner, since asking the user to supply a rate for it would invite overriding a real bill. llm_cost_usd is labelled by token_type and the provider reports one number per turn, so the charge is distributed across the buckets in the rate table's own proportions. Only the total is authoritative; the split stays an estimate, which is why this scales the existing buckets rather than inventing a label. estimateCostBreakdown does the same, keeping the stacked chart summing to the headline figure instead of silently under-drawing every Grok row. Co-authored-by: J <agent@multica.ai> Co-authored-by: multica-agent <github@multica.ai> * docs(changelog): say Grok cost now follows xAI's actual charge The earlier wording scoped the claim down to catalog pricing and cached input because the long-context tier was still unhandled. It is handled now — the cost comes from what xAI charged for the turn — so the entry can say so. Co-authored-by: J <agent@multica.ai> Co-authored-by: multica-agent <github@multica.ai> * fix(usage): keep the provider's cost when the model has no rate row Both cost consumers bailed out before reading the authoritative figure when the rate table had no row for the model. A `grok-composer-*` turn — in the Grok Build catalog, absent from xAI's price sheet — was therefore reported as $0 spend even though xAI told us exactly what it charged. Worse on the client: estimateCost returned the real cost while estimateCostBreakdown returned zeros, so the headline and the stacked chart disagreed on precisely the rows whose cost is exact — and the unmapped-models banner was (correctly) hidden, so nothing explained the discrepancy. Handle the charge before the rate lookup in both places. Without rates there is nothing to split a total by, so it lands whole in the `input` bucket, the same fallback distributeAuthoritativeCost already uses when it has no shape to scale. Tokens with no rate keep going to llm_unpriced_tokens: "unpriced" describes the rate table, not the money. Co-authored-by: J <agent@multica.ai> Co-authored-by: multica-agent <github@multica.ai> * perf(usage): drop the historical rewrite from the cost-split migration Migration 213 rewrote every existing task_usage_hourly row to seed the uncosted counters. That is a full-table UPDATE inside a schema migration — lock time, WAL and bloat all scaling with table size — for rows this issue explicitly does not care about. Deleting the UPDATE alone would have zeroed historical cost: with `NOT NULL DEFAULT 0`, an untouched row asserts "nothing here needs estimating", so every pre-split bucket would report $0 until the rollup happened to touch it. Make the uncosted columns nullable with no default instead. NULL means "never recomputed since the split existed", readers COALESCE it to the row's own token total ("estimate all of it"), and the pre-split behaviour is preserved exactly — with nothing to seed, so no rewrite. A bare ADD COLUMN is metadata-only, so this is now fast DDL. Rows heal into the split naturally as the rollup recomputes their buckets. Verified on a fresh database: a legacy-shaped row reads back as its full tokens to estimate, and a group mixing legacy and post-split buckets sums to the authoritative cost plus both rows' estimable tokens. Co-authored-by: J <agent@multica.ai> Co-authored-by: multica-agent <github@multica.ai> --------- Co-authored-by: Bohan-J <bohan@devv.ai> Co-authored-by: J <agent@multica.ai> Co-authored-by: multica-agent <github@multica.ai>
325 lines
12 KiB
Go
325 lines
12 KiB
Go
package handler
|
|
|
|
import (
|
|
"context"
|
|
"net/http"
|
|
|
|
"github.com/jackc/pgx/v5/pgtype"
|
|
"github.com/multica-ai/multica/server/internal/util"
|
|
db "github.com/multica-ai/multica/server/pkg/db/generated"
|
|
)
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Workspace / Project dashboard
|
|
//
|
|
// Three read endpoints power the workspace dashboard:
|
|
//
|
|
// GET /api/dashboard/usage/daily per-(date, model) token rows
|
|
// GET /api/dashboard/usage/by-agent per-(agent, model) token rows
|
|
// GET /api/dashboard/agent-runtime per-agent run-time + task counts
|
|
// GET /api/dashboard/runtime/daily per-date run-time + task counts
|
|
//
|
|
// All three accept ?days=N (defaults to 30, capped at 365) and an optional
|
|
// ?project_id=<uuid> to scope the rollup to a single project. With no
|
|
// project_id the data spans the whole workspace.
|
|
//
|
|
// Cost is computed client-side from a per-model pricing table — the model
|
|
// dimension is intentionally preserved on the wire (same convention as the
|
|
// per-runtime usage endpoints).
|
|
//
|
|
// Access control: workspace membership only — we don't filter by per-agent
|
|
// visibility on the dashboard because token spend / run time are workspace-
|
|
// level operational metrics. Agent-detail pages still gate on per-agent
|
|
// access (see GetWorkspaceAgentRunCounts).
|
|
// ---------------------------------------------------------------------------
|
|
|
|
// parseProjectIDParam reads ?project_id=<uuid> off the URL. Returns a
|
|
// pgtype.UUID with Valid=false when the param is absent so sqlc's nullable
|
|
// argument resolves to SQL NULL and the WHERE clause degrades to "no
|
|
// project filter". On a malformed UUID it writes a 400 and returns
|
|
// ok=false; callers must return immediately.
|
|
func parseProjectIDParam(w http.ResponseWriter, r *http.Request) (pgtype.UUID, bool) {
|
|
raw := r.URL.Query().Get("project_id")
|
|
if raw == "" {
|
|
return pgtype.UUID{}, true
|
|
}
|
|
u, err := util.ParseUUID(raw)
|
|
if err != nil {
|
|
writeError(w, http.StatusBadRequest, "invalid project_id")
|
|
return pgtype.UUID{}, false
|
|
}
|
|
return u, true
|
|
}
|
|
|
|
// DashboardUsageDailyResponse is one (date, provider, model) bucket. Cost-side
|
|
// math happens on the client from a per-model pricing table; provider + model
|
|
// stay on the wire so the client can disambiguate bare model ids that collide
|
|
// across providers (e.g. Cursor's `auto`).
|
|
type DashboardUsageDailyResponse struct {
|
|
Date string `json:"date"`
|
|
Provider string `json:"provider"`
|
|
Model string `json:"model"`
|
|
InputTokens int64 `json:"input_tokens"`
|
|
OutputTokens int64 `json:"output_tokens"`
|
|
CacheReadTokens int64 `json:"cache_read_tokens"`
|
|
CacheWriteTokens int64 `json:"cache_write_tokens"`
|
|
// Cost split: `CostUSDTicks` is what the provider itself charged for the
|
|
// rows behind this aggregate (1e-10 USD), and the `Uncosted*` token
|
|
// counts are the tokens from rows the provider did NOT price. The client
|
|
// reports authoritative + estimate(uncosted), so a window mixing both
|
|
// kinds of row stays whole. See migration 213.
|
|
CostUSDTicks int64 `json:"cost_usd_ticks"`
|
|
UncostedInputTokens int64 `json:"uncosted_input_tokens"`
|
|
UncostedOutputTokens int64 `json:"uncosted_output_tokens"`
|
|
UncostedCacheReadTokens int64 `json:"uncosted_cache_read_tokens"`
|
|
UncostedCacheWriteTokens int64 `json:"uncosted_cache_write_tokens"`
|
|
TaskCount int32 `json:"task_count"`
|
|
}
|
|
|
|
// GetDashboardUsageDaily returns per-(date, model) token rows for the
|
|
// workspace, optionally scoped to a project. Backed by task_usage_hourly,
|
|
// sliced into calendar days under the viewer's tz.
|
|
func (h *Handler) GetDashboardUsageDaily(w http.ResponseWriter, r *http.Request) {
|
|
workspaceID := h.resolveWorkspaceID(r)
|
|
if _, ok := h.workspaceMember(w, r, workspaceID); !ok {
|
|
return
|
|
}
|
|
projectID, ok := parseProjectIDParam(w, r)
|
|
if !ok {
|
|
return
|
|
}
|
|
tz := h.resolveViewingTZ(r)
|
|
since := parseSinceParamInTZ(r, 30, tz)
|
|
|
|
resp, err := h.listDashboardUsageDaily(r.Context(), parseUUID(workspaceID), tz, since, projectID)
|
|
if err != nil {
|
|
writeError(w, http.StatusInternalServerError, "failed to list usage")
|
|
return
|
|
}
|
|
writeJSON(w, http.StatusOK, resp)
|
|
}
|
|
|
|
func (h *Handler) listDashboardUsageDaily(
|
|
ctx context.Context,
|
|
workspaceID pgtype.UUID,
|
|
tz string,
|
|
since pgtype.Timestamptz,
|
|
projectID pgtype.UUID,
|
|
) ([]DashboardUsageDailyResponse, error) {
|
|
rows, err := h.Queries.ListDashboardUsageDaily(ctx, db.ListDashboardUsageDailyParams{
|
|
WorkspaceID: workspaceID,
|
|
Tz: tz,
|
|
Since: since,
|
|
ProjectID: projectID,
|
|
})
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
resp := make([]DashboardUsageDailyResponse, len(rows))
|
|
for i, row := range rows {
|
|
resp[i] = DashboardUsageDailyResponse{
|
|
Date: row.Date.Time.Format("2006-01-02"),
|
|
Provider: row.Provider,
|
|
Model: row.Model,
|
|
InputTokens: row.InputTokens,
|
|
OutputTokens: row.OutputTokens,
|
|
CacheReadTokens: row.CacheReadTokens,
|
|
CacheWriteTokens: row.CacheWriteTokens,
|
|
CostUSDTicks: row.CostUsdTicks,
|
|
UncostedInputTokens: row.UncostedInputTokens,
|
|
UncostedOutputTokens: row.UncostedOutputTokens,
|
|
UncostedCacheReadTokens: row.UncostedCacheReadTokens,
|
|
UncostedCacheWriteTokens: row.UncostedCacheWriteTokens,
|
|
TaskCount: row.TaskCount,
|
|
}
|
|
}
|
|
return resp, nil
|
|
}
|
|
|
|
// DashboardUsageByAgentResponse is one (agent, provider, model) row. provider
|
|
// rides along for the same cross-provider pricing disambiguation as the daily
|
|
// response; the client folds by agent_id and sums cost.
|
|
type DashboardUsageByAgentResponse struct {
|
|
AgentID string `json:"agent_id"`
|
|
Provider string `json:"provider"`
|
|
Model string `json:"model"`
|
|
InputTokens int64 `json:"input_tokens"`
|
|
OutputTokens int64 `json:"output_tokens"`
|
|
CacheReadTokens int64 `json:"cache_read_tokens"`
|
|
CacheWriteTokens int64 `json:"cache_write_tokens"`
|
|
// Cost split: `CostUSDTicks` is what the provider itself charged for the
|
|
// rows behind this aggregate (1e-10 USD), and the `Uncosted*` token
|
|
// counts are the tokens from rows the provider did NOT price. The client
|
|
// reports authoritative + estimate(uncosted), so a window mixing both
|
|
// kinds of row stays whole. See migration 213.
|
|
CostUSDTicks int64 `json:"cost_usd_ticks"`
|
|
UncostedInputTokens int64 `json:"uncosted_input_tokens"`
|
|
UncostedOutputTokens int64 `json:"uncosted_output_tokens"`
|
|
UncostedCacheReadTokens int64 `json:"uncosted_cache_read_tokens"`
|
|
UncostedCacheWriteTokens int64 `json:"uncosted_cache_write_tokens"`
|
|
TaskCount int32 `json:"task_count"`
|
|
}
|
|
|
|
// GetDashboardUsageByAgent returns per-(agent, model) token aggregates
|
|
// for the workspace, optionally scoped to a project. Backed by
|
|
// task_usage_hourly with the viewer's tz applied to the `?days=` cutoff.
|
|
func (h *Handler) GetDashboardUsageByAgent(w http.ResponseWriter, r *http.Request) {
|
|
workspaceID := h.resolveWorkspaceID(r)
|
|
if _, ok := h.workspaceMember(w, r, workspaceID); !ok {
|
|
return
|
|
}
|
|
projectID, ok := parseProjectIDParam(w, r)
|
|
if !ok {
|
|
return
|
|
}
|
|
// "By agent" has no date grouping in the SQL — tz only determines
|
|
// the cutoff boundary, not the bucket axis.
|
|
tz := h.resolveViewingTZ(r)
|
|
since := parseSinceParamInTZ(r, 30, tz)
|
|
|
|
resp, err := h.listDashboardUsageByAgent(r.Context(), parseUUID(workspaceID), since, projectID)
|
|
if err != nil {
|
|
writeError(w, http.StatusInternalServerError, "failed to list usage by agent")
|
|
return
|
|
}
|
|
writeJSON(w, http.StatusOK, resp)
|
|
}
|
|
|
|
func (h *Handler) listDashboardUsageByAgent(
|
|
ctx context.Context,
|
|
workspaceID pgtype.UUID,
|
|
since pgtype.Timestamptz,
|
|
projectID pgtype.UUID,
|
|
) ([]DashboardUsageByAgentResponse, error) {
|
|
rows, err := h.Queries.ListDashboardUsageByAgent(ctx, db.ListDashboardUsageByAgentParams{
|
|
WorkspaceID: workspaceID,
|
|
Since: since,
|
|
ProjectID: projectID,
|
|
})
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
resp := make([]DashboardUsageByAgentResponse, len(rows))
|
|
for i, row := range rows {
|
|
resp[i] = DashboardUsageByAgentResponse{
|
|
AgentID: uuidToString(row.AgentID),
|
|
Provider: row.Provider,
|
|
Model: row.Model,
|
|
InputTokens: row.InputTokens,
|
|
OutputTokens: row.OutputTokens,
|
|
CacheReadTokens: row.CacheReadTokens,
|
|
CacheWriteTokens: row.CacheWriteTokens,
|
|
CostUSDTicks: row.CostUsdTicks,
|
|
UncostedInputTokens: row.UncostedInputTokens,
|
|
UncostedOutputTokens: row.UncostedOutputTokens,
|
|
UncostedCacheReadTokens: row.UncostedCacheReadTokens,
|
|
UncostedCacheWriteTokens: row.UncostedCacheWriteTokens,
|
|
TaskCount: row.TaskCount,
|
|
}
|
|
}
|
|
return resp, nil
|
|
}
|
|
|
|
// DashboardAgentRunTimeResponse is one agent's total terminal-task run time
|
|
// over the window. Includes failed tasks so the dashboard can surface how
|
|
// much execution time was spent on runs that didn't succeed.
|
|
type DashboardAgentRunTimeResponse struct {
|
|
AgentID string `json:"agent_id"`
|
|
TotalSeconds int64 `json:"total_seconds"`
|
|
TaskCount int32 `json:"task_count"`
|
|
FailedCount int32 `json:"failed_count"`
|
|
}
|
|
|
|
// GetDashboardAgentRunTime returns per-agent total task run time (seconds)
|
|
// and task counts for the workspace, optionally scoped to a project. Only
|
|
// terminal tasks (completed or failed) with both started_at and
|
|
// completed_at populated contribute, since queued/running tasks have no
|
|
// finite duration.
|
|
func (h *Handler) GetDashboardAgentRunTime(w http.ResponseWriter, r *http.Request) {
|
|
workspaceID := h.resolveWorkspaceID(r)
|
|
if _, ok := h.workspaceMember(w, r, workspaceID); !ok {
|
|
return
|
|
}
|
|
projectID, ok := parseProjectIDParam(w, r)
|
|
if !ok {
|
|
return
|
|
}
|
|
// Cutoff in the viewer's tz so the "last N days" window matches the
|
|
// per-agent cost card (GetDashboardUsageByAgent).
|
|
tz := h.resolveViewingTZ(r)
|
|
since := parseSinceParamInTZ(r, 30, tz)
|
|
|
|
rows, err := h.Queries.ListDashboardAgentRunTime(r.Context(), db.ListDashboardAgentRunTimeParams{
|
|
WorkspaceID: parseUUID(workspaceID),
|
|
Since: since,
|
|
ProjectID: projectID,
|
|
})
|
|
if err != nil {
|
|
writeError(w, http.StatusInternalServerError, "failed to list agent runtime")
|
|
return
|
|
}
|
|
|
|
resp := make([]DashboardAgentRunTimeResponse, len(rows))
|
|
for i, row := range rows {
|
|
resp[i] = DashboardAgentRunTimeResponse{
|
|
AgentID: uuidToString(row.AgentID),
|
|
TotalSeconds: row.TotalSeconds,
|
|
TaskCount: row.TaskCount,
|
|
FailedCount: row.FailedCount,
|
|
}
|
|
}
|
|
writeJSON(w, http.StatusOK, resp)
|
|
}
|
|
|
|
// DashboardRunTimeDailyResponse is one (date) bucket of terminal-task run
|
|
// time and counts. Powers the workspace dashboard's daily Time and Tasks
|
|
// charts — same toggle as Tokens / Cost, different metric.
|
|
type DashboardRunTimeDailyResponse struct {
|
|
Date string `json:"date"`
|
|
TotalSeconds int64 `json:"total_seconds"`
|
|
TaskCount int32 `json:"task_count"`
|
|
FailedCount int32 `json:"failed_count"`
|
|
}
|
|
|
|
// GetDashboardRunTimeDaily returns per-date total task run time and task
|
|
// counts for the workspace, optionally scoped to a project. Only terminal
|
|
// tasks (completed or failed) with both started_at and completed_at
|
|
// populated contribute. Bucketed by completed_at so the day boundaries
|
|
// line up with the per-agent run-time card.
|
|
func (h *Handler) GetDashboardRunTimeDaily(w http.ResponseWriter, r *http.Request) {
|
|
workspaceID := h.resolveWorkspaceID(r)
|
|
if _, ok := h.workspaceMember(w, r, workspaceID); !ok {
|
|
return
|
|
}
|
|
projectID, ok := parseProjectIDParam(w, r)
|
|
if !ok {
|
|
return
|
|
}
|
|
// Slice day buckets in the viewer's tz so the Time / Tasks charts cut
|
|
// their calendar day identically to the Cost / Tokens charts.
|
|
tz := h.resolveViewingTZ(r)
|
|
since := parseSinceParamInTZ(r, 30, tz)
|
|
|
|
rows, err := h.Queries.ListDashboardRunTimeDaily(r.Context(), db.ListDashboardRunTimeDailyParams{
|
|
WorkspaceID: parseUUID(workspaceID),
|
|
Tz: tz,
|
|
Since: since,
|
|
ProjectID: projectID,
|
|
})
|
|
if err != nil {
|
|
writeError(w, http.StatusInternalServerError, "failed to list daily runtime")
|
|
return
|
|
}
|
|
|
|
resp := make([]DashboardRunTimeDailyResponse, len(rows))
|
|
for i, row := range rows {
|
|
resp[i] = DashboardRunTimeDailyResponse{
|
|
Date: row.Date.Time.Format("2006-01-02"),
|
|
TotalSeconds: row.TotalSeconds,
|
|
TaskCount: row.TaskCount,
|
|
FailedCount: row.FailedCount,
|
|
}
|
|
}
|
|
writeJSON(w, http.StatusOK, resp)
|
|
}
|