From ffa8e16369e9007d70227ee3497d7d64ef4beabd Mon Sep 17 00:00:00 2001 From: Bohan Jiang <52446949+Bohan-J@users.noreply.github.com> Date: Fri, 24 Jul 2026 01:42:08 +0800 Subject: [PATCH] MUL-5228 fix(usage): bill Grok at xAI's reported cost, fix $0 resumed sessions (#5841) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(agent): attribute Grok usage from the turn's own model id A resumed Grok session with no configured model recorded its entire spend under the model id "unknown", which matches no pricing row — so the task reported $0 cost instead of its real spend. grok.go only learned the model from the session handshake, and ACP's `session/load` carries no model id (only `session/new` does). When neither the agent nor MULTICA_GROK_MODEL pins a model, `daemon.go` legitimately passes an empty model, leaving nothing to attribute the usage to. Every Grok turn stamps `result._meta.modelId` with what it actually billed against. Parse it in the shared ACP result parser and use it as the fallback in grok.go. Other ACP backends are untouched — they keep whatever the handshake gave them. Co-authored-by: J Co-authored-by: multica-agent * fix(metrics): price the Grok catalog in server-side cost metrics server/internal/metrics/pricing.go carried no Grok rows at all, so RecordLLMUsage took the unpriced branch for every Grok turn: llm_cost_usd reported zero Grok spend while the tokens accumulated in llm_unpriced_tokens. Internal cost monitoring simply could not see Grok. Add the six SKUs xAI publishes rates for, mirroring the frontend table in packages/views/runtimes/utils.ts. Aliases are anchored exact matches like the gpt-5.6 rows, so `grok-composer-*` (in the catalog, absent from the price sheet) stays unmapped instead of inheriting a guessed rate. Short-context tier on purpose: xAI bills a request at 2x once its prompt reaches 200K tokens, but a usage record aggregates every model call in a turn and cannot say which tier an individual request hit. A regression test re-derives the cost of a real grok 0.2.106 turn from the table and checks it against the costUsdTicks xAI returned for that turn. Co-authored-by: J Co-authored-by: multica-agent * docs(changelog): scope the Grok cost claim to what was actually fixed The v0.4.9 entry promised "accurate cost" in all four languages, but the fix corrected catalog pricing and cached-input double-counting — it did not implement xAI's 2x long-context tier, so a turn whose requests reach 200K prompt tokens still under-reports by up to 50%. Say what was fixed instead. Also correct two stale claims in the pricing comment: the daemon tags usage rows with the runtime provider `grok`, not `xai` (the bare `grok-*` keys are what make them resolve), and record why thresholding the long-context tier on an aggregated row would be worse than not pricing it at all. Co-authored-by: J Co-authored-by: multica-agent * feat(usage): carry the provider's own cost through to the usage record Cost has always been derived client-side as tokens x a static rate, which cannot express request-level pricing rules. xAI bills a Grok request at 2x once its prompt reaches 200K tokens, and a task_usage row aggregates every model call in a turn — so the stored token counts genuinely cannot say which tier any individual request hit. Thresholding on the aggregate would be worse than the status quo: it turns a bounded 50% under-estimate into an unbounded over-estimate for turns made of many short requests. Grok already reports what it charged, per turn, in `_meta.usage.costUsdTicks`. Parse it, carry it through agent -> daemon -> API, and store it on task_usage as a nullable BIGINT of 1e-10 USD ticks (integer, so sub-cent turns stay exact end to end). NULL means the provider reported no cost — every pre-existing row and every provider that doesn't return one. No backfill: there is no authoritative figure to recover for those, and inventing one is the guess this removes. A single hourly bucket can mix rows that carry a cost with rows that don't, so task_usage_hourly gains both halves: `cost_usd_ticks` sums the authoritative side, and `uncosted_*_tokens` carry exactly the tokens that still need a rate-table estimate. Consumers report authoritative + estimate(uncosted), which degrades to today's behaviour when nothing in the bucket is authoritative. The existing token columns keep covering every row, so token displays are untouched. The new columns are additive with defaults, so the unique key, the dirty-queue shape, and migration 102's triggers are unaffected. Co-authored-by: J Co-authored-by: multica-agent * feat(usage): prefer the provider's own cost over the rate table With the authoritative figure now stored, both cost consumers use it: the usage dashboard (estimateCost / estimateCostBreakdown) and the server-side llm_cost_usd metric. Each reports `authoritative + estimate(uncosted tokens)`, so a row or bucket that mixes priced and unpriced sources stays whole. The static rate tables remain, but for Grok they are now a fallback — they still price usage recorded by a daemon too old to report cost, and every provider that reports none. Custom pricing overrides likewise apply only to the estimated half: they are a user's guess at a rate, and the authoritative half is not a guess. A model with no rate-table row but a provider-reported cost now also drops out of the "unmapped models" banner, since asking the user to supply a rate for it would invite overriding a real bill. llm_cost_usd is labelled by token_type and the provider reports one number per turn, so the charge is distributed across the buckets in the rate table's own proportions. Only the total is authoritative; the split stays an estimate, which is why this scales the existing buckets rather than inventing a label. estimateCostBreakdown does the same, keeping the stacked chart summing to the headline figure instead of silently under-drawing every Grok row. Co-authored-by: J Co-authored-by: multica-agent * docs(changelog): say Grok cost now follows xAI's actual charge The earlier wording scoped the claim down to catalog pricing and cached input because the long-context tier was still unhandled. It is handled now — the cost comes from what xAI charged for the turn — so the entry can say so. Co-authored-by: J Co-authored-by: multica-agent * fix(usage): keep the provider's cost when the model has no rate row Both cost consumers bailed out before reading the authoritative figure when the rate table had no row for the model. A `grok-composer-*` turn — in the Grok Build catalog, absent from xAI's price sheet — was therefore reported as $0 spend even though xAI told us exactly what it charged. Worse on the client: estimateCost returned the real cost while estimateCostBreakdown returned zeros, so the headline and the stacked chart disagreed on precisely the rows whose cost is exact — and the unmapped-models banner was (correctly) hidden, so nothing explained the discrepancy. Handle the charge before the rate lookup in both places. Without rates there is nothing to split a total by, so it lands whole in the `input` bucket, the same fallback distributeAuthoritativeCost already uses when it has no shape to scale. Tokens with no rate keep going to llm_unpriced_tokens: "unpriced" describes the rate table, not the money. Co-authored-by: J Co-authored-by: multica-agent * perf(usage): drop the historical rewrite from the cost-split migration Migration 213 rewrote every existing task_usage_hourly row to seed the uncosted counters. That is a full-table UPDATE inside a schema migration — lock time, WAL and bloat all scaling with table size — for rows this issue explicitly does not care about. Deleting the UPDATE alone would have zeroed historical cost: with `NOT NULL DEFAULT 0`, an untouched row asserts "nothing here needs estimating", so every pre-split bucket would report $0 until the rollup happened to touch it. Make the uncosted columns nullable with no default instead. NULL means "never recomputed since the split existed", readers COALESCE it to the row's own token total ("estimate all of it"), and the pre-split behaviour is preserved exactly — with nothing to seed, so no rewrite. A bare ADD COLUMN is metadata-only, so this is now fast DDL. Rows heal into the split naturally as the rollup recomputes their buckets. Verified on a fresh database: a legacy-shaped row reads back as its full tokens to estimate, and a group mixing legacy and post-split buckets sums to the authoritative cost plus both rows' estimable tokens. Co-authored-by: J Co-authored-by: multica-agent --------- Co-authored-by: Bohan-J Co-authored-by: J Co-authored-by: multica-agent --- apps/web/features/landing/i18n/en.ts | 2 +- apps/web/features/landing/i18n/ja.ts | 2 +- apps/web/features/landing/i18n/ko.ts | 2 +- apps/web/features/landing/i18n/zh.ts | 2 +- packages/core/api/schemas.ts | 25 ++ packages/core/types/agent.ts | 43 ++++ packages/views/runtimes/utils.test.ts | 214 +++++++++++++++++ packages/views/runtimes/utils.ts | 169 +++++++++++-- server/internal/daemon/daemon.go | 2 + server/internal/daemon/types.go | 5 + server/internal/handler/daemon.go | 27 ++- server/internal/handler/dashboard.go | 66 ++++-- server/internal/handler/runtime.go | 95 ++++++-- server/internal/metrics/business.go | 59 ++++- server/internal/metrics/business_test.go | 142 ++++++++++- server/internal/metrics/pricing.go | 32 +++ server/internal/metrics/pricing_test.go | 102 ++++++++ server/internal/service/task.go | 6 +- ...213_task_usage_authoritative_cost.down.sql | 141 +++++++++++ .../213_task_usage_authoritative_cost.up.sql | 223 ++++++++++++++++++ server/pkg/agent/agent.go | 15 ++ server/pkg/agent/grok.go | 13 + server/pkg/agent/grok_test.go | 56 ++++- server/pkg/agent/hermes.go | 35 +++ server/pkg/agent/hermes_test.go | 72 ++++++ server/pkg/db/generated/models.go | 9 + server/pkg/db/generated/runtime_usage.sql.go | 91 +++++-- server/pkg/db/generated/task_usage.sql.go | 100 ++++++-- server/pkg/db/queries/runtime_usage.sql | 17 +- server/pkg/db/queries/task_usage.sql | 23 +- 30 files changed, 1657 insertions(+), 133 deletions(-) create mode 100644 server/migrations/213_task_usage_authoritative_cost.down.sql create mode 100644 server/migrations/213_task_usage_authoritative_cost.up.sql diff --git a/apps/web/features/landing/i18n/en.ts b/apps/web/features/landing/i18n/en.ts index 65ab02fe4..e5423a9e2 100644 --- a/apps/web/features/landing/i18n/en.ts +++ b/apps/web/features/landing/i18n/en.ts @@ -308,7 +308,7 @@ export function createEnDict(allowSignup: boolean): LandingDict { "Grouping and filters now behave the same across List, Board, and Swimlane views.", "The chat box now grows with the window so you can see more of a long draft.", "Getting started now takes fewer steps.", - "Grok usage now shows accurate cost, and your saved custom prices stay editable.", + "Grok cost now comes from what xAI actually charged for each turn, including long-context requests, and your saved custom prices stay editable.", "Animations across the app are now smoother.", ], fixes: [ diff --git a/apps/web/features/landing/i18n/ja.ts b/apps/web/features/landing/i18n/ja.ts index 443020024..cbd033fb2 100644 --- a/apps/web/features/landing/i18n/ja.ts +++ b/apps/web/features/landing/i18n/ja.ts @@ -284,7 +284,7 @@ export function createJaDict(allowSignup: boolean): LandingDict { "リスト・ボード・スイムレーンの各ビューでグループ化と絞り込みの挙動がそろいました。", "チャット入力欄がウィンドウに合わせて広がり、長い下書きも見やすくなりました。", "初回セットアップの手順が少なくなりました。", - "Grok の使用量が正確な費用で表示され、保存したカスタム価格も編集し続けられます。", + "Grok の費用が xAI の実際の課金額に基づくようになり、長いコンテキストのリクエストも正確になりました。保存したカスタム価格も編集し続けられます。", "アプリ全体のアニメーションがなめらかになりました。", ], fixes: [ diff --git a/apps/web/features/landing/i18n/ko.ts b/apps/web/features/landing/i18n/ko.ts index c9ae61d1f..dc768edb7 100644 --- a/apps/web/features/landing/i18n/ko.ts +++ b/apps/web/features/landing/i18n/ko.ts @@ -283,7 +283,7 @@ export function createKoDict(allowSignup: boolean): LandingDict { "리스트, 보드, 스윔레인 뷰에서 그룹화와 필터가 동일하게 동작합니다.", "채팅 입력창이 창 크기에 맞춰 커져 긴 초안도 더 잘 보입니다.", "시작하기 설정 단계가 더 줄었습니다.", - "Grok 사용량이 정확한 비용으로 표시되고, 저장한 맞춤 가격도 계속 편집할 수 있습니다.", + "Grok 비용이 xAI가 실제로 청구한 금액을 그대로 반영하며, 긴 컨텍스트 요청도 정확합니다. 저장한 맞춤 가격도 계속 편집할 수 있습니다.", "앱 전반의 애니메이션이 더 부드러워졌습니다.", ], fixes: [ diff --git a/apps/web/features/landing/i18n/zh.ts b/apps/web/features/landing/i18n/zh.ts index 9dc709d3b..c972fa4df 100644 --- a/apps/web/features/landing/i18n/zh.ts +++ b/apps/web/features/landing/i18n/zh.ts @@ -308,7 +308,7 @@ export function createZhDict(allowSignup: boolean): LandingDict { "列表、看板和泳道视图的分组与筛选现在表现一致了。", "聊天输入框现在会随窗口变大，长草稿一眼看得更全。", "初次上手现在步骤更少了。", - "Grok 用量现在会显示准确费用，保存后的自定义价格也能再改。", + "Grok 费用现在直接采用 xAI 每轮实际收取的金额，长上下文请求也算得准，保存后的自定义价格也能再改。", "全站的界面动画更顺滑了。", ], fixes: [ diff --git a/packages/core/api/schemas.ts b/packages/core/api/schemas.ts index 9953a52cf..f1a84b8b6 100644 --- a/packages/core/api/schemas.ts +++ b/packages/core/api/schemas.ts @@ -742,6 +742,26 @@ export const EMPTY_CLOUD_RUNTIME_NODE: CloudRuntimeNode = { // only that row instead of dropping the whole array to the `[]` fallback. // --------------------------------------------------------------------------- +// Cost split carried by every usage row. `cost_usd_ticks` is what the provider +// itself charged for the rows behind this aggregate (1e-10 USD); the +// `uncosted_*` counts are the tokens from rows the provider did NOT price, and +// so are the only ones the client should run through its rate table. +// +// The `uncosted_*` fields are deliberately `.optional()` rather than +// `.default(0)`: a backend that predates them sends nothing, and defaulting +// those rows to "0 tokens left to estimate" would silently zero their cost. +// `undefined` means "this backend doesn't split", and the consumer falls back +// to the full token counts — i.e. exactly the old behaviour. A real 0 from a +// current backend means "everything here is already priced", which is a +// different thing and must stay distinguishable. +const CostSplitShape = { + cost_usd_ticks: z.number().optional(), + uncosted_input_tokens: z.number().optional(), + uncosted_output_tokens: z.number().optional(), + uncosted_cache_read_tokens: z.number().optional(), + uncosted_cache_write_tokens: z.number().optional(), +}; + const DashboardUsageDailySchema = z.object({ date: z.string().default(""), provider: z.string().default(""), @@ -750,6 +770,7 @@ const DashboardUsageDailySchema = z.object({ output_tokens: z.number().default(0), cache_read_tokens: z.number().default(0), cache_write_tokens: z.number().default(0), + ...CostSplitShape, task_count: z.number().default(0), }).loose(); @@ -763,6 +784,7 @@ const DashboardUsageByAgentSchema = z.object({ output_tokens: z.number().default(0), cache_read_tokens: z.number().default(0), cache_write_tokens: z.number().default(0), + ...CostSplitShape, task_count: z.number().default(0), }).loose(); @@ -802,6 +824,7 @@ const RuntimeUsageSchema = z.object({ output_tokens: z.number().default(0), cache_read_tokens: z.number().default(0), cache_write_tokens: z.number().default(0), + ...CostSplitShape, }).loose(); export const RuntimeUsageListSchema = z.array(RuntimeUsageSchema); @@ -821,6 +844,7 @@ const RuntimeUsageByAgentSchema = z.object({ output_tokens: z.number().default(0), cache_read_tokens: z.number().default(0), cache_write_tokens: z.number().default(0), + ...CostSplitShape, task_count: z.number().default(0), }).loose(); @@ -833,6 +857,7 @@ const RuntimeUsageByHourSchema = z.object({ output_tokens: z.number().default(0), cache_read_tokens: z.number().default(0), cache_write_tokens: z.number().default(0), + ...CostSplitShape, task_count: z.number().default(0), }).loose(); diff --git a/packages/core/types/agent.ts b/packages/core/types/agent.ts index f201dcf46..41200b7a1 100644 --- a/packages/core/types/agent.ts +++ b/packages/core/types/agent.ts @@ -791,9 +791,27 @@ export interface IssueUsageSummary { total_output_tokens: number; total_cache_read_tokens: number; total_cache_write_tokens: number; + // Optional unlike the usage-row types: `getIssueUsage` returns this shape + // unvalidated (no zod schema), so nothing guarantees the field is present + // when the backend is older than the cost split. + cost_usd_ticks?: number; + uncosted_input_tokens?: number; + uncosted_output_tokens?: number; + uncosted_cache_read_tokens?: number; + uncosted_cache_write_tokens?: number; task_count: number; } +// `cost_usd_ticks` + `uncosted_*`: the cost split every usage row carries. +// All five are optional: a backend older than the split sends none of them, +// and `undefined` has to stay distinguishable from a real 0 (see below). +// The provider priced the rows behind `cost_usd_ticks` itself (1e-10 USD); +// `uncosted_*` are the tokens it did not price, and are the only ones that +// should go through the client's rate table. The `uncosted_*` fields are +// optional because a backend older than the split omits them — `undefined` +// there means "estimate from the full token counts", which is not the same as +// a real 0 ("nothing left to estimate"). See estimateCost in +// packages/views/runtimes/utils.ts. export interface RuntimeUsage { runtime_id: string; date: string; @@ -803,6 +821,11 @@ export interface RuntimeUsage { output_tokens: number; cache_read_tokens: number; cache_write_tokens: number; + cost_usd_ticks?: number; + uncosted_input_tokens?: number; + uncosted_output_tokens?: number; + uncosted_cache_read_tokens?: number; + uncosted_cache_write_tokens?: number; } export interface RuntimeHourlyActivity { @@ -823,6 +846,11 @@ export interface RuntimeUsageByAgent { output_tokens: number; cache_read_tokens: number; cache_write_tokens: number; + cost_usd_ticks?: number; + uncosted_input_tokens?: number; + uncosted_output_tokens?: number; + uncosted_cache_read_tokens?: number; + uncosted_cache_write_tokens?: number; task_count: number; } @@ -836,6 +864,11 @@ export interface RuntimeUsageByHour { output_tokens: number; cache_read_tokens: number; cache_write_tokens: number; + cost_usd_ticks?: number; + uncosted_input_tokens?: number; + uncosted_output_tokens?: number; + uncosted_cache_read_tokens?: number; + uncosted_cache_write_tokens?: number; task_count: number; } @@ -853,6 +886,11 @@ export interface DashboardUsageDaily { output_tokens: number; cache_read_tokens: number; cache_write_tokens: number; + cost_usd_ticks?: number; + uncosted_input_tokens?: number; + uncosted_output_tokens?: number; + uncosted_cache_read_tokens?: number; + uncosted_cache_write_tokens?: number; task_count: number; } @@ -867,6 +905,11 @@ export interface DashboardUsageByAgent { output_tokens: number; cache_read_tokens: number; cache_write_tokens: number; + cost_usd_ticks?: number; + uncosted_input_tokens?: number; + uncosted_output_tokens?: number; + uncosted_cache_read_tokens?: number; + uncosted_cache_write_tokens?: number; task_count: number; } diff --git a/packages/views/runtimes/utils.test.ts b/packages/views/runtimes/utils.test.ts index 1bf16d1d5..f1305dddf 100644 --- a/packages/views/runtimes/utils.test.ts +++ b/packages/views/runtimes/utils.test.ts @@ -507,6 +507,220 @@ describe("estimateCost", () => { ).toBeCloseTo(3, 5); }); + // ------------------------------------------------------------------------- + // Provider-reported cost. `cost_usd_ticks` is what the provider actually + // charged (1e-10 USD) for the rows behind an aggregate; `uncosted_*` are the + // tokens it did not price and which therefore still need the rate table. + // ------------------------------------------------------------------------- + + it("uses the provider's own cost instead of the rate table when it reports one", () => { + // Real grok 0.2.106 turn: 2049 uncached input + 10880 cache read + 29 + // output, costUsdTicks 75360000 = $0.007536. Here the rate table would + // agree, which is what makes it a clean check that the authoritative + // number is the one being used rather than added to an estimate. + expect( + estimateCost({ + ...zeroUsage, + provider: "grok", + model: "grok-4.5", + input_tokens: 2049, + cache_read_tokens: 10880, + output_tokens: 29, + cost_usd_ticks: 75_360_000, + uncosted_input_tokens: 0, + uncosted_output_tokens: 0, + uncosted_cache_read_tokens: 0, + uncosted_cache_write_tokens: 0, + }), + ).toBeCloseTo(0.007536, 10); + }); + + it("keeps the long-context surcharge the rate table cannot express", () => { + // xAI bills a request at 2x once its prompt reaches 200K tokens. The same + // tokens priced from the table give the short-context figure; the + // provider's own number carries the surcharge, and must not be quietly + // replaced by the cheaper local estimate. + const tokens = { + ...zeroUsage, + provider: "grok", + model: "grok-4.5", + input_tokens: 1_000_000, + output_tokens: 1_000_000, + }; + const shortContext = estimateCost(tokens); + expect(shortContext).toBeCloseTo(8, 5); + + const longContext = estimateCost({ + ...tokens, + cost_usd_ticks: 16 * 10_000_000_000, // $16 — the 2x tier + uncosted_input_tokens: 0, + uncosted_output_tokens: 0, + uncosted_cache_read_tokens: 0, + uncosted_cache_write_tokens: 0, + }); + expect(longContext).toBeCloseTo(16, 5); + }); + + it("adds an estimate for the tokens the provider did not price", () => { + // A bucket can mix rows that carry a provider cost with rows that don't — + // two providers in one aggregate, or Grok either side of a CLI upgrade. + // Reporting only the authoritative half would under-report the bucket. + expect( + estimateCost({ + ...zeroUsage, + provider: "grok", + model: "grok-4.5", + input_tokens: 1_000_000, + output_tokens: 1_000_000, + cost_usd_ticks: 4 * 10_000_000_000, // $4 for the priced half + uncosted_input_tokens: 1_000_000, // $2 at the table rate + uncosted_output_tokens: 1_000_000, // $6 at the table rate + uncosted_cache_read_tokens: 0, + uncosted_cache_write_tokens: 0, + }), + ).toBeCloseTo(12, 5); + }); + + it("falls back to estimating the full row when the backend omits the split", () => { + // A backend older than the cost split sends no `uncosted_*` fields. + // Treating that as "nothing left to estimate" would report $0 for every + // row, so an absent split must estimate the full token counts. + expect( + estimateCost({ + ...zeroUsage, + provider: "grok", + model: "grok-4.5", + input_tokens: 1_000_000, + output_tokens: 1_000_000, + }), + ).toBeCloseTo(8, 5); + }); + + it("does not double-charge a cost that arrives without its token split", () => { + // Defensive: an authoritative cost with no `uncosted_*` must not also get + // a full-row estimate stacked on top. + expect( + estimateCost({ + ...zeroUsage, + provider: "grok", + model: "grok-4.5", + input_tokens: 1_000_000, + output_tokens: 1_000_000, + cost_usd_ticks: 16 * 10_000_000_000, + }), + ).toBeCloseTo(16, 5); + }); + + it("reports provider cost even for a model with no rate-table row", () => { + // `grok-composer-*` has no published rate, but a turn the provider priced + // itself needs no rate — the money is known exactly. + expect( + estimateCost({ + ...zeroUsage, + provider: "grok", + model: "grok-composer-2.5-fast", + input_tokens: 500, + output_tokens: 100, + cost_usd_ticks: 12_345_678_900, + uncosted_input_tokens: 0, + uncosted_output_tokens: 0, + uncosted_cache_read_tokens: 0, + uncosted_cache_write_tokens: 0, + }), + ).toBeCloseTo(1.23456789, 8); + }); + + it("keeps the breakdown and the headline agreeing on an unpriced model", () => { + // `grok-composer-*` has no rate row, so there is nothing to split by — but + // the provider priced the turn. If the breakdown returned zeros here the + // stacked chart would read $0 while the headline read the real cost, and + // the unmapped banner (correctly) would not be shown to explain it. + const usage = { + ...zeroUsage, + provider: "grok", + model: "grok-composer-2.5-fast", + input_tokens: 500, + output_tokens: 100, + cost_usd_ticks: 12_345_678_900, + uncosted_input_tokens: 0, + uncosted_output_tokens: 0, + uncosted_cache_read_tokens: 0, + uncosted_cache_write_tokens: 0, + }; + const b = estimateCostBreakdown(usage); + expect(b.input + b.output + b.cacheRead + b.cacheWrite).toBeCloseTo( + estimateCost(usage), + 8, + ); + expect(b.input).toBeCloseTo(1.23456789, 8); + }); + + it("reports no cost for an unpriced model the provider did not price either", () => { + // The control for the case above: no rates and no provider cost must stay + // at zero rather than inventing a figure. + const usage = { + ...zeroUsage, + provider: "grok", + model: "grok-composer-2.5-fast", + input_tokens: 500, + output_tokens: 100, + }; + expect(estimateCost(usage)).toBe(0); + const b = estimateCostBreakdown(usage); + expect(b.input + b.output + b.cacheRead + b.cacheWrite).toBe(0); + // ...and it still asks the user for a rate, because one would help here. + expect(collectUnmappedModels([usage])).toEqual(["grok/grok-composer-2.5-fast"]); + }); + + it("keeps the cost breakdown summing to the total on provider-priced rows", () => { + // The stacked chart is drawn from the breakdown while the headline uses + // estimateCost; if the authoritative charge were dropped from the split + // the two would silently disagree on every Grok row. + const usage = { + ...zeroUsage, + provider: "grok", + model: "grok-4.5", + input_tokens: 1_000_000, + output_tokens: 1_000_000, + cost_usd_ticks: 16 * 10_000_000_000, + uncosted_input_tokens: 0, + uncosted_output_tokens: 0, + uncosted_cache_read_tokens: 0, + uncosted_cache_write_tokens: 0, + }; + const b = estimateCostBreakdown(usage); + expect(b.input + b.output + b.cacheRead + b.cacheWrite).toBeCloseTo( + estimateCost(usage), + 5, + ); + // Split follows the rate table's own proportions ($2 input : $6 output). + expect(b.input).toBeCloseTo(4, 5); + expect(b.output).toBeCloseTo(12, 5); + }); + + it("drops a fully provider-priced model from the unmapped diagnostic", () => { + // The banner asks the user to supply a missing rate. A row the provider + // priced in full needs no rate, so prompting for one would invite + // overriding a real bill with a guess. + const row = { + ...zeroUsage, + provider: "grok", + model: "grok-composer-2.5-fast", + input_tokens: 500, + cost_usd_ticks: 12_345_678_900, + uncosted_input_tokens: 0, + uncosted_output_tokens: 0, + uncosted_cache_read_tokens: 0, + uncosted_cache_write_tokens: 0, + }; + expect(collectUnmappedModels([row])).toEqual([]); + // ...but the same model still surfaces while any of its tokens are + // unpriced, because those genuinely need a rate. + expect( + collectUnmappedModels([{ ...row, uncosted_input_tokens: 500 }]), + ).toEqual(["grok/grok-composer-2.5-fast"]); + }); + it("leaves Grok SKUs that xAI does not publish a price for unmapped", () => { // No startsWith fallback: `grok-composer-2.5-fast` is in the Grok Build // catalog but absent from docs.x.ai/developers/pricing, so it must NOT diff --git a/packages/views/runtimes/utils.ts b/packages/views/runtimes/utils.ts index 954fe0319..50b614473 100644 --- a/packages/views/runtimes/utils.ts +++ b/packages/views/runtimes/utils.ts @@ -256,15 +256,20 @@ const MODEL_PRICING: Record< "glm-4.5-flash": { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }, // -- xAI Grok (docs.x.ai/developers/pricing). Rates below are the - // short-context tier. xAI bills a request at 2x ("long context") once - // its prompt reaches 200K tokens, but aggregated usage rows carry no - // per-request prompt sizes, so we price at the standard tier — the same - // trade-off the Anthropic `[1m]` context tag takes (see `resolvePricing`). + // short-context tier, and are now only a FALLBACK for Grok: xAI reports + // its own price per turn and `estimateCost` prefers it. That matters + // because xAI bills a request at 2x once its prompt reaches 200K tokens, + // and a usage row aggregates every model call in a turn — so these rates + // cannot tell which tier a request hit, while xAI's own figure already + // has it priced in. These rows still apply to Grok usage recorded by a + // daemon too old to report cost (the same trade-off the Anthropic `[1m]` + // context tag takes, see `resolvePricing`). // `cacheRead` is xAI's published "Cached" input rate; there is no // separate cache-write rate on the page (writes bill as normal input), // so cacheWrite mirrors input per the header note. Grok ids are - // vendor-prefixed, so these keys stay unqualified even though the - // daemon tags the rows with provider `xai`. + // vendor-prefixed, so these keys stay unqualified — which is what makes + // them resolve at all, since the daemon tags the rows with the runtime + // provider `grok`, not `xai`. // `grok-composer-*` ships in the Grok Build catalog // (server/pkg/agent/models.go) but is absent from the price sheet; it // deliberately stays unmapped rather than inheriting a guessed rate. -- @@ -468,12 +473,22 @@ export function isModelPriced(model: string, provider?: string): boolean { // the row carries a provider, so the same bare model id reported by two // providers surfaces as two distinct entries the user can price separately. // Empty when everything's priced or there are no rows. +// A row the provider priced in full needs no rate-table entry, so it must not +// raise the "we can't price this model" warning — its cost is already exact, +// and asking the user to supply a rate for it would be asking them to override +// a real bill with a guess. export function collectUnmappedModels(rows: readonly Priceable[]): string[] { const set = new Set(); for (const r of rows) { - if (r.model && !isModelPriced(r.model, r.provider)) { - set.add(pricingKey(r.model, r.provider)); - } + if (!r.model || isModelPriced(r.model, r.provider)) continue; + const uncosted = uncostedTokens(r); + const needsEstimate = + uncosted.input > 0 || + uncosted.output > 0 || + uncosted.cacheRead > 0 || + uncosted.cacheWrite > 0; + if (!needsEstimate && (r.cost_usd_ticks ?? 0) > 0) continue; + set.add(pricingKey(r.model, r.provider)); } return Array.from(set).toSorted(); } @@ -488,18 +503,85 @@ export function collectUnmappedModels(rows: readonly Priceable[]): string[] { // DashboardUsageDaily / DashboardUsageByAgent all carry it on the wire. type Priceable = Pick< RuntimeUsage, - "model" | "input_tokens" | "output_tokens" | "cache_read_tokens" | "cache_write_tokens" -> & { provider?: string }; + | "model" + | "input_tokens" + | "output_tokens" + | "cache_read_tokens" + | "cache_write_tokens" +> & { + provider?: string; + cost_usd_ticks?: number; + uncosted_input_tokens?: number; + uncosted_output_tokens?: number; + uncosted_cache_read_tokens?: number; + uncosted_cache_write_tokens?: number; +}; +// Providers report cost in ticks of 1e-10 USD (xAI's unit), which keeps +// sub-cent turn costs exact as integers all the way from the agent to here. +const COST_USD_TICKS_PER_USD = 10_000_000_000; + +// The tokens in a row that still need pricing from the table above, i.e. the +// ones no provider priced for us. +// +// A backend older than the cost split sends no `uncosted_*` at all. Treating +// that as "0 tokens left to estimate" would report $0 for every row, so +// `undefined` falls back to the full token counts — exactly the pre-split +// behaviour. The one exception is a row that DOES carry an authoritative cost +// without the split: adding a full-token estimate on top would double-charge +// it, so the authoritative figure stands alone. A current backend always sends +// both, so this only guards against version drift. +function uncostedTokens(usage: Priceable): { + input: number; + output: number; + cacheRead: number; + cacheWrite: number; +} { + if (usage.uncosted_input_tokens === undefined) { + if ((usage.cost_usd_ticks ?? 0) > 0) { + return { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }; + } + return { + input: usage.input_tokens, + output: usage.output_tokens, + cacheRead: usage.cache_read_tokens, + cacheWrite: usage.cache_write_tokens, + }; + } + return { + input: usage.uncosted_input_tokens, + output: usage.uncosted_output_tokens ?? 0, + cacheRead: usage.uncosted_cache_read_tokens ?? 0, + cacheWrite: usage.uncosted_cache_write_tokens ?? 0, + }; +} + +// Cost of a usage row: what the provider actually charged, plus a rate-table +// estimate for whatever it didn't charge for. +// +// The rate table cannot express request-level pricing rules — xAI bills a Grok +// request at 2x once its prompt reaches 200K tokens, and these rows aggregate +// every model call in a turn, so the token counts genuinely cannot say which +// tier a given request hit. Where the provider tells us its own price, that +// number is the bill and no estimate can improve on it. +// +// Both halves are summed rather than one winning outright because a single row +// can aggregate both kinds of source row (two providers in a bucket, or Grok +// either side of a CLI upgrade). Custom pricing overrides still apply — but +// only to the estimated half, since they are a user's guess at a rate and the +// authoritative half is not a guess. export function estimateCost(usage: Priceable): number { + const authoritative = (usage.cost_usd_ticks ?? 0) / COST_USD_TICKS_PER_USD; const pricing = resolvePricing(usage.model, usage.provider); - if (!pricing) return 0; + if (!pricing) return authoritative; + const uncosted = uncostedTokens(usage); return ( - (usage.input_tokens * pricing.input + - usage.output_tokens * pricing.output + - usage.cache_read_tokens * pricing.cacheRead + - usage.cache_write_tokens * pricing.cacheWrite) / - 1_000_000 + authoritative + + (uncosted.input * pricing.input + + uncosted.output * pricing.output + + uncosted.cacheRead * pricing.cacheRead + + uncosted.cacheWrite * pricing.cacheWrite) / + 1_000_000 ); } @@ -510,16 +592,59 @@ export interface CostBreakdown { cacheWrite: number; } +// Per-token-type split of `estimateCost`. The estimated half splits naturally; +// the authoritative half arrives as one number per row, so it is distributed +// across the buckets in the same proportions the rate table would have charged. +// Only the total is authoritative — the split is presentation, and doing it +// this way keeps the stacked chart summing to the headline figure instead of +// silently under-drawing every Grok row. export function estimateCostBreakdown(usage: Priceable): CostBreakdown { const pricing = resolvePricing(usage.model, usage.provider); if (!pricing) { - return { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }; + // No rates to split by, but the provider may still have priced the turn + // itself. Returning zeros here would make the stacked chart disagree with + // the headline `estimateCost` on exactly the rows whose cost is EXACT, so + // the charge lands whole in one bucket instead. + return { + input: (usage.cost_usd_ticks ?? 0) / COST_USD_TICKS_PER_USD, + output: 0, + cacheRead: 0, + cacheWrite: 0, + }; } + const uncosted = uncostedTokens(usage); + const breakdown: CostBreakdown = { + input: (uncosted.input * pricing.input) / 1_000_000, + output: (uncosted.output * pricing.output) / 1_000_000, + cacheRead: (uncosted.cacheRead * pricing.cacheRead) / 1_000_000, + cacheWrite: (uncosted.cacheWrite * pricing.cacheWrite) / 1_000_000, + }; + + const authoritative = (usage.cost_usd_ticks ?? 0) / COST_USD_TICKS_PER_USD; + if (authoritative <= 0) return breakdown; + + // Shape the authoritative charge like the rate table would have priced the + // tokens it covers — the row's full tokens minus the estimated ones. + const shape = { + input: ((usage.input_tokens - uncosted.input) * pricing.input) / 1_000_000, + output: ((usage.output_tokens - uncosted.output) * pricing.output) / 1_000_000, + cacheRead: + ((usage.cache_read_tokens - uncosted.cacheRead) * pricing.cacheRead) / 1_000_000, + cacheWrite: + ((usage.cache_write_tokens - uncosted.cacheWrite) * pricing.cacheWrite) / 1_000_000, + }; + const shapeTotal = shape.input + shape.output + shape.cacheRead + shape.cacheWrite; + if (shapeTotal <= 0) { + // Nothing to shape it with (unpriced tokens, or a row carrying cost but no + // tokens). Keep the money in the total rather than dropping it. + return { ...breakdown, input: breakdown.input + authoritative }; + } + const scale = authoritative / shapeTotal; return { - input: (usage.input_tokens * pricing.input) / 1_000_000, - output: (usage.output_tokens * pricing.output) / 1_000_000, - cacheRead: (usage.cache_read_tokens * pricing.cacheRead) / 1_000_000, - cacheWrite: (usage.cache_write_tokens * pricing.cacheWrite) / 1_000_000, + input: breakdown.input + shape.input * scale, + output: breakdown.output + shape.output * scale, + cacheRead: breakdown.cacheRead + shape.cacheRead * scale, + cacheWrite: breakdown.cacheWrite + shape.cacheWrite * scale, }; } diff --git a/server/internal/daemon/daemon.go b/server/internal/daemon/daemon.go index beea6baa3..4eaa8fc33 100644 --- a/server/internal/daemon/daemon.go +++ b/server/internal/daemon/daemon.go @@ -4779,6 +4779,7 @@ func (d *Daemon) runTask(ctx context.Context, task Task, provider string, slot i OutputTokens: u.OutputTokens, CacheReadTokens: u.CacheReadTokens, CacheWriteTokens: u.CacheWriteTokens, + CostUSDTicks: u.CostUSDTicks, }) } @@ -5454,6 +5455,7 @@ func mergeUsage(a, b map[string]agent.TokenUsage) map[string]agent.TokenUsage { existing.OutputTokens += u.OutputTokens existing.CacheReadTokens += u.CacheReadTokens existing.CacheWriteTokens += u.CacheWriteTokens + existing.CostUSDTicks += u.CostUSDTicks merged[model] = existing } return merged diff --git a/server/internal/daemon/types.go b/server/internal/daemon/types.go index a4611e547..0b4da92ad 100644 --- a/server/internal/daemon/types.go +++ b/server/internal/daemon/types.go @@ -241,6 +241,11 @@ type TaskUsageEntry struct { OutputTokens int64 `json:"output_tokens"` CacheReadTokens int64 `json:"cache_read_tokens"` CacheWriteTokens int64 `json:"cache_write_tokens"` + // CostUSDTicks is the provider's own price for this usage, in 1e-10 USD. + // Omitted when the agent reports no cost, which is the common case — the + // server then leaves the column NULL and the client estimates from the + // pricing table instead. See agent.TokenUsage.CostUSDTicks. + CostUSDTicks int64 `json:"cost_usd_ticks,omitempty"` } // TaskResult is the outcome of executing a task. diff --git a/server/internal/handler/daemon.go b/server/internal/handler/daemon.go index 93a28cf9c..da83f2c43 100644 --- a/server/internal/handler/daemon.go +++ b/server/internal/handler/daemon.go @@ -3344,6 +3344,22 @@ type TaskUsagePayload struct { OutputTokens int64 `json:"output_tokens"` CacheReadTokens int64 `json:"cache_read_tokens"` CacheWriteTokens int64 `json:"cache_write_tokens"` + // CostUSDTicks is the provider's own price for this usage in 1e-10 USD. + // Absent or 0 from every daemon that doesn't have one (older builds, and + // every provider except Grok today) — stored as NULL so the reader knows + // to fall back to rate-table estimation rather than reading a real $0. + CostUSDTicks int64 `json:"cost_usd_ticks"` +} + +// authoritativeCostTicks converts a reported cost into the nullable column. +// Only a positive figure is authoritative: 0 is what a daemon that knows +// nothing about cost sends, and storing it would claim a genuine $0 spend and +// suppress the estimate. Negative is nonsense from a malformed report. +func authoritativeCostTicks(ticks int64) pgtype.Int8 { + if ticks <= 0 { + return pgtype.Int8{} + } + return pgtype.Int8{Int64: ticks, Valid: true} } func (h *Handler) ReportTaskUsage(w http.ResponseWriter, r *http.Request) { @@ -3391,11 +3407,12 @@ func (h *Handler) ReportTaskUsage(w http.ResponseWriter, r *http.Request) { OutputTokens: u.OutputTokens, CacheReadTokens: u.CacheReadTokens, CacheWriteTokens: u.CacheWriteTokens, + CostUsdTicks: authoritativeCostTicks(u.CostUSDTicks), }); err != nil { slog.Warn("upsert task usage failed", "task_id", taskID, "model", u.Model, "error", err) continue } - h.TaskService.CaptureTaskUsage(r.Context(), task, provider, u.Model, u.InputTokens, u.OutputTokens, u.CacheReadTokens, u.CacheWriteTokens) + h.TaskService.CaptureTaskUsage(r.Context(), task, provider, u.Model, u.InputTokens, u.OutputTokens, u.CacheReadTokens, u.CacheWriteTokens, u.CostUSDTicks) // Surface prompt-cache effectiveness per run so cache hit rates are // observable in logs, not just queryable from runtime_usage. The ratio @@ -3796,7 +3813,13 @@ func (h *Handler) GetIssueUsage(w http.ResponseWriter, r *http.Request) { "total_output_tokens": row.TotalOutputTokens, "total_cache_read_tokens": row.TotalCacheReadTokens, "total_cache_write_tokens": row.TotalCacheWriteTokens, - "task_count": row.TaskCount, + // Cost split — see the note on DashboardUsageDailyResponse. + "cost_usd_ticks": row.TotalCostUsdTicks, + "uncosted_input_tokens": row.UncostedInputTokens, + "uncosted_output_tokens": row.UncostedOutputTokens, + "uncosted_cache_read_tokens": row.UncostedCacheReadTokens, + "uncosted_cache_write_tokens": row.UncostedCacheWriteTokens, + "task_count": row.TaskCount, }) } diff --git a/server/internal/handler/dashboard.go b/server/internal/handler/dashboard.go index d3f44af87..d25f47484 100644 --- a/server/internal/handler/dashboard.go +++ b/server/internal/handler/dashboard.go @@ -63,7 +63,17 @@ type DashboardUsageDailyResponse struct { OutputTokens int64 `json:"output_tokens"` CacheReadTokens int64 `json:"cache_read_tokens"` CacheWriteTokens int64 `json:"cache_write_tokens"` - TaskCount int32 `json:"task_count"` + // Cost split: `CostUSDTicks` is what the provider itself charged for the + // rows behind this aggregate (1e-10 USD), and the `Uncosted*` token + // counts are the tokens from rows the provider did NOT price. The client + // reports authoritative + estimate(uncosted), so a window mixing both + // kinds of row stays whole. See migration 213. + CostUSDTicks int64 `json:"cost_usd_ticks"` + UncostedInputTokens int64 `json:"uncosted_input_tokens"` + UncostedOutputTokens int64 `json:"uncosted_output_tokens"` + UncostedCacheReadTokens int64 `json:"uncosted_cache_read_tokens"` + UncostedCacheWriteTokens int64 `json:"uncosted_cache_write_tokens"` + TaskCount int32 `json:"task_count"` } // GetDashboardUsageDaily returns per-(date, model) token rows for the @@ -108,14 +118,19 @@ func (h *Handler) listDashboardUsageDaily( resp := make([]DashboardUsageDailyResponse, len(rows)) for i, row := range rows { resp[i] = DashboardUsageDailyResponse{ - Date: row.Date.Time.Format("2006-01-02"), - Provider: row.Provider, - Model: row.Model, - InputTokens: row.InputTokens, - OutputTokens: row.OutputTokens, - CacheReadTokens: row.CacheReadTokens, - CacheWriteTokens: row.CacheWriteTokens, - TaskCount: row.TaskCount, + Date: row.Date.Time.Format("2006-01-02"), + Provider: row.Provider, + Model: row.Model, + InputTokens: row.InputTokens, + OutputTokens: row.OutputTokens, + CacheReadTokens: row.CacheReadTokens, + CacheWriteTokens: row.CacheWriteTokens, + CostUSDTicks: row.CostUsdTicks, + UncostedInputTokens: row.UncostedInputTokens, + UncostedOutputTokens: row.UncostedOutputTokens, + UncostedCacheReadTokens: row.UncostedCacheReadTokens, + UncostedCacheWriteTokens: row.UncostedCacheWriteTokens, + TaskCount: row.TaskCount, } } return resp, nil @@ -132,7 +147,17 @@ type DashboardUsageByAgentResponse struct { OutputTokens int64 `json:"output_tokens"` CacheReadTokens int64 `json:"cache_read_tokens"` CacheWriteTokens int64 `json:"cache_write_tokens"` - TaskCount int32 `json:"task_count"` + // Cost split: `CostUSDTicks` is what the provider itself charged for the + // rows behind this aggregate (1e-10 USD), and the `Uncosted*` token + // counts are the tokens from rows the provider did NOT price. The client + // reports authoritative + estimate(uncosted), so a window mixing both + // kinds of row stays whole. See migration 213. + CostUSDTicks int64 `json:"cost_usd_ticks"` + UncostedInputTokens int64 `json:"uncosted_input_tokens"` + UncostedOutputTokens int64 `json:"uncosted_output_tokens"` + UncostedCacheReadTokens int64 `json:"uncosted_cache_read_tokens"` + UncostedCacheWriteTokens int64 `json:"uncosted_cache_write_tokens"` + TaskCount int32 `json:"task_count"` } // GetDashboardUsageByAgent returns per-(agent, model) token aggregates @@ -177,14 +202,19 @@ func (h *Handler) listDashboardUsageByAgent( resp := make([]DashboardUsageByAgentResponse, len(rows)) for i, row := range rows { resp[i] = DashboardUsageByAgentResponse{ - AgentID: uuidToString(row.AgentID), - Provider: row.Provider, - Model: row.Model, - InputTokens: row.InputTokens, - OutputTokens: row.OutputTokens, - CacheReadTokens: row.CacheReadTokens, - CacheWriteTokens: row.CacheWriteTokens, - TaskCount: row.TaskCount, + AgentID: uuidToString(row.AgentID), + Provider: row.Provider, + Model: row.Model, + InputTokens: row.InputTokens, + OutputTokens: row.OutputTokens, + CacheReadTokens: row.CacheReadTokens, + CacheWriteTokens: row.CacheWriteTokens, + CostUSDTicks: row.CostUsdTicks, + UncostedInputTokens: row.UncostedInputTokens, + UncostedOutputTokens: row.UncostedOutputTokens, + UncostedCacheReadTokens: row.UncostedCacheReadTokens, + UncostedCacheWriteTokens: row.UncostedCacheWriteTokens, + TaskCount: row.TaskCount, } } return resp, nil diff --git a/server/internal/handler/runtime.go b/server/internal/handler/runtime.go index 8f850c71f..709932714 100644 --- a/server/internal/handler/runtime.go +++ b/server/internal/handler/runtime.go @@ -90,6 +90,16 @@ type RuntimeUsageResponse struct { OutputTokens int64 `json:"output_tokens"` CacheReadTokens int64 `json:"cache_read_tokens"` CacheWriteTokens int64 `json:"cache_write_tokens"` + // Cost split: `CostUSDTicks` is what the provider itself charged for the + // rows behind this aggregate (1e-10 USD), and the `Uncosted*` token + // counts are the tokens from rows the provider did NOT price. The client + // reports authoritative + estimate(uncosted), so a window mixing both + // kinds of row stays whole. See migration 213. + CostUSDTicks int64 `json:"cost_usd_ticks"` + UncostedInputTokens int64 `json:"uncosted_input_tokens"` + UncostedOutputTokens int64 `json:"uncosted_output_tokens"` + UncostedCacheReadTokens int64 `json:"uncosted_cache_read_tokens"` + UncostedCacheWriteTokens int64 `json:"uncosted_cache_write_tokens"` } // GetRuntimeUsage returns daily token usage for a runtime, aggregated from @@ -141,14 +151,19 @@ func (h *Handler) listRuntimeUsage(ctx context.Context, runtimeID pgtype.UUID, t resp := make([]RuntimeUsageResponse, len(rows)) for i, row := range rows { resp[i] = RuntimeUsageResponse{ - RuntimeID: resolvedRuntimeID, - Date: row.Date.Time.Format("2006-01-02"), - Provider: row.Provider, - Model: row.Model, - InputTokens: row.InputTokens, - OutputTokens: row.OutputTokens, - CacheReadTokens: row.CacheReadTokens, - CacheWriteTokens: row.CacheWriteTokens, + RuntimeID: resolvedRuntimeID, + Date: row.Date.Time.Format("2006-01-02"), + Provider: row.Provider, + Model: row.Model, + InputTokens: row.InputTokens, + OutputTokens: row.OutputTokens, + CacheReadTokens: row.CacheReadTokens, + CacheWriteTokens: row.CacheWriteTokens, + CostUSDTicks: row.CostUsdTicks, + UncostedInputTokens: row.UncostedInputTokens, + UncostedOutputTokens: row.UncostedOutputTokens, + UncostedCacheReadTokens: row.UncostedCacheReadTokens, + UncostedCacheWriteTokens: row.UncostedCacheWriteTokens, } } return resp, nil @@ -208,7 +223,17 @@ type RuntimeUsageByAgentResponse struct { OutputTokens int64 `json:"output_tokens"` CacheReadTokens int64 `json:"cache_read_tokens"` CacheWriteTokens int64 `json:"cache_write_tokens"` - TaskCount int32 `json:"task_count"` + // Cost split: `CostUSDTicks` is what the provider itself charged for the + // rows behind this aggregate (1e-10 USD), and the `Uncosted*` token + // counts are the tokens from rows the provider did NOT price. The client + // reports authoritative + estimate(uncosted), so a window mixing both + // kinds of row stays whole. See migration 213. + CostUSDTicks int64 `json:"cost_usd_ticks"` + UncostedInputTokens int64 `json:"uncosted_input_tokens"` + UncostedOutputTokens int64 `json:"uncosted_output_tokens"` + UncostedCacheReadTokens int64 `json:"uncosted_cache_read_tokens"` + UncostedCacheWriteTokens int64 `json:"uncosted_cache_write_tokens"` + TaskCount int32 `json:"task_count"` } // GetRuntimeUsageByAgent returns per-agent token aggregates for a runtime @@ -247,14 +272,19 @@ func (h *Handler) GetRuntimeUsageByAgent(w http.ResponseWriter, r *http.Request) resp := make([]RuntimeUsageByAgentResponse, len(rows)) for i, row := range rows { resp[i] = RuntimeUsageByAgentResponse{ - AgentID: uuidToString(row.AgentID), - Provider: row.Provider, - Model: row.Model, - InputTokens: row.InputTokens, - OutputTokens: row.OutputTokens, - CacheReadTokens: row.CacheReadTokens, - CacheWriteTokens: row.CacheWriteTokens, - TaskCount: row.TaskCount, + AgentID: uuidToString(row.AgentID), + Provider: row.Provider, + Model: row.Model, + InputTokens: row.InputTokens, + OutputTokens: row.OutputTokens, + CacheReadTokens: row.CacheReadTokens, + CacheWriteTokens: row.CacheWriteTokens, + CostUSDTicks: row.CostUsdTicks, + UncostedInputTokens: row.UncostedInputTokens, + UncostedOutputTokens: row.UncostedOutputTokens, + UncostedCacheReadTokens: row.UncostedCacheReadTokens, + UncostedCacheWriteTokens: row.UncostedCacheWriteTokens, + TaskCount: row.TaskCount, } } @@ -271,7 +301,17 @@ type RuntimeUsageByHourResponse struct { OutputTokens int64 `json:"output_tokens"` CacheReadTokens int64 `json:"cache_read_tokens"` CacheWriteTokens int64 `json:"cache_write_tokens"` - TaskCount int32 `json:"task_count"` + // Cost split: `CostUSDTicks` is what the provider itself charged for the + // rows behind this aggregate (1e-10 USD), and the `Uncosted*` token + // counts are the tokens from rows the provider did NOT price. The client + // reports authoritative + estimate(uncosted), so a window mixing both + // kinds of row stays whole. See migration 213. + CostUSDTicks int64 `json:"cost_usd_ticks"` + UncostedInputTokens int64 `json:"uncosted_input_tokens"` + UncostedOutputTokens int64 `json:"uncosted_output_tokens"` + UncostedCacheReadTokens int64 `json:"uncosted_cache_read_tokens"` + UncostedCacheWriteTokens int64 `json:"uncosted_cache_write_tokens"` + TaskCount int32 `json:"task_count"` } // GetRuntimeUsageByHour returns hourly (0..23) token aggregates for a @@ -313,13 +353,18 @@ func (h *Handler) GetRuntimeUsageByHour(w http.ResponseWriter, r *http.Request) resp := make([]RuntimeUsageByHourResponse, len(rows)) for i, row := range rows { resp[i] = RuntimeUsageByHourResponse{ - Hour: int(row.Hour), - Model: row.Model, - InputTokens: row.InputTokens, - OutputTokens: row.OutputTokens, - CacheReadTokens: row.CacheReadTokens, - CacheWriteTokens: row.CacheWriteTokens, - TaskCount: row.TaskCount, + Hour: int(row.Hour), + Model: row.Model, + InputTokens: row.InputTokens, + OutputTokens: row.OutputTokens, + CacheReadTokens: row.CacheReadTokens, + CacheWriteTokens: row.CacheWriteTokens, + CostUSDTicks: row.CostUsdTicks, + UncostedInputTokens: row.UncostedInputTokens, + UncostedOutputTokens: row.UncostedOutputTokens, + UncostedCacheReadTokens: row.UncostedCacheReadTokens, + UncostedCacheWriteTokens: row.UncostedCacheWriteTokens, + TaskCount: row.TaskCount, } } diff --git a/server/internal/metrics/business.go b/server/internal/metrics/business.go index 548bfae45..f25008ff8 100644 --- a/server/internal/metrics/business.go +++ b/server/internal/metrics/business.go @@ -250,7 +250,11 @@ func (m *BusinessMetrics) RecordTaskLeaseExpired(source string) { m.taskLeaseExpired.WithLabelValues(NormalizeTaskSource(source)).Inc() } -func (m *BusinessMetrics) RecordLLMUsage(source, runtimeMode, rawProvider, modelAlias string, inputTokens, outputTokens, cacheReadTokens, cacheWriteTokens int64) { +// costUSDTicks is the provider's own price for this usage in 1e-10 USD, or 0 +// when it reported none. When present it wins over the rate table: the table +// cannot express request-level rules such as xAI's 2x surcharge above a 200K +// prompt, so for those providers the local estimate is structurally low. +func (m *BusinessMetrics) RecordLLMUsage(source, runtimeMode, rawProvider, modelAlias string, inputTokens, outputTokens, cacheReadTokens, cacheWriteTokens, costUSDTicks int64) { if m == nil { return } @@ -264,17 +268,62 @@ func (m *BusinessMetrics) RecordLLMUsage(source, runtimeMode, rawProvider, model m.recordUnpricedTokens(provider, alias, "output", outputTokens) m.recordUnpricedTokens(provider, alias, "cache_read", cacheReadTokens) m.recordUnpricedTokens(provider, alias, "cache_write", cacheWriteTokens) + // Having no rate row does not mean having no cost: the provider may + // have priced the turn itself (`grok-composer-*` is in the Grok Build + // catalog but absent from xAI's price sheet). Dropping the charge here + // would under-report real spend purely for lack of a rate we no longer + // need. Without rates there is nothing to split the total by, so it + // lands whole in the `input` bucket — the same fallback + // distributeAuthoritativeCost uses when it has no shape to scale. + if costUSDTicks > 0 { + m.llmCostUSD. + WithLabelValues(provider, alias, NormalizeTokenType("input"), runtimeMode, source). + Add(float64(costUSDTicks) / CostUSDTicksPerUSD) + } m.llmRequests.WithLabelValues(provider, "unknown", runtimeMode).Inc() return } - m.recordPricedTokens(price.Provider, price.Model, "input", runtimeMode, source, inputTokens, tokenCostUSD(inputTokens, price.InputPerM)) - m.recordPricedTokens(price.Provider, price.Model, "output", runtimeMode, source, outputTokens, tokenCostUSD(outputTokens, price.OutputPerM)) - m.recordPricedTokens(price.Provider, price.Model, "cache_read", runtimeMode, source, cacheReadTokens, tokenCostUSD(cacheReadTokens, price.CacheReadPerM)) - m.recordPricedTokens(price.Provider, price.Model, "cache_write", runtimeMode, source, cacheWriteTokens, tokenCostUSD(cacheWriteTokens, price.CacheWritePerM)) + costs := [4]float64{ + tokenCostUSD(inputTokens, price.InputPerM), + tokenCostUSD(outputTokens, price.OutputPerM), + tokenCostUSD(cacheReadTokens, price.CacheReadPerM), + tokenCostUSD(cacheWriteTokens, price.CacheWritePerM), + } + if costUSDTicks > 0 { + costs = distributeAuthoritativeCost(float64(costUSDTicks)/CostUSDTicksPerUSD, costs) + } + + m.recordPricedTokens(price.Provider, price.Model, "input", runtimeMode, source, inputTokens, costs[0]) + m.recordPricedTokens(price.Provider, price.Model, "output", runtimeMode, source, outputTokens, costs[1]) + m.recordPricedTokens(price.Provider, price.Model, "cache_read", runtimeMode, source, cacheReadTokens, costs[2]) + m.recordPricedTokens(price.Provider, price.Model, "cache_write", runtimeMode, source, cacheWriteTokens, costs[3]) m.llmRequests.WithLabelValues(price.Provider, price.Model, runtimeMode).Inc() } +// distributeAuthoritativeCost rescales the per-token-type estimates so they +// sum to the provider's actual charge. `llm_cost_usd` is broken down by +// token_type and the provider reports one number for the whole turn, so the +// split has to come from somewhere; the rate table's own proportions are the +// best available guess and keep the total exact. Only the total is +// authoritative — the per-type split remains an estimate, which is why this +// scales rather than inventing a new label value. +// +// A zero estimate (unknown rates, or a turn recorded with no tokens) has no +// proportions to scale, so the charge lands on `input` to avoid dropping real +// spend from the total. +func distributeAuthoritativeCost(actual float64, estimated [4]float64) [4]float64 { + total := estimated[0] + estimated[1] + estimated[2] + estimated[3] + if total <= 0 { + return [4]float64{actual, 0, 0, 0} + } + scale := actual / total + for i := range estimated { + estimated[i] *= scale + } + return estimated +} + func (m *BusinessMetrics) recordPricedTokens(provider, model, tokenType, runtimeMode, source string, tokens int64, cost float64) { if tokens <= 0 { return diff --git a/server/internal/metrics/business_test.go b/server/internal/metrics/business_test.go index 691643d98..ebcabb51a 100644 --- a/server/internal/metrics/business_test.go +++ b/server/internal/metrics/business_test.go @@ -1,6 +1,7 @@ package metrics import ( + "math" "strconv" "testing" @@ -68,7 +69,7 @@ func TestBusinessMetricsFailureReasonUsesCanonicalClassifier(t *testing.T) { func TestBusinessMetricsLLMPricingAndUnpricedTokens(t *testing.T) { m := NewBusinessMetrics() - m.RecordLLMUsage("chat", "cloud", "codex", "gpt-5.4", 1_000_000, 2_000_000, 3_000_000, 4_000_000) + m.RecordLLMUsage("chat", "cloud", "codex", "gpt-5.4", 1_000_000, 2_000_000, 3_000_000, 4_000_000, 0) if got := testutil.ToFloat64(m.llmTokens.WithLabelValues("openai", "gpt-5.4", "input", "cloud", "chat")); got != 1_000_000 { t.Fatalf("priced input tokens = %v, want 1000000", got) @@ -86,7 +87,7 @@ func TestBusinessMetricsLLMPricingAndUnpricedTokens(t *testing.T) { t.Fatalf("priced request counter = %v, want 1", got) } - m.RecordLLMUsage("issue", "local", "custom-provider", "Free Model!!", 7, 0, 0, 0) + m.RecordLLMUsage("issue", "local", "custom-provider", "Free Model!!", 7, 0, 0, 0, 0) if got := testutil.ToFloat64(m.llmUnpricedTokens.WithLabelValues("other", "free_model_", "input")); got != 7 { t.Fatalf("unpriced input tokens = %v, want 7", got) } @@ -107,8 +108,8 @@ func TestBusinessMetricsRegistryExposesAllFamilies(t *testing.T) { m.RecordTaskFailed("issue", "local", taskfailure.ReasonTimeout.String()) m.RecordTaskQueuedExpired("issue", "local") m.RecordTaskLeaseExpired("issue") - m.RecordLLMUsage("issue", "local", "codex", "gpt-5.4", 1, 1, 1, 1) - m.RecordLLMUsage("issue", "local", "custom-provider", "custom-model", 1, 0, 0, 0) + m.RecordLLMUsage("issue", "local", "codex", "gpt-5.4", 1, 1, 1, 1, 0) + m.RecordLLMUsage("issue", "local", "custom-provider", "custom-model", 1, 0, 0, 0, 0) // PR3 funnel / community / commercial events. Drive every counter // with one synthetic value so the gather loop below sees the family. @@ -169,3 +170,136 @@ func exerciseEvent(m *BusinessMetrics, name string, props map[string]any) { } m.IncForEvent(analytics.Event{Name: name, Properties: props}) } + +// TestBusinessMetricsPrefersProviderReportedCost pins the rule that a +// provider's own charge wins over the rate table. The table cannot express +// request-level pricing (xAI bills a Grok request at 2x above a 200K prompt), +// so for those turns the local estimate is structurally low and recording it +// would under-report real spend. +func TestBusinessMetricsPrefersProviderReportedCost(t *testing.T) { + m := NewBusinessMetrics() + + // 1M input + 1M output on grok-4.5 estimates to $2 + $6 = $8 from the + // table. The provider says the turn cost $16 — the long-context tier. + const actualUSD = 16.0 + m.RecordLLMUsage("issue", "local", "grok", "grok-4.5", + 1_000_000, 1_000_000, 0, 0, int64(actualUSD*CostUSDTicksPerUSD)) + + input := testutil.ToFloat64(m.llmCostUSD.WithLabelValues("xai", "grok-4.5", "input", "local", "issue")) + output := testutil.ToFloat64(m.llmCostUSD.WithLabelValues("xai", "grok-4.5", "output", "local", "issue")) + if total := input + output; math.Abs(total-actualUSD) > 1e-9 { + t.Fatalf("recorded cost = %v, want %v (the provider's own charge)", total, actualUSD) + } + // The per-type split is presentation only, but it must follow the table's + // own proportions ($2 input : $6 output) so the labels stay meaningful. + if math.Abs(input-4) > 1e-9 { + t.Errorf("input cost = %v, want 4 (a quarter of the charge)", input) + } + if math.Abs(output-12) > 1e-9 { + t.Errorf("output cost = %v, want 12 (three quarters of the charge)", output) + } + // Token counters are unaffected by the cost source. + if got := testutil.ToFloat64(m.llmTokens.WithLabelValues("xai", "grok-4.5", "input", "local", "issue")); got != 1_000_000 { + t.Errorf("input tokens = %v, want 1000000", got) + } +} + +// TestBusinessMetricsFallsBackToRateTableWithoutProviderCost is the other half +// of the rule: every provider that reports no cost must keep being estimated. +func TestBusinessMetricsFallsBackToRateTableWithoutProviderCost(t *testing.T) { + m := NewBusinessMetrics() + + m.RecordLLMUsage("issue", "local", "grok", "grok-4.5", 1_000_000, 1_000_000, 0, 0, 0) + + input := testutil.ToFloat64(m.llmCostUSD.WithLabelValues("xai", "grok-4.5", "input", "local", "issue")) + output := testutil.ToFloat64(m.llmCostUSD.WithLabelValues("xai", "grok-4.5", "output", "local", "issue")) + if math.Abs(input-2) > 1e-9 || math.Abs(output-6) > 1e-9 { + t.Fatalf("estimated cost = (%v, %v), want (2, 6) from the rate table", input, output) + } +} + +func TestDistributeAuthoritativeCost(t *testing.T) { + t.Parallel() + + for _, tc := range []struct { + name string + actual float64 + estimated [4]float64 + want [4]float64 + }{ + { + name: "scales the estimate to the real charge", + actual: 16, + estimated: [4]float64{2, 6, 0, 0}, + want: [4]float64{4, 12, 0, 0}, + }, + { + name: "scales down when the estimate was too high", + actual: 4, + estimated: [4]float64{2, 6, 0, 0}, + want: [4]float64{1, 3, 0, 0}, + }, + { + // No proportions to shape it with — the charge still has to land + // somewhere, or real spend silently vanishes from the total. + name: "keeps the charge when there is nothing to scale", + actual: 7, + estimated: [4]float64{0, 0, 0, 0}, + want: [4]float64{7, 0, 0, 0}, + }, + } { + t.Run(tc.name, func(t *testing.T) { + t.Parallel() + got := distributeAuthoritativeCost(tc.actual, tc.estimated) + var sum float64 + for i := range got { + if math.Abs(got[i]-tc.want[i]) > 1e-9 { + t.Errorf("bucket %d = %v, want %v", i, got[i], tc.want[i]) + } + sum += got[i] + } + if math.Abs(sum-tc.actual) > 1e-9 { + t.Errorf("buckets sum to %v, want the actual charge %v", sum, tc.actual) + } + }) + } +} + +// TestBusinessMetricsRecordsProviderCostForUnpricedModel covers the gap where +// a model has no rate row but the provider priced the turn itself +// (`grok-composer-*` is in the Grok Build catalog and absent from xAI's price +// sheet). Returning early on "no rates" would drop real spend from +// llm_cost_usd for want of a rate the charge does not need. +func TestBusinessMetricsRecordsProviderCostForUnpricedModel(t *testing.T) { + m := NewBusinessMetrics() + + const actualUSD = 1.23456789 + m.RecordLLMUsage("issue", "local", "grok", "grok-composer-2.5-fast", + 500, 100, 0, 0, int64(actualUSD*CostUSDTicksPerUSD)) + + // No rates means no way to split by token type, so the whole charge lands + // in one bucket — but it must be the whole charge. + got := testutil.ToFloat64(m.llmCostUSD.WithLabelValues( + "grok", "grok-composer-2.5-fast", "input", "local", "issue")) + if math.Abs(got-actualUSD) > 1e-9 { + t.Fatalf("recorded cost = %v, want %v", got, actualUSD) + } + // The tokens still have no rate, so they stay in the unpriced counter — + // "unpriced" describes the rate table, not the money. + if got := testutil.ToFloat64(m.llmUnpricedTokens.WithLabelValues("grok", "grok-composer-2.5-fast", "input")); got != 500 { + t.Errorf("unpriced input tokens = %v, want 500", got) + } +} + +// TestBusinessMetricsUnpricedModelWithoutCostStaysAtZero is the control: no +// rates and no provider cost must record no spend, not a fabricated one. +func TestBusinessMetricsUnpricedModelWithoutCostStaysAtZero(t *testing.T) { + m := NewBusinessMetrics() + + m.RecordLLMUsage("issue", "local", "grok", "grok-composer-2.5-fast", 500, 100, 0, 0, 0) + + if got := testutil.ToFloat64(m.llmCostUSD.WithLabelValues( + "grok", "grok-composer-2.5-fast", "input", "local", "issue")); got != 0 { + t.Fatalf("recorded cost = %v, want 0", got) + } +} diff --git a/server/internal/metrics/pricing.go b/server/internal/metrics/pricing.go index 9358b004b..8f312393f 100644 --- a/server/internal/metrics/pricing.go +++ b/server/internal/metrics/pricing.go @@ -5,6 +5,12 @@ import ( "strings" ) +// CostUSDTicksPerUSD is the scale of provider-reported costs: xAI reports +// whole ticks of 1e-10 USD. Declared here rather than imported from pkg/agent +// (which owns the wire-format parsing) so this package keeps no dependency on +// the agent runtime for a physical unit; the two must stay equal. +const CostUSDTicksPerUSD = 10_000_000_000 + type ModelPrice struct { Provider string Model string @@ -51,6 +57,22 @@ var modelPrices = map[string]ModelPrice{ "google:gemini-3.1-pro": {Provider: "google", Model: "gemini-3.1-pro", InputPerM: 2.00, CacheReadPerM: 0.20, CacheWritePerM: 2.00, OutputPerM: 12.00}, "google:gemini-2.5-pro": {Provider: "google", Model: "gemini-2.5-pro", InputPerM: 1.25, CacheReadPerM: 0.31, CacheWritePerM: 1.25, OutputPerM: 10.00}, "google:gemini-2.5-flash": {Provider: "google", Model: "gemini-2.5-flash", InputPerM: 0.30, CacheReadPerM: 0.03, CacheWritePerM: 0.30, OutputPerM: 2.50}, + // xAI Grok (docs.x.ai/developers/pricing). Short-context tier: xAI bills + // a request at 2x once its prompt reaches 200K tokens, but a usage record + // aggregates every model call in a turn, so it cannot say which tier any + // individual request hit — pricing the standard tier under-estimates a + // long-context turn by at most 50% instead of over-estimating a short one + // by 100%. xAI publishes no separate cache-write rate (writes bill as + // normal input), so CacheWrite mirrors Input. These rows mirror + // packages/views/runtimes/utils.ts; keep the two tables in sync. + // `grok-composer-*` ships in the Grok Build catalog but is absent from the + // price sheet, so it stays unmapped rather than inheriting a guessed rate. + "xai:grok-4.5": {Provider: "xai", Model: "grok-4.5", InputPerM: 2.00, CacheReadPerM: 0.30, CacheWritePerM: 2.00, OutputPerM: 6.00}, + "xai:grok-4.3": {Provider: "xai", Model: "grok-4.3", InputPerM: 1.25, CacheReadPerM: 0.20, CacheWritePerM: 1.25, OutputPerM: 2.50}, + "xai:grok-build-0.1": {Provider: "xai", Model: "grok-build-0.1", InputPerM: 1.00, CacheReadPerM: 0.20, CacheWritePerM: 1.00, OutputPerM: 2.00}, + "xai:grok-4.20-multi-agent-0309": {Provider: "xai", Model: "grok-4.20-multi-agent-0309", InputPerM: 1.25, CacheReadPerM: 0.20, CacheWritePerM: 1.25, OutputPerM: 2.50}, + "xai:grok-4.20-0309-reasoning": {Provider: "xai", Model: "grok-4.20-0309-reasoning", InputPerM: 1.25, CacheReadPerM: 0.20, CacheWritePerM: 1.25, OutputPerM: 2.50}, + "xai:grok-4.20-0309-non-reasoning": {Provider: "xai", Model: "grok-4.20-0309-non-reasoning", InputPerM: 1.25, CacheReadPerM: 0.20, CacheWritePerM: 1.25, OutputPerM: 2.50}, } var modelAliasRules = []struct { @@ -89,6 +111,16 @@ var modelAliasRules = []struct { {regexp.MustCompile(`gemini-3[.]1-pro`), "google:gemini-3.1-pro"}, {regexp.MustCompile(`gemini-2[.]5-pro`), "google:gemini-2.5-pro"}, {regexp.MustCompile(`gemini-2[.]5-flash`), "google:gemini-2.5-flash"}, + // Anchored exact-match, dotted spelling only — same rule as the gpt-5.6 + // rows above. The frontend resolver does not dash-normalize non-Anthropic + // ids, so a dashed `grok-4-5` must surface as unmapped on both sides + // rather than silently borrowing a tier here. + {regexp.MustCompile(`(^|/|:)grok-4\.5$`), "xai:grok-4.5"}, + {regexp.MustCompile(`(^|/|:)grok-4\.3$`), "xai:grok-4.3"}, + {regexp.MustCompile(`(^|/|:)grok-build-0\.1$`), "xai:grok-build-0.1"}, + {regexp.MustCompile(`(^|/|:)grok-4\.20-multi-agent-0309$`), "xai:grok-4.20-multi-agent-0309"}, + {regexp.MustCompile(`(^|/|:)grok-4\.20-0309-reasoning$`), "xai:grok-4.20-0309-reasoning"}, + {regexp.MustCompile(`(^|/|:)grok-4\.20-0309-non-reasoning$`), "xai:grok-4.20-0309-non-reasoning"}, } func PriceForModelAlias(model string) (ModelPrice, bool) { diff --git a/server/internal/metrics/pricing_test.go b/server/internal/metrics/pricing_test.go index d2325daed..2fcc47d72 100644 --- a/server/internal/metrics/pricing_test.go +++ b/server/internal/metrics/pricing_test.go @@ -94,3 +94,105 @@ func TestPriceForModelAliasCodexGPT56(t *testing.T) { } } } + +// TestPriceForModelAliasGrok pins the xAI catalog to the published rates +// (docs.x.ai/developers/pricing). Before these rows existed every Grok token +// took the unpriced branch in RecordLLMUsage, so llm_cost_usd reported zero +// Grok spend while the tokens piled up in llm_unpriced_tokens. +func TestPriceForModelAliasGrok(t *testing.T) { + cases := []struct { + model string + want ModelPrice + }{ + { + model: "grok-4.5", + want: ModelPrice{Provider: "xai", Model: "grok-4.5", InputPerM: 2, CacheReadPerM: 0.3, CacheWritePerM: 2, OutputPerM: 6}, + }, + { + model: "xai:grok-4.5", + want: ModelPrice{Provider: "xai", Model: "grok-4.5", InputPerM: 2, CacheReadPerM: 0.3, CacheWritePerM: 2, OutputPerM: 6}, + }, + { + model: "xai/grok-4.5", + want: ModelPrice{Provider: "xai", Model: "grok-4.5", InputPerM: 2, CacheReadPerM: 0.3, CacheWritePerM: 2, OutputPerM: 6}, + }, + { + model: "grok-4.3", + want: ModelPrice{Provider: "xai", Model: "grok-4.3", InputPerM: 1.25, CacheReadPerM: 0.2, CacheWritePerM: 1.25, OutputPerM: 2.5}, + }, + { + model: "grok-build-0.1", + want: ModelPrice{Provider: "xai", Model: "grok-build-0.1", InputPerM: 1, CacheReadPerM: 0.2, CacheWritePerM: 1, OutputPerM: 2}, + }, + { + model: "grok-4.20-multi-agent-0309", + want: ModelPrice{Provider: "xai", Model: "grok-4.20-multi-agent-0309", InputPerM: 1.25, CacheReadPerM: 0.2, CacheWritePerM: 1.25, OutputPerM: 2.5}, + }, + { + model: "grok-4.20-0309-reasoning", + want: ModelPrice{Provider: "xai", Model: "grok-4.20-0309-reasoning", InputPerM: 1.25, CacheReadPerM: 0.2, CacheWritePerM: 1.25, OutputPerM: 2.5}, + }, + { + model: "grok-4.20-0309-non-reasoning", + want: ModelPrice{Provider: "xai", Model: "grok-4.20-0309-non-reasoning", InputPerM: 1.25, CacheReadPerM: 0.2, CacheWritePerM: 1.25, OutputPerM: 2.5}, + }, + } + + for _, tc := range cases { + got, ok := PriceForModelAlias(tc.model) + if !ok { + t.Fatalf("PriceForModelAlias(%q) did not resolve", tc.model) + } + if got != tc.want { + t.Fatalf("PriceForModelAlias(%q) = %+v, want %+v", tc.model, got, tc.want) + } + } + + // `grok-composer-*` ships in the Grok Build catalog but xAI publishes no + // rate for it, so it must stay unmapped rather than inherit grok-4.5's. + // Suffixed and dash-spelled variants must miss for the same reason the + // gpt-5.6 rows do: the frontend resolver is an exact match that does not + // dash-normalize non-Anthropic ids, so both sides agree on "unmapped". + for _, model := range []string{ + "grok-composer-2.5-fast", + "grok-composer-2.5", + "grok-4.5-fast", + "grok-4-5", + "grok-4.20-0309", + "grok", + "unknown", + } { + if got, ok := PriceForModelAlias(model); ok { + t.Fatalf("PriceForModelAlias(%q) unexpectedly resolved to %+v; want unmapped", model, got) + } + } +} + +// TestGrokPricingMatchesRecordedTurn re-derives the cost of a real +// grok 0.2.106 turn from the table and checks it against the costUsdTicks xAI +// returned for that same turn (1 tick = 1e-10 USD). This is the end-to-end +// proof that both the rates and the cached-input bucketing are right. +func TestGrokPricingMatchesRecordedTurn(t *testing.T) { + // Captured payload: inputTokens 12929, cachedReadTokens 10880, + // outputTokens 29, totalTokens 12958, costUsdTicks 75360000. Grok counts + // the cached prefix inside inputTokens, so the uncached remainder is + // 12929 - 10880 = 2049 (see excludeACPCachedInput in pkg/agent/hermes.go). + const ( + uncachedInput = int64(2049) + cacheRead = int64(10880) + output = int64(29) + wantUSD = 75360000 / 1e10 + ) + + price, ok := PriceForModelAlias("grok-4.5") + if !ok { + t.Fatal("grok-4.5 did not resolve") + } + got := tokenCostUSD(uncachedInput, price.InputPerM) + + tokenCostUSD(cacheRead, price.CacheReadPerM) + + tokenCostUSD(output, price.OutputPerM) + + if diff := got - wantUSD; diff > 1e-12 || diff < -1e-12 { + t.Fatalf("recomputed cost = %.10f, want %.10f (xAI costUsdTicks)", got, wantUSD) + } +} diff --git a/server/internal/service/task.go b/server/internal/service/task.go index 741d28d4b..1e28bc344 100644 --- a/server/internal/service/task.go +++ b/server/internal/service/task.go @@ -668,12 +668,14 @@ func (s *TaskService) captureTaskCancelled(ctx context.Context, task db.AgentTas } } -func (s *TaskService) CaptureTaskUsage(ctx context.Context, task db.AgentTaskQueue, provider, model string, inputTokens, outputTokens, cacheReadTokens, cacheWriteTokens int64) { +// costUSDTicks is the provider's own price for this usage in 1e-10 USD, or 0 +// when it reported none — the metrics layer prefers it over its rate table. +func (s *TaskService) CaptureTaskUsage(ctx context.Context, task db.AgentTaskQueue, provider, model string, inputTokens, outputTokens, cacheReadTokens, cacheWriteTokens, costUSDTicks int64) { if s.Metrics == nil { return } source, runtimeMode, _ := s.taskMetricsContext(ctx, task) - s.Metrics.RecordLLMUsage(source, runtimeMode, provider, model, inputTokens, outputTokens, cacheReadTokens, cacheWriteTokens) + s.Metrics.RecordLLMUsage(source, runtimeMode, provider, model, inputTokens, outputTokens, cacheReadTokens, cacheWriteTokens, costUSDTicks) } func (s *TaskService) CaptureQueuedExpiredTasks(ctx context.Context, tasks []db.AgentTaskQueue) { diff --git a/server/migrations/213_task_usage_authoritative_cost.down.sql b/server/migrations/213_task_usage_authoritative_cost.down.sql new file mode 100644 index 000000000..2c1a160d1 --- /dev/null +++ b/server/migrations/213_task_usage_authoritative_cost.down.sql @@ -0,0 +1,141 @@ +-- Restore migration 102's rollup window function verbatim, then drop the +-- columns. Order matters: the function must stop referencing the columns +-- before they go away. +CREATE OR REPLACE FUNCTION rollup_task_usage_hourly_window( + p_from TIMESTAMPTZ, + p_to TIMESTAMPTZ +) +RETURNS BIGINT +LANGUAGE plpgsql +AS $$ +DECLARE + v_rows BIGINT; +BEGIN + IF p_from >= p_to THEN + RETURN 0; + END IF; + + WITH + dirty_from_updates AS ( + SELECT DISTINCT + task_usage_hour_bucket(tu.created_at) AS bucket_hour, + a.workspace_id AS workspace_id, + atq.runtime_id AS runtime_id, + atq.agent_id AS agent_id, + i.project_id AS project_id, + tu.provider AS provider, + tu.model AS model + FROM task_usage tu + JOIN agent_task_queue atq ON atq.id = tu.task_id + JOIN agent a ON a.id = atq.agent_id + LEFT JOIN issue i ON i.id = atq.issue_id + WHERE atq.runtime_id IS NOT NULL + AND ( + (tu.updated_at >= p_from AND tu.updated_at < p_to) + OR (tu.updated_at IS NULL + AND tu.created_at >= p_from + AND tu.created_at < p_to) + ) + ), + dirty_from_queue AS ( + SELECT bucket_hour, workspace_id, runtime_id, agent_id, + project_id, provider, model + FROM task_usage_hourly_dirty + WHERE enqueued_at < p_to + ), + dirty_keys AS ( + SELECT * FROM dirty_from_updates + UNION + SELECT * FROM dirty_from_queue + ), + recomputed AS ( + SELECT + dk.bucket_hour, + dk.workspace_id, + dk.runtime_id, + dk.agent_id, + dk.project_id, + dk.provider, + dk.model, + SUM(tu.input_tokens)::bigint AS input_tokens, + SUM(tu.output_tokens)::bigint AS output_tokens, + SUM(tu.cache_read_tokens)::bigint AS cache_read_tokens, + SUM(tu.cache_write_tokens)::bigint AS cache_write_tokens, + COUNT(DISTINCT tu.task_id)::bigint AS task_count, + COUNT(*)::bigint AS event_count + FROM dirty_keys dk + JOIN agent_task_queue atq ON atq.runtime_id = dk.runtime_id + AND atq.agent_id = dk.agent_id + JOIN agent a ON a.id = atq.agent_id + AND a.workspace_id = dk.workspace_id + LEFT JOIN issue i ON i.id = atq.issue_id + JOIN task_usage tu ON tu.task_id = atq.id + AND tu.provider = dk.provider + AND tu.model = dk.model + AND task_usage_hour_bucket(tu.created_at) = dk.bucket_hour + WHERE (i.project_id IS NOT DISTINCT FROM dk.project_id) + GROUP BY 1, 2, 3, 4, 5, 6, 7 + ), + upserted AS ( + INSERT INTO task_usage_hourly AS d ( + bucket_hour, workspace_id, runtime_id, agent_id, + project_id, provider, model, + input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, + task_count, event_count + ) + SELECT + bucket_hour, workspace_id, runtime_id, agent_id, + project_id, provider, model, + input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, + task_count, event_count + FROM recomputed + ON CONFLICT ON CONSTRAINT uq_task_usage_hourly_key DO UPDATE + SET input_tokens = EXCLUDED.input_tokens, + output_tokens = EXCLUDED.output_tokens, + cache_read_tokens = EXCLUDED.cache_read_tokens, + cache_write_tokens = EXCLUDED.cache_write_tokens, + task_count = EXCLUDED.task_count, + event_count = EXCLUDED.event_count, + updated_at = now() + RETURNING 1 + ), + deleted_empty AS ( + DELETE FROM task_usage_hourly d + USING dirty_keys dk + WHERE d.bucket_hour = dk.bucket_hour + AND d.workspace_id = dk.workspace_id + AND d.runtime_id = dk.runtime_id + AND d.agent_id = dk.agent_id + AND d.project_id IS NOT DISTINCT FROM dk.project_id + AND d.provider = dk.provider + AND d.model = dk.model + AND NOT EXISTS ( + SELECT 1 FROM recomputed r + WHERE r.bucket_hour = dk.bucket_hour + AND r.workspace_id = dk.workspace_id + AND r.runtime_id = dk.runtime_id + AND r.agent_id = dk.agent_id + AND r.project_id IS NOT DISTINCT FROM dk.project_id + AND r.provider = dk.provider + AND r.model = dk.model + ) + RETURNING 1 + ) + SELECT (SELECT COUNT(*) FROM upserted) + (SELECT COUNT(*) FROM deleted_empty) + INTO v_rows; + + DELETE FROM task_usage_hourly_dirty WHERE enqueued_at < p_to; + + RETURN v_rows; +END; +$$; + +ALTER TABLE task_usage_hourly + DROP COLUMN IF EXISTS uncosted_cache_write_tokens, + DROP COLUMN IF EXISTS uncosted_cache_read_tokens, + DROP COLUMN IF EXISTS uncosted_output_tokens, + DROP COLUMN IF EXISTS uncosted_input_tokens, + DROP COLUMN IF EXISTS cost_usd_ticks; + +ALTER TABLE task_usage + DROP COLUMN IF EXISTS cost_usd_ticks; diff --git a/server/migrations/213_task_usage_authoritative_cost.up.sql b/server/migrations/213_task_usage_authoritative_cost.up.sql new file mode 100644 index 000000000..2235c49bc --- /dev/null +++ b/server/migrations/213_task_usage_authoritative_cost.up.sql @@ -0,0 +1,223 @@ +-- Provider-reported cost for a usage record, and the rollup columns needed to +-- mix it with rows that have none. +-- +-- WHY: cost has always been derived client-side as tokens x a static rate. +-- That cannot express request-level pricing rules. xAI bills a Grok request at +-- 2x once its prompt reaches 200K tokens, and a `task_usage` row aggregates +-- every model call in a turn — so the stored token counts genuinely cannot say +-- which tier any individual request hit. Thresholding on the aggregate would +-- turn a bounded 50% under-estimate into an unbounded over-estimate. Grok +-- already returns its own price per turn (`_meta.usage.costUsdTicks`); store +-- it and the arithmetic stops being a guess. +-- +-- UNIT: ticks of 1e-10 USD, as xAI reports them. An integer keeps sub-cent +-- turn costs exact end-to-end instead of drifting through float64. BIGINT +-- holds ~9.2e8 USD, which is not a reachable spend for one usage record. +-- +-- NULL means "the provider reported no cost" — every pre-existing row, and +-- every provider that doesn't return one. Those rows keep being estimated from +-- the pricing table. No backfill: historical rows have no authoritative figure +-- to recover, and inventing one would be the same guess we're removing. +ALTER TABLE task_usage + ADD COLUMN IF NOT EXISTS cost_usd_ticks BIGINT; + +COMMENT ON COLUMN task_usage.cost_usd_ticks IS + 'Provider-reported cost in 1e-10 USD. NULL when the provider reports none; those rows are priced client-side from the static rate table.'; + +-- Rollup side. A single hourly bucket can mix rows that carry an authoritative +-- cost with rows that don't (two providers, or Grok before and after a CLI +-- upgrade). Summing only the authoritative side would silently under-report +-- the bucket, so the rollup carries both halves: +-- +-- cost_usd_ticks - sum over rows that HAVE a provider cost +-- uncosted_*_tokens - tokens from rows that do NOT, i.e. exactly the +-- tokens still needing a rate-table estimate +-- +-- The consumer then reports `authoritative + estimate(uncosted tokens)`, which +-- degrades to today's behaviour when nothing in the bucket is authoritative. +-- The existing token columns keep their meaning (every row in the bucket) so +-- token displays are untouched. +-- +-- The `uncosted_*` columns are NULLABLE WITH NO DEFAULT, and that is load- +-- bearing: NULL means "this bucket has never been recomputed since the split +-- existed", which is every pre-existing row. Readers COALESCE it to the +-- bucket's own token total, i.e. "estimate all of it" — exactly what those +-- rows did before. A `NOT NULL DEFAULT 0` would instead assert "nothing here +-- needs estimating" and collapse every historical bucket's cost to $0 until +-- the rollup happened to touch it again, which is why this migration does NOT +-- rewrite history to seed them: with NULL there is nothing to seed. A bare +-- ADD COLUMN is metadata-only, so this stays a fast DDL with no table rewrite, +-- no WAL churn, and no lock held proportional to table size. +-- +-- The rollup writes concrete values for every bucket it recomputes, so rows +-- heal into the split naturally as they are touched. +-- +-- `cost_usd_ticks` is NOT NULL DEFAULT 0 because 0 is the honest identity for +-- a sum of "no rows here were priced by their provider", which is true of +-- every historical bucket. +-- +-- All additive, so the unique key, the dirty-queue key shape, and every +-- trigger in migration 102 are unaffected. +ALTER TABLE task_usage_hourly + ADD COLUMN IF NOT EXISTS cost_usd_ticks BIGINT NOT NULL DEFAULT 0, + ADD COLUMN IF NOT EXISTS uncosted_input_tokens BIGINT, + ADD COLUMN IF NOT EXISTS uncosted_output_tokens BIGINT, + ADD COLUMN IF NOT EXISTS uncosted_cache_read_tokens BIGINT, + ADD COLUMN IF NOT EXISTS uncosted_cache_write_tokens BIGINT; + +COMMENT ON COLUMN task_usage_hourly.cost_usd_ticks IS + 'Sum of provider-reported cost (1e-10 USD) over the rows in this bucket that had one; 0 when none did.'; +COMMENT ON COLUMN task_usage_hourly.uncosted_input_tokens IS + 'Input tokens from rows with no provider-reported cost — the portion still priced from the static rate table. NULL on buckets not yet recomputed since this column existed; readers COALESCE to input_tokens.'; + +-- Teach the rollup to populate them. Body is migration 102's, with the two +-- new expression groups in `recomputed` and the matching SET list on +-- conflict; everything else — dirty discovery, the empty-bucket delete, the +-- queue drain, the idempotency contract — is unchanged. +CREATE OR REPLACE FUNCTION rollup_task_usage_hourly_window( + p_from TIMESTAMPTZ, + p_to TIMESTAMPTZ +) +RETURNS BIGINT +LANGUAGE plpgsql +AS $$ +DECLARE + v_rows BIGINT; +BEGIN + IF p_from >= p_to THEN + RETURN 0; + END IF; + + WITH + dirty_from_updates AS ( + SELECT DISTINCT + task_usage_hour_bucket(tu.created_at) AS bucket_hour, + a.workspace_id AS workspace_id, + atq.runtime_id AS runtime_id, + atq.agent_id AS agent_id, + i.project_id AS project_id, + tu.provider AS provider, + tu.model AS model + FROM task_usage tu + JOIN agent_task_queue atq ON atq.id = tu.task_id + JOIN agent a ON a.id = atq.agent_id + LEFT JOIN issue i ON i.id = atq.issue_id + WHERE atq.runtime_id IS NOT NULL + AND ( + (tu.updated_at >= p_from AND tu.updated_at < p_to) + -- Legacy updated_at-NULL rows; partial index from 078. + OR (tu.updated_at IS NULL + AND tu.created_at >= p_from + AND tu.created_at < p_to) + ) + ), + dirty_from_queue AS ( + SELECT bucket_hour, workspace_id, runtime_id, agent_id, + project_id, provider, model + FROM task_usage_hourly_dirty + WHERE enqueued_at < p_to + ), + dirty_keys AS ( + SELECT * FROM dirty_from_updates + UNION + SELECT * FROM dirty_from_queue + ), + recomputed AS ( + SELECT + dk.bucket_hour, + dk.workspace_id, + dk.runtime_id, + dk.agent_id, + dk.project_id, + dk.provider, + dk.model, + SUM(tu.input_tokens)::bigint AS input_tokens, + SUM(tu.output_tokens)::bigint AS output_tokens, + SUM(tu.cache_read_tokens)::bigint AS cache_read_tokens, + SUM(tu.cache_write_tokens)::bigint AS cache_write_tokens, + -- Authoritative half: only rows the provider priced. + COALESCE(SUM(tu.cost_usd_ticks), 0)::bigint AS cost_usd_ticks, + -- Estimated half: tokens from rows the provider did not price. + COALESCE(SUM(tu.input_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_input_tokens, + COALESCE(SUM(tu.output_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_output_tokens, + COALESCE(SUM(tu.cache_read_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_cache_read_tokens, + COALESCE(SUM(tu.cache_write_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_cache_write_tokens, + COUNT(DISTINCT tu.task_id)::bigint AS task_count, + COUNT(*)::bigint AS event_count + FROM dirty_keys dk + JOIN agent_task_queue atq ON atq.runtime_id = dk.runtime_id + AND atq.agent_id = dk.agent_id + JOIN agent a ON a.id = atq.agent_id + AND a.workspace_id = dk.workspace_id + LEFT JOIN issue i ON i.id = atq.issue_id + JOIN task_usage tu ON tu.task_id = atq.id + AND tu.provider = dk.provider + AND tu.model = dk.model + AND task_usage_hour_bucket(tu.created_at) = dk.bucket_hour + WHERE (i.project_id IS NOT DISTINCT FROM dk.project_id) + GROUP BY 1, 2, 3, 4, 5, 6, 7 + ), + upserted AS ( + INSERT INTO task_usage_hourly AS d ( + bucket_hour, workspace_id, runtime_id, agent_id, + project_id, provider, model, + input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, + cost_usd_ticks, + uncosted_input_tokens, uncosted_output_tokens, + uncosted_cache_read_tokens, uncosted_cache_write_tokens, + task_count, event_count + ) + SELECT + bucket_hour, workspace_id, runtime_id, agent_id, + project_id, provider, model, + input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, + cost_usd_ticks, + uncosted_input_tokens, uncosted_output_tokens, + uncosted_cache_read_tokens, uncosted_cache_write_tokens, + task_count, event_count + FROM recomputed + ON CONFLICT ON CONSTRAINT uq_task_usage_hourly_key DO UPDATE + SET input_tokens = EXCLUDED.input_tokens, + output_tokens = EXCLUDED.output_tokens, + cache_read_tokens = EXCLUDED.cache_read_tokens, + cache_write_tokens = EXCLUDED.cache_write_tokens, + cost_usd_ticks = EXCLUDED.cost_usd_ticks, + uncosted_input_tokens = EXCLUDED.uncosted_input_tokens, + uncosted_output_tokens = EXCLUDED.uncosted_output_tokens, + uncosted_cache_read_tokens = EXCLUDED.uncosted_cache_read_tokens, + uncosted_cache_write_tokens = EXCLUDED.uncosted_cache_write_tokens, + task_count = EXCLUDED.task_count, + event_count = EXCLUDED.event_count, + updated_at = now() + RETURNING 1 + ), + deleted_empty AS ( + DELETE FROM task_usage_hourly d + USING dirty_keys dk + WHERE d.bucket_hour = dk.bucket_hour + AND d.workspace_id = dk.workspace_id + AND d.runtime_id = dk.runtime_id + AND d.agent_id = dk.agent_id + AND d.project_id IS NOT DISTINCT FROM dk.project_id + AND d.provider = dk.provider + AND d.model = dk.model + AND NOT EXISTS ( + SELECT 1 FROM recomputed r + WHERE r.bucket_hour = dk.bucket_hour + AND r.workspace_id = dk.workspace_id + AND r.runtime_id = dk.runtime_id + AND r.agent_id = dk.agent_id + AND r.project_id IS NOT DISTINCT FROM dk.project_id + AND r.provider = dk.provider + AND r.model = dk.model + ) + RETURNING 1 + ) + SELECT (SELECT COUNT(*) FROM upserted) + (SELECT COUNT(*) FROM deleted_empty) + INTO v_rows; + + DELETE FROM task_usage_hourly_dirty WHERE enqueued_at < p_to; + + RETURN v_rows; +END; +$$; diff --git a/server/pkg/agent/agent.go b/server/pkg/agent/agent.go index 178a15f90..66ae73937 100644 --- a/server/pkg/agent/agent.go +++ b/server/pkg/agent/agent.go @@ -140,8 +140,23 @@ type TokenUsage struct { OutputTokens int64 CacheReadTokens int64 CacheWriteTokens int64 + // CostUSDTicks is the provider's own statement of what this usage cost, + // in ticks of 1e-10 USD. Zero means "not reported" — only a few agents + // return it (xAI Grok Build does, via `_meta.usage.costUsdTicks`). + // + // It matters because a token-times-rate estimate cannot reproduce + // request-level pricing rules. xAI bills a request at 2x once its prompt + // reaches 200K tokens, and a usage record aggregates every model call in + // a turn — so the stored token counts cannot say which tier any single + // request hit. The provider's own figure already has that priced in. + CostUSDTicks int64 } +// CostUSDTicksPerUSD is the scale of the provider-reported cost unit: xAI +// reports whole ticks of 1e-10 USD, which keeps sub-cent turn costs exact in +// int64 all the way to the database instead of drifting through float64. +const CostUSDTicksPerUSD = 10_000_000_000 + // Result is the final outcome after an agent session completes. type Result struct { Status string // "completed", "failed", "aborted", "timeout", "cancelled" diff --git a/server/pkg/agent/grok.go b/server/pkg/agent/grok.go index 43fe06b6e..39f7bff82 100644 --- a/server/pkg/agent/grok.go +++ b/server/pkg/agent/grok.go @@ -443,10 +443,23 @@ func (b *grokBackend) Execute(ctx context.Context, prompt string, opts ExecOptio finalStatus = "aborted" finalError = "grok cancelled the prompt" } + // `session/load` carries no model id (only `session/new` + // does), so a resumed session with no configured model would + // otherwise bucket its whole spend under "unknown" — which + // prices at $0 because no pricing row matches. The turn's + // own `_meta.modelId` is authoritative; use it. + if effectiveModel == "" { + effectiveModel = pr.modelID + } c.usageMu.Lock() c.usage.InputTokens += pr.usage.InputTokens c.usage.OutputTokens += pr.usage.OutputTokens c.usage.CacheReadTokens += pr.usage.CacheReadTokens + // xAI prices the turn itself and reports the result here. + // Carrying it through is the only way the ≥200K long-context + // surcharge reaches the bill — token counts alone cannot + // reconstruct which tier a request hit. + c.usage.CostUSDTicks += pr.usage.CostUSDTicks c.usageMu.Unlock() default: } diff --git a/server/pkg/agent/grok_test.go b/server/pkg/agent/grok_test.go index f650cd4f0..2acbe6883 100644 --- a/server/pkg/agent/grok_test.go +++ b/server/pkg/agent/grok_test.go @@ -109,7 +109,7 @@ while IFS= read -r line; do if [ -n "$GROK_USAGE" ]; then # Match live Grok Build ACP (0.2.x): metering lives under result._meta, # not a top-level usage field or sessionUpdate=usage_update. - printf '{"jsonrpc":"2.0","id":%s,"result":{"stopReason":"end_turn","_meta":{"sessionId":"ses_new","modelId":"grok-4.5","inputTokens":120,"outputTokens":30,"cachedReadTokens":20,"usage":{"inputTokens":120,"outputTokens":30,"totalTokens":150,"cachedReadTokens":20,"modelCalls":1}}}}\n' "$id" + printf '{"jsonrpc":"2.0","id":%s,"result":{"stopReason":"end_turn","_meta":{"sessionId":"ses_new","modelId":"grok-4.5","inputTokens":120,"outputTokens":30,"cachedReadTokens":20,"usage":{"inputTokens":120,"outputTokens":30,"totalTokens":150,"cachedReadTokens":20,"modelCalls":1,"costUsdTicks":98765}}}}\n' "$id" else printf '{"jsonrpc":"2.0","id":%s,"result":{"stopReason":"end_turn"}}\n' "$id" fi @@ -616,6 +616,60 @@ func TestGrokPropagatesMCPAndUsage(t *testing.T) { if usage.InputTokens != 100 || usage.OutputTokens != 30 || usage.CacheReadTokens != 20 { t.Fatalf("unexpected usage: %+v", usage) } + // xAI's own price for the turn has to survive the whole backend, not just + // the parser: it is the only figure carrying the ≥200K prompt surcharge, + // and everything downstream falls back to a rate-table guess without it. + if usage.CostUSDTicks != 98765 { + t.Fatalf("cost ticks = %d, want 98765", usage.CostUSDTicks) + } +} + +// TestGrokAttributesUsageOnResumeWithoutConfiguredModel pins the model +// attribution on the resume path. `session/load` reports no model id (only +// `session/new` does), so when neither the agent nor the runtime pins a model +// the turn's own `_meta.modelId` is the only source left. Without it the whole +// run buckets under "unknown", which matches no pricing row and reports $0 +// spend for the task. +func TestGrokAttributesUsageOnResumeWithoutConfiguredModel(t *testing.T) { + t.Parallel() + tempDir := t.TempDir() + fakePath := filepath.Join(tempDir, "grok") + writeTestExecutable(t, fakePath, []byte(fakeGrokACPScript())) + backend, err := New("grok", Config{ + ExecutablePath: fakePath, + Logger: slog.Default(), + Env: map[string]string{"GROK_USAGE": "1"}, + }) + if err != nil { + t.Fatalf("new grok backend: %v", err) + } + // No Model: the daemon leaves it empty whenever neither the agent nor + // MULTICA_GROK_MODEL pins one (see daemon.go resolveModel). + session, err := backend.Execute(context.Background(), "continue", ExecOptions{ + ResumeSessionID: "ses_existing", + Timeout: 5 * time.Second, + }) + if err != nil { + t.Fatalf("execute: %v", err) + } + go func() { + for range session.Messages { + } + }() + result := <-session.Result + if result.Status != "completed" { + t.Fatalf("expected completed, got status=%q error=%q", result.Status, result.Error) + } + if _, unknown := result.Usage["unknown"]; unknown { + t.Fatalf("resumed usage fell back to the unpriced \"unknown\" bucket: %+v", result.Usage) + } + usage, ok := result.Usage["grok-4.5"] + if !ok { + t.Fatalf("usage missing grok-4.5 key: %+v", result.Usage) + } + if usage.InputTokens != 100 || usage.OutputTokens != 30 || usage.CacheReadTokens != 20 { + t.Fatalf("unexpected usage: %+v", usage) + } } func TestGrokTimeoutAndCancellation(t *testing.T) { diff --git a/server/pkg/agent/hermes.go b/server/pkg/agent/hermes.go index c3ce4a855..0367dc2d9 100644 --- a/server/pkg/agent/hermes.go +++ b/server/pkg/agent/hermes.go @@ -682,6 +682,11 @@ func waitForHermesPipeDrain(readerDone, stderrDone <-chan struct{}, timeout time type hermesPromptResult struct { stopReason string usage TokenUsage + // modelID is the model the agent actually billed this turn against, as + // reported on `result._meta.modelId`. Empty for agents that don't report + // it. Backends use it to attribute usage when the session handshake + // didn't surface a model id (see grok.go). + modelID string } type hermesClient struct { @@ -1115,6 +1120,7 @@ func (c *hermesClient) extractPromptResult(data json.RawMessage) { pr := hermesPromptResult{ stopReason: resp.StopReason, + modelID: parseACPModelIDFromMeta(resp.Meta), } if len(resp.Usage) > 0 && string(resp.Usage) != "null" { pr.usage = parseACPTokenUsage(resp.Usage) @@ -1164,6 +1170,28 @@ func parseACPTokenUsageFromMeta(meta json.RawMessage) TokenUsage { return parseACPTokenUsage(meta) } +// parseACPModelIDFromMeta pulls the model id off an ACP result `_meta` +// object. Grok Build stamps every turn with `_meta.modelId`, which is the +// only authoritative statement of what the turn was billed against — the +// session handshake reports a model id on `session/new` but NOT on +// `session/load`, so a resumed session has no other source. +func parseACPModelIDFromMeta(meta json.RawMessage) string { + if len(meta) == 0 || string(meta) == "null" { + return "" + } + var r struct { + ModelID string `json:"modelId"` + ModelIDSnake string `json:"model_id"` + } + if err := json.Unmarshal(meta, &r); err != nil { + return "" + } + if id := strings.TrimSpace(r.ModelID); id != "" { + return id + } + return strings.TrimSpace(r.ModelIDSnake) +} + func (c *hermesClient) handleNotification(raw map[string]json.RawMessage) { var method string _ = json.Unmarshal(raw["method"], &method) @@ -1638,6 +1666,9 @@ func (c *hermesClient) handleUsageUpdate(data json.RawMessage) { if usage.CacheWriteTokens > c.usage.CacheWriteTokens { c.usage.CacheWriteTokens = usage.CacheWriteTokens } + if usage.CostUSDTicks > c.usage.CostUSDTicks { + c.usage.CostUSDTicks = usage.CostUSDTicks + } c.usageMu.Unlock() } @@ -1665,6 +1696,10 @@ func parseACPTokenUsage(data json.RawMessage) TokenUsage { "cache_write_tokens", "cache_creation_input_tokens", ), + // The provider's own price for this turn, already inclusive of + // request-level pricing rules we cannot reconstruct from token + // counts (see TokenUsage.CostUSDTicks). + CostUSDTicks: acpUsageInt64(fields, "costUsdTicks", "cost_usd_ticks"), } return excludeACPCachedInput(usage, acpUsageInt64(fields, "totalTokens", "total_tokens")) } diff --git a/server/pkg/agent/hermes_test.go b/server/pkg/agent/hermes_test.go index fee47b925..c92b380df 100644 --- a/server/pkg/agent/hermes_test.go +++ b/server/pkg/agent/hermes_test.go @@ -5,6 +5,7 @@ import ( "encoding/json" "fmt" "log/slog" + "math" "os" "path/filepath" "runtime" @@ -1578,6 +1579,77 @@ func TestHermesClientExtractPromptResultMetaUsage(t *testing.T) { if got.usage.CacheReadTokens != 10880 { t.Errorf("cacheReadTokens: got %d, want 10880", got.usage.CacheReadTokens) } + // The turn stamps the model it billed against. A resumed session has no + // other source for this (session/load reports no model id), so losing it + // buckets the whole run under "unknown", which prices at $0. + if got.modelID != "grok-4.5" { + t.Errorf("modelID: got %q, want %q", got.modelID, "grok-4.5") + } + // xAI's own price for the turn, in ticks of 1e-10 USD. This is the only + // figure that carries request-level pricing rules (the ≥200K prompt + // surcharge); dropping it forces a rate-table guess downstream. + if got.usage.CostUSDTicks != 75360000 { + t.Errorf("costUsdTicks: got %d, want 75360000", got.usage.CostUSDTicks) + } + if usd := float64(got.usage.CostUSDTicks) / CostUSDTicksPerUSD; math.Abs(usd-0.007536) > 1e-12 { + t.Errorf("cost in USD: got %v, want 0.007536", usd) + } +} + +// TestParseACPTokenUsageCostIsOptional guards the common case: almost no agent +// reports a cost, and a zero must stay zero so the reader knows to estimate +// rather than recording a real $0 spend. +func TestParseACPTokenUsageCostIsOptional(t *testing.T) { + t.Parallel() + + for _, tc := range []struct { + name string + raw string + want int64 + }{ + {name: "camelCase", raw: `{"inputTokens":10,"costUsdTicks":4200}`, want: 4200}, + {name: "snake_case", raw: `{"input_tokens":10,"cost_usd_ticks":4200}`, want: 4200}, + {name: "absent", raw: `{"inputTokens":10,"outputTokens":2}`, want: 0}, + {name: "null", raw: `{"inputTokens":10,"costUsdTicks":null}`, want: 0}, + {name: "string number", raw: `{"inputTokens":10,"costUsdTicks":"4200"}`, want: 4200}, + } { + t.Run(tc.name, func(t *testing.T) { + t.Parallel() + if got := parseACPTokenUsage(json.RawMessage(tc.raw)).CostUSDTicks; got != tc.want { + t.Errorf("CostUSDTicks: got %d, want %d", got, tc.want) + } + }) + } +} + +// TestParseACPModelIDFromMeta covers the shapes the `_meta` model id can +// arrive in, and confirms a missing one degrades to empty rather than +// inventing an attribution. +func TestParseACPModelIDFromMeta(t *testing.T) { + t.Parallel() + + for _, tc := range []struct { + name string + raw string + want string + }{ + {name: "camelCase", raw: `{"modelId":"grok-4.5"}`, want: "grok-4.5"}, + {name: "snake_case", raw: `{"model_id":"grok-4.3"}`, want: "grok-4.3"}, + {name: "camelCase wins over snake_case", raw: `{"modelId":"grok-4.5","model_id":"grok-4.3"}`, want: "grok-4.5"}, + {name: "whitespace trimmed", raw: `{"modelId":" grok-4.5 "}`, want: "grok-4.5"}, + {name: "absent", raw: `{"sessionId":"ses_1"}`, want: ""}, + {name: "empty string", raw: `{"modelId":""}`, want: ""}, + {name: "null meta", raw: `null`, want: ""}, + {name: "malformed", raw: `{"modelId":`, want: ""}, + {name: "wrong type degrades to empty", raw: `{"modelId":42}`, want: ""}, + } { + t.Run(tc.name, func(t *testing.T) { + t.Parallel() + if got := parseACPModelIDFromMeta(json.RawMessage(tc.raw)); got != tc.want { + t.Errorf("parseACPModelIDFromMeta(%s): got %q, want %q", tc.raw, got, tc.want) + } + }) + } } // TestHermesClientExtractPromptResultMetaFlatOnly covers agents that put diff --git a/server/pkg/db/generated/models.go b/server/pkg/db/generated/models.go index 996a2baba..4223f57eb 100644 --- a/server/pkg/db/generated/models.go +++ b/server/pkg/db/generated/models.go @@ -938,6 +938,8 @@ type TaskUsage struct { CacheWriteTokens int64 `json:"cache_write_tokens"` CreatedAt pgtype.Timestamptz `json:"created_at"` UpdatedAt pgtype.Timestamptz `json:"updated_at"` + // Provider-reported cost in 1e-10 USD. NULL when the provider reports none; those rows are priced client-side from the static rate table. + CostUsdTicks pgtype.Int8 `json:"cost_usd_ticks"` } type TaskUsageHourly struct { @@ -955,6 +957,13 @@ type TaskUsageHourly struct { TaskCount int64 `json:"task_count"` EventCount int64 `json:"event_count"` UpdatedAt pgtype.Timestamptz `json:"updated_at"` + // Sum of provider-reported cost (1e-10 USD) over the rows in this bucket that had one; 0 when none did. + CostUsdTicks int64 `json:"cost_usd_ticks"` + // Input tokens from rows with no provider-reported cost — the portion still priced from the static rate table. NULL on buckets not yet recomputed since this column existed; readers COALESCE to input_tokens. + UncostedInputTokens pgtype.Int8 `json:"uncosted_input_tokens"` + UncostedOutputTokens pgtype.Int8 `json:"uncosted_output_tokens"` + UncostedCacheReadTokens pgtype.Int8 `json:"uncosted_cache_read_tokens"` + UncostedCacheWriteTokens pgtype.Int8 `json:"uncosted_cache_write_tokens"` } type TaskUsageHourlyDirty struct { diff --git a/server/pkg/db/generated/runtime_usage.sql.go b/server/pkg/db/generated/runtime_usage.sql.go index 43bd185c5..71b3f5b29 100644 --- a/server/pkg/db/generated/runtime_usage.sql.go +++ b/server/pkg/db/generated/runtime_usage.sql.go @@ -61,6 +61,11 @@ SELECT SUM(tu.output_tokens)::bigint AS output_tokens, SUM(tu.cache_read_tokens)::bigint AS cache_read_tokens, SUM(tu.cache_write_tokens)::bigint AS cache_write_tokens, + COALESCE(SUM(tu.cost_usd_ticks), 0)::bigint AS cost_usd_ticks, + COALESCE(SUM(tu.input_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_input_tokens, + COALESCE(SUM(tu.output_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_output_tokens, + COALESCE(SUM(tu.cache_read_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_cache_read_tokens, + COALESCE(SUM(tu.cache_write_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_cache_write_tokens, COUNT(DISTINCT tu.task_id)::int AS task_count FROM task_usage tu JOIN agent_task_queue atq ON atq.id = tu.task_id @@ -77,13 +82,18 @@ type GetRuntimeUsageByHourParams struct { } type GetRuntimeUsageByHourRow struct { - Hour int32 `json:"hour"` - Model string `json:"model"` - InputTokens int64 `json:"input_tokens"` - OutputTokens int64 `json:"output_tokens"` - CacheReadTokens int64 `json:"cache_read_tokens"` - CacheWriteTokens int64 `json:"cache_write_tokens"` - TaskCount int32 `json:"task_count"` + Hour int32 `json:"hour"` + Model string `json:"model"` + InputTokens int64 `json:"input_tokens"` + OutputTokens int64 `json:"output_tokens"` + CacheReadTokens int64 `json:"cache_read_tokens"` + CacheWriteTokens int64 `json:"cache_write_tokens"` + CostUsdTicks int64 `json:"cost_usd_ticks"` + UncostedInputTokens int64 `json:"uncosted_input_tokens"` + UncostedOutputTokens int64 `json:"uncosted_output_tokens"` + UncostedCacheReadTokens int64 `json:"uncosted_cache_read_tokens"` + UncostedCacheWriteTokens int64 `json:"uncosted_cache_write_tokens"` + TaskCount int32 `json:"task_count"` } // Per-(hour, model) token aggregates (hour ∈ 0..23) for a runtime since a @@ -110,6 +120,11 @@ func (q *Queries) GetRuntimeUsageByHour(ctx context.Context, arg GetRuntimeUsage &i.OutputTokens, &i.CacheReadTokens, &i.CacheWriteTokens, + &i.CostUsdTicks, + &i.UncostedInputTokens, + &i.UncostedOutputTokens, + &i.UncostedCacheReadTokens, + &i.UncostedCacheWriteTokens, &i.TaskCount, ); err != nil { return nil, err @@ -130,7 +145,12 @@ SELECT SUM(input_tokens)::bigint AS input_tokens, SUM(output_tokens)::bigint AS output_tokens, SUM(cache_read_tokens)::bigint AS cache_read_tokens, - SUM(cache_write_tokens)::bigint AS cache_write_tokens + SUM(cache_write_tokens)::bigint AS cache_write_tokens, + SUM(cost_usd_ticks)::bigint AS cost_usd_ticks, + SUM(COALESCE(uncosted_input_tokens, input_tokens))::bigint AS uncosted_input_tokens, + SUM(COALESCE(uncosted_output_tokens, output_tokens))::bigint AS uncosted_output_tokens, + SUM(COALESCE(uncosted_cache_read_tokens, cache_read_tokens))::bigint AS uncosted_cache_read_tokens, + SUM(COALESCE(uncosted_cache_write_tokens, cache_write_tokens))::bigint AS uncosted_cache_write_tokens FROM task_usage_hourly WHERE runtime_id = $1 AND bucket_hour >= $3::timestamptz @@ -145,13 +165,18 @@ type ListRuntimeUsageParams struct { } type ListRuntimeUsageRow struct { - Date pgtype.Date `json:"date"` - Provider string `json:"provider"` - Model string `json:"model"` - InputTokens int64 `json:"input_tokens"` - OutputTokens int64 `json:"output_tokens"` - CacheReadTokens int64 `json:"cache_read_tokens"` - CacheWriteTokens int64 `json:"cache_write_tokens"` + Date pgtype.Date `json:"date"` + Provider string `json:"provider"` + Model string `json:"model"` + InputTokens int64 `json:"input_tokens"` + OutputTokens int64 `json:"output_tokens"` + CacheReadTokens int64 `json:"cache_read_tokens"` + CacheWriteTokens int64 `json:"cache_write_tokens"` + CostUsdTicks int64 `json:"cost_usd_ticks"` + UncostedInputTokens int64 `json:"uncosted_input_tokens"` + UncostedOutputTokens int64 `json:"uncosted_output_tokens"` + UncostedCacheReadTokens int64 `json:"uncosted_cache_read_tokens"` + UncostedCacheWriteTokens int64 `json:"uncosted_cache_write_tokens"` } // Reads from the UTC-bucketed `task_usage_hourly` rollup table, @@ -182,6 +207,11 @@ func (q *Queries) ListRuntimeUsage(ctx context.Context, arg ListRuntimeUsagePara &i.OutputTokens, &i.CacheReadTokens, &i.CacheWriteTokens, + &i.CostUsdTicks, + &i.UncostedInputTokens, + &i.UncostedOutputTokens, + &i.UncostedCacheReadTokens, + &i.UncostedCacheWriteTokens, ); err != nil { return nil, err } @@ -202,6 +232,11 @@ SELECT SUM(tu.output_tokens)::bigint AS output_tokens, SUM(tu.cache_read_tokens)::bigint AS cache_read_tokens, SUM(tu.cache_write_tokens)::bigint AS cache_write_tokens, + COALESCE(SUM(tu.cost_usd_ticks), 0)::bigint AS cost_usd_ticks, + COALESCE(SUM(tu.input_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_input_tokens, + COALESCE(SUM(tu.output_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_output_tokens, + COALESCE(SUM(tu.cache_read_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_cache_read_tokens, + COALESCE(SUM(tu.cache_write_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_cache_write_tokens, COUNT(DISTINCT tu.task_id)::int AS task_count FROM task_usage tu JOIN agent_task_queue atq ON atq.id = tu.task_id @@ -217,14 +252,19 @@ type ListRuntimeUsageByAgentParams struct { } type ListRuntimeUsageByAgentRow struct { - AgentID pgtype.UUID `json:"agent_id"` - Provider string `json:"provider"` - Model string `json:"model"` - InputTokens int64 `json:"input_tokens"` - OutputTokens int64 `json:"output_tokens"` - CacheReadTokens int64 `json:"cache_read_tokens"` - CacheWriteTokens int64 `json:"cache_write_tokens"` - TaskCount int32 `json:"task_count"` + AgentID pgtype.UUID `json:"agent_id"` + Provider string `json:"provider"` + Model string `json:"model"` + InputTokens int64 `json:"input_tokens"` + OutputTokens int64 `json:"output_tokens"` + CacheReadTokens int64 `json:"cache_read_tokens"` + CacheWriteTokens int64 `json:"cache_write_tokens"` + CostUsdTicks int64 `json:"cost_usd_ticks"` + UncostedInputTokens int64 `json:"uncosted_input_tokens"` + UncostedOutputTokens int64 `json:"uncosted_output_tokens"` + UncostedCacheReadTokens int64 `json:"uncosted_cache_read_tokens"` + UncostedCacheWriteTokens int64 `json:"uncosted_cache_write_tokens"` + TaskCount int32 `json:"task_count"` } // Per-(agent, provider, model) token aggregates for a runtime since a cutoff. Powers @@ -255,6 +295,11 @@ func (q *Queries) ListRuntimeUsageByAgent(ctx context.Context, arg ListRuntimeUs &i.OutputTokens, &i.CacheReadTokens, &i.CacheWriteTokens, + &i.CostUsdTicks, + &i.UncostedInputTokens, + &i.UncostedOutputTokens, + &i.UncostedCacheReadTokens, + &i.UncostedCacheWriteTokens, &i.TaskCount, ); err != nil { return nil, err diff --git a/server/pkg/db/generated/task_usage.sql.go b/server/pkg/db/generated/task_usage.sql.go index 707283191..fe08dce77 100644 --- a/server/pkg/db/generated/task_usage.sql.go +++ b/server/pkg/db/generated/task_usage.sql.go @@ -17,6 +17,11 @@ SELECT COALESCE(SUM(tu.output_tokens), 0)::bigint AS total_output_tokens, COALESCE(SUM(tu.cache_read_tokens), 0)::bigint AS total_cache_read_tokens, COALESCE(SUM(tu.cache_write_tokens), 0)::bigint AS total_cache_write_tokens, + COALESCE(SUM(tu.cost_usd_ticks), 0)::bigint AS total_cost_usd_ticks, + COALESCE(SUM(tu.input_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_input_tokens, + COALESCE(SUM(tu.output_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_output_tokens, + COALESCE(SUM(tu.cache_read_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_cache_read_tokens, + COALESCE(SUM(tu.cache_write_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_cache_write_tokens, COUNT(DISTINCT tu.task_id)::int AS task_count FROM task_usage tu JOIN agent_task_queue atq ON atq.id = tu.task_id @@ -24,11 +29,16 @@ WHERE atq.issue_id = $1 ` type GetIssueUsageSummaryRow struct { - TotalInputTokens int64 `json:"total_input_tokens"` - TotalOutputTokens int64 `json:"total_output_tokens"` - TotalCacheReadTokens int64 `json:"total_cache_read_tokens"` - TotalCacheWriteTokens int64 `json:"total_cache_write_tokens"` - TaskCount int32 `json:"task_count"` + TotalInputTokens int64 `json:"total_input_tokens"` + TotalOutputTokens int64 `json:"total_output_tokens"` + TotalCacheReadTokens int64 `json:"total_cache_read_tokens"` + TotalCacheWriteTokens int64 `json:"total_cache_write_tokens"` + TotalCostUsdTicks int64 `json:"total_cost_usd_ticks"` + UncostedInputTokens int64 `json:"uncosted_input_tokens"` + UncostedOutputTokens int64 `json:"uncosted_output_tokens"` + UncostedCacheReadTokens int64 `json:"uncosted_cache_read_tokens"` + UncostedCacheWriteTokens int64 `json:"uncosted_cache_write_tokens"` + TaskCount int32 `json:"task_count"` } func (q *Queries) GetIssueUsageSummary(ctx context.Context, issueID pgtype.UUID) (GetIssueUsageSummaryRow, error) { @@ -39,13 +49,18 @@ func (q *Queries) GetIssueUsageSummary(ctx context.Context, issueID pgtype.UUID) &i.TotalOutputTokens, &i.TotalCacheReadTokens, &i.TotalCacheWriteTokens, + &i.TotalCostUsdTicks, + &i.UncostedInputTokens, + &i.UncostedOutputTokens, + &i.UncostedCacheReadTokens, + &i.UncostedCacheWriteTokens, &i.TaskCount, ) return i, err } const getTaskUsage = `-- name: GetTaskUsage :many -SELECT id, task_id, provider, model, input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, created_at, updated_at FROM task_usage +SELECT id, task_id, provider, model, input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, created_at, updated_at, cost_usd_ticks FROM task_usage WHERE task_id = $1 ORDER BY model ` @@ -70,6 +85,7 @@ func (q *Queries) GetTaskUsage(ctx context.Context, taskID pgtype.UUID) ([]TaskU &i.CacheWriteTokens, &i.CreatedAt, &i.UpdatedAt, + &i.CostUsdTicks, ); err != nil { return nil, err } @@ -237,6 +253,11 @@ SELECT SUM(output_tokens)::bigint AS output_tokens, SUM(cache_read_tokens)::bigint AS cache_read_tokens, SUM(cache_write_tokens)::bigint AS cache_write_tokens, + SUM(cost_usd_ticks)::bigint AS cost_usd_ticks, + SUM(COALESCE(uncosted_input_tokens, input_tokens))::bigint AS uncosted_input_tokens, + SUM(COALESCE(uncosted_output_tokens, output_tokens))::bigint AS uncosted_output_tokens, + SUM(COALESCE(uncosted_cache_read_tokens, cache_read_tokens))::bigint AS uncosted_cache_read_tokens, + SUM(COALESCE(uncosted_cache_write_tokens, cache_write_tokens))::bigint AS uncosted_cache_write_tokens, SUM(task_count)::int AS task_count FROM task_usage_hourly WHERE workspace_id = $1 @@ -253,14 +274,19 @@ type ListDashboardUsageByAgentParams struct { } type ListDashboardUsageByAgentRow struct { - AgentID pgtype.UUID `json:"agent_id"` - Provider string `json:"provider"` - Model string `json:"model"` - InputTokens int64 `json:"input_tokens"` - OutputTokens int64 `json:"output_tokens"` - CacheReadTokens int64 `json:"cache_read_tokens"` - CacheWriteTokens int64 `json:"cache_write_tokens"` - TaskCount int32 `json:"task_count"` + AgentID pgtype.UUID `json:"agent_id"` + Provider string `json:"provider"` + Model string `json:"model"` + InputTokens int64 `json:"input_tokens"` + OutputTokens int64 `json:"output_tokens"` + CacheReadTokens int64 `json:"cache_read_tokens"` + CacheWriteTokens int64 `json:"cache_write_tokens"` + CostUsdTicks int64 `json:"cost_usd_ticks"` + UncostedInputTokens int64 `json:"uncosted_input_tokens"` + UncostedOutputTokens int64 `json:"uncosted_output_tokens"` + UncostedCacheReadTokens int64 `json:"uncosted_cache_read_tokens"` + UncostedCacheWriteTokens int64 `json:"uncosted_cache_write_tokens"` + TaskCount int32 `json:"task_count"` } // Per-(agent, provider, model) token aggregates from `task_usage_hourly`. No @@ -294,6 +320,11 @@ func (q *Queries) ListDashboardUsageByAgent(ctx context.Context, arg ListDashboa &i.OutputTokens, &i.CacheReadTokens, &i.CacheWriteTokens, + &i.CostUsdTicks, + &i.UncostedInputTokens, + &i.UncostedOutputTokens, + &i.UncostedCacheReadTokens, + &i.UncostedCacheWriteTokens, &i.TaskCount, ); err != nil { return nil, err @@ -315,6 +346,11 @@ SELECT SUM(output_tokens)::bigint AS output_tokens, SUM(cache_read_tokens)::bigint AS cache_read_tokens, SUM(cache_write_tokens)::bigint AS cache_write_tokens, + SUM(cost_usd_ticks)::bigint AS cost_usd_ticks, + SUM(COALESCE(uncosted_input_tokens, input_tokens))::bigint AS uncosted_input_tokens, + SUM(COALESCE(uncosted_output_tokens, output_tokens))::bigint AS uncosted_output_tokens, + SUM(COALESCE(uncosted_cache_read_tokens, cache_read_tokens))::bigint AS uncosted_cache_read_tokens, + SUM(COALESCE(uncosted_cache_write_tokens, cache_write_tokens))::bigint AS uncosted_cache_write_tokens, SUM(task_count)::int AS task_count FROM task_usage_hourly WHERE workspace_id = $1 @@ -332,14 +368,19 @@ type ListDashboardUsageDailyParams struct { } type ListDashboardUsageDailyRow struct { - Date pgtype.Date `json:"date"` - Provider string `json:"provider"` - Model string `json:"model"` - InputTokens int64 `json:"input_tokens"` - OutputTokens int64 `json:"output_tokens"` - CacheReadTokens int64 `json:"cache_read_tokens"` - CacheWriteTokens int64 `json:"cache_write_tokens"` - TaskCount int32 `json:"task_count"` + Date pgtype.Date `json:"date"` + Provider string `json:"provider"` + Model string `json:"model"` + InputTokens int64 `json:"input_tokens"` + OutputTokens int64 `json:"output_tokens"` + CacheReadTokens int64 `json:"cache_read_tokens"` + CacheWriteTokens int64 `json:"cache_write_tokens"` + CostUsdTicks int64 `json:"cost_usd_ticks"` + UncostedInputTokens int64 `json:"uncosted_input_tokens"` + UncostedOutputTokens int64 `json:"uncosted_output_tokens"` + UncostedCacheReadTokens int64 `json:"uncosted_cache_read_tokens"` + UncostedCacheWriteTokens int64 `json:"uncosted_cache_write_tokens"` + TaskCount int32 `json:"task_count"` } // Daily per-(date, provider, model) token aggregates for the workspace, served @@ -381,6 +422,11 @@ func (q *Queries) ListDashboardUsageDaily(ctx context.Context, arg ListDashboard &i.OutputTokens, &i.CacheReadTokens, &i.CacheWriteTokens, + &i.CostUsdTicks, + &i.UncostedInputTokens, + &i.UncostedOutputTokens, + &i.UncostedCacheReadTokens, + &i.UncostedCacheWriteTokens, &i.TaskCount, ); err != nil { return nil, err @@ -394,14 +440,15 @@ func (q *Queries) ListDashboardUsageDaily(ctx context.Context, arg ListDashboard } const upsertTaskUsage = `-- name: UpsertTaskUsage :exec -INSERT INTO task_usage (task_id, provider, model, input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, updated_at) -VALUES ($1, $2, $3, $4, $5, $6, $7, now()) +INSERT INTO task_usage (task_id, provider, model, input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, cost_usd_ticks, updated_at) +VALUES ($1, $2, $3, $4, $5, $6, $7, $8, now()) ON CONFLICT (task_id, provider, model) DO UPDATE SET input_tokens = EXCLUDED.input_tokens, output_tokens = EXCLUDED.output_tokens, cache_read_tokens = EXCLUDED.cache_read_tokens, cache_write_tokens = EXCLUDED.cache_write_tokens, + cost_usd_ticks = EXCLUDED.cost_usd_ticks, updated_at = now() ` @@ -413,12 +460,16 @@ type UpsertTaskUsageParams struct { OutputTokens int64 `json:"output_tokens"` CacheReadTokens int64 `json:"cache_read_tokens"` CacheWriteTokens int64 `json:"cache_write_tokens"` + CostUsdTicks pgtype.Int8 `json:"cost_usd_ticks"` } // Bumps `updated_at` on INSERT and on conflict so the hourly-rollup worker // detects the row as dirty and re-aggregates its bucket. // Without the conflict-side bump, a correction to historical token counts // would never propagate to the rollup. +// cost_usd_ticks is the provider's own price for this usage (1e-10 USD), NULL +// when it reports none. It is overwritten like the token counters so a +// corrected report replaces the previous figure rather than accumulating. func (q *Queries) UpsertTaskUsage(ctx context.Context, arg UpsertTaskUsageParams) error { _, err := q.db.Exec(ctx, upsertTaskUsage, arg.TaskID, @@ -428,6 +479,7 @@ func (q *Queries) UpsertTaskUsage(ctx context.Context, arg UpsertTaskUsageParams arg.OutputTokens, arg.CacheReadTokens, arg.CacheWriteTokens, + arg.CostUsdTicks, ) return err } diff --git a/server/pkg/db/queries/runtime_usage.sql b/server/pkg/db/queries/runtime_usage.sql index ea5550eb3..315f0a8be 100644 --- a/server/pkg/db/queries/runtime_usage.sql +++ b/server/pkg/db/queries/runtime_usage.sql @@ -17,7 +17,12 @@ SELECT SUM(input_tokens)::bigint AS input_tokens, SUM(output_tokens)::bigint AS output_tokens, SUM(cache_read_tokens)::bigint AS cache_read_tokens, - SUM(cache_write_tokens)::bigint AS cache_write_tokens + SUM(cache_write_tokens)::bigint AS cache_write_tokens, + SUM(cost_usd_ticks)::bigint AS cost_usd_ticks, + SUM(COALESCE(uncosted_input_tokens, input_tokens))::bigint AS uncosted_input_tokens, + SUM(COALESCE(uncosted_output_tokens, output_tokens))::bigint AS uncosted_output_tokens, + SUM(COALESCE(uncosted_cache_read_tokens, cache_read_tokens))::bigint AS uncosted_cache_read_tokens, + SUM(COALESCE(uncosted_cache_write_tokens, cache_write_tokens))::bigint AS uncosted_cache_write_tokens FROM task_usage_hourly WHERE runtime_id = $1 AND bucket_hour >= sqlc.arg('since')::timestamptz @@ -55,6 +60,11 @@ SELECT SUM(tu.output_tokens)::bigint AS output_tokens, SUM(tu.cache_read_tokens)::bigint AS cache_read_tokens, SUM(tu.cache_write_tokens)::bigint AS cache_write_tokens, + COALESCE(SUM(tu.cost_usd_ticks), 0)::bigint AS cost_usd_ticks, + COALESCE(SUM(tu.input_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_input_tokens, + COALESCE(SUM(tu.output_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_output_tokens, + COALESCE(SUM(tu.cache_read_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_cache_read_tokens, + COALESCE(SUM(tu.cache_write_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_cache_write_tokens, COUNT(DISTINCT tu.task_id)::int AS task_count FROM task_usage tu JOIN agent_task_queue atq ON atq.id = tu.task_id @@ -79,6 +89,11 @@ SELECT SUM(tu.output_tokens)::bigint AS output_tokens, SUM(tu.cache_read_tokens)::bigint AS cache_read_tokens, SUM(tu.cache_write_tokens)::bigint AS cache_write_tokens, + COALESCE(SUM(tu.cost_usd_ticks), 0)::bigint AS cost_usd_ticks, + COALESCE(SUM(tu.input_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_input_tokens, + COALESCE(SUM(tu.output_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_output_tokens, + COALESCE(SUM(tu.cache_read_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_cache_read_tokens, + COALESCE(SUM(tu.cache_write_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_cache_write_tokens, COUNT(DISTINCT tu.task_id)::int AS task_count FROM task_usage tu JOIN agent_task_queue atq ON atq.id = tu.task_id diff --git a/server/pkg/db/queries/task_usage.sql b/server/pkg/db/queries/task_usage.sql index f697deff7..8e86a0262 100644 --- a/server/pkg/db/queries/task_usage.sql +++ b/server/pkg/db/queries/task_usage.sql @@ -3,14 +3,18 @@ -- detects the row as dirty and re-aggregates its bucket. -- Without the conflict-side bump, a correction to historical token counts -- would never propagate to the rollup. -INSERT INTO task_usage (task_id, provider, model, input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, updated_at) -VALUES ($1, $2, $3, $4, $5, $6, $7, now()) +-- cost_usd_ticks is the provider's own price for this usage (1e-10 USD), NULL +-- when it reports none. It is overwritten like the token counters so a +-- corrected report replaces the previous figure rather than accumulating. +INSERT INTO task_usage (task_id, provider, model, input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, cost_usd_ticks, updated_at) +VALUES ($1, $2, $3, $4, $5, $6, $7, sqlc.narg('cost_usd_ticks'), now()) ON CONFLICT (task_id, provider, model) DO UPDATE SET input_tokens = EXCLUDED.input_tokens, output_tokens = EXCLUDED.output_tokens, cache_read_tokens = EXCLUDED.cache_read_tokens, cache_write_tokens = EXCLUDED.cache_write_tokens, + cost_usd_ticks = EXCLUDED.cost_usd_ticks, updated_at = now(); -- name: GetTaskUsage :many @@ -24,6 +28,11 @@ SELECT COALESCE(SUM(tu.output_tokens), 0)::bigint AS total_output_tokens, COALESCE(SUM(tu.cache_read_tokens), 0)::bigint AS total_cache_read_tokens, COALESCE(SUM(tu.cache_write_tokens), 0)::bigint AS total_cache_write_tokens, + COALESCE(SUM(tu.cost_usd_ticks), 0)::bigint AS total_cost_usd_ticks, + COALESCE(SUM(tu.input_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_input_tokens, + COALESCE(SUM(tu.output_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_output_tokens, + COALESCE(SUM(tu.cache_read_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_cache_read_tokens, + COALESCE(SUM(tu.cache_write_tokens) FILTER (WHERE tu.cost_usd_ticks IS NULL), 0)::bigint AS uncosted_cache_write_tokens, COUNT(DISTINCT tu.task_id)::int AS task_count FROM task_usage tu JOIN agent_task_queue atq ON atq.id = tu.task_id @@ -55,6 +64,11 @@ SELECT SUM(output_tokens)::bigint AS output_tokens, SUM(cache_read_tokens)::bigint AS cache_read_tokens, SUM(cache_write_tokens)::bigint AS cache_write_tokens, + SUM(cost_usd_ticks)::bigint AS cost_usd_ticks, + SUM(COALESCE(uncosted_input_tokens, input_tokens))::bigint AS uncosted_input_tokens, + SUM(COALESCE(uncosted_output_tokens, output_tokens))::bigint AS uncosted_output_tokens, + SUM(COALESCE(uncosted_cache_read_tokens, cache_read_tokens))::bigint AS uncosted_cache_read_tokens, + SUM(COALESCE(uncosted_cache_write_tokens, cache_write_tokens))::bigint AS uncosted_cache_write_tokens, SUM(task_count)::int AS task_count FROM task_usage_hourly WHERE workspace_id = $1 @@ -86,6 +100,11 @@ SELECT SUM(output_tokens)::bigint AS output_tokens, SUM(cache_read_tokens)::bigint AS cache_read_tokens, SUM(cache_write_tokens)::bigint AS cache_write_tokens, + SUM(cost_usd_ticks)::bigint AS cost_usd_ticks, + SUM(COALESCE(uncosted_input_tokens, input_tokens))::bigint AS uncosted_input_tokens, + SUM(COALESCE(uncosted_output_tokens, output_tokens))::bigint AS uncosted_output_tokens, + SUM(COALESCE(uncosted_cache_read_tokens, cache_read_tokens))::bigint AS uncosted_cache_read_tokens, + SUM(COALESCE(uncosted_cache_write_tokens, cache_write_tokens))::bigint AS uncosted_cache_write_tokens, SUM(task_count)::int AS task_count FROM task_usage_hourly WHERE workspace_id = $1