Files
multica/server/pkg/db/queries/github_snapshot.sql
Bohan Jiang ecce589867 MUL-5265: GitHub API-snapshot PR cards — CI status + mergeability (#5889)
* feat(github): API-snapshot PR cards — CI status + mergeability (MUL-5265)

Fetch each linked PR's CI checks and mergeability from the GitHub GraphQL
API as the single source of truth (Plan C). Webhooks, page visits and a
bounded TTL sweep are refresh triggers only; nothing is inferred from
webhook payloads anymore.

Backend (server/internal/integrations/ghsnapshot):
- installation-token cache + GraphQL client (private key / tokens never logged)
- one paginated pullRequest query -> normalized per-check snapshot
- outbound queue: (installation,repo,PR) dedup + single in-flight per PR,
  bounded worker pool, Retry-After / rate-limit backoff, jitter
- head-SHA-guarded atomic batch replace (a slow response for an old head
  can never overwrite a newer head's snapshot)
- bounded chase window (30s->5m, stops on terminal/closed) + page-visit +
  TTL refresh; clean degradation when no App private key is configured

Removes the old suite-level webhook aggregation display path (query +
handlers + tests). check_suite / check_run / status are now pure triggers.

Frontend: PR card shows two independent tri-state elements (CI status +
mergeability). "Ready to merge" only when merge state is clean; no-checks
and unknown-mergeable never assert a positive verdict; progress strip
removed; four locales; stale marker.

Docs: github-integration + environment-variables (four languages) — now
required App private key, read-only Checks/Commit-statuses permissions,
new event subscriptions, capability boundaries and troubleshooting.

Co-authored-by: multica-agent <github@multica.ai>

* fix(github): address PR snapshot review blockers

Co-authored-by: multica-agent <github@multica.ai>

* fix(github): bound snapshot refresh scheduling

Co-authored-by: multica-agent <github@multica.ai>

* fix(github): concurrent check-run index migration + singleflight token mint

Address Elon's third-round review on the MUL-5265 PR snapshot pipeline.

Must-fix — migration built a non-concurrent index. The
github_pull_request_check_run table declared PRIMARY KEY (pr_id, ordinal)
inside CREATE TABLE, which builds a unique index synchronously and violates
the repo rule that every migration-created index (including on a new table)
use CREATE UNIQUE INDEX CONCURRENTLY in its own single-statement file. Split:
222 now creates the table without a primary key; new 223 adds the
(pr_id, ordinal) unique index CONCURRENTLY. The atomic delete-all/insert
write path already guarantees ordinal uniqueness, so a plain unique index is
sufficient; the index also serves the pr_id-prefix list aggregation and the
workspace/PR cleanup deletes.

Nit — token mint now singleflights per installation. installationToken
released the lock before minting, so the N workers of one installation could
mint N tokens on a cold cache or a simultaneous renew. Concurrent callers for
the same installation are now collapsed via singleflight into one HTTP mint;
added a -race concurrent-mint test asserting a single mint under 16 callers.

Verified: fresh DB migrates through 223 (table has no PK, concurrent unique
index present); ghsnapshot suite + new test pass under -race; migration lint
and handler github/workspace-delete tests pass; sqlc produced no diff;
go build / vet / gofmt / git diff --check clean.

Co-authored-by: multica-agent <github@multica.ai>

---------

Co-authored-by: Bohan-J <bohan@devv.ai>
Co-authored-by: multica-agent <github@multica.ai>
2026-07-24 18:30:20 +08:00

95 lines
4.2 KiB
SQL

-- =====================
-- GitHub API snapshot (MUL-5265, Plan C)
--
-- These queries back the API-snapshot refresh pipeline. The GitHub GraphQL
-- response is the single source of truth; each successful fetch is written as
-- one atomic batch replace (guarded update of the PR row + full replace of the
-- per-check rows) inside a single transaction.
-- =====================
-- name: ListGitHubPRRowsByAddress :many
-- One (installation, owner, repo, number) address can map to several
-- github_pull_request rows — the same installation can be bound to multiple
-- workspaces (#4823/#4855), each mirroring its own row. A single API fetch is
-- applied to every matching row (each guarded by its own head_sha).
SELECT id, workspace_id, head_sha, state
FROM github_pull_request
WHERE installation_id = $1 AND repo_owner = $2 AND repo_name = $3 AND pr_number = $4;
-- name: UpdateGitHubPRSnapshot :execrows
-- Head-SHA anti-stale write (acceptance criterion 1): the snapshot is written
-- only when the row's current head_sha still equals the head the snapshot was
-- fetched for. If the head advanced (a newer push landed while this request was
-- in flight, mirrored by the pull_request webhook), 0 rows are updated and the
-- caller discards the whole response — the per-check replace is skipped too.
UPDATE github_pull_request
SET api_mergeable = sqlc.narg('api_mergeable'),
api_merge_state_status = sqlc.narg('api_merge_state_status'),
checks_rollup_state = sqlc.narg('checks_rollup_state'),
snapshot_head_sha = sqlc.arg('head_sha'),
snapshot_fetched_at = sqlc.arg('fetched_at'),
updated_at = now()
WHERE id = sqlc.arg('pr_id') AND head_sha = sqlc.arg('head_sha');
-- name: DeleteGitHubPRCheckRuns :exec
-- First half of the atomic per-check replace. Runs inside the same transaction
-- as UpdateGitHubPRSnapshot and the inserts below.
DELETE FROM github_pull_request_check_run WHERE pr_id = $1;
-- name: InsertGitHubPRCheckRun :exec
INSERT INTO github_pull_request_check_run (
pr_id, head_sha, ordinal, name, status, conclusion, details_url, is_status_context
) VALUES (
$1, $2, $3, $4, $5, sqlc.narg('conclusion'), sqlc.narg('details_url'), $6
);
-- name: ListStaleUndecidedGitHubPRs :many
-- TTL / safety-net sweep source. Returns distinct addresses of open/draft PRs
-- whose snapshot is both stale and undecided. A decided snapshot leaves the
-- periodic refresh set; later webhook or view activity can still refresh it.
-- The caller advances an address cursor after each bounded batch. Rows after
-- the cursor sort first, followed by a wrap to the start, so even perpetually
-- failing addresses cannot pin the same first LIMIT rows forever.
WITH candidates AS (
SELECT installation_id, repo_owner, repo_name, pr_number
FROM github_pull_request AS pr
WHERE state IN ('open', 'draft')
AND (snapshot_fetched_at IS NULL OR snapshot_fetched_at < sqlc.arg('older_than'))
AND (
snapshot_fetched_at IS NULL
OR api_mergeable IS NULL
OR api_mergeable = 'UNKNOWN'
OR checks_rollup_state IN ('PENDING', 'EXPECTED')
OR EXISTS (
SELECT 1
FROM github_pull_request_check_run AS cr
WHERE cr.pr_id = pr.id AND cr.status <> 'completed'
)
)
GROUP BY installation_id, repo_owner, repo_name, pr_number
)
SELECT installation_id, repo_owner, repo_name, pr_number
FROM candidates
ORDER BY (
ROW(installation_id, repo_owner, repo_name, pr_number) >
ROW(
sqlc.arg('after_installation_id')::BIGINT,
sqlc.arg('after_repo_owner')::TEXT,
sqlc.arg('after_repo_name')::TEXT,
sqlc.arg('after_pr_number')::INTEGER
)
) DESC,
installation_id, repo_owner, repo_name, pr_number
LIMIT sqlc.arg('max_rows');
-- name: ListGitHubPRNumbersByHeadSHA :many
-- Resolves a commit SHA to the PR numbers whose head it is. `status` webhook
-- events (legacy commit statuses) carry a SHA + repo but no PR number, so we
-- map back through the mirrored head_sha to find which PR(s) to refresh.
SELECT DISTINCT pr_number
FROM github_pull_request
WHERE installation_id = $1 AND repo_owner = $2 AND repo_name = $3 AND head_sha = $4;
-- name: GetGitHubPullRequestByID :one
SELECT * FROM github_pull_request WHERE id = $1;