mirror of
https://github.com/multica-ai/multica.git
synced 2026-08-03 19:20:07 +02:00
perf(ci): cache turbo task results across runs (MUL-5347)
Every CI run rebuilt all 16 frontend tasks cold -- turbo reported `Remote caching disabled` / `Cached: 0 cached, 16 total` on every run, because only the pnpm store was cached. Persist turbo's filesystem cache with actions/cache instead. No Vercel remote-cache credentials exist in this repo, so this is the local cache keyed per job. Measured, simulating a fresh checkout against a warm cache: frontend-build 54.7s -> 184ms (12/12 cached) frontend-test 115.7s -> 50ms (7/7 cached) Cache footprint is 20MB for the build job and ~60KB for the test job -- `test` declares no outputs, so turbo stores exit codes and logs, yet a hit still skips the most expensive task in the graph. Turning the cache on promotes two latent hashing bugs into real ones, so both are fixed here: `build` declared narrow `inputs` globs that matched neither `content/**/*.mdx` (compiled by fumadocs-mdx during the build), `public/**` assets, nor `package.json` / `tsconfig.json`. Editing any of them left the task hash byte-identical, which is inert when nothing is ever restored and a stale-build bug the moment something is. Dropped in favour of turbo's default input set. `test` regains its `^typecheck` edge, reverting part of the previous commit. That change was justified on the basis that the edge only imposed ordering, which was true without a cache: a turbo task hash covers a workspace dependency's sources only if a task edge reaches them, so with no edge, editing packages/views or packages/ui left `@multica/web#test` and `@multica/desktop#test` unchanged in hash and a cached pass would be replayed over changed code. `typecheck` is the only task every package defines, so it is the edge that closes the gap; `^test` would miss packages/ui, which has no test script. `lint` keeps no edge -- eslint here is not type-aware, so it cannot observe a dependency's sources. Also adds .github/workflows/ci.yml to globalDependencies: it pins the Node version, and without it a toolchain bump would replay results produced by the old runtime behind a green check. Co-authored-by: multica-agent <github@multica.ai>
This commit is contained in:
33
.github/workflows/ci.yml
vendored
33
.github/workflows/ci.yml
vendored
@@ -90,6 +90,9 @@ jobs:
|
||||
frontend-build:
|
||||
needs: changes
|
||||
runs-on: ubuntu-latest
|
||||
env:
|
||||
# Pin turbo's filesystem cache somewhere actions/cache can address.
|
||||
TURBO_CACHE_DIR: .turbo/cache
|
||||
steps:
|
||||
- name: Checkout
|
||||
if: ${{ needs.changes.outputs.frontend == 'true' }}
|
||||
@@ -110,6 +113,21 @@ jobs:
|
||||
if: ${{ needs.changes.outputs.frontend == 'true' }}
|
||||
run: pnpm install
|
||||
|
||||
# Cache entries are immutable, so the key carries the commit SHA to make
|
||||
# every run publish a fresh one and `restore-keys` falls back to the most
|
||||
# recent prefix match. The two frontend jobs run disjoint task sets, so
|
||||
# they get their own prefixes rather than racing to save one key.
|
||||
# GitHub scopes caches by branch: a PR reads main's entries (so unchanged
|
||||
# tasks hit on the first push) and writes its own (so re-pushes hit too).
|
||||
- name: Restore turbo cache
|
||||
if: ${{ needs.changes.outputs.frontend == 'true' }}
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: .turbo/cache
|
||||
key: turbo-build-${{ runner.os }}-${{ github.sha }}
|
||||
restore-keys: |
|
||||
turbo-build-${{ runner.os }}-
|
||||
|
||||
- name: Test self-host env derivation
|
||||
if: ${{ needs.changes.outputs.frontend == 'true' }}
|
||||
run: bash scripts/selfhost-config.test.sh
|
||||
@@ -136,6 +154,8 @@ jobs:
|
||||
frontend-test:
|
||||
needs: changes
|
||||
runs-on: ubuntu-latest
|
||||
env:
|
||||
TURBO_CACHE_DIR: .turbo/cache
|
||||
steps:
|
||||
- name: Checkout
|
||||
if: ${{ needs.changes.outputs.frontend == 'true' }}
|
||||
@@ -156,6 +176,19 @@ jobs:
|
||||
if: ${{ needs.changes.outputs.frontend == 'true' }}
|
||||
run: pnpm install
|
||||
|
||||
# See frontend-build for the key strategy. These entries are tiny (~60KB
|
||||
# measured): `test` declares no outputs, so turbo caches exit codes and
|
||||
# logs rather than artifacts -- yet a hit still skips the whole suite,
|
||||
# which is the single most expensive task in the graph.
|
||||
- name: Restore turbo cache
|
||||
if: ${{ needs.changes.outputs.frontend == 'true' }}
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: .turbo/cache
|
||||
key: turbo-test-${{ runner.os }}-${{ github.sha }}
|
||||
restore-keys: |
|
||||
turbo-test-${{ runner.os }}-
|
||||
|
||||
- name: Test
|
||||
if: ${{ needs.changes.outputs.frontend == 'true' }}
|
||||
# Same filter rationale as frontend-build. Type errors are not this
|
||||
|
||||
35
turbo.json
35
turbo.json
@@ -1,5 +1,10 @@
|
||||
{
|
||||
"$schema": "https://turbo.build/schema.json",
|
||||
// The CI workflow pins the toolchain these tasks run under (Node 22 today).
|
||||
// Without it in the global hash, bumping that version would replay cached
|
||||
// typecheck/test results produced by the old runtime and hide an
|
||||
// incompatibility behind a green check.
|
||||
"globalDependencies": [".github/workflows/ci.yml"],
|
||||
"globalEnv": [
|
||||
"DATABASE_URL",
|
||||
"PORT",
|
||||
@@ -18,7 +23,13 @@
|
||||
"tasks": {
|
||||
"build": {
|
||||
"dependsOn": ["^build"],
|
||||
"inputs": ["src/**", "app/**", "**/*.ts", "**/*.tsx", "**/*.css"],
|
||||
// No explicit `inputs`: turbo's default (every git-tracked file in the
|
||||
// package) is the only safe hash source now that the cache is actually
|
||||
// restored in CI. The previous narrow glob list matched neither
|
||||
// `content/**/*.mdx` (compiled by fumadocs-mdx during the build),
|
||||
// `public/**` assets, nor `package.json` / `tsconfig.json`, so the task
|
||||
// hash was unchanged by edits to any of them -- harmless while nothing
|
||||
// was ever cached, a stale-build bug the moment it is.
|
||||
"outputs": [".next/**", "!.next/cache/**", "dist/**", "out/**"]
|
||||
},
|
||||
"dev": {
|
||||
@@ -32,13 +43,21 @@
|
||||
"typecheck": {
|
||||
"dependsOn": ["^typecheck"]
|
||||
},
|
||||
// `test` and `lint` deliberately have no `^typecheck` dependency. No
|
||||
// package emits build artifacts (core/ui/views export raw .ts that the
|
||||
// consumer transpiles), so depending on a dependency's `tsc --noEmit`
|
||||
// bought nothing but head-of-line blocking: the heaviest task in the
|
||||
// graph, `@multica/views:test`, sat idle until core/ui typechecked.
|
||||
// Type errors still fail CI through the `typecheck` task itself.
|
||||
"test": {},
|
||||
// `^typecheck` is load-bearing here, but not for ordering: a turbo task
|
||||
// hash only covers a workspace dependency's sources if a task edge
|
||||
// reaches them. Without this edge, editing packages/views left
|
||||
// `@multica/web#test` and `@multica/desktop#test` byte-identical in hash,
|
||||
// so a cached pass would be replayed for code that changed underneath.
|
||||
// `typecheck` is the one task every package defines, which is what makes
|
||||
// it the usable edge -- `^test` would miss packages/ui, which has no test
|
||||
// script. Harmless to omit while nothing was cached; not once it is.
|
||||
"test": {
|
||||
"dependsOn": ["^typecheck"]
|
||||
},
|
||||
// `lint` keeps no dependency edge: eslint here is not type-aware (no
|
||||
// `projectService` / `parserOptions.project` anywhere in
|
||||
// packages/eslint-config), so it only ever reads its own package's files
|
||||
// and a dependency's source cannot change its result.
|
||||
"lint": {}
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user