In this commit, we replace the basic inline cleanup command in the release
workflow with the comprehensive cleanup-space action that was previously
only used in the main CI workflow. The previous release workflow cleanup
simply removed the hostedtoolcache directory, which freed only a few
gigabytes and proved insufficient for multi-platform release builds.
By switching to the cleanup-space action (now enhanced to free 20-25GB),
the release workflow will have substantially more disk space available
before beginning the build process. This should resolve the disk space
exhaustion issues that were occurring during the Windows ARM build phase,
which is one of the final platforms in the 15-platform build sequence.
In this commit, we significantly expand the cleanup-space GitHub Actions
workflow to free up substantially more disk space on GitHub runners. The
previous cleanup only removed three large toolsets (dotnet, android,
ghc), which should free ~14GB. This enhancement adds removal of several
additional large packages and caches, bringing the total freed space to
approximately 20-25GB.
The specific additions include removing Swift and Julia language runtimes,
the hosted toolcache directory, all Docker images, numerous large apt
packages (aspnetcore, llvm, php, mongodb, mysql, azure-cli, browsers, and
development tools), and various cache directories. We also add disk space
reporting before and after cleanup to provide visibility into how much
space is actually being freed during workflow runs.
This enhancement was motivated by release builds running out of disk space
when building for all 15 supported platforms (darwin, freebsd, linux,
netbsd, openbsd, windows across multiple architectures). The sequential
builds with verbose output were consuming more space than the basic cleanup
could provide.
In this commit, we revamp the way our skip-labels worked to instead mark
certain stages as green, instead of skipping them. Skipped changes
didn't count towards the set of required checks, which slowed down PR
velocity.
Previously, when PRs were labeled with 'no-itest' or 'no-changelog',
the corresponding CI jobs would be completely skipped. This caused the
GitHub checks to show as skipped rather than successful, which could
be confusing and prevented certain merge rules from working properly.
This commit changes the behavior so that these jobs still run but
immediately report success when the special labels are detected. Each
affected job now starts with a label check step that sets a skip
flag, and all subsequent steps are conditionally executed based on
this flag. When skipped, the jobs add a notice to the GitHub step
summary explaining that tests were auto-passed due to the label.
The change affects five jobs: basic-integration-test, integration-test,
windows-integration-test, macos-integration-test, and milestone-check.
This commit adds a more descriptive docs about the caching process. In
addition, the `github.job` is now removed from the cache key
construction to allow jobs sharing the same cache. In addition, given we
now use `inputs.key-prefix`, the `github.job` is no longer relevant as
the `key-prefix` already creates a unique name for the cache.
We also update the lint and check commits job to use the same cache from
the unit test, given they are running in the same environment.
In this commit, a new `test_native_sql` build flag is defined. If this
build flag is used along with the `--use-native-sql` config option, then
the SQLStore implementation of the graphdb.V1Store will be initialised.
This is then used to run our itest suite against the new SQLStore graph
implementation.
NOTE that this only works for new nodes currently - the migration from
kv-to-sql is yet to be implemeneted.
In this commit, we implement the postgres and sqlite versions of the
NewTestDB function. We add the various build flags so that only one of
the three versions of this function can be active at a time.
We also introduce the SQLStore struct which is the SQL implementation of
the V1Store interface.
NOTE: it currently temporarily embeds the KVStore struct so that we can
implement the V1Store interface incrementally. For any method not
implemented, things will fall back to the KVStore. This is ONLY the
case for the time being while this struct is purely used in unit tests
only. Once all the methods have been implemented, the KVStore field will
be removed from the SQLStore struct.
In an attempt to fix the "out of disk space" build error during release
builds, we first disable using Golang caches (which can be quite large)
and then remove a bunch of pre-installed tools and their caches to
provide some additional disk storage.
Turns out that actions/setup-go starting with @v4 also adds caching.
With that, our cache size on disk has almost doubled, leading to the
GitHub runner running out of space in certain situation.
We fix that by disabling the automated caching since we already have our
own, custom-tailored version.
Sometimes only the "Send coverage" step of a CI job will fail. This
commit turns this step into a "best effort" step instead so that it
does not block a CI job from passing.
It can, for example, often happen that a single job is re-run from the
GH UI, it then passes but the "Send coverage" step fails due to the
"Can't add a job to a build that is already closed." error meaning that
the only way to get the CI step to pass is to re-push and retrigger a
full CI run.
