Github-Pull: #32989
Rebased-From: 3f339e99e0
Previously jobs were running on a large multi-core server where 10 jobs
as default made sense (or may even have been on the low side).
Using hosted runners with fixed (and lower) numbers of vCPUs we should
adapt compilation to match the number of cpus we have dynamically.
This is cross-platform compatible with macos and linux only.
Github-Pull: #32989
Rebased-From: cc1735d777
To remove multiple occurances of the respository name, against which we
compare `${{ github.repository }}` to check if we should use Cirrus
Runners, introduce a helper job which can check a single environment
variable and output this as an input to subsequent jobs.
Forks can maintain a trivial patch of their repo name against the
`REPO_USE_CIRRUS_RUNNERS` variable in ci.yml if they have Cirrus Runners
of their own, which will then enable cache actions and docker build
cache to use Cirrus Cache.
It's not possible to use `${{ env.USE_CIRRUS_RUNNERS }}` in the
`runs-on:` directive as the context is not supported by GitHub.
If it was, this job would no longer be necessary.
Github-Pull: #32989
Rebased-From: 020069e6b7
Whilst the action cirruslabs/actions/cache will automatically set this
host, the docker `gha` build cache backend will not be aware of it.
Set the value here, which will later be used in the docker build args to
enable docker build cache on the cirrus cache.
Github-Pull: #32989
Rebased-From: fdf64e5532
Another action to reduce boilerplate in the main ci.yml file.
This action will set up a docker builder compatible with caching build
layers to a container registry using the `gha` build driver.
It will then configure the docker build cache args.
Github-Pull: #32989
Rebased-From: 33ba073df7
If set, Cirrus runners will be used on pushes to, and pull requests
against, this repository.
Forks can set this if they have their own cirrus runners.
Github-Pull: #32989
Rebased-From: b232b0fa5e
Add "Restore" and "Save" caching actions.
These actions reduce boilerplate in the main ci.yml configuration file.
These actions are implemented so that caches will be saved on `push`
only.
When a pull request is opened it will cache hit on the caches from the
lastest push, or in the case of depends will hit on any matching depends
hash, falling back to partial matches.
Depends caches are hashed using
`$(git ls-tree HEAD depends "ci/test/$FILE_ENV" | sha256sum | cut -d' ' -f1)`
and this hash is passed in as an input to the actions. This means we
direct cache hit in cases where depends would not be re-built, otherwise
falling back to a partial match.
Previous releases cache is hashed similarly to depends, but using the
test/get_previous_releases.py file.
The cirruslabs cache action will fallback transparently to GitHub's
cache in the case that the job is not being run on a Cirrus Runner,
making these compatible with running on forks (on free GH hardware).
The ci "test-each-commit" job fetches the PR branch being tested with a depth of (# of commits in PR + 2), and then tries to run tests on commits after the most recent merge commit.
When a PR is opened against a bitcoin core branch, a merge commit is always guaranteed to exist within the fetch depth, because bitcoin core branches always point at merge commits.
However, in fork repositories, pull requests can be opened based on other branches that don't contain recent merge commits, and this will currently cause git rev-list to fail with fatal: bad revision '^^@'.
Work around this problem by not requiring a recent merge commit, and just testing on all fetched commits if a merge commit can't be found.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
DANGER_CI_ON_HOST_CACHE_FOLDERS if set will mount caches in directories on the host rather than in docker volumes. Supports saving and restoring caches on Github Actions.
9eea51d905 ci: move Asan / LSan / USDT job to Github Actions (Max Edwards)
4b527fa93b ci: add IPV6 network to ci container (Max Edwards)
Pull request description:
PR for moving the ASAN + LSAN + USDT + friends job to github actions from Cirrus.
The motivation for this PR is that this task needs a full VM (or bare metal) to function, because of the tracepoints. It can not run in a container on an arbitrary Linux, because the outside machine must exactly match the specification of the distro used in the CI task config. This requires more maintenance for the persistent worker, and I think moving to GHA will reduce the maintenance burden, or at least make it possible for anyone to work on.
Also, it makes it easier to run the task on forks (bitcoin-inquisition, bitcoin-knots, devel forks, ...) without having to set-up a real machine.
ACKs for top commit:
maflcko:
review ACK 9eea51d905
achow101:
ACK 9eea51d905
hebasto:
ACK 9eea51d905.
Tree-SHA512: 1111c1c9e3a11e725dff1344643fff3c91fb9b4d7c1cc9a7d507a8f146f5223316a00272030b41ae37ecb59d044f2e90e1cd907450049b25f094f0b60643d4c7
GHA Windows images previously had multiple VC Build Tools installed,
which required specifying the `VCPKG_PLATFORM_TOOLSET_VERSION`
explicitly to avoid linker errors. This issue has been resolved as per
https://github.com/actions/runner-images/issues/9701.
115c283516 ci: add print of powershell version to win64 job (Max Edwards)
Pull request description:
Extraction of just printing powershell version from closed PR: https://github.com/bitcoin/bitcoin/pull/29581
See https://github.com/bitcoin/bitcoin/pull/29581#issuecomment-1984212990 for the cause of a CI failure which was a powershell update.
This PR will make it easier to notice in the future that PS has changed.
ACKs for top commit:
hebasto:
ACK 115c283516. We still use PowerShell in some steps of the "Win64 native" CI job.
Tree-SHA512: 4c7ba9df4f0a98491120326f05e877a995f43a387fe9bbd193549b32f5a4488f85f83e472c9277db457110a7deda04f08832fe6e8129aff4b0b7278be23d4e35
Homebrew attempts to check for outdated dependents or those with broken
linkage. Such behavior might lead to failures when Homebrew updates them
on old macOS images.
This change prevents such behavior.
The GHA VS installation includes its own vcpkg package manager, which is
available since VS 17.6. This change avoids any ambiguity about which
copy of vcpkg we run.
