Github-Pull: #32989
Rebased-From: 2aa288efdd
Docker currently warns that we are missing a default value.
Set this to scratch which will error if an appropriate image tag is not
passed in to silence the warning.
Github-Pull: #32989
Rebased-From: 3f339e99e0
Previously jobs were running on a large multi-core server where 10 jobs
as default made sense (or may even have been on the low side).
Using hosted runners with fixed (and lower) numbers of vCPUs we should
adapt compilation to match the number of cpus we have dynamically.
This is cross-platform compatible with macos and linux only.
Github-Pull: #32989
Rebased-From: 2c990d84a3
When using hosted runners in combination with cached docker images,
there is the possibility that the host runner image is updated,
rendering the linux-headers package (stored in the cached docker image)
incompatible.
Fix this by doing a re-install of the headers package in
03_test_script.sh.
If the underlying runner kernel has not changed thie has no effect, but
prevents the job from failing if it has.
Github-Pull: #32989
Rebased-From: 9c2b96e0d0
This sets the build dir at build time so that Apple SDK gets installed
in the correct/expected location for the runtime to find it.
Co-authored-by: Max Edwards <youwontforgetthis@gmail.com>
Github-Pull: #32989
Rebased-From: 18f6be09d0
Reverts: e87429a2d0
This was added in PR #31545 with the intention that self-hosted runners
might use it to save build cache.
As we are not using hosted runners with a registry build cache, the bulk
of this commit can be reverted, simply using the value of
$DOCKER_BUILD_CACHE_ARG in the script.
link: https://github.com/bitcoin/bitcoin/pull/31545
Github-Pull: #32989
Rebased-From: 94a0932547
Using buildx is required to properly load the correct driver, for use
with registry caching. Neither build, nor BUILDKIT=1 currently do this
properly.
Use of `docker buildx build` is compatible with podman.
Rather than trying to match the apt installed clang version, which is
prone to intermittent issues. i.e #33345.
Github-Pull: #33364
Rebased-From: b736052e39
Also, use update-alternatives to avoid having to manually specify
clang-${APT_LLVM_V} or llvm-symbolizer-${APT_LLVM_V} everywhere.
Github-Pull: #32999
Rebased-From: fad040a578
9b033bebb1 cmake: rename Kernel component to bitcoinkernel for consistency (Cory Fields)
2e0c92558e cmake: add and use install_binary_component (Cory Fields)
0264c5d86c cmake: use per-target components for bitcoin-qt and bitcoin-gui (Cory Fields)
fb0546b1c5 ci: don't try to install for a fuzz build (Cory Fields)
Pull request description:
This makes it possible to build/install only the desired binaries regardless of the configuration.
For consistency, the component names match the binary names. `Kernel` and `GUI` have been renamed.
Additionally it fixes#31762 by installing only the manpages for the configured targets (and includes them in the component installs for each).
Also fixes#31745.
Alternative to #31765 which is (imo) more correct/thorough.
Can be tested using (for ex):
```bash
$ cmake -B build
$ cmake --build build -t bitcoind -t bitcoin-cli
$ cmake --install build --component bitcoind
$ cmake --install build --component bitcoin-cli
```
ACKs for top commit:
hebasto:
ACK 9b033bebb1.
TheCharlatan:
Re-ACK 9b033bebb1
stickies-v:
re-ACK 9b033bebb1
Tree-SHA512: fd4818e76f190dbeafbf0c246b466f829771902c9d6d7111ed917093b811c8a5536a4a45e20708f73e7f581d6cb77c8e61cfa69e065788dcf0886792f553a355
fa952acdb6 ci: Skip read-write of default env vars (MarcoFalke)
Pull request description:
If they remain unset, they use the default anyway. Except for `USER`, but this seems unused anyway.
Can be checked via:
```
sh-5.2# touch /tmp/empty_env
sh-5.2# podman run --rm --env-file /tmp/empty_env 'ubuntu:24.04' env
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
container=podman
HOME=/root
HOSTNAME=19ece5c9e052
ACKs for top commit:
0xB10C:
ACK fa952acdb6
Prabhat1308:
utACK [fa952ac](fa952acdb6)
Tree-SHA512: fe0c173b23cfda3025306303a44ffe32ecc57c2e0e1a2376594696f9887ed22f5105da84e898e790041bf15a4aa42a365fba016710ad269d439dda691977be90
Currently the manpages are installed, but that is a bug. An upcoming commit
will avoid installing manpages for targets that aren't configured, which
removes the "install" target for fuzz builds.
fa5a02bcfa ci: Use clang-20 for sanitizer tasks (MarcoFalke)
Pull request description:
A new clang version generally comes with bugfixes, new (sanitizer) features, and deprecations.
Upgrade the sanitizer tasks to use the new version.
This was also suggested in https://github.com/bitcoin/bitcoin/pull/31691#issuecomment-2602517116
ACKs for top commit:
fanquake:
ACK fa5a02bcfa - tested 20 in some other infra, we just needed to fix the same deprecation warnings we'd seen, in cryptofuzz: 09ca550c3e.
Tree-SHA512: 6114d790b5d7145eb5f019e02da6c2c833342707ad67fb9f9c09506001afbef0c9b9beee7e51321f17f12ea692509d6428e6072ad105dba51e4d54cd057621cd
faf8fc5487 lint: Call lint_commit_msg from test_runner (MarcoFalke)
fa99728b0c lint: Move commit range printing to test_runner (MarcoFalke)
fa673cf344 lint: Call lint_scripted_diff from test_runner (MarcoFalke)
Pull request description:
The lint `commit-script-check.sh` can not be called from the test_runner at all and must be called manually. Also, some checks require `COMMIT_RANGE` to be set.
Fix all issues by moving two lint checks into the test_runner. Also, the proper commit range is passed to the checks by the test_runner, so that the user no longer has to do it.
ACKs for top commit:
kevkevinpal:
reACK [faf8fc5](faf8fc5487)
willcl-ark:
tACK faf8fc5487
Tree-SHA512: 78018adc618d997508c226c9eee0a4fada3899cdfd91587132ab1c0389aea69127bafc3a900e90e30aca2c6bae9dcd6e6188ef287e91413bc63ee66fb078b1af
8888ee4403 ci: Allow build dir on CI host (MarcoFalke)
Pull request description:
This is required to pass cross builds on to a different machine after the build.
See for example https://github.com/bitcoin/bitcoin/pull/31176, but this pull will also allow someone to implement it outside this repo.
ACKs for top commit:
davidgumberg:
lgtm ACK 8888ee4403
hebasto:
re-ACK 8888ee4403.
Tree-SHA512: a1e2c32bc1b95efbd0b48287ac5b49e0e1bacbf5a5800845be5352bbdd3e17fa478e90348b2e94e95cf3ae863cdf75ab444089376588f6f8eec438f73a4b5b97
e87429a2d0 ci: optionally use local docker build cache (0xb10c)
Pull request description:
By setting `DANGER_DOCKER_BUILD_CACHE_HOST_DIR`, the task-specific docker images built during the CI run can be cached. This allows, for example, ephemeral CI runners to reuse the docker images (or layers of it) from earlier runs, by persisting the image cache before the ephemeral CI runner is shut down. The cache keyed by `CONTAINER_NAME`.
As `--cache-to` doesn't remove old cache files, the existing cache is removed after a successful `docker build` and the newly cached image is moved to it's location to avoid the cache from growing indefinitely with old, unused layers.
When `--cache-from` doesn't find the directory, the cached version is a cache-miss, or the cache can't be imported for whatever other reason, it warns and `docker build` continues by building the docker image.
This feature is opt-in. The documentation for the docker build cache of `type=local` can be found on https://docs.docker.com/build/cache/backends/local/
This replaces https://github.com/bitcoin/bitcoin/pull/31377 - some of the discussion there might provide more context.
ACKs for top commit:
maflcko:
I haven't tested this, and it looks harmless and is easy to revert, if needed. So lgtm ACK e87429a2d0
achow101:
ACK e87429a2d0
TheCharlatan:
tACK e87429a2d0
willcl-ark:
ACK e87429a2d0
Tree-SHA512: 0887c395dee2e2020394933246d4c1bfb6dde7165219cbe93eccfe01379e05c75dce8920b6edd7df07364c703fcee7be4fba8fa45fd0e0e89da9e24759f67a71
fa8ade300f refactor: Avoid GCC false positive error (MarcoFalke)
fa40807fa8 ci: Enable DEBUG=1 for one GCC-12+ build to catch 117966 regressions (MarcoFalke)
Pull request description:
It is possible that someone accidentally removes the workaround in fa9e0489f5, or more likely that someone accidentally adds new code without the workaround.
Avoid this by adding a temporary CI check.
This can be tested by reverting the workaround and observing a failure.
ACKs for top commit:
hebasto:
ACK fa8ade300f, I've tested locally on Ubuntu 24.04.
Tree-SHA512: 7ee1538fd5304a5ab91ac8c7619a573548d7e0345592a1e9d38b3b73729e09e7c77a9ee703d64cf02a8218de3148376d7836e294abb939aa7533034ba36dfb6c
Allowing to call the check from the test_runner allows for consistent
error messages and better UX by having a single test_runner for all
checks.
This requires the env var to be set for now. The next commit makes the
commit range optional.
faaabfaea7 ci: Bump centos stream 10 (MarcoFalke)
Pull request description:
This is a follow-up to fa47baa03b, which bumped the gcc version to avoid a warning bloat in the CI log. However, it is also required to bump python3, see https://github.com/bitcoin/bitcoin/issues/31476#issue-2735206340
> This will uncover an issue in the centos task that the correct python version is missing. I guess this should be fixed by installing and activating an acceptable python version.
Instead of bumping the packages individually in centos stream 9, just bump to stream 10.
ACKs for top commit:
fanquake:
ACK faaabfaea7
Tree-SHA512: a564ff3a2a0dc4d39874e87540e67072f293bbed82c8eca22266fcadc16c5571e0e41d38576a63e466b64d13f7e3acbd95be10cf2420de33127aa420eca3b928
fabefd9915 ci: Turn CentOS task into native one (MarcoFalke)
Pull request description:
Cross-compiling to `i686-pc-linux-gnu` on CentOS in the CI is mostly redundant with the `ci/test/00_setup_env_i686_multiprocess.sh` task (albeit it using clang):
35bf426e02/ci/test/00_setup_env_i686_multiprocess.sh (L9-L12)
One task seems sufficient as a sanity check, given that there seems to be no real demand for this architecture anyway.
Turning the task into a native one makes it possible to run the task natively on aarch64 or any other supported architecture.
Also, remove the install of the `lbzip2` package, which is unused since commit a46065e36c
Also, remove the `CONFIG_SHELL` env var, which is unused since the cmake migration. (`CONFIG_SHELL` in depends is still kept).
ACKs for top commit:
davidgumberg:
ACK fabefd9915
hebasto:
ACK fabefd9915, tested locally on Ubuntu 24.10.
Tree-SHA512: 5a7b3131b379d11ef602e5821165861e9bdf61d605014bf8fcb33b8e12d8823450798af2d3289b96f7559dfa47b839bf939ddc0b3725efecfeac7ae570a981e7
According to the description for pkg-config, "pkgconf is a
replacement for pkg-config, providing additional functionality
while also maintaining compatibility. This package only provides
a dependency link to the pkgconf package to help with package
upgrades. It can be safely removed."
Thus several scripts and markdown files are updated.
For the task `MSan, depends (Cirrus CI)` we build a custom libc++ for
which we already use `-DLIBCXX_HARDENING_MODE=debug`. Compile it also
with `_LIBCPP_ABI_BOUNDED_*` to enable further checks.
Docs at: https://libcxx.llvm.org/Hardening.html#abi-options
By setting DANGER_DOCKER_BUILD_CACHE_HOST_DIR, the task-specific
docker images built during the CI run can be cached. This allows,
for example, ephemeral CI runners to reuse the docker images (or
layers of it) from earlier runs, by persisting the image cache
before the ephemeral CI runner is shut down. The cache keyed by
`CONTAINER_NAME`.
As --cache-to doesn't remove old cache files, the existing cache
is removed after a successful `docker build` and the newly cached
image is moved to it's location to avoid the cache from growing
indefinitly with old, unused layers.
When --cache-from doesn't find the directory, the cached version is
a cache-miss, or the cache can't be imported for whatever other reason,
it warns and `docker build` continues by building the docker image.
This feature is opt-in. The documentation for the cache type=local
can be found https://docs.docker.com/build/cache/backends/local/
This replaces https://github.com/bitcoin/bitcoin/pull/31377
31e59d94c6 iwyu: Drop backported mapping (Hennadii Stepanov)
fe9bc5abef ci: Update Clang in "tidy" job (Hennadii Stepanov)
Pull request description:
This PR switches to the latest [IWYU 0.23](https://github.com/include-what-you-use/include-what-you-use/releases/tag/0.23), which is compatible with Clang 19.
New "bugprone-use-after-move" and "modernize-use-starts-ends-with" warnings that emerged have been addressed.
ACKs for top commit:
maflcko:
lgtm ACK 31e59d94c6
l0rinc:
ACK 31e59d94c6
theuni:
ACK 31e59d94c6
Tree-SHA512: ae0ca150673e1bfa78664f2ef35dbc965094b32374cafeeae390c6d368c28169a7f7790debe9a6eeb5efc39c9a468f5032d92f30cc4032b09d8265f6a75de882
