fa70b9ebaa ci: Temporarily use clang in valgrind tasks (MarcoFalke)
faf3ef4ee7 ci: Clarify why valgrind task has gui disabled (MarcoFalke)
fadb77169b test: Scale feature_dbcrash.py timeout with factor (MarcoFalke)
Pull request description:
valgrind currently does not work on GCC compiled executables, due to an upstream bug. https://bugs.kde.org/show_bug.cgi?id=472329
So temporarily switch to clang, so that a long term solution can be figured out in the meantime.
ACKs for top commit:
l0rinc:
ACK fa70b9ebaa
fanquake:
ACK fa70b9ebaa - also checked that it doesn't currently work under aarch64.
Tree-SHA512: 2e7c7a709311efa7bf29c3f9b1db60886b189b2d2bfebb516062163d65f0d7e8de3b6fc21c94cd62f6bd7e786e9c36fba55c4bae956b849851eb8b08e772c03e
valgrind currently does not work on GCC -O2 compiled executables, which
contain std::optional use, due to an upstream bug. See
https://bugs.kde.org/show_bug.cgi?id=472329
One workaround could be to use -O1. However, that seems brittle, as
variantions of the bug were seen with -O1 as well.
So temporarily use clang in the valgrind CI tasks, because this also
allows to drop a false-positive suppression for:
-DCMAKE_CXX_FLAGS='-Wno-error=array-bounds'
Also, update the comment in contrib/valgrind.supp to mention the
background:
* GCC -O2 wasn't tested with the suppressions file, due to the mentioned
bug.
* Clang-17 (or later) on aarch64 wasn't tested due to bug
https://github.com/bitcoin/bitcoin/issues/29635 and the minimum
supported clang version is clang-17 right now.
* GUI isn't tested, because it requires a debug build, see the prior
commit.
This means the only tested config right now is the one mentioned in the
suppression file.
A build with system libs (or with a normal depends build) will fail
with:
```sh
$ valgrind --exit-on-first-error=yes --error-exitcode=1 --quiet ./bld-cmake/bin/test_bitcoin-qt
Detected locale "C" with character encoding "ANSI_X3.4-1968", which is not UTF-8.
Qt depends on a UTF-8 locale, and has switched to "C.UTF-8" instead.
If this causes problems, reconfigure your locale. See the locale(1) manual
for more information.
********* Start testing of AppTests *********
Config: Using QtTest library 6.10.2, Qt 6.10.2 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 15.2.0), ubuntu 26.04
PASS : AppTests::initTestCase()
QINFO : AppTests::appTests() Backing up GUI settings to "/tmp/test_common bitcoin/60d474ffae390f81657d/regtest/guisettings.ini.bak"
==18007== Conditional jump or move depends on uninitialised value(s)
==18007== at 0x12655E26: ???
==18007== by 0xCB28E7F: ???
==18007==
==18007==
==18007== Exit program on first error (--exit-on-first-error=yes)
```
A DEBUG=1 depends build would work, but that seems tedious for
questionable benefit.
Replace the hardcoded x86_64 binary name with $(uname --machine) so the
correct binary is downloaded when building the lint container, where at
all possible.
fa90b21430 test: Remove unused feature_segwit.py functions (MarcoFalke)
fa6b05c96f test: Remove unused CUSTOM_._COUNT (MarcoFalke)
fa7bac94d8 test: Remove unused wait_for_addr, firstAddrnServices, on_addr (MarcoFalke)
fa388a3585 test: Remove unused self.p2p_conn_index = 1 (MarcoFalke)
fa803710e2 test: Remove unused AddressType (MarcoFalke)
fab5072ce1 ci: Remove vulture (MarcoFalke)
Pull request description:
Currently, `vulture` is run with `--min-confidence=100`, which reduces its checks to dead code after control statements, which is nice, but not really a common nor severe issue. See the discussion in https://github.com/bitcoin/bitcoin/issues/34810#issuecomment-4045927137 and commit 5c005363a8, which had to remove dead code manually.
Reducing the confidence has shown to be too brittle/tedious in the past, so remove the tool for now from CI.
Of course, removing this from CI does not prevent anyone from running it locally and removing dead code.
Fixes https://github.com/bitcoin/bitcoin/issues/34810
ACKs for top commit:
fanquake:
ACK fa90b21430
willcl-ark:
ACK fa90b21430
Tree-SHA512: 6a5998470dae3a17baec29b70b02333f4cd9b81bc4c6a05b56085ff1ba527ed7bdeccd17b09d9ad785ae03c97982ee1f3147e4df3bd537c66b02e9a44d0e5f15
d03e3be246 ci: check macos bundle structure and codesigning (fanquake)
66d80d57b4 macdeploy: use plugins dir to find plugins (fanquake)
ab137cbfe2 macdeploy: subprocess out to zip rather than shutil.make_archive (fanquake)
Pull request description:
Fix bundle format.
Add a CI check that codesigning works.
Fixes#34744.
ACKs for top commit:
Sjors:
tACK d03e3be246
hebasto:
ACK d03e3be246, tested on macOS Tahoe 26.3.1.
sedited:
ACK d03e3be246
Tree-SHA512: 5a7db896952edf338ff4fe8c934f1e1c992642850a99d5fafbb1212c6979601b3b72b6f3af880fb6f6ac8759cd4102e9f01792abb05410ceaf36cbffaec48e47
fafdb8f635 ci: Allow running iwyu ci in worktree (MarcoFalke)
fab73e213d ci: Reject unsafe execution of shell scripts (MarcoFalke)
Pull request description:
Currently, the iwyu CI fails to run in a git-worktree, or git-archive. This is due to the use of `git diff`.
Fix this by force-initializing a dummy git repo with a single dummy commit.
It may be possible to detect when `git diff` is not available in the directory, and only apply the fallback when needed, but the git history is not needed and it is easier to unconditionally apply the git init.
ACKs for top commit:
willcl-ark:
reACK fafdb8f635
hebasto:
ACK fafdb8f635, I have reviewed the code and it looks OK. Tested on Fedora 43.
sedited:
ACK fafdb8f635
Tree-SHA512: 572f1e2b9e215c2804095382498abb5b8636e3a49d5ba2a736b975e06afa2881d815b854a8a593d0f187c7c6b55034688e11f46d6814edfe7c29505197e80b18
9f3752c437 ci: use latest versions of lint deps (fanquake)
Pull request description:
Use the latest available versions, except for LIEF, which is changed with Guix.
ACKs for top commit:
hebasto:
ACK 9f3752c437, I've verified the releases against https://pypi.org and https://github.com/becheran/mlc.
Tree-SHA512: e6ed79bb7dc8601ed0708eb7b53cbf4cf843b69829c073c41e9d97be690b4b2bf9ea5ecf250e05cbacba4ad35df06aa3e2cb2ff319145a34e1a7831cf182ec21
24699fec84 doc: Add initial asmap data documentation (Fabian Jahr)
bab085d282 ci: Use without embedded asmap build option in one ci job (Fabian Jahr)
e53934422a doc: Expand documentation on asmap feature and tooling (Fabian Jahr)
6244212a55 init, net: Implement usage of binary-embedded asmap data (Fabian Jahr)
6202b50fb9 build: Generate ip_asn.dat.h during build process (Fabian Jahr)
634cd60dc8 build: Add embedded asmap data (Fabian Jahr)
Pull request description:
This is the final in a series of PRs that implement the necessary changes for embedding of asmap data into the binary. This last part add the initial asmap data, implements the build changes and adds further documentation.
Currently an asmap file needs to be acquired by there user from some location or the user needs to generate one themselves. Then they need to move the file to the right place in datadir or pass the path to the file as `-asmap=PATH` in order to use the asmap feature. The change here allows for builds to embed asmap data into the bitcoind binary which makes it possible to use the feature without handling of the asmap file by the user. If the user starts bitcoind with `-asmap` the embedded data will be used for bucketing of nodes.
The data lives in the repository at `src/node/data/ip_asn.dat` and can be replaced with a new version at any time. The idea is that the data should be updated with every release. By default the data at that location is embedded into the binary but there is also a build option to prevent this (`-DWITH_EMBEDDED_ASMAP=OFF`). In this case the original behavior of the `-asmap` option is maintained.
ACKs for top commit:
achow101:
ACK 24699fec84
sipa:
ACK 24699fec84
hodlinator:
ACK 24699fec84
Tree-SHA512: c2e33dbeea387efdfd3d415432bf8fa64de80f272c1207015ea53b85bb77f5c29f1dae5644513a23c844a98fb0a4bb257bf765f38b15bfc4c41984f0315b4c6a
The shell scripts are inherently unsafe, because they will install new
software packages, modify global configuration settings, write to the
root / or $HOME, and possibly modify the git repo.
The only safe way to run them is through the CI system itself, that is
the ci_exec python function.
The ci_exec funtion ensures that the user has set up a sandbox
externally and set DANGER_RUN_CI_ON_HOST=1 at their own risk, or that a
sandbox was set up with the given container_id, in which case it is safe
to set DANGER_RUN_CI_ON_HOST=1 for that sandbox.
Also, it is safe to set DANGER_RUN_CI_ON_HOST=1 when building the
sandbox image in ci/test_imagefile.
Then, the two shell scripts can reject early if unsafe execution is
detected.
faba426b3b lint: Flatten lint image entry points (MarcoFalke)
1111fff91c lint: Add missing --platform=linux to docker build command (MarcoFalke)
Pull request description:
Two fixups to the lint container:
* Add a missing `--platform=linux` to avoid running a non-native arch, like s390x, which happens with podman if such a container was most recently used.
* Flatten the entry points to remove the bash-based one:
Previously, an additional entry point into the container that spawned a bash was supported. The bash had an alias `lint` to run all lint scripts. However, such a use-case seems limited (because it only runs inside the container), inflexible (because it only allows running all lint scripts), and possibly brittle (because it can miss re-building the image when the cache is stale). So remove it and just offer the single entry point via the `./ci/lint.py` script.
If there is a use-case to skip the image building, it should be trivial to add an env var setting the the lint Python script like `DANGER_SKIP_IMAGE_RE_BUILD=1` (or so) in the future.
ACKs for top commit:
willcl-ark:
ACK faba426b3b
Tree-SHA512: 9afda16723c215602c6c42fa3a286d1828c887c8f6ff9512c8ec162ec8997789695f0c464d389cae94e67acf8b5e0f1a55e2ee0d60131a2eee091cf281f91514
d79249d279 ci: add chimera Linux LTO CI job (fanquake)
Pull request description:
Adds a CI config based on using [Chimera Linux](https://chimera-linux.org/). This might be interesting for any of the following:
* Chimera is based on LLVM & musl libc - we test both of these in isolation, but not together.
* No GNU components. I don't think we have an existing Linux CI job that doesn't have a gcc/stdlibc++ install. This exercises the depends logic for a fully LLVM/Clang/lld only build, including building the native tools (related to #33902).
* We don't currently have a job with LTO enabled (here using CMakes `CMAKE_INTERPROCEDURAL_OPTIMIZATION`, which is `-flto=thin` for LLVM/Clang). I think this is worth having generally (we do use LTO in some other places, like oss-fuzz). If runtime is too much of an issue, then it could also be dropped. (Chimera itself is also compiled with LTO).
QT in depends doesn't build (#32744), so is excluded for now.
Chimera has pointed out at least a few quirks, i.e #34390, #34408 and https://github.com/bitcoin/bitcoin/pull/29963#discussion_r2707922298.
ACKs for top commit:
maflcko:
lgtm ACK d79249d279
hebasto:
ACK d79249d279.
Tree-SHA512: 1174a7462bf2e7433a2c27a6cf398e94b05db42bb414629c71cf9f9a297ca269e173ae1b7517b30510b494b4397f918eef706d3c75c4286767c5557aeb6db4c7
b65a3d8009 iwyu: Fix patch to prefer `<cstdint>` (Hennadii Stepanov)
Pull request description:
The goal of the [patch](https://github.com/bitcoin/bitcoin/blob/master/ci/test/01_iwyu.patch) is to suggest C++ headers rather than their C counterparts. However, for fixed width integer types, the patched IWYU currently suggests `<cinttypes>` where `<cstdint>` is sufficient.
This PR fixes this behavior.
ACKs for top commit:
maflcko:
lgtm ACK b65a3d8009
furszy:
utACK b65a3d8009
willcl-ark:
utACK b65a3d8009
Tree-SHA512: 695efdd44b92a642401738572e49c8b6591aa4463d387107fdf3d2f7c9c4b39f4097cb82413752caf9e8890dcca7246a894e562a1dd17023b05a7e455705beac
-Werror is added to the previous releases job, given it runs on Ubuntu
22.04, which uses an older CMake.
`--compile-no-warning-as-error` can be used, if needed, in future, to
suppress the `CMAKE_COMPILE_WARNING_AS_ERROR` behaviour from a CI
config.
CMAKE_COMPILE_WARNING_AS_ERROR was added to CMake in 3.24.
See https://cmake.org/cmake/help/latest/prop_tgt/COMPILE_WARNING_AS_ERROR.html.
Co-authored-by: willcl-ark <will8clark@gmail.com>
The goal of the patch is to suggest C++ headers rather than their C
counterparts. However, for fixed width integer types, the patched IWYU
currently suggests `<cinttypes>` where `<cstdint>` is sufficient.
This change fixes this behavior.
bbbb78a4f2 ci: Print verbose build error message in test-each-commit (MarcoFalke)
2222dadabb ci: [refactor] Allow overwriting check option in run helper (MarcoFalke)
Pull request description:
Currently, the build error in the test-each-commit task is not too nice. E.g. https://github.com/bitcoin/bitcoin/actions/runs/21509735101/job/61973587699#step:8:10464:
```
...
[ 75%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/txvalidation_tests.cpp.o
[ 75%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/txvalidationcache_tests.cpp.o
[ 75%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/validation_block_tests.cpp.o
[ 75%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/validation_chainstate_tests.cpp.o
[ 75%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/validation_chainstatemanager_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/validation_flush_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/validation_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/wallet_test_fixture.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/db_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/coinselector_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/coinselection_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/feebumper_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/group_outputs_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/ismine_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/psbt_wallet_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/scriptpubkeyman_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/spend_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/wallet_rpc_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/wallet_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/wallet_transaction_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/walletdb_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/wallet/test/walletload_tests.cpp.o
[ 76%] Building CXX object src/test/CMakeFiles/test_bitcoin.dir/__/ipc/test/ipc_tests.cpp.o
[ 77%] Linking CXX executable ../../bin/test_bitcoin
[ 87%] Built target test_bitcoin
gmake: *** [Makefile:146: all] Error 2
Command '['cmake', '--build', 'ci_build', '-j', '4']' returned non-zero exit status 2.
error: cannot rebase: Your index contains uncommitted changes.
warning: execution failed: git merge --no-commit origin/master && python3 ./.github/ci-test-each-commit-exec.py && git reset --hard
and made changes to the index and/or the working tree.
You can fix the problem, and then run
git rebase --continue
Error: Process completed with exit code 1.
```
Fix it by just using the same approach that the other CI tasks are using:
01651324f4/ci/test/03_test_script.sh (L143-L146)
ACKs for top commit:
willcl-ark:
ACK bbbb78a4f2
Tree-SHA512: ae54ee62f53e060ed42f76ca59daf7a017bd12495e171efed03d1f5dda969db4f7e2e2c9ed7a178ff5fa9a5baa55ab6a7b30b3ab6b81d5279fe937006ac8228f
7528d18796 ci: show more verbose ccache stats (will)
580e9eefe3 ci: bump CCACHE_MAXSIZE to 2G (will)
Pull request description:
Currently some CI jobs don't have great ccache hitrates which we should try to improve: https://willcl-ark.github.io/bitcoin-core-ci-stats/graph/ccache/
- bump ccache maxsize to 2GB in all jobs. We have 150GB shared cache to use, so this should be OK at maximum of 36GB total (current jobset).
- print more verbose ccache stats in the CI logs
The idea is that increasing the cache size to > 2x needed size should eliminate any cache thrashing which might be taking place on master builds when we save the cache. Additionally, larger caches result in more hits in general.
ACKs for top commit:
maflcko:
lgtm ACK 7528d18796
Tree-SHA512: ae00a05159e3f38d24aebc50a2576c5f11241b1196058a4ca2f5f78909795b891bef20cdc4412f512a2ba09cc9ec65afa5132ac0509b54845dd84f933528500c
3c8f5e48f7 ci: Treat SHA1 LLVM signing key as warning (will)
Pull request description:
The current SHA1 LLVM signing key is considered not secure since
2026-02-01T00:00:00Z which makes this run fail when downloading
packages.
See: https://github.com/llvm/llvm-project/issues/153385
Apply the fix from the issue to temporarily to treat this error as a
warning, until the upstream key can be updated.
This PR should be reverted once the upstream key is updated.
ACKs for top commit:
hebasto:
ACK 3c8f5e48f7, tested by running the "iwyu" CI job locally on Ubuntu 25.10 after burning all podman's caches.
Tree-SHA512: fbccf98bfd73cb338670f1ceea994d277d746acbc88b9b90a403d9a59d82abda0f3ba34c4d484b70926340c2d0c873259f930c36ccd4f9d18bb1d22d49ee70c4
The current SHA1 LLVM signing key is considered not secure since
2026-02-01T00:00:00Z which makes this run fail when downloading
packages.
See: https://github.com/llvm/llvm-project/issues/153385
Apply the fix from the issue to temporarily to treat this error as a
warning, until the upstream key can be updated.
This PR should be reverted once the upstream key is updated.
With 15 runners we get 150GB of cache space to use, and we currently
have 18 jobs using ccache.
Although each run only generates ~ 200-300 MB of cache data on each run,
the small the small size may be contributing to poor hitrate.
Bump ccache to 2GB per job ~ --> 36GB of the total 150GB cache space to
try and reduce any thrashaing and generally increase hitrate.
Also, use str(e) consistently in all run helpers.
This refactor does not change any behavior.
This can be reviewed by checking that all instances are exactly
identical code now:
$ git grep --function-context 'def run(cmd'
