The GUI manifest has a distinct Guix profile and consequently produces
different depends build IDs. Previously this mean that the gui build
would invalidate the build cache for the non-guix build.
Move guix builds under a GUIX/ root, and further BUILD/ and GUI/ trees,
so that local developer builds, GUI and non GUI guix builds can all
co-exist together.
There will end up being more copies of build depends packages, (local
builds, and two guix variants), but these all use different toolchains
and toolsets so this is desirable.
de9b436ba3 depends: Switch from multilib to platform-specific toolchains (Hennadii Stepanov)
Pull request description:
Using the multilib GCC toolchain, as currently documented in [`depends/README.md`](4c1906a500/depends/README.md), has several issues, such as:
1. The [`g++-multilib`](https://packages.ubuntu.com/noble/g++-multilib) package conflicts with platform-specific cross-compiler packages. This means it is not possible to cross compile for `i686` and other platforms using the same set of installed packages.
2. The [`g++-multilib`](https://packages.ubuntu.com/noble/g++-multilib) package is not available for `arm64`:
```sh
$ sudo apt install g++-multilib
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package g++-multilib
```
3. Managing the multilib GCC toolchain requires additional code in both depends and Guix scripts.
This PR addresses all the issues mentioned above by switching from multilib to platform-specific toolchains.
Also see https://github.com/bitcoin/bitcoin/pull/22456.
---
Here are examples of building for different scenarions:
- Linux, `x86_64` or `arm64`, building with depends natively:
```sh
$ gmake -C depends -j $(nproc)
$ cmake -B build --toolchain depends/$(./depends/config.sub $(./depends/config.guess))/toolchain.cmake
$ cmake --build build -j $(nproc)
```
- Linux, `x86_64` or `arm64`, cross compiling for `i686-pc-linux-gnu`:
```sh
$ sudo apt install g++-i686-linux-gnu binutils-i686-linux-gnu
$ export HOST=i686-linux-gnu
$ gmake -C depends -j $(nproc)
$ cmake -B build-${HOST} --toolchain depends/${HOST}/toolchain.cmake
$ cmake --build build-${HOST} -j $(nproc)
```
- Linux, `x86_64`, cross compiling for `arm64`:
```sh
$ sudo apt install g++-aarch64-linux-gnu binutils-aarch64-linux-gnu
$ export HOST=aarch64-linux-gnu
$ gmake -C depends -j $(nproc)
$ cmake -B build-${HOST} --toolchain depends/${HOST}/toolchain.cmake
$ cmake --build build-${HOST} -j $(nproc)
```
- Linux, `arm64`, cross compiling for `x86_64`:
```sh
$ sudo apt install g++-x86-64-linux-gnu binutils-x86-64-linux-gnu
$ export HOST=x86_64-linux-gnu
$ gmake -C depends -j $(nproc)
$ cmake -B build-${HOST} --toolchain depends/${HOST}/toolchain.cmake
$ cmake --build build-${HOST} -j $(nproc)
```
ACKs for top commit:
fanquake:
ACK de9b436ba3
BrandonOdiwuor:
ACK de9b436ba3
Tree-SHA512: 453b4744974cdf56d6edfdbe93bb11e3bae3f9bc9cd99b9c57aee74e65fcdd3ac011a1dcc19f485ea3be427f4e9c6c6b0d704881369f719620f0cb299123e561
Otherwise the build will fail with warnings about missing Python (removed
in the next commit). However the non-gui build does not need Python.
Note that #25573 contains the same change, because the use of -static-pie
causes warnings when combined with our/CMakes own PIE handling.
Can do this now that the GUI has been split out.
riscv64-linux-gnu failus due to
https://github.com/boostorg/test/issues/345:
```bash
[102%] Linking CXX executable ../../bin/test_bitcoin
/gnu/store/r03804zpq5i6wsalx0yaqrr5jb7pqrmv-binutils-cross-riscv64-linux-gnu-2.46.0/bin/riscv64-linux-gnu-ld: CMakeFiles/test_bitcoin.dir/main.cpp.o: in function `boost::fpe::disable(unsigned int)':
/bitcoin/depends/riscv64-linux-gnu/boost/include/boost/test/impl/execution_monitor.ipp:1538:(.text+0x9dc8): warning: fedisableexcept is not implemented and will always fail
/gnu/store/r03804zpq5i6wsalx0yaqrr5jb7pqrmv-binutils-cross-riscv64-linux-gnu-2.46.0/bin/riscv64-linux-gnu-ld: CMakeFiles/test_bitcoin.dir/main.cpp.o: in function `boost::fpe::enable(unsigned int)':
/bitcoin/depends/riscv64-linux-gnu/boost/include/boost/test/impl/execution_monitor.ipp:1502:(.text+0x9d76): warning: feenableexcept is not implemented and will always fail
collect2: error: ld returned 1 exit status
```
Darwin could be done after something like
https://github.com/bitcoin/bitcoin/pull/35756.
2cb3bfa8df scripted-diff: Use long form of shell options in Guix scripts (Hennadii Stepanov)
711eb10f08 guix: Add copyright headers to Guix scripts (Hennadii Stepanov)
80f831494e guix: Fix `glibc` version in comment (Hennadii Stepanov)
8916f7967e scripted-diff: Use C.UTF-8 locale in Guix scripts (Hennadii Stepanov)
Pull request description:
The C.UTF-8 locale is set by default in `guix shell`, and there is no reason to avoid it nowadays. This PR also silences superfluous warnings from Qt tools, making build logs cleaner and other issues easier to spot. For example:
```
Detected locale "C" with character encoding "ANSI_X3.4-1968", which is not UTF-8.
Qt depends on a UTF-8 locale, and has switched to "C.UTF-8" instead.
If this causes problems, reconfigure your locale. See the locale(1) manual
for more information.
```
Locales in the `guix-*` launch scripts have been updated as well for consistency with the rest of the codebase.
Additionally, the headers of the Guix scripts have been adjusted for [uniformity](https://github.com/bitcoin/bitcoin/pull/35775#discussion_r3636959268).
ACKs for top commit:
fanquake:
ACK 2cb3bfa8df
Tree-SHA512: 9e21d4ad50f5d583efdd8f79d9f96d45a92f1982ea3a422565bceea38aa50bbd9a9360972b37a1b49907e67523d68f3d1f889cb26179ce735453eb4fae4d3fa4
The C.UTF-8 locale is set by default in `guix shell`, and there is no
reason to avoid it nowadays. This change also silences superfluous
warnings from Qt tools, making build logs cleaner and other issues
easier to spot.
Locales in the `guix-*` launch scripts have been updated as well for
consistency with the rest of the codebase.
-BEGIN VERIFY SCRIPT-
sed -i "s/\<export LC_ALL=C\>/export LC_ALL=C.UTF-8/g" \
$( git grep -l "export LC_ALL=C" ./contrib/guix/* )
-END VERIFY SCRIPT-
Using shell parameter expansion ensures that on non-Linux hosts the
build script won't pass an empty string argument to CMake. This resolves
the "Ignoring empty string" warning.
d62f46eed4 doc: Add `INSTALL.md` to Linux release tarballs (Hennadii Stepanov)
Pull request description:
Closes https://github.com/bitcoin/bitcoin/issues/32097:
> Better I think would be to add instructions for the most popular desktop distros.
ACKs for top commit:
willcl-ark:
utACK d62f46eed4
achow101:
ACK d62f46eed4
janb84:
re ACK d62f46eed4
w0xlt:
ACK d62f46eed4
sedited:
ACK d62f46eed4
Tree-SHA512: eab6904b6acc63eaf0e13112f52d856ee630ce4110fb368f4f5f170655def91706bb7c7f70c123186a51461ea3d48ef026a4e026d665675f1fd76f2d0bbca68c
2594d5a189 build: Remove CMAKE_SKIP_BUILD_RPATH and SKIP_BUILD_RPATH settings (Henry Romp)
Pull request description:
Remove CMAKE_SKIP_BUILD_RPATH and SKIP_BUILD_RPATH settings that are no longer needed after reordering the Guix build script to perform binary checks after installation.
This PR also removes the unused CMake maintenance targets (`check-security` and `check-symbols`) and updates the Guix security checks to include binaries in the `libexec/` directory (added in PR #31679).
ACKs for top commit:
purpleKarrot:
ACK 2594d5a189
hebasto:
ACK 2594d5a189.
Tree-SHA512: ed451a298f5aae05c177b0033b092faaa7536caeaa3d84da9b8b611e2aa905e1dd337e57aef0efd69ce6ce6ac0cf77dc57adf175079b95bf53dd96d5d0c8118b
Remove CMake settings that are no longer needed after reordering Guix build script to perform binary checks after installation.
Also removes unused CMake maintenance targets (check-security and check-symbols) and updates security checks to include libexec/ directory binaries (see PR #31679).
59c4898994 guix: remove python-pydantic-core input from LIEF (fanquake)
9f2a6927d3 guix: use Clang & LLVM 19 for macOS build (fanquake)
9570ddbec9 guix: update time-machine to 5cb84f2013c5b1e48a7d0e617032266f1e6059e2 (fanquake)
7b5cc276aa guix: patch around riscv issue with newer (2.40+) binutils (fanquake)
91b5cbaabb ci: use Debian Trixie for macOS cross job (fanquake)
Pull request description:
5cb84f2013 isn't super recent, but it's enough to get access to some newer packages, such as LLVM 19, and avoids having to add any further work arounds for things that we know are fixed later (i.e nsis). Once things upstream have stabilized a bit more (the `core-updates` branch was fairly recently merged), we could look at bumping to something newer.
Package updates:
(base) glibc 2.35 -> 2.39
binutils 2.38 -> 2.41
diffutils 3.8 -> 3.10
gawk 5.2.1 -> 5.3.0
git-minimal 2.45.2 -> 2.46.0
grep 3.8 -> 3.11
gzip 1.12 -> 1.13
linux-headers 6.1.106 -> 6.1.119
make 4.3 -> 4.4.1
xz 5.2.8 -> 5.4.5
CMake 3.30 becomes available.
Clang/LLVM 19 becomes available.
Could be used for #32764.
ACKs for top commit:
hebasto:
re-ACK 59c4898994.
willcl-ark:
ACK 59c4898994
Tree-SHA512: c44965d5a315e4c862f5e40d8e98c645713405fec72a61055f95b6c68b7d2dcc69a61a084e397a4556d4c1df18f1cfa7a905234643fe4a7df9c58d486e26c097
4b41f99d57 build: Move CMAKE_SKIP_INSTALL_RPATH from CMake to Guix script (Henry Romp)
Pull request description:
Remove `CMAKE_SKIP_INSTALL_RPATH` from CMakeLists.txt and add `CMAKE_SKIP_RPATH` to the Guix build script. This keeps build-environment-specific settings in the build scripts rather than hardcoded in the CMake configuration.
ACKs for top commit:
purpleKarrot:
ACK 4b41f99d57
janb84:
re ACK 4b41f99d57
Tree-SHA512: 74d6af382476d731f10f9833978d670e9981c160ba306d0e9d4b1ad1e9b9960b8d03a3b9b608e234edb1c0c2c7a2b4f9f606a2a7887b7a153792159e71ae9b21
Remove CMAKE_SKIP_INSTALL_RPATH from CMakeLists.txt and add CMAKE_SKIP_RPATH to the Guix build script. This keeps build-environment-specific settings in the build scripts rather than hardcoded in the CMake configuration.
8e434a8499 macdeploy: rename macOS output to bitcoin-macos-app.zip (fanquake)
05353d9cf0 macdeploy: combine appname & -zip arguments (fanquake)
Pull request description:
Output `bitcoin-macos-app.zip`, similar to what we do for Windows: `bitcoin-win64-setup.exe`.
ACKs for top commit:
hodlinator:
re-ACK 8e434a8499
willcl-ark:
ACK 8e434a8499
Tree-SHA512: e762c9866630c4f8c577027ee9492d74a5c7f4b194df73876d702703b9100c356a30986c2f209ba3f3e2d483017f5e61596a2a7cdfae0a684f8dc244420cd108
f49840dd90 doc: Fix typo in files.md (Ryan Ofsky)
f5cf0b1ccc bitcoin wrapper: improve help output (Ryan Ofsky)
c810b168b8 doc: Add description of installed files to files.md (Ryan Ofsky)
94ffd01a02 doc: Add release notes describing libexec/ binaries (Ryan Ofsky)
cd97905ebc cmake: Move internal binaries from bin/ to libexec/ (Ryan Ofsky)
Pull request description:
This change moves binaries that are not typically invoked directly by users from the `bin/` directory to the `libexec/` directory in CMake installs and binary releases. The goal of the PR is to introduce a distinction between internal and external binaries so starting with #31802, we can use IPC to implement features in new binaries without adding those binaries to the CLI. The change also helps reduce clutter in `bin/`, making it easier for users to identify useful tools to run. Summary of changes:
- For **source builds** (i.e. developer builds) — There are no changes.
- For **source installs** (i.e. `cmake --install` result) — `test_bitcoin`, `test_bitcoin-qt`, and `bench_bitcoin` are installed in `${CMAKE_PREFIX_PATH}/libexec` instead of `${CMAKE_PREFIX_PATH}/bin`, so they are no longer on the system `PATH`. However, they can still be invoked from the `libexec/` directory, or from the CLI as `bitcoin test`, `bitcoin test-gui`, and `bitcoin bench`, respectively.
- For **binary releases** — Since `test_bitcoin` is the only test binary enabled in releases, the only change is moving `test_bitcoin` from `bin/` to `libexec/`.
<details><summary>Details</summary>
<p>
The table below shows the install location of each binary after this change, and the availability of each binary.
| Binary | Location | Availability | Change |
|----------------------|--------------|----------------------|-------------------------------|
| `bitcoin` | `bin/` | 📦 Binary release (since #31375) | Unchanged |
| `bitcoin-cli` | `bin/` | 📦 Binary release | Unchanged |
| `bitcoind` | `bin/` | 📦 Binary release | Unchanged |
| `bitcoin-qt` | `bin/` | 📦 Binary release | Unchanged |
| `bitcoin-tx` | `bin/` | 📦 Binary release | Unchanged |
| `bitcoin-util` | `bin/` | 📦 Binary release | Unchanged |
| `bitcoin-wallet` | `bin/` | 📦 Binary release | Unchanged |
| `bench_bitcoin` | `libexec/` | 🛠 Source build only | Moved from `bin/` |
| `bitcoin-chainstate` | `libexec/` | 🛠 Source build only | Newly installed (was built) |
| `bitcoin-gui` | `libexec/` | 🛠 Source build only (until #31802) | Moved from `bin/` |
| `bitcoin-node` | `libexec/` | 🛠 Source build only (until #31802) | Moved from `bin/` |
| `test_bitcoin` | `libexec/` | 📦 Binary release | Moved from `bin/` |
| `test_bitcoin-qt` | `libexec/` | 🛠 Source build only | Moved from `bin/` |
</p>
</details>
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).
ACKs for top commit:
l0rinc:
re-ACK f49840dd90
Sjors:
re-ACK f49840dd90
achow101:
ACK f49840dd90
janb84:
re ACK f49840dd90
BrandonOdiwuor:
Tested ACK f49840dd90
hodlinator:
re-ACK f49840dd90
willcl-ark:
utACK f49840dd90
Tree-SHA512: 858a2e1a53db11ee3c5c759bfdeea566f242b9ce5e8a898fa435222e41662b8184577c0dc2c4c058294b4de41d8cb3ba3e5d24c748c280efa4a3f84e3ec4344d
Currently there is a warning for this in guix-build, but we also need
one in guix-codesign, otherwise the codesigned hashes are not
reproducible.
Move common functionality into prelude and call the function in both
guix actions.
This change moves binaries that are not typically invoked directly by users
from the `bin/` directory to the `libexec/` directory in CMake installs and
binary releases. The goal is to simplify the contents of `bin/` for end users
while still making all binaries available when needed. After this change, the
binaries remaining in `bin/` are:
- bitcoin
- bitcoin-cli
- bitcoind
- bitcoin-qt
- bitcoin-tx
- bitcoin-util
- bitcoin-wallet
And the binaries that are moved to `libexec/` are:
- bench_bitcoin
- bitcoin-chainstate(*)
- bitcoin-gui(***)
- bitcoin-node(***)
- test_bitcoin(**)
- test_bitcoin-qt
(*) bitcoin-chainstate was previously missing an install rule and was actually
not installed even when it was enabled.
(**) test_bitcoin is the only libexec/ binary that is currently included in
bitcoin binary releases. The others are only installed when building from
source with relevant cmake options enabled.
(***) bitcoin-node and bitcoin-gui are not currently built by default or
included in binary releases but both of these changes are planned and
implemented in #31802
