df9ebbf659 depends: use "mkdir -p" when installing xproto (fanquake)
Pull request description:
It looks like the mkdir detection in xproto is broken on Alpine. Ensure we always use `mkdir -p`.
Fixes#32494.
ACKs for top commit:
hebasto:
ACK df9ebbf659, I have reviewed the code and it looks OK.
janb84:
ACK df9ebbf659
willcl-ark:
ACK df9ebbf659
Tree-SHA512: 0f23b1096ffdf5ffa13115665dc42b65835b78bb0ab04a8be8f210980356953ab518e1273302fe4c9239361201f4f9ac737c0ebf10625f4817f81b65b3b25572
415650cea9 guix: move *-check.py scripts under contrib/guix (fanquake)
Pull request description:
These scripts are not meant for general developer usage. They are for use on the release binaries, which have been compiled in an environment that makes various assumptions in regards to c library, compiler options, hardening options, dependency patching etc.
Anyone is free to run these scripts against self-compiled binaries, but this isn't something we want to modify/generalize the scripts to support.
ACKs for top commit:
laanwj:
Code review ACK 415650cea9
TheCharlatan:
ACK 415650cea9
Tree-SHA512: 469cc5da9df014b4a9dc39080f31c3283641691209497cdb7cc5dc4767681a7f4b67425768e81838153ce39a26547ed94cd3be4dc27352d33960765bfac9d0e5
These scripts are not meant for general developer usage. They are for
use on the release binaries, which have been compiled in an environment
that makes various assumptions in regards to c library, compiler
options, hardening options, patching etc.
Using helper variables has two issues:
1. They contaminate the global namespace of the main build script.
2. They can be used as `set(var)`, effectively exposing a cache variable
`var`, which makes the toolchain file susceptible to the build
environment.
Native compilation is explained before cross-compilation. Move
install and (g)make steps up.
In the Configuring section, use Linux native compilation as the
example instead of Windows cross-compile.
With newly introduced libmultiprocess subtree, there's no need for depends
system to download and track changes to the upstream repository.
Note that adding the libmultiprocess subtree does not allow dropping
libmultiprocess packages from the depends build, because libmultiprocess
includes a code generation tool called mpgen, and in cross-compiled builds,
bitcoin core's cmake build system doesn't have access to a native toolchain and
can't build mpgen itself, so the depends system (or the native environment if
not using depends) needs to supply it.
Move parts of the int_get_build_id into a new int_get_build_properties
function. There is no change in behavior. This just organizes assignments
better so some build properties can be used to help compute build ids in the
next commit.
When ENABLE_IPC option is on, build with libmultiprocess subtree and
`add_subdirectory(src/ipc/libmultiprocess)` instead of external package
and `find_package(Libmultiprocess)` by default.
Behavior can be toggled with `WITH_EXTERNAL_LIBMULTIPROCESS` option. Using a
subtree should be more convenient for most bitcoin developers, but using an
external package is more convenient for developing in the libmultiprocess
repository.
The `WITH_EXTERNAL_LIBMULTIPROCESS` option is also used to avoid needing to
changing the depends build here. But in later commits, the depends build is
switched to use the add_subdirectory build as well.
Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
Rename WITH_MULTIPROCESS to ENABLE_IPC, because ENABLE_IPC is a more accurate
name for the feature. It controls whether the src/ipc/ directory is built and
whether IPC features like -ipcbind, -ipcconnect, and -ipcfd are available. It
does NOT currently enable multiprocess features which are implemented in #10102
building on top of the IPC features. It will also no longer (as of the next
commit), control whether a find_package call is made so the "WITH_" prefix is
also inappropriate.
-BEGIN VERIFY SCRIPT-
git grep -l WITH_MULTIPROCESS | xargs sed -i s/WITH_MULTIPROCESS/ENABLE_IPC/g
-END VERIFY SCRIPT-
According to the CMake docs, this is the correct way to setup a
toolchain file for cross-compilation using Clang. See
https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html#cross-compiling-using-clang
Internally it looks like CMake will only take this variable into account
if it detects the compiler to be Clang, so this shouldn't effect other
builds, but in the case of our Apple cross builds, we'd end up with a
duplicated `--target=arm64-apple-darwin` on the compiler line, given we
are already setting `--target` for Darwin builds.
Would fix#31748.
From the GNU make 3.82 release announcement:
* The 'define' make directive now allows a variable assignment operator
after the variable name, to allow for simple, conditional, or appending
multi-line variable assignment.
macOS ships with 3.81. This caused the multiprocess config options
to be ignored.
Fixes#32068
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
36b6f36ac4 build: require sqlite when building the wallet (Sjors Provoost)
Pull request description:
Require that sqlite is available in order to compile the wallet. Removes instances of `USE_SQLITE` since it is no longer possible to not have sqlite available.
The `NO_SQLITE` option is dropped from depends.
This is another step towards dropping the legacy wallet, extracted from #31250.
ACKs for top commit:
m3dwards:
ACK 36b6f36ac4
davidgumberg:
crACK 36b6f36ac4
hebasto:
re-ACK 36b6f36ac4.
Tree-SHA512: 870a0135671c80c4f28602119eb8637a1ed43b51b1673bfe88425782fb62ec6ef0f3d6baf0d5984d6a243779b0f63423fd4c4dc324ef87bffba13d63e05ad793
5dfef6b9b3 depends: remove NO_HARDEN option (fanquake)
Pull request description:
This was only needed to work around a (Libtool related iirc) Windows issue, when hardening was disabled. I can no-longer recreate this failure, so it'd be good to remove this Windows carveout.
ACKs for top commit:
davidgumberg:
crACK 5dfef6b9b3
laanwj:
Code review ACK 5dfef6b9b3
Tree-SHA512: 38657f09c537ba02ecaf0676d47087a835283cabfc81ad9b2d5e68858dcd7a610b6a1df6730920d40b48be2bbc55a45d6b8aea4364884b5f1c1bd12126940f5b
Require that sqlite is available in order to compile the wallet. Removes
instances of USE_SQLITE since it is no longer possible to not have
sqlite available.
The NO_SQLITE option is dropped from depends.
Co-authored-by: Ava Chow <github@achow101.com>
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
This only existed to workaround a (iirc libtool related) windows issue
that only occured when compiling without hardening. We no-longer use
libtool, and I can no-longer create the failure.
Similar to #31840, currently our Linux toolchain file contains:
```bash
set(CMAKE_AR "aarch64-linux-gnu-ar")
set(CMAKE_RANLIB "aarch64-linux-gnu-ranlib")
set(CMAKE_STRIP "aarch64-linux-gnu-strip")
set(CMAKE_OBJCOPY "aarch64-linux-gnu-objcopy")
set(CMAKE_OBJDUMP "")
```
`objdump` is currently only used for the macOS cross build, where it's
`llvm-objdump`, but we should be consistent in producing a toolchain
file that points to actual tools, rather than leaving variables unset.
3edaf0b428 depends: add missing Darwin objcopy (fanquake)
Pull request description:
Our CMake toolchain for a Darwin cross build currently contains:
```bash
set(CMAKE_AR "/usr/bin/llvm-ar")
set(CMAKE_RANLIB "/usr/bin/llvm-ranlib")
set(CMAKE_STRIP "/usr/bin/llvm-strip")
set(CMAKE_OBJCOPY "arm64-apple-darwin-objcopy")
set(CMAKE_OBJDUMP "/usr/bin/llvm-objdump")
```
`objcopy` isn't currently used for the Darwin build (only for Linux and splitting the debug symbols), but we shouldn't be producing a toolchain file that refers to nonexistent tools.
ACKs for top commit:
laanwj:
Code review ACK 3edaf0b428
theuni:
utACK 3edaf0b428
Tree-SHA512: b74deb9f3f053c37d03505e698419d4a14131131f12a042dab698a81f2ad76b71fd55c1d1afd5f5822cc50fdaad5acdab15a8b20626c56f705179add1165449f
f89f16846e depends: Fix compiling `libevent` package on NetBSD (Hennadii Stepanov)
Pull request description:
Libevent [introduced](https://github.com/libevent/libevent/pull/909) the [`typeof`](https://gcc.gnu.org/onlinedocs/gcc/Typeof.html) C language extension in the NetBSD-specific code, which was pulled into our depends in https://github.com/bitcoin/bitcoin/pull/21991.
However, GCC [states](https://gcc.gnu.org/onlinedocs/gcc/Alternate-Keywords.html):
> the various `-std` options disable certain keywords.
Due to our use of b042c4f053/depends/hosts/netbsd.mk (L1)
the `typeof` keyword is disabled, resulting in a compilation error:
```
$ gmake -C depends libevent CC=/usr/pkg/gcc14/bin/gcc CXX=/usr/pkg/gcc14/bin/g++
<snip>
[ 37%] Building C object CMakeFiles/event_core_static.dir/kqueue.c.o
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c: In function 'kq_setup_kevent':
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c:56:27: error: implicit declaration of function 'typeof' [-Wimplicit-function-declaration]
56 | #define INT_TO_UDATA(x) ((typeof(((struct kevent *)0)->udata))(intptr_t)(x))
| ^~~~~~
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c:190:30: note: in expansion of macro 'INT_TO_UDATA'
190 | out->udata = INT_TO_UDATA(ADD_UDATA);
| ^~~~~~~~~~~~
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c:56:64: error: expected expression before 'intptr_t'
56 | #define INT_TO_UDATA(x) ((typeof(((struct kevent *)0)->udata))(intptr_t)(x))
| ^~~~~~~~
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c:190:30: note: in expansion of macro 'INT_TO_UDATA'
190 | out->udata = INT_TO_UDATA(ADD_UDATA);
| ^~~~~~~~~~~~
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c:56:27: error: called object is not a function or function pointer
56 | #define INT_TO_UDATA(x) ((typeof(((struct kevent *)0)->udata))(intptr_t)(x))
| ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c:190:30: note: in expansion of macro 'INT_TO_UDATA'
190 | out->udata = INT_TO_UDATA(ADD_UDATA);
| ^~~~~~~~~~~~
gmake[3]: *** [CMakeFiles/event_core_static.dir/build.make:328: CMakeFiles/event_core_static.dir/kqueue.c.o] Error 1
<snip>
```
This PR resolves this issue by following GCC's [recommendation](https://gcc.gnu.org/onlinedocs/gcc/Typeof.html):
> write `__typeof__` instead of `typeof`.
ACKs for top commit:
fanquake:
ACK f89f16846e
Tree-SHA512: c0d2e535408db120535781f8518c616b0f5a39b1c6babb2a74e8e0565348aaf00b0f5a93cac0af7cf6d6bf028d5d58763fe71b3969ed9c7059fa7c3dca9d084c
b28917be36 depends: Make default `host` and `build` comparable (Hennadii Stepanov)
Pull request description:
To detect cross-compiling, the host and build platforms are compared. The `build` variable is always an output of `config.sub`, but the `host` is not. This can lead to false results. For example, on OpenBSD:
- host=amd64-unknown-openbsd7.5
- build=x86_64-unknown-openbsd7.5
This PR sets the default value of the `host` variable to the value of `build`, ensuring cross-compiling won't be triggered when the `HOST` variable is not set.
This PR fixes needless triggering of cross-compiling for CMake-built packages in depends on OpenBSD due to this code:eb85cacd29/depends/funcs.mk (L193-L197)
No changes in Guix build.
ACKs for top commit:
laanwj:
Concept and code review ACK b28917be36
theuni:
utACK b28917be36.
Tree-SHA512: 8c5835cb8b739355b71f7cb161b350ad8b038a00e6b1def36354ba228cea3dcb9883df3c9a8e79d7d0143241f6f054129fe90772b1b2579702db51237f9d66ff
Our CMake toolchain for a Darwin cross build currently contains:
```bash
set(CMAKE_AR "/usr/bin/llvm-ar")
set(CMAKE_RANLIB "/usr/bin/llvm-ranlib")
set(CMAKE_STRIP "/usr/bin/llvm-strip")
set(CMAKE_OBJCOPY "arm64-apple-darwin-objcopy")
set(CMAKE_OBJDUMP "/usr/bin/llvm-objdump")
```
`objcopy` isn't currently used for the Darwin build (only for Linux and
splitting the debug symbols), but we shouldn't be producing a toolchain
file that refers to nonexistent tools.
The `-ffile-prefix-map` compiler option implicitly enables
`-fprofile-prefix-map` in GCC or `-fcoverage-prefix-map` in Clang, which
can cause issues with coverage builds.
This change ensures that only the options necessary for build
reproducibility are applied.
d9c8aacce3 depends, refactor: Avoid hardcoding `host_prefix` in toolchain file (Hennadii Stepanov)
Pull request description:
This PR allows the entire `depends/<host_prefix>` directory to be relocatable.
Only `libevent` package configuration files are non-relocatable for the version `2.1.12-stable` we use now. However, this issue has been fixed upstream in 1f1593ff27 and friends.
Fixes https://github.com/bitcoin/bitcoin/issues/31050.
ACKs for top commit:
theuni:
Neat. utACK d9c8aacce3
ryanofsky:
Code review ACK d9c8aacce3
Tree-SHA512: c4c340722e63fc1da36fba2b15f025a6e1d477da1991194d678f546f2f3ea9454e2f0ff054aae6ae6c332a0781a597c3ce63b9018b46b8c258033f0d19efbef3
The `libevent` package defaults to the "Release" build type, which
overrides our per-build-type optimization flags with `-O3`.
To prevent this behavior, set `CMAKE_BUILD_TYPE` to "None", consistent
with how other packages are handled.
01df180bfb depends: add mold & ld.lld to gen_id (fanquake)
d032ac8063 depends: add *FLAGS to gen_id (fanquake)
Pull request description:
The depends cache should be busted when flags change, the same as any other tooling change. I'd also like to start passing `*FLAGS` into depends inside the Guix env, which, without this change, doesn't bust the cache.
ACKs for top commit:
hebasto:
ACK 01df180bfb.
Tree-SHA512: 3809359fe763af9dde484e0c6bd3e262c4c09fcbe2f96ccf64194f5f9f840f5476b9c9929cf7bda7b8c14efeffd369cdb8c233625b79a944e1380df20698246f
31a0e5f090 depends: Qt 5.15.16 (fanquake)
Pull request description:
Contains a handful of miscellaneous bug fixes.
We can drop a few of our patches.
See https://github.com/qt/qtbase/compare/v5.15.14-lts-lgpl...v5.15.16-lts-lgpl.
ACKs for top commit:
hebasto:
ACK 31a0e5f090.
TheCharlatan:
ACK 31a0e5f090
Tree-SHA512: dd7b3332dd6ecb95189bc72364883425fb8869e03850791d2ee92555a37046c7abaaee16575a0396f1ce9674856b894563dbd36868c2cf46f9fee48028fd967b
According to the description for pkg-config, "pkgconf is a
replacement for pkg-config, providing additional functionality
while also maintaining compatibility. This package only provides
a dependency link to the pkgconf package to help with package
upgrades. It can be safely removed."
Thus several scripts and markdown files are updated.
