Merge bitcoin/bitcoin#30454: build: Introduce CMake-based build system

41051290ab cmake: Ignore build subdirectories within source directory (Hennadii Stepanov) 6ce50fd9d0 doc: Update for CMake-based build system (Hennadii Stepanov) 9730288a0c ci: Migrate CI scripts to CMake (Hennadii Stepanov) c360837ca5 cmake, lint: Adjust `lint_includes_build_config` (Hennadii Stepanov) 3885441ee0 cmake: Add presets for native Windows builds (Hennadii Stepanov) 7681746b20 cmake: Add vcpkg manifest file (Hennadii Stepanov) 8b6f1c4353 cmake: Add `Coverage` and `CoverageFuzz` scripts (Hennadii Stepanov) 65bdbc1ff2 cmake: Add `docs` build target (Hennadii Stepanov) fb75ebbc33 cmake: Add compiler diagnostic flags (Hennadii Stepanov) e821f0a37a cmake: Migrate Guix build scripts to CMake (Hennadii Stepanov) 747adb6ffe cmake: Add `Maintenance` module (Hennadii Stepanov) 1f60b30df0 cmake: Add `APPEND_{CPP,C,CXX,LD}FLAGS` cache variables (Hennadii Stepanov) 2b43c45b13 cmake: Add `AddWindowsResources` module (Hennadii Stepanov) 973a3b0c5d cmake: Implement `install` build target (Hennadii Stepanov) 84ac35cfd4 cmake: Add cross-compiling support (Hennadii Stepanov) 0d01c228a7 build: Generate `toolchain.cmake` in depends (Hennadii Stepanov) 91a799247d depends: Add host-specific `cmake_system_version` variables (Hennadii Stepanov) 9b31209b4c depends: Rename `cmake_system` -> `cmake_system_name` (Hennadii Stepanov) 4a5208a81d Revert "build, qt: Do not install *.prl files" (Hennadii Stepanov) 6522af62af depends: Amend handling flags environment variables (Hennadii Stepanov) 90cec4d251 cmake: Add `MULTIPROCESS` option (Hennadii Stepanov) bb1a450dcb cmake: Build `bitcoin-chainstate` executable (Hennadii Stepanov) aed38ea58c cmake: Build `bitcoinkernel` library (Hennadii Stepanov) 975d67369b cmake: Build `test_bitcoin-qt` executable (Hennadii Stepanov) 10fcc668a3 cmake: Add `WITH_DBUS` option (Hennadii Stepanov) 5bb5a4bc75 cmake: Add `libqrencode` optional package support (Hennadii Stepanov) 57a6e2ef4a cmake: Build `bitcoin-qt` executable (Hennadii Stepanov) 30f642952c cmake: Add `WERROR` option (Hennadii Stepanov) c98d4a4c34 cmake: Add `REDUCE_EXPORTS` option (Hennadii Stepanov) a01cb6e63f cmake: Add `HARDENING` option (Hennadii Stepanov) a8a2e364ac cmake: Add Python-based tests (Hennadii Stepanov) 3d85379570 cmake: Add fuzzing options (Hennadii Stepanov) 908530e312 cmake: Add `SANITIZERS` option (Hennadii Stepanov) 8bb0e85631 cmake: Build `bench_bitcoin` executable (Hennadii Stepanov) 801735163a cmake: Add external signer support (Hennadii Stepanov) 353e0c9e96 cmake: Add `systemtap-sdt` optional package support (Hennadii Stepanov) d2fda82b49 cmake: Add `libzmq` optional package support (Hennadii Stepanov) ae7b39a0e1 cmake: Add `libminiupnpc` optional package support (Hennadii Stepanov) 6480e1dcdb cmake: Add `libnatpmp` optional package support (Hennadii Stepanov) e73e9304a1 cmake: Build `bitcoin-util` executable (Hennadii Stepanov) 027c6d7caa cmake: Build `bitcoin-tx` executable (Hennadii Stepanov) d10c5c34c3 cmake: Add wallet functionality (Hennadii Stepanov) ab2e99b0d9 cmake: Create test suite for `ctest` (Hennadii Stepanov) 959370bd76 cmake: Build `test_bitcoin` executable (Hennadii Stepanov) b27bf9700d cmake: Build `bitcoin-cli` executable (Hennadii Stepanov) a9813df826 cmake: Build `bitcoind` executable (Hennadii Stepanov) 97829ce2d5 cmake: Add `FindLibevent` module (Hennadii Stepanov) 3118e40c61 cmake: Build `bitcoin_consensus` library (Hennadii Stepanov) 809a2f1929 cmake: Build `bitcoin_util` static library (Hennadii Stepanov) 0a9a521a70 cmake: Build `bitcoin_crypto` library (Hennadii Stepanov) 958971f476 cmake: Build `univalue` static library (Hennadii Stepanov) 752747fda8 cmake: Generate `obj/build.h` header (Hennadii Stepanov) 1f0a78edf3 cmake: Build `minisketch` static library (Hennadii Stepanov) 12bfbc8154 cmake: Build `leveldb` static library (Hennadii Stepanov) 51985c5304 cmake: Build `crc32c` static library (Hennadii Stepanov) db7a198f29 cmake: Build `secp256k1` subtree (Hennadii Stepanov) dbb7ed14e8 cmake: Add `ccache` support (Hennadii Stepanov) cedfdf6c72 cmake: Redefine/adjust per-configuration flags (Hennadii Stepanov) b6b5e732c8 cmake: Add global compiler and linker flags (Hennadii Stepanov) f98327931b cmake: Add `TryAppendLinkerFlag` module (Hennadii Stepanov) 4a0af29697 cmake: Add `TryAppendCXXFlags` module (Hennadii Stepanov) 35cffc497d cmake: Add POSIX threads support (Hennadii Stepanov) fd72d00ffe cmake: Add position independent code support (Hennadii Stepanov) 07069e2bb0 cmake: Add introspection module (Hennadii Stepanov) 27d687fc1f cmake: Add `config/bitcoin-config.h` support (Hennadii Stepanov) fe5cdace5f cmake: Print compiler and linker flags in summary (Hennadii Stepanov) 70683884c5 cmake: Introduce interface libraries to encapsulate common flags (Hennadii Stepanov) a2317e27b7 cmake: Add root `CMakeLists.txt` file (Hennadii Stepanov) Pull request description: This PR introduces a new CMake-based build system, which is a drop-in replacement for the current Autotools-based build system. ML announcement: https://groups.google.com/g/bitcoindev/c/hgKkfQWzrTo As discussed during the recent CoreDev meetup in April, the switch from Autotools to CMake is intended to happen as soon as possible after branching 28.x off, which means that 29.0 will be built using CMake. This PR branch is essentially the [staging branch](https://github.com/hebasto/bitcoin/tree/cmake-staging), with every change reviewed and tested by a group of contributors, including (in alphabetical order): - [**achow101**](https://github.com/achow101) - [**fanquake**](https://github.com/fanquake) - [**maflcko**](https://github.com/maflcko) - [**m3dwards**](https://github.com/m3dwards) - [**pablomartin4btc**](https://github.com/pablomartin4btc) - [**real-or-random**](https://github.com/real-or-random) - [**ryanofsky**](https://github.com/ryanofsky) - [**sipsorcery**](https://github.com/sipsorcery) - [**TheCharlatan**](https://github.com/TheCharlatan) - [**theStack**](https://github.com/theStack) - [**theuni**](https://github.com/theuni) - [**vasild**](https://github.com/vasild) Reviewing in a separate staging repo was suggested in https://github.com/bitcoin/bitcoin/pull/27060#issuecomment-1431798320. The accompanying changes to the OSS-Fuzz project are available in https://github.com/hebasto/oss-fuzz/pull/8. Please refer to the [build options parity table](https://gist.github.com/hebasto/2ef97d3a726bfce08ded9df07f7dab5e). The "auto" value is no longer available; non-default values must be specified explicitly. Additionally, the new default values have been chosen to suit the everyday build experience for the majority of developers. System requirements for using the CMake-based build system: - CMake >= 3.22 (if not available in your system's repository, it can be downloaded from https://cmake.org/download/) - a build tool of your choice: - any Make (GNU Make is no longer a requirement); GNU Make is still required to build depends - Ninja (https://ninja-build.org/) - MSBuild - Xcode A note for Windows users: The default installation of the latest version of MSVC 17.10.4 includes both CMake 3.28.3 and the vcpkg package manager). --- We, the build system developers, kindly ask reviewers to refrain from making suggestions that are not directly related to the migration process or can be implemented separately. Bugs in the scripts and errors in the updated documentation should be the focus of this PR. Please be advised that comments not aligned with this PR's goal may be ignored. Thank you all for your understanding. ACKs for top commit: maflcko: review ACK 41051290ab 🐥 sipsorcery: ACK 41051290ab. vasild: ACK 41051290ab TheCharlatan: ACK 41051290ab pablomartin4btc: tACK 41051290ab i-am-yuvi: tACK [`4105129`](41051290ab) theuni: ACK 41051290ab. fanquake: ACK 41051290ab Tree-SHA512: 6c1445054436c6c00ad63bfa0f19d64091a2b25c9bd694f85bf2218ac358ffb774d6c000685b3ca1e9b50401babed989fa2a0694b774c211d226bfd1944c9b39
2026-03-25 06:55:32 +01:00 · 2024-08-28 10:51:24 +01:00
parent 80f00cafde 41051290ab
commit 338bc2cd26
113 changed files with 5878 additions and 524 deletions
--- a/test/CMakeLists.txt
+++ b/test/CMakeLists.txt
@@ -0,0 +1,48 @@
+# Copyright (c) 2023-present The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or https://opensource.org/license/mit/.
+
+function(create_test_config)
+  set(abs_top_srcdir ${PROJECT_SOURCE_DIR})
+  set(abs_top_builddir ${PROJECT_BINARY_DIR})
+  set(EXEEXT ${CMAKE_EXECUTABLE_SUFFIX})
+
+  macro(set_configure_variable var conf_var)
+    if(${var})
+      set(${conf_var}_TRUE "")
+    else()
+      set(${conf_var}_TRUE "#")
+    endif()
+  endmacro()
+
+  set_configure_variable(ENABLE_WALLET ENABLE_WALLET)
+  set_configure_variable(WITH_SQLITE USE_SQLITE)
+  set_configure_variable(WITH_BDB USE_BDB)
+  set_configure_variable(BUILD_CLI BUILD_BITCOIN_CLI)
+  set_configure_variable(BUILD_UTIL BUILD_BITCOIN_UTIL)
+  set_configure_variable(BUILD_WALLET_TOOL BUILD_BITCOIN_WALLET)
+  set_configure_variable(BUILD_DAEMON BUILD_BITCOIND)
+  set_configure_variable(BUILD_FUZZ_BINARY ENABLE_FUZZ_BINARY)
+  set_configure_variable(WITH_ZMQ ENABLE_ZMQ)
+  set_configure_variable(ENABLE_EXTERNAL_SIGNER ENABLE_EXTERNAL_SIGNER)
+  set_configure_variable(WITH_USDT ENABLE_USDT_TRACEPOINTS)
+
+  configure_file(config.ini.in config.ini @ONLY)
+endfunction()
+
+create_test_config()
+
+file(MAKE_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/functional)
+file(MAKE_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/fuzz)
+file(MAKE_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/util)
+
+file(GLOB_RECURSE functional_tests RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} functional/*)
+foreach(script ${functional_tests} fuzz/test_runner.py util/rpcauth-test.py util/test_runner.py)
+  if(CMAKE_HOST_WIN32)
+    set(symlink)
+  else()
+    set(symlink SYMBOLIC)
+  endif()
+  file(CREATE_LINK ${CMAKE_CURRENT_SOURCE_DIR}/${script} ${CMAKE_CURRENT_BINARY_DIR}/${script} COPY_ON_ERROR ${symlink})
+endforeach()
+unset(functional_tests)
--- a/test/README.md
+++ b/test/README.md
@@ -13,13 +13,15 @@ interfaces.
 - [util](/test/util) which tests the utilities (bitcoin-util, bitcoin-tx, ...).
 - [lint](/test/lint/) which perform various static analysis checks.

-The util tests are run as part of `make check` target. The fuzz tests, functional
+The util tests are run as part of `ctest` invocation. The fuzz tests, functional
 tests and lint scripts can be run as explained in the sections below.

 # Running tests locally

 Before tests can be run locally, Bitcoin Core must be built.  See the [building instructions](/doc#building) for help.

+The following examples assume that the build directory is named `build`.
+
 ## Fuzz tests

 See [/doc/fuzzing.md](/doc/fuzzing.md)
@@ -45,19 +47,19 @@ set PYTHONUTF8=1
 Individual tests can be run by directly calling the test script, e.g.:

 ```
-test/functional/feature_rbf.py
+build/test/functional/feature_rbf.py
 ```

 or can be run through the test_runner harness, eg:

 ```
-test/functional/test_runner.py feature_rbf.py
+build/test/functional/test_runner.py feature_rbf.py
 ```

 You can run any combination (incl. duplicates) of tests by calling:

 ```
-test/functional/test_runner.py <testname1> <testname2> <testname3> ...
+build/test/functional/test_runner.py <testname1> <testname2> <testname3> ...
 ```

 Wildcard test names can be passed, if the paths are coherent and the test runner
@@ -65,34 +67,34 @@ is called from a `bash` shell or similar that does the globbing. For example,
 to run all the wallet tests:

 ```
-test/functional/test_runner.py test/functional/wallet*
-functional/test_runner.py functional/wallet* (called from the test/ directory)
-test_runner.py wallet* (called from the test/functional/ directory)
+build/test/functional/test_runner.py test/functional/wallet*
+functional/test_runner.py functional/wallet*  # (called from the build/test/ directory)
+test_runner.py wallet*  # (called from the build/test/functional/ directory)
 ```

 but not

 ```
-test/functional/test_runner.py wallet*
+build/test/functional/test_runner.py wallet*
 ```

 Combinations of wildcards can be passed:

 ```
-test/functional/test_runner.py ./test/functional/tool* test/functional/mempool*
+build/test/functional/test_runner.py ./test/functional/tool* test/functional/mempool*
 test_runner.py tool* mempool*
 ```

 Run the regression test suite with:

 ```
-test/functional/test_runner.py
+build/test/functional/test_runner.py
 ```

 Run all possible tests with

 ```
-test/functional/test_runner.py --extended
+build/test/functional/test_runner.py --extended
 ```

 In order to run backwards compatibility tests, first run:
@@ -107,7 +109,7 @@ By default, up to 4 tests will be run in parallel by test_runner. To specify
 how many jobs to run, append `--jobs=n`

 The individual tests and the test_runner harness have many command-line
-options. Run `test/functional/test_runner.py -h` to see them all.
+options. Run `build/test/functional/test_runner.py -h` to see them all.

 #### Speed up test runs with a RAM disk

@@ -130,7 +132,7 @@ For example running the test suite with `--jobs=100` might need a 4 GiB RAM disk
 To use, run the test suite specifying the RAM disk as the `cachedir` and `tmpdir`:

 ```bash
-test/functional/test_runner.py --cachedir=/mnt/tmp/cache --tmpdir=/mnt/tmp
+build/test/functional/test_runner.py --cachedir=/mnt/tmp/cache --tmpdir=/mnt/tmp
 ```

 Once finished with the tests and the disk, and to free the RAM, simply unmount the disk:
@@ -151,7 +153,7 @@ Configure the RAM disk size, expressed as the number of blocks, at the end of th
 (`4096 MiB * 2048 blocks/MiB = 8388608 blocks` for 4 GiB). To run the tests using the RAM disk:

 ```bash
-test/functional/test_runner.py --cachedir=/Volumes/ramdisk/cache --tmpdir=/Volumes/ramdisk/tmp
+build/test/functional/test_runner.py --cachedir=/Volumes/ramdisk/cache --tmpdir=/Volumes/ramdisk/tmp
 ```

 To unmount:
@@ -193,14 +195,14 @@ pkill -9 bitcoind
 ##### Data directory cache

 A pre-mined blockchain with 200 blocks is generated the first time a
-functional test is run and is stored in test/cache. This speeds up
+functional test is run and is stored in build/test/cache. This speeds up
 test startup times since new blockchains don't need to be generated for
 each test. However, the cache may get into a bad state, in which case
 tests will fail. If this happens, remove the cache directory (and make
 sure bitcoind processes are stopped as above):

 ```bash
-rm -rf test/cache
+rm -rf build/test/cache
 killall bitcoind
 ```

@@ -236,7 +238,7 @@ aggregate log by running the `combine_logs.py` script. The output can be plain
 text, colorized text or html. For example:

 ```
-test/functional/combine_logs.py -c <test data directory> | less -r
+build/test/functional/combine_logs.py -c <test data directory> | less -r
 ```

 will pipe the colorized logs from the test into less.
@@ -297,7 +299,7 @@ See this link for considerations: https://www.kernel.org/doc/Documentation/secur

 Often while debugging RPC calls in functional tests, the test might time out before the
 process can return a response. Use `--timeout-factor 0` to disable all RPC timeouts for that particular
-functional test. Ex: `test/functional/wallet_hd.py --timeout-factor 0`.
+functional test. Ex: `build/test/functional/wallet_hd.py --timeout-factor 0`.

 ##### Profiling

@@ -321,7 +323,7 @@ For ways to generate more granular profiles, see the README in

 ### Util tests

-Util tests can be run locally by running `test/util/test_runner.py`.
+Util tests can be run locally by running `build/test/util/test_runner.py`.
 Use the `-v` option for verbose output.

 ### Lint tests
--- a/test/functional/test_runner.py
+++ b/test/functional/test_runner.py
@@ -595,7 +595,7 @@ def run_tests(*, test_list, src_dir, build_dir, tmpdir, jobs=1, enable_coverage=
        print(f"{BOLD[1]}WARNING!{BOLD[0]} There may be insufficient free space in {tmpdir} to run the Bitcoin functional test suite. "
              f"Running the test suite with fewer than {min_space // (1024 * 1024)} MB of free space might cause tests to fail.")

-    tests_dir = src_dir + '/test/functional/'
+    tests_dir = build_dir + '/test/functional/'
    # This allows `test_runner.py` to work from an out-of-source build directory using a symlink,
    # a hard link or a copy on any platform. See https://github.com/bitcoin/bitcoin/pull/27561.
    sys.path.append(tests_dir)
--- a/test/lint/test_runner/src/main.rs
+++ b/test/lint/test_runner/src/main.rs
@@ -318,20 +318,14 @@ Please add any false positives, such as subtrees, or externally sourced files to
 }

 fn lint_includes_build_config() -> LintResult {
-    let config_path = "./src/config/bitcoin-config.h.in";
-    if !Path::new(config_path).is_file() {
-        assert!(Command::new("./autogen.sh")
-            .status()
-            .expect("command error")
-            .success());
-    }
+    let config_path = "./cmake/bitcoin-config.h.in";
    let defines_regex = format!(
        r"^\s*(?!//).*({})",
-        check_output(Command::new("grep").args(["undef ", "--", config_path]))
+        check_output(Command::new("grep").args(["define", "--", config_path]))
            .expect("grep failed")
            .lines()
            .map(|line| {
-                line.split("undef ")
+                line.split_whitespace()
                    .nth(1)
                    .unwrap_or_else(|| panic!("Could not extract name in line: {line}"))
            })