30073e6b3a24cbe417c45cd5df6a3a2de0251e9d multiprocess: Add -ipcbind option to bitcoin-node (Russell Yanofsky)
73fe7d723084653671f2178ea1177a8627edfafa multiprocess: Add unit tests for connect, serve, and listen functions (Ryan Ofsky)
955d4077aac621697246bcb20a854ba97e37c519 multiprocess: Add IPC connectAddress and listenAddress methods (Russell Yanofsky)
4da20434d4d68b7933e39aca36faa6fd2264cc45 depends: Update libmultiprocess library for CustomMessage function and ThreadContext bugfix (Ryan Ofsky)
Pull request description:
Add `-ipcbind` option to `bitcoin-node` to make it listen on a unix socket and accept connections from other processes. The default socket path is `<datadir>/node.sock`, but this can be customized.
This option lets potential wallet, gui, index, and mining processes connect to the node and control it. See examples in #19460, #19461, and #30437.
Motivation for this PR, in combination with #30510, is be able to release a bitcoin core node binary that can generate block templates for a separate Stratum v2 mining service, like the one being implemented in https://github.com/Sjors/bitcoin/pull/48, that connects over IPC.
Other things to know about this PR:
- While the `-ipcbind` option lets other processes to connect to the `bitcoin-node` process, the only thing they can actually do after connecting is call methods on the [`Init`](https://github.com/bitcoin/bitcoin/blob/master/src/ipc/capnp/init.capnp#L17-L20) interface which is currently very limited and doesn't do much. But PRs [#30510](https://github.com/bitcoin/bitcoin/pull/30510), [#29409](https://github.com/bitcoin/bitcoin/pull/29409), and [#10102](https://github.com/bitcoin/bitcoin/pull/10102) expand the `Init` interface to expose mining, wallet, and gui functionality respectively.
- This PR is not needed for [#10102](https://github.com/bitcoin/bitcoin/pull/10102), which runs GUI, node, and wallet code in different processes, because [#10102](https://github.com/bitcoin/bitcoin/pull/10102) does not use unix sockets or allow outside processes to connect to existing processes. [#10102](https://github.com/bitcoin/bitcoin/pull/10102) lets parent and child processes communicate over internal socketpairs, not externally accessible sockets.
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).
ACKs for top commit:
achow101:
ACK 30073e6b3a24cbe417c45cd5df6a3a2de0251e9d
TheCharlatan:
Re-ACK 30073e6b3a24cbe417c45cd5df6a3a2de0251e9d
itornaza:
Code review ACK 30073e6b3a24cbe417c45cd5df6a3a2de0251e9d
Tree-SHA512: 2b766e60535f57352e8afda9c3748a32acb5a57b2827371b48ba865fa9aa1df00f340732654f2e300c6823dbc6f3e14377fca87e4e959e613fe85a6d2312d9c8
c2b779da4e7f1bf1a5c5d67ec94cba3027b42ee7 refactor: Manage dumptxoutset RAII classes with std::optional (Fabian Jahr)
4b5bf335adabd1586043caa72a98356a8255bc29 test: Add coverage for failing dumptxoutset behavior (Fabian Jahr)
Pull request description:
This adds a test that checks that network activity is not suspended if dumptxoutset fails in the middle of its process which is implemented with the `NetworkDisable` RAII class. I would have liked to add coverage for the `TemporaryRollback` RAII class but that seems a lot more tricky since the failure needs to happen at some point after the rollback and on the scale of our test chain here I couldn't find a way to do it yet. This was requested by pablomartin4btc here: https://github.com/bitcoin/bitcoin/pull/30808#pullrequestreview-2280450117. To test the test you can comment out the content of the destructor of `NetworkDisable`.
It also addresses the feedback by ryanofsky to use `std::optional` instead of `std::unique_ptr` for the management of the RAII object: https://github.com/bitcoin/bitcoin/pull/30808#discussion_r1744149228
ACKs for top commit:
achow101:
ACK c2b779da4e7f1bf1a5c5d67ec94cba3027b42ee7
pablomartin4btc:
cr & tACK c2b779da4e7f1bf1a5c5d67ec94cba3027b42ee7
tdb3:
ACK c2b779da4e7f1bf1a5c5d67ec94cba3027b42ee7
BrandonOdiwuor:
Code Review ACK c2b779da4e7f1bf1a5c5d67ec94cba3027b42ee7
theStack:
ACK c2b779da4e7f1bf1a5c5d67ec94cba3027b42ee7
Tree-SHA512: 9556e75014a2599bb870b70faf887608b332f2312626333f771d4ec11c04f863a2cf17e223ec473d4e8b0c9e8008394a4e0c321561f7ef3a2eec713dcfaea58a
ee47ca29d6ef55650a0af63bca817c5d494f31ef init: fix fatal error on '-wallet' negated option value (furszy)
Pull request description:
Currently, if users provide a double negated value such as '-nowallet=0' or a non-boolean
convertible value to a negated option such as '-nowallet=not_a_boolean', the initialization
process results in a fatal error, causing an unclean shutdown and displaying a poorly
descriptive error message:
"JSON value of type bool is not of expected type string." (On bitcoind. The GUI
does not display any error msg - upcoming PR -).
This PR fixes the issue by ensuring that only string values are returned in the
the "wallet" settings list, failing otherwise. It also improves the clarity of the
returned error message.
Note:
This bug was introduced in https://github.com/bitcoin/bitcoin/pull/22217. Where the `GetArgs("-wallet")` call was
replaced by `GetSettingsList("-wallet")`.
ACKs for top commit:
achow101:
ACK ee47ca29d6ef55650a0af63bca817c5d494f31ef
ryanofsky:
Code review ACK ee47ca29d6ef55650a0af63bca817c5d494f31ef, just adding the suggested test since last review
TheCharlatan:
ACK ee47ca29d6ef55650a0af63bca817c5d494f31ef
ismaelsadeeq:
Tested ACK ee47ca29d6ef55650a0af63bca817c5d494f31ef
Tree-SHA512: 5f01076f74a048019bb70791160f0accc2db7a457d969cb23687bed81ccbbdec1dda68311e7c6e2dd56250e23e8d926d4066e5014b2a99a2fc202e24ed264fbd
27c976d11a68d66db97d9a7a30c6a6a71c6ab586 fix: increase consistency of rpcauth parsing (tdb3)
2ad3689512a36eaff957df9ac28e65b2fedbc36c test: add norpcauth test (tdb3)
67df0dec1abe547773e532aa60aff0317e018123 test: blank rpcauth CLI interaction (tdb3)
ecc98ccff25b7e758337e764e59d764076772fec test: add cases for blank rpcauth (tdb3)
Pull request description:
The current `rpcauth` parsing behavior is inconsistent and unintuitive (see https://github.com/bitcoin/bitcoin/pull/29141#issuecomment-1972085251 and additional details below).
The current behavior inconsistently treats empty `rpcauth` as an error (or not) depending on the location within CLI/bitcoin.conf and the location of adjacent valid `rpcauth` params.
Empty `rpcauth` is now consistently treated as an error and prevents bitcoind from starting.
Continuation of the upforgrabs PR #29141.
### Additional details:
Current `rpcauth` behavior is nonsensical:
- If an empty `rpcauth` argument was specified as the last command line argument, it would cause all other `rpcauth` arguments to be ignored.
- If an empty `rpcauth` argument was specified on the command line followed by any nonempty `rpcauth` argument, it would cause an error.
- If an empty `rpcauth=` line was specified after non-empty rpcauth line in the config file it would cause an error.
- If an empty `rpcauth=` line in a config file was first it would cause other rpcauth entries in the config file to be ignored, unless there were `-rpcauth` command line arguments and the last one was nonempty, in which case it would cause an error.
New behavior is simple:
- If an empty rpcauth config line or command line argument is used it will cause an error
ACKs for top commit:
naiyoma:
Tested ACK [27c976d11a)
achow101:
ACK 27c976d11a68d66db97d9a7a30c6a6a71c6ab586
ryanofsky:
Code review ACK 27c976d11a68d66db97d9a7a30c6a6a71c6ab586. Since last review commit message was just tweaked to clarify previous behavior.
Tree-SHA512: af2e9dd60d1ad030409ae2c3805ab139c7435327823d9f8bbeede815f376cb696a5929b08a6e8c8b5f7278ed49cfb231789f9041bd57f1f03ec96501b669da5b
30803a35d54acda19ded88474c205f8954fea5e1 cmake: decouple FORTIFY_SOURCE check from Debug build type (fanquake)
Pull request description:
`FORTIFY_SOURCE` should be used if `ENABLE_HARDENING=ON` and optimisations are being used. This should not be coupled to any particular build type, because even if the build type is `Debug`, optimisations might still be in use.
Fixes: #30800.
Also somewhat of a followup to https://github.com/bitcoin/bitcoin/pull/30778#discussion_r1742257436.
ACKs for top commit:
ryanofsky:
Code review ACK 30803a35d54acda19ded88474c205f8954fea5e1
TheCharlatan:
ACK 30803a35d54acda19ded88474c205f8954fea5e1
Tree-SHA512: 298f8805a5bb2f1ff54e51ea31324d712c2070cc3eba26561c31001ace4bfa37ae6d18531cbd45e2faf610a0a1b83b420fcde6e329e17f02b021d26563583913
1f054eca4e779cfa5f9f6e9278071adf65e5eafe cmake: add USE_SOURCE_PERMISSIONS to all configure_file usage (fanquake)
Pull request description:
`USE_SOURCE_PERMISSIONS` is the default, so this should not change behaviour. However, being explicit makes it clear what we are doing.
Related to #30815.
See https://cmake.org/cmake/help/latest/command/configure_file.html#options.
ACKs for top commit:
hebasto:
ACK 1f054eca4e779cfa5f9f6e9278071adf65e5eafe.
TheCharlatan:
ACK 1f054eca4e779cfa5f9f6e9278071adf65e5eafe
Tree-SHA512: efed91b8aa0813100304ee58e169bbf5cfbb7db465ec4f7e6cbbae6053f09a36757bf96b4d1cb9ddf4c1cab0ceb3ab18805ebefa122535518ffb501c9b489d3d
7a669fde183b2f98c4faeff3eedaf95a1cba3b09 docs: Fix minor typo (Gutflo)
Pull request description:
Fix typo in doc/build-windows-msvc.md:
- "Micsrosoft" -> Microsoft
No test required.
ACKs for top commit:
l0rinc:
ACK 7a669fde183b2f98c4faeff3eedaf95a1cba3b09
Tree-SHA512: fd3815ebf449885e8a27d4f21e61a4482a7983ccfe40b13a4d658a304845775acd2f4f8acec2e85c24b6179223bb21baaecf1dd0a4d2921427686148ac1ed208
fa9d7d5d205ada8915cbbc29599ab8e7bf1fffe0 test: Work around boost compilation error (MarcoFalke)
fa3ecdf778bcde7713004ff8bb86d3fdfc916969 Revert "build: work around issue with Boost <= 1.80 and Clang >= 18" (MarcoFalke)
Pull request description:
There seems to be an issue compiling the `chainstatemanager_rebalance_caches` test case with some specific versions of Boost in combination with some specific versions of Clang. For example, Boost 1.74 may fail in combination with Clang 18. [1]
The error stems from a mixed-type closeness comparison. Given that the comparison is using floating point, and isn't meant to be exact, work around the compile error by ensuring both sides of the comparison are using the same type (`double`).
This also allows to drop a previous workaround.
[1] Error:
```
In file included from /usr/include/boost/mpl/integral_c.hpp:32:
/usr/include/boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for the enumeration type 'udt_builtin_mixture_enum' [-Wenum-constexpr-conversion]
73 | typedef AUX_WRAPPER_INST( BOOST_MPL_AUX_STATIC_CAST(AUX_WRAPPER_VALUE_TYPE, (value - 1)) ) prior;
| ^
/usr/include/boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
24 | # define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)
| ^
In file included from ../../../src/test/validation_chainstatemanager_tests.cpp:8:
In file included from ../../../src/node/chainstatemanager_args.h:9:
In file included from ../../../src/validation.h:28:
In file included from ../../../src/txmempool.h:26:
In file included from /usr/include/boost/multi_index/hashed_index.hpp:38:
In file included from /usr/include/boost/multi_index/detail/node_handle.hpp:22:
In file included from /usr/include/boost/multi_index_container_fwd.hpp:18:
In file included from /usr/include/boost/multi_index/indexed_by.hpp:17:
In file included from /usr/include/boost/mpl/vector.hpp:36:
In file included from /usr/include/boost/mpl/vector/vector20.hpp:18:
In file included from /usr/include/boost/mpl/vector/vector10.hpp:18:
In file included from /usr/include/boost/mpl/vector/vector0.hpp:24:
In file included from /usr/include/boost/mpl/vector/aux_/clear.hpp:18:
In file included from /usr/include/boost/mpl/vector/aux_/vector0.hpp:22:
In file included from /usr/include/boost/mpl/vector/aux_/iterator.hpp:19:
In file included from /usr/include/boost/mpl/plus.hpp:19:
In file included from /usr/include/boost/mpl/aux_/arithmetic_op.hpp:17:
In file included from /usr/include/boost/mpl/integral_c.hpp:32:
/usr/include/boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for the enumeration type 'int_float_mixture_enum' [-Wenum-constexpr-conversion]
/usr/include/boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
24 | # define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)
| ^
2 errors generated.
ACKs for top commit:
hebasto:
ACK fa9d7d5d205ada8915cbbc29599ab8e7bf1fffe0.
fanquake:
ACK fa9d7d5d205ada8915cbbc29599ab8e7bf1fffe0
Tree-SHA512: 4964b23162f2351c7d3cf7e9efa7860d62f3b6717c3cc5be967d286f1ddb3539c2637247c79aa83123d36ff111ba77df22be2a25487ddd94dc1321d5e751dc70
Add `-ipcbind` option to `bitcoin-node` to listen on an IPC socket and accept
connections from other processes. In the future, there will be an `-ipcconnect`
option added to `bitcoin-wallet` and `bitcoin-node` to allow wallet and gui
processes to connect to the node and access it.
Example usage:
src/bitcoin-node -regtest -debug -ipcbind=unix
src/bitcoin-wallet -regtest -ipcconnect=unix info
src/bitcoin-gui -regtest -ipcconnect=unix
src/bitcoin-mine -regtest -ipcconnect=unix
3ae35b427fe59bc9ab24d07c1adb46faa702de20 ci: run check-deps.sh as part of clang-tidy job (Ryan Ofsky)
0aaa1298a08f898318916661f2317b2e755206e6 contrib: fix check-deps.sh when libraries do not import symbols (Ryan Ofsky)
3c99f5a38a47e4e10a0daab3a114b5e476fcacfa contrib: fix check-deps.sh to check for weak symbols (Ryan Ofsky)
86c80e9cf296f6560f2f846bd4e7286f7b958b93 contrib: make check-deps.sh script work with cmake (Ryan Ofsky)
Pull request description:
Fix check-deps.sh to check for weak symbols so it can detect when an exported template function like is used from another library.
Also update the script to work with cmake and configure it to run as part of CI.
Problem was reported by hebasto in https://github.com/bitcoin/bitcoin/pull/29015#issuecomment-2209258843
ACKs for top commit:
TheCharlatan:
Re-ACK 3ae35b427fe59bc9ab24d07c1adb46faa702de20
hebasto:
ACK 3ae35b427fe59bc9ab24d07c1adb46faa702de20, I have reviewed the code and it looks OK. Also I've tested it locally.
Tree-SHA512: c3b58175450b675e6e848549b81bcfe42930ea9bcd693063867ce3f0ac3999c98cd2c3e961f163ff06641e8288f3a4e81530936a296a83d45d33364f27489521
fadbcd51fc77a3f4e877851463f3c7425fb751d2 bench: Remove redundant logging benchmarks (MarcoFalke)
fa8dd952e279a87f6027ddd2e2119bf2ae2f9943 bench: Use LogInfo instead of the deprecated alias LogPrintf (MarcoFalke)
Pull request description:
`LogPrint*ThreadNames` is redundant with `LogWith(out)ThreadNames`,
because they all measure toggling the thread names (and check that it
has no effect on performance).
Fix it by removing the redundant ones. This also allows to drop a deprecated logging alias.
ACKs for top commit:
stickies-v:
ACK fadbcd51fc77a3f4e877851463f3c7425fb751d2
Tree-SHA512: 4fe137f374aa4ee1aa0e1da4a1f9839c0e52c23dbb93198ecafee98de39d311cc47304bba4191f3807aa00c51b1eae543e3f270f03d341c84910e5e341a1d475
fa84f9decd224ea1c25dc7095bad70a48fa1a534 test: Pin and document TEST_DIR_PATH_ELEMENT (MarcoFalke)
2222f7a87404078984c7189768a3422deb114302 test: Rename SeedRand::SEED to FIXED_SEED for clarity (MarcoFalke)
Pull request description:
Two small test changes:
* A refactor to update the name and documentation around `SeedRand::FIXED_SEED`.
* A change to extract and document `TEST_DIR_PATH_ELEMENT`, and to change its value to better match the `TMPDIR_PREFIX` in functional tests. The value previously included `PACKAGE_NAME`, which is cute, but doesn't explain why it was used (to include a space). So just use `test_common bitcoin` to achieve the same with less effort.
ACKs for top commit:
hodlinator:
ACK fa84f9decd224ea1c25dc7095bad70a48fa1a534
ryanofsky:
Code review ACK fa84f9decd224ea1c25dc7095bad70a48fa1a534
Tree-SHA512: eb35d6598bb08f9b996e3a4762d8f26b2441c0ca00780798e473015af735dfc9997120895a922b94d4b6ada45adadba4a686e9cf9c285ddf688848e764c64840
a7a4e11db8722c85175bcc4d9f73a713e9e61513 cmake: scope Boost Test check to vcpkg (fanquake)
Pull request description:
This check was added for `vcpkg`, given how it packages Boost. However, we don't need to run the check for other platforms, and it's quite slow. So, scope it to just `vcpkg`.
On my machine, this reduces the time to run `time cmake -B build` from ~12 seconds, to ~6 seconds.
Fixes: #30787.
ACKs for top commit:
kevkevinpal:
lgtm ACK [a7a4e11](a7a4e11db8)
maflcko:
review ACK a7a4e11db8722c85175bcc4d9f73a713e9e61513
davidgumberg:
Tested ACK a7a4e11db8
hebasto:
re-ACK a7a4e11db8722c85175bcc4d9f73a713e9e61513.
Tree-SHA512: 67cf3908a5381e21aeaa168a6f76b6e066d64a8ad2127d5ae9fe71a0f04bccf58a400726d9d4e228b3bdb6fca799034fd05a38388278fea30a1a841f6adac017
cd062d6684908d526be7423f8f1057b891254a3c build: work around issue with Boost <= 1.80 and Clang >= 18 (fanquake)
Pull request description:
Our current minimum supported Boost is `1.73.0`. However, when compiling with Boost `1.74.0` (Debian Stable), using Clang `18`, compilation fails with:
```bash
In file included from /usr/include/boost/mpl/integral_c.hpp:32:
/usr/include/boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for the enumeration type 'udt_builtin_mixture_enum' [-Wenum-constexpr-conversion]
73 | typedef AUX_WRAPPER_INST( BOOST_MPL_AUX_STATIC_CAST(AUX_WRAPPER_VALUE_TYPE, (value - 1)) ) prior;
| ^
/usr/include/boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
24 | # define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)
| ^
In file included from ../../../src/test/validation_chainstatemanager_tests.cpp:8:
In file included from ../../../src/node/chainstatemanager_args.h:9:
In file included from ../../../src/validation.h:28:
In file included from ../../../src/txmempool.h:26:
In file included from /usr/include/boost/multi_index/hashed_index.hpp:38:
In file included from /usr/include/boost/multi_index/detail/node_handle.hpp:22:
In file included from /usr/include/boost/multi_index_container_fwd.hpp:18:
In file included from /usr/include/boost/multi_index/indexed_by.hpp:17:
In file included from /usr/include/boost/mpl/vector.hpp:36:
In file included from /usr/include/boost/mpl/vector/vector20.hpp:18:
In file included from /usr/include/boost/mpl/vector/vector10.hpp:18:
In file included from /usr/include/boost/mpl/vector/vector0.hpp:24:
In file included from /usr/include/boost/mpl/vector/aux_/clear.hpp:18:
In file included from /usr/include/boost/mpl/vector/aux_/vector0.hpp:22:
In file included from /usr/include/boost/mpl/vector/aux_/iterator.hpp:19:
In file included from /usr/include/boost/mpl/plus.hpp:19:
In file included from /usr/include/boost/mpl/aux_/arithmetic_op.hpp:17:
In file included from /usr/include/boost/mpl/integral_c.hpp:32:
/usr/include/boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for the enumeration type 'int_float_mixture_enum' [-Wenum-constexpr-conversion]
/usr/include/boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
24 | # define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)
| ^
2 errors generated.
```
Work around this issue by ignoring this diagnostic for this include. I did attempt to just downgrade the error into a warning, but that did not seem to work. Not a huge fan of inline warning/issue suppression, but this seems like the cleanest thing to do here (and easy to backport to `28.x`).
Can be tested with something like:
```bash
docker pull debian:bookworm
docker run -it debian:bookworm /bin/bash
apt update && apt install ccache cmake git pkg-config libboost-dev libevent-dev python3 libsqlite3-dev lsb-release wget software-properties-common gnupg
git clone https://github.com/bitcoin/bitcoin
wget https://apt.llvm.org/llvm.sh
chmod +x llvm.sh
./llvm.sh 18
cd bitcoin
cmake -B build -DCMAKE_C_COMPILER=clang-18 -DCMAKE_CXX_COMPILER=clang++-18
cmake --build build -j17
<snip>
In file included from /usr/include/boost/mpl/integral_c.hpp:32:
/usr/include/boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for the enumeration type 'int_float_mixture_enum' [-Wenum-constexpr-conversion]
/usr/include/boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
24 | # define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)
| ^
2 errors generated.
Apply the patch
cmake --build build -j17
ctest --test-dir build -j17
```
Fixes#30751.
ACKs for top commit:
achow101:
ACK cd062d6684908d526be7423f8f1057b891254a3c
hebasto:
ACK cd062d6684908d526be7423f8f1057b891254a3c, tested on Fedora 40 using the downloaded [Boost 1.74](https://archives.boost.io/release/1.74.0/source/) and commands as follows:
Tree-SHA512: 13e5b3a544496ed2a6529ad45d03a2d872ebf41caaa06d0eec23a639d678ae1c55d73f2d4b164a4cc9e2c163264e736cd85eae90fde8089ca999cd810b16ecb5
`FORTIFY_SOURCE` should be used if `ENABLE_HARDENING=ON` and optimisations
are being used. This should not be coupled to any particular build type,
because even if the build type is `Debug`, optimisations might still
be in use.
Fixes: #30800.
This check was added for vcpkg, given how it packages Boost. However, we
don't need to run the check for other platforms, and it's quite slow.
So, scope it to VCPKG. On my machine, this reduces the time to run
`cmake -B build` from ~12 seconds, to ~6 seconds.
Fixes: #30787
fa3a7ebe5b5db63e4cb4fb6974c028cc7af0b898 lint: Check for release note snippets in the wrong folder (MarcoFalke)
Pull request description:
It is a common mistake to place the snippets in the wrong folder, where they could be missed. For example https://github.com/bitcoin/bitcoin/pull/30719#pullrequestreview-2262535007 or commit 84900ac34f6888b7a851d0a6a5885192155f865c.
Fix all issues by adding a simple lint check.
Can be tested by reverting a prior commit that violated the rule and then running the new check:
```
git revert 35ef34eab7b36e3c53ed438d74a9b783cbcaec27
( cd ./test/lint/test_runner/ && RUST_BACKTRACE=1 cargo run -- --lint=doc_release_note_snippets )
ACKs for top commit:
l0rinc:
ACK fa3a7ebe5b5db63e4cb4fb6974c028cc7af0b898
TheCharlatan:
Re-ACK fa3a7ebe5b5db63e4cb4fb6974c028cc7af0b898
Tree-SHA512: 65a13696178aa8f94daa12a767cc74861293c631c19da9ca23c0fd43cedd47e7928d0ef14ad9ad83a434c1ac0e006f5a632ba9679756e071dea65b3cbf927c2d
Our current minimum supported Boost is `1.73.0`. However, when compiling
with Boost `1.74.0` (Debian Stable), using Clang `18`, compilation fails
with:
```bash
In file included from /usr/include/boost/mpl/integral_c.hpp:32:
/usr/include/boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for the enumeration type 'udt_builtin_mixture_enum' [-Wenum-constexpr-conversion]
73 | typedef AUX_WRAPPER_INST( BOOST_MPL_AUX_STATIC_CAST(AUX_WRAPPER_VALUE_TYPE, (value - 1)) ) prior;
| ^
/usr/include/boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
24 | # define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)
| ^
In file included from ../../../src/test/validation_chainstatemanager_tests.cpp:8:
In file included from ../../../src/node/chainstatemanager_args.h:9:
In file included from ../../../src/validation.h:28:
In file included from ../../../src/txmempool.h:26:
In file included from /usr/include/boost/multi_index/hashed_index.hpp:38:
In file included from /usr/include/boost/multi_index/detail/node_handle.hpp:22:
In file included from /usr/include/boost/multi_index_container_fwd.hpp:18:
In file included from /usr/include/boost/multi_index/indexed_by.hpp:17:
In file included from /usr/include/boost/mpl/vector.hpp:36:
In file included from /usr/include/boost/mpl/vector/vector20.hpp:18:
In file included from /usr/include/boost/mpl/vector/vector10.hpp:18:
In file included from /usr/include/boost/mpl/vector/vector0.hpp:24:
In file included from /usr/include/boost/mpl/vector/aux_/clear.hpp:18:
In file included from /usr/include/boost/mpl/vector/aux_/vector0.hpp:22:
In file included from /usr/include/boost/mpl/vector/aux_/iterator.hpp:19:
In file included from /usr/include/boost/mpl/plus.hpp:19:
In file included from /usr/include/boost/mpl/aux_/arithmetic_op.hpp:17:
In file included from /usr/include/boost/mpl/integral_c.hpp:32:
/usr/include/boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for the enumeration type 'int_float_mixture_enum' [-Wenum-constexpr-conversion]
/usr/include/boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
24 | # define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)
| ^
2 errors generated.
```
Work around this issue by ignoring this diagnostic for this include.
I did attempt to just downgrade the error into a warning, but that did
not seem to work.
See https://github.com/bitcoin/bitcoin/issues/30751 for further
discussion.
7346b0109208c67798bdbd451e509048308eff5d qt, build: remove unneeded `Q_IMPORT_PLUGIN` macro calls (Sebastian Falbesoner)
Pull request description:
After the recent full removal of Autotools (PR [#30664](https://github.com/bitcoin/bitcoin/pull/30664)), these macros are not needed anymore in the .cpp files according to the TODO in qt's CMakeLists.txt. Tested building on OpenBSD 7.5, where the XCB plugin was still imported according to the debug log:
```
2024-09-02T21:13:27Z Bitcoin Core version v28.99.0-7346b0109208 (release build)
2024-09-02T21:13:27Z Qt 5.15.12 (dynamic), plugin=xcb
2024-09-02T21:13:27Z No static plugins.
2024-09-02T21:13:27Z Style: fusion / QFusionStyle
2024-09-02T21:13:27Z System: OpenBSD 7.5, x86_64-little_endian-lp64
```
ACKs for top commit:
hebasto:
ACK 7346b0109208c67798bdbd451e509048308eff5d.
Tree-SHA512: ffa033fc6e0412a99d2c167044cc7af89512a731172d6911db71434f5353e811802c268d853a76d3732e9da954444cf6c39a852aeb25938c435826e117a16fca
faecca9a85c1c1917d024f55cca34d19cc94f3b9 test: Use span for raw data (MarcoFalke)
fac973647d69dd14089cee9972e8dfa0074c8a97 test: Use string_view for json_tests (MarcoFalke)
Pull request description:
The build system converts raw data into a C++ header file for tests.
This change modernizes the code to use the convenience wrappers `std::span` and `std::string_view`, so that redundant copies can be avoided.
ACKs for top commit:
fjahr:
re-ACK faecca9a85
TheCharlatan:
ACK faecca9a85c1c1917d024f55cca34d19cc94f3b9
stickies-v:
ACK faecca9a85c1c1917d024f55cca34d19cc94f3b9
hebasto:
ACK faecca9a85c1c1917d024f55cca34d19cc94f3b9, I have reviewed the code and the generated headers.
Tree-SHA512: 1f4951c54aff11ba27c41fb70f2821bdb79e06ca0abae734b970bd0d64dda9d8cced824a891fd51b3e9d4e5715ee9eb49ed5d369010a45eca7c3bec9f8641235
d9fcbfc3727e06e6f57d4ab09861f3212d558426 build: Add `JOBS` variable support to `CoverageFuzz.cmake` script (Hennadii Stepanov)
e7cf4a6f27d1c7fb38b212b93fe0120b8c4ef2a2 build: Add missed `-g` for "Coverage" build configuration (Hennadii Stepanov)
fe2003ab1246948151bfe3dfe125395fcbc4f606 build: Add `COMMAND_ERROR_IS_FATAL` to every process in coverage scrips (Hennadii Stepanov)
Pull request description:
The first commit ensures early error catching.
The second commit adds the `-g` flag that was missed during the migration from Autotools.
This PR is intended to be tested with GCC compiler (as clang support is still under [scrutiny](https://github.com/hebasto/bitcoin/issues/341)). Depending on the `lcov` version, additional flags `-DCMAKE_C_FLAGS="-fprofile-update=atomic" -DCMAKE_CXX_FLAGS="-fprofile-update=atomic"` may be required.
ACKs for top commit:
maflcko:
review ACK d9fcbfc3727e06e6f57d4ab09861f3212d558426
tdb3:
cr re ACK d9fcbfc3727e06e6f57d4ab09861f3212d558426
Tree-SHA512: 0998411dc1ccd60d7bd6b36f4e2881f699202c65dcc8c177b46380d0f255d291d9537f1dc6fb35478b632f3515d3484d8e7d2877126c57e3f02b21f90160f1eb
556775408797d8e27154c3edaf139820b0979cce depends: build libevent with -D_GNU_SOURCE (fanquake)
Pull request description:
Currently, builds of libevent in depends, using CMake, fail on some systems, like Alpine, with the following:
```bash
/bitcoin/depends/work/build/aarch64-unknown-linux-musl/libevent/2.1.12-stable-1516ed47ea8/evmap.c: In function 'evmap_signal_add_':
/bitcoin/depends/work/build/aarch64-unknown-linux-musl/libevent/2.1.12-stable-1516ed47ea8/evmap.c:456:31: error: 'NSIG' undeclared (first use in this function)
456 | if (sig < 0 || sig >= NSIG)
```
From what I can tell the `GNU_SOURCE` "detection" in libevents CMake build system, never? really worked, primarily relies on looking for a deprecated define, and it's not clear what a nice fix is. For now, always build with `_GNU_SOURCE`, to match the autotools behaviour.
ACKs for top commit:
TheCharlatan:
ACK 556775408797d8e27154c3edaf139820b0979cce
Tree-SHA512: 4552b4a92867e8fa2af0ffa39b2be6c994bf739de7ce6a7c581590be486da81f7d93fca816854548c1e912347d33a35218c441b5058c3cbd3e82c74a9b7c78d9
This change allows to drop brittle sizeof calls in favor of the
std::span::size method.
Other improvements include:
* Use of a namespace to mark test and bench data
* Use of the modern std::byte
* Drop of a no longer used std::vector copy and the bench/data module
In case of a failure to create the dump, the node should not be left in an inconsistent state like deactivated network activity or an invalidated blockchain.
e5f7272ad322aa4eb7a7d6f531419a4d7aee4802 doc: fix assumeutxo design doc link (marcofleon)
Pull request description:
A correction to a link as I was exploring Assumeutxo stuff.
ACKs for top commit:
fjahr:
ACK e5f7272ad322aa4eb7a7d6f531419a4d7aee4802
tdb3:
ACK e5f7272ad322aa4eb7a7d6f531419a4d7aee4802
MarnixCroes:
ACK e5f7272ad322aa4eb7a7d6f531419a4d7aee4802
Tree-SHA512: b7380d884a196b89eed32bc14af8ca11191c9f2bcb1c5c163bb627fd87a6231dbd86da6e659baddb8c652961b5e44f80509606fee9ae38a53e90ebb05d082404
LogPrint*ThreadNames is redundant with LogWith(out)ThreadNames, because
they all measure toggling the thread names (and check that it has no
effect on performance).
This also allows to remove unused and deprecated macros.
8131bf7483c0ea10d3573c9f2e977d19d8569b7f ci: parse TEST_RUNNER_EXTRA into an array (Max Edwards)
c4762b0aa06f2654d108bc7ca05887ffd88cf6f8 test: allow excluding func test by name and arg (Max Edwards)
Pull request description:
While working on CI I wanted to disable some functional tests so I used the `TEST_RUNNER_EXTRA` var. The problem I had was tests that have flags such as `rpc_bind.py --ipv6` must be passed in quotes otherwise the `--ipv6` portion will be considered an argument to `test_runner.py` rather than a test name.
This change allows proper parsing of quotes and complex values such as:
```shell
TEST_RUNNER_EXTRA='--exclude "rpc_bind.py --ipv6,feature_proxy.py"'
```
Update:
While testing this it was noticed that `test_runner.py` when given `--exclude "rpc_bind.py --ipv6"` will exclude all `rpc_bind.py` tests so this PR has been updated to include a change to the test runner to only exclude the specific test if you pass an arg or exclude all tests of that name if you do not pass an arg. `--exclude rpc_bind.py` will exclude all three variants and `--exclude rpc_bind --ipv6` will only exclude the IPV6 variant.
ACKs for top commit:
maflcko:
ACK 8131bf7483c0ea10d3573c9f2e977d19d8569b7f
achow101:
ACK 8131bf7483c0ea10d3573c9f2e977d19d8569b7f
hebasto:
ACK 8131bf7483c0ea10d3573c9f2e977d19d8569b7f, tested on Ubuntu 23.10 and Windows 11.
Tree-SHA512: 82b73f12d627f533d8e5be4a518d455ef4427a755bbe03ccd11d0bb70c7ff3cee76220b0264fcfb236661c4cf5deba034cbfc2372b96d5861f3436c21eae8264
fafdb7df34507eee735893aa871da6ae529e6372 lint: Speed up flake8 checks (MarcoFalke)
faf17df7fb88590d936d10c471a9ea6a2ce4454d lint: Document missing py_lint dependency (MarcoFalke)
faebeb828f5f0ec68d90e7f76add66bc562f6fa3 lint: Remove python whitespace and shadowing lint rules (MarcoFalke)
77770478355ce6c1ab077dbc12ec898875ec5620 lint: Remove python lint rules that are SyntaxError (MarcoFalke)
faaf3e53f09c73278e36674db0af14a262f0bd94 test: [refactor] Fix F841 flake8 (MarcoFalke)
444421db69539b74077306b6d0cb23e82afeb891 test: [refactor] Fix E714 pycodestyle (MarcoFalke)
Pull request description:
The checks have many issues:
* Some checks that could in theory hide bugs are not applied -> Fix them and apply them going forward
* Some checks are redundant Python 2 checks, or of low value -> Remove them
* The checks are slow -> Speed them up from ~10 seconds to about ~20 milliseconds
ACKs for top commit:
davidgumberg:
review and tested reACK fafdb7df34
kevkevinpal:
ACK [fafdb7d](fafdb7df34)
achow101:
ACK fafdb7df34507eee735893aa871da6ae529e6372
Tree-SHA512: a0488b722cfaf7071bd6848cd3be002e0b6c38af80d8b5cbb08613c0b174ef63277289f960db8ac31adb09fe563a4973203b8fb10b83cbcfdc6f0ef39bd04410
Script was failing when called on libraries that do not import symbols, because
bash pipefail option was specified, and grep was used in some pipelines to
filter symbols, and grep returns status 1 when it doesn't match any lines. This
could cause the script to fail on some systems and configurations, such as the
clang-tidy CI configuration
https://cirrus-ci.com/task/4801670352207872?logs=ci#L6191 where the
libbitcoin_crypto_x86_shani.a library does not import symbols.
Fix check-deps.sh to check for weak symbols so it can detect when an exported
template function is used from another library.
In a previous version of this commit, this change caused an invalid dependency
in the consensus library on the TryParseHex template function from the util
library to be detected, and a suppression was added here. But #30377 removed
the invalid dependency so the suppression is no longer needed.
The invalid dependency and problem detecting weak symbol usage was originally
reported by Hennadii Stepanov in
https://github.com/bitcoin/bitcoin/pull/29015#issuecomment-2209258843
