fa09451f8e6799682d7e7c863f25334fd1c7dce3 Add lint check for bitcoin-config.h include IWYU pragma (MarcoFalke)
dddd40ba8267dea11a3eb03d5cf8b51dbb99be5d scripted-diff: Add IWYU pragma keep to bitcoin-config.h includes (MarcoFalke)
Pull request description:
The `bitcoin-config.h` includes have issues:
* The header is incompatible with iwyu, because symbols may be defined or not defined. So the `IWYU pragma: keep` is needed to keep the include when a symbol is not defined on a platform. Compare the previous discussion in https://github.com/bitcoin/bitcoin/pull/29408#issuecomment-1948959711
* Guarding the includes by `HAVE_CONFIG_H` is verbose and brittle. Now that all build config dependencies have been removed from low level headers, the benefits are questionable, and the guard can be removed. The linter could also be tricked by guarding the include by `#if defined(HAVE_C0NFIG_H)` (`O` replaced by `0`). Compare the previous discussion in https://github.com/bitcoin/bitcoin/pull/29404#discussion_r1483189853 .
ACKs for top commit:
achow101:
ACK fa09451f8e6799682d7e7c863f25334fd1c7dce3
TheCharlatan:
ACK fa09451f8e6799682d7e7c863f25334fd1c7dce3
hebasto:
re-ACK fa09451f8e6799682d7e7c863f25334fd1c7dce3, only rebased since my recent [review](https://github.com/bitcoin/bitcoin/pull/29494#pullrequestreview-2028864535) (`timedata.cpp` removed in https://github.com/bitcoin/bitcoin/pull/29623).
Tree-SHA512: 47cb973f7f24bc625acc4e78683371863675d186780236d55d886cf4130e05a78bb04f1d731aae7088313b8e963a9677cc77cf518187dbd99d776f6421ca9b52
bd2de7ac591d7704b79304089ad1fb57e085da8b refactor, test: Always initialize pointer (Hennadii Stepanov)
Pull request description:
This change fixes MSVC warning [C4703](https://learn.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-4-c4703).
All `DisableSpecificWarnings` dropped from `test_bitcoin.vcxproj` as all remained are inherited from `common.init.vcxproj`.
Required to simplify warning suppression porting to the CMake-based build system.
ACKs for top commit:
maflcko:
utACK bd2de7ac591d7704b79304089ad1fb57e085da8b
sipsorcery:
utACK bd2de7ac591d7704b79304089ad1fb57e085da8b.
ryanofsky:
Code review ACK bd2de7ac591d7704b79304089ad1fb57e085da8b
Tree-SHA512: 006db041d3c3697a77d9df14de86cf7c8a10804b45789df01268b2236cf6452e77dc57e89f5d5a6bc26d4b5cd483f0722d6035649c8a523b57954bb1fc810d0c
The `test\util\setup_common.cpp` and `wallet\test\util.cpp` sources are
already compiled and included in the `libtest_util` library, which is
linked to the `test_bitcoin-qt.exe` binary. This change follows the same
logic as `Makefile.qttest.include`.
6acec6b9ff02b91de132bb1575d75908a8a2d27b multiprocess: Add type conversion code for UniValue types (Ryan Ofsky)
0cc74fce72e0c79849109ee5d7afe707991b3512 multiprocess: Add type conversion code for serializable types (Ryan Ofsky)
4aaee239211a5287fbc361c0eb158b105ae8c8db test: add ipc test to test multiprocess type conversion code (Ryan Ofsky)
Pull request description:
Add type conversion hooks to allow `UniValue` objects, and objects that have `CDataStream` `Serialize` and `Unserialize` methods to be used as arguments and return values in Cap'nProto interface methods. Also add unit test to verify the hooks are working and data can be round-tripped correctly.
The non-test code in this PR was previously part of #10102 and has been split off for easier review, but the test code is new.
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).
ACKs for top commit:
achow101:
ACK 6acec6b9ff02b91de132bb1575d75908a8a2d27b
dergoegge:
reACK 6acec6b9ff02b91de132bb1575d75908a8a2d27b
Tree-SHA512: 5d2cbc5215d488b876d34420adf91205dabf09b736183dcc85aa86255e3804c2bac5bab6792dacd585ef99a1d92cf29c8afb3eb65e4d953abc7ffe41994340c6
308aec3e5655327d98e0428d8205d246f24d6af5 build: disable external-signer for Windows (fanquake)
35537318a19360ddf1ea8f0c1e6d8ad49e635516 ci: remove --enable-external-signer from win64 job (fanquake)
Pull request description:
It's come to light that Boost ASIO (a Boost Process sub dep) has in some
instances, been quietly initialising our network stack on Windows (see
PR https://github.com/bitcoin/bitcoin/pull/28486 and discussion in https://github.com/bitcoin/bitcoin/issues/28940).
This has been shielding a bug in our own code, but the larger issue
is that Boost Process/ASIO is running code before main, and doing things
like setting up networking. This undermines our own assumptions about
how our binary works, happens before we run any sanity checks,
and before we call our own code to setup networking. Note that ASIO also
calls WSAStartup with version `2.0`, whereas we call with `2.2`.
It's also not clear why a feature like external signer would have a
dependency that would be doing anything network/socket related,
given it only exists to spawn a local process.
See also the discussion in https://github.com/bitcoin/bitcoin/issues/24907. Note that the maintaince of Boost Process in general,
has not really improved. For example, rather than fixing bugs like https://github.com/boostorg/process/issues/111,
i.e, https://github.com/boostorg/process/pull/317, the maintainer chooses to just wrap exception causing overflows
in try-catch blocks: 0c42a58eac. These changes get merged in large,
unreviewed PRs, i.e https://github.com/boostorg/process/pull/319.
This PR disables external-signer on Windows for now. If, in future, someone
changes how Boost Process works, or replaces it entirely with some
properly reviewed and maintained code, we could reenable this feature on
Windows.
ACKs for top commit:
hebasto:
re-ACK 308aec3e5655327d98e0428d8205d246f24d6af5.
TheCharlatan:
ACK 308aec3e5655327d98e0428d8205d246f24d6af5
Tree-SHA512: 7405f7fc9833eeaacd6836c4e5b1c1a7845a40c1fdd55c1060152f8d8189e4777464fde650e11eb1539556a75dddf49667105987078b1457493ee772945da66e
1a5dae630df1eef9eac51557b2f1c5dba0924953 msvc: Define the same `QT_...` macros as in Autotools builds (Hennadii Stepanov)
Pull request description:
There are no reasons to have such a diversion.
Also it fixes https://github.com/bitcoin/bitcoin/pull/28960#issuecomment-1847971114.
ACKs for top commit:
sipsorcery:
tACK 1a5dae630df1eef9eac51557b2f1c5dba0924953.
TheCharlatan:
ACK 1a5dae630df1eef9eac51557b2f1c5dba0924953
Tree-SHA512: 75be5eabb8fec974b8d77a023c72323015a3d95fbc13b7fd85e5f25c250ae67850ddf0bcaef143828d75fe35a49e7c9b1966976b74f3ce7d14465174e6585ceb
6e0f1d2abbb700d4fd4b956a7d1f9505b653653c msvc: Optimize "Release" builds (Hennadii Stepanov)
Pull request description:
It is awkward not using optimization.
In addition to the obvious benefits for Windows users, this PR reduces the duration of functional tests by an hour.
Picked from https://github.com/bitcoin/bitcoin/pull/24773.
ACKs for top commit:
sipsorcery:
tACK 6e0f1d2abbb700d4fd4b956a7d1f9505b653653c.
Tree-SHA512: 5aa7fd38cb1a81d58ea3206756a8099891866c82a747d3b8079cab0b2afa1f40ba53adff2f32eb233efcd1227babee80ab175e35a83678fafa8a4f63c356e5ca
It's come to light that Boost ASIO (a Boost Process sub dep) has in some
instances, been queitly initialising our network stack on Windows (see
PR #28486 and discussion in #28940).
This has been shielding a bug in our own code, but the larger issue
is that Boost Process/ASIO is running code before main, and doing things
like setting up networking. This undermines our own assumptions about
how our binary works, happens before we get to run any sanity checks,
and also runs before we call our own code to setup networking.
It's also not clear why a feature like external signer would have a
dependency that would be doing anything network/socket related, given it
only exists to spawn a local process.
Add unit test to test IPC method calls and type conversion between bitcoin c++
types and capnproto messages.
Right now there are custom type hooks in bitcoin IPC code, so the test is
simple, but in upcoming commits, code will be added to convert bitcoin types to
capnproto messages, and the test will be expanded.
fa5831bd6f940c4afb43ff625ba4fa6c641e999a build: Do not define `ENABLE_ZMQ` when ZMQ is not available (Hennadii Stepanov)
Pull request description:
A new behavior is consistent with the other optional dependencies.
The source code contains `#if ENABLE_ZMQ` lines only:
```
$ git grep ENABLE_ZMQ -- src/*.cpp
src/init.cpp:#if ENABLE_ZMQ
src/init.cpp:#if ENABLE_ZMQ
src/init.cpp:#if ENABLE_ZMQ
src/init.cpp:#if ENABLE_ZMQ
src/init.cpp:#if ENABLE_ZMQ
```
Change in description line -- "Define to 1..." --> "Define this symbol.." -- is motivated by the fact that the actual value of the defined `ENABLE_ZMQ` macro does not matter at all.
Related to:
- https://github.com/bitcoin/bitcoin/issues/16419
- https://github.com/bitcoin/bitcoin/pull/25302
ACKs for top commit:
TheCharlatan:
ACK fa5831bd6f940c4afb43ff625ba4fa6c641e999a
jarolrod:
ACK fa5831bd6f940c4afb43ff625ba4fa6c641e999a
Tree-SHA512: 5e72ff0d34c4b33205338daea0aae8d7aa0e48fd633e21af01af32b7ddb0532ef68dd3dd74deb2c1d2599691929617e8c09676bcbaaf7d669b88816f866f1db2
b8ed95127b23873db973b2ef4f68d9f04e9c23ae msvc: Provide `ObjectFileName` explicitly (Hennadii Stepanov)
Pull request description:
This PR is a follow-up to https://github.com/bitcoin/bitcoin/pull/26715.
Fixes intermittent MSVC link [errors](https://cirrus-ci.com/task/6646912535756800).
ACKs for top commit:
sipsorcery:
ACK b8ed95127b23873db973b2ef4f68d9f04e9c23ae.
Tree-SHA512: 4319ecf61b578ce66d240998d089b9bb0a42f89dcfcb1a73f648cd3915f566c773721dcff1feba27d393a743d121334ccb890b1a519173e35a156d6135821ef4
a94d75fa81bab8f4695ab1756524e639af0ff69c msvc: Do not define `HAVE_CONSENSUS_LIB` (Hennadii Stepanov)
cf6ff1031bd577c18fe8eb2dd168a6d0d1c16e0d msvc: Clean up `libbitcoin_consensus` source files (Hennadii Stepanov)
30aee016f15b43b90efb023d9bd04aa84cf58ade scripted-diff: Rename `libbitcoinconsensus` to `libbitcoin_consensus` (Hennadii Stepanov)
Pull request description:
The current Autotools-based build system operates with two build artifacts:
- [`LIBBITCOIN_CONSENSUS`](3777c75d14/src/Makefile.am (L31)) which is [defined as](https://github.com/bitcoin/bitcoin/blob/master/doc/design/libraries.md) "Stable, backwards-compatible consensus functionality used by _libbitcoin_node_ and _libbitcoin_wallet_"
- [`LIBBITCOINCONSENSUS`](3777c75d14/src/Makefile.am (L42)) which is [defined as](https://github.com/bitcoin/bitcoin/blob/master/doc/design/libraries.md) "Shared library build of static _libbitcoin_consensus_ library"
The way how the `libbitcoinconsensus.vcxproj` project is used in the MSVC build system obviously shows that it is the former use case.
This PR makes the related adjustments to the MSVC build system.
ACKs for top commit:
sipsorcery:
ACK a94d75fa81bab8f4695ab1756524e639af0ff69c.
Tree-SHA512: 1144e13ee2b428ce14be8f76729744830c502a07814eb03e2aa6b8e009d8936fd13743e3f36ef3f31fac0e3979eb9af23e6a1364f151df49b3ae18dbd14cbf99
33e2b82a4fc990253ff77655f437c7aed336bc55 wallet, bench: Remove unused database options from WalletBenchLoading (Andrew Chow)
80ace042d8fece9be50bfef1be64c6e5720e87e6 tests: Modify records directly in wallet ckey loading test (Andrew Chow)
b3bb17d5d07f51ac2e501e4a7a3bbcd17144070f tests: Update DuplicateMockDatabase for MockableDatabase (Andrew Chow)
f0eecf5e408238c64b77b0a4974ba2b9edb17487 scripted-diff: Replace CreateMockWalletDB with CreateMockableWalletDB (Andrew Chow)
075962bc25a90661612fe4613cd50ea1cae21f52 wallet, tests: Include wallet/test/util.h (Andrew Chow)
14aa4cb1e44f089a6022a2b14a98bca4a7dd9a01 wallet: Move DummyDatabase to salvage (Andrew Chow)
f67a385556c60b2e4788a378196a395fca0539f5 wallet, tests: Replace usage of dummy db with mockable db (Andrew Chow)
33c6245ac1ecdfe25b1ee4fd9e93c43393634ae3 Introduce MockableDatabase for wallet unit tests (Andrew Chow)
Pull request description:
For the wallet's unit tests, we currently use either `DummyDatabase` or memory-only versions of either BDB or SQLite. The tests that use `DummyDatabase` just need a `WalletDatabase` so that the `CWallet` can be constructed, while the tests using the memory-only databases just need a backing data store. There is also a `FailDatabase` that is similar to `DummyDatabase` except it fails be default or can have a configured return value. Having all of these different database types can make it difficult to write tests, particularly tests that work when either BDB or SQLite is disabled.
This PR unifies all of these different unit test database classes into a single `MockableDatabase`. Like `DummyDatabase`, most functions do nothing and just return true. Like `FailDatabase`, the return value of some functions can be configured on the fly to test various failure cases. Like the memory-only databases, records can actually be written to the `MockableDatabase` and be retrieved later, but all of this is still held in memory. Using `MockableDatabase` completely removes the need for having BDB or SQLite backed wallets in the unit tests for the tests that are not actually testing specific database behaviors.
Because `MockableDatabase`s can be created by each unit test, we can also control what records are stored in the database. Records can be added and removed externally from the typical database modification functions. This will give us greater ability to test failure conditions, particularly those involving corrupted records.
Possible alternative to #26644
ACKs for top commit:
furszy:
ACK 33e2b82
TheCharlatan:
ACK 33e2b82a4fc990253ff77655f437c7aed336bc55
Tree-SHA512: c2b09eff9728d063d2d4aea28a0f0e64e40b76483e75dc53f08667df23bd25834d52656cd4eafb02e552db0b9e619cfdb1b1c65b26b5436ee2c971d804768bcc
MockableDatabase is a WalletDatabase that allows us to interact with the
records to change them independently from the wallet, as well as
changing the return values from within the tests. This will give us
greater flexibility in testing the wallet.
