1e5d3b4f0d doc: add release note for mining option validation (Sjors Provoost)
0317f52022 ci: enforce iwyu for touched files (Sjors Provoost)
8c58f63578 refactor: have mining files include what they use (Sjors Provoost)
3bb6498fb0 mining: store block create options in NodeContext (Sjors Provoost)
4637cd157d mining: reject invalid block create options (Sjors Provoost)
8daac1d6eb mining: add block create option helpers (Sjors Provoost)
128da7c3ff miner: add block_max_weight to BlockCreateOptions (Sjors Provoost)
fa81e51eae mining: parse block creation args in mining_args (Sjors Provoost)
020166080c mining: use interface for tests, bench and fuzzers (Sjors Provoost)
44082bea47 interfaces: make Mining use const NodeContext (Sjors Provoost)
d4368e059c move-only: add node/mining_types.h (Sjors Provoost)
6aeb1fbea2 test: cover IPC blockmaxweight policy (Sjors Provoost)
63b23ea1e9 test: regression test for waitNext mining policy (Sjors Provoost)
24750f8b31 test: add createNewBlock failure helper (Sjors Provoost)
63ee9cd15b test: misc interface_ipc_mining.py improvements (Sjors Provoost)
Pull request description:
Although this PR is primarily a refactor, _there are behavior changes_ documented in the release note:
- the IPC mining interface now rejects out-of-range block template options instead of silently clamping them;
- startup now rejects `-blockmaxweight` values lower than `-blockreservedweight`, instead of allowing them to be clamped later.
The interaction between node startup options like `-blockreservedweight` and runtime options, especially those passed via IPC, is confusing.
They're combined in `BlockAssembler::Options`, which this PR gets rid of in favour of `BlockCreateOptions`.
`BlockCreateOptions` is used by interface clients. As before, IPC clients have access to a safe / sane subset, whereas RPC and test code can use all fields. The same type is also used to store mining defaults parsed once during node startup in `NodeContext`.
The maximum block weight setting (`block_max_weight`) is optional. When read from startup options it matches `-blockmaxweight`; when provided by callers it is a runtime override. `Merge()` fills unset fields from startup defaults while preserving caller-provided values.
This all happens in commits `mining: add block create option helpers` and `mining: store block create options in NodeContext`, and requires some preparation to keep things easy to review.
We get rid of `BlockAssembler::Options` but this is used in many tests. Since large churn is inevitable, we might as well switch all tests, bench and fuzzers over to the Mining interface. The `mining: use interface for tests, bench and fuzzers` commit does that, dramatically reducing direct use of `BlockAssembler`. Two exceptions are documented in the commit message. Because `test_block_validity` wasn't available via the interface and the block_assemble benchmark needs it, it's moved from `BlockAssembler::Options` to `BlockCreateOptions` (still not exposed via IPC).
We need access to mining related structs from both the miner and node initialization code. To avoid having to pull in all of `BlockAssembler` for the latter, the `move-only: add node/mining_types.h` commit introduces `node/mining_types.h` and moves `BlockCreateOptions`, `BlockWaitOptions` and `BlockCheckOptions` there from `src/node/types.h`.
I considered also moving `DEFAULT_BLOCK_MAX_WEIGHT`, `DEFAULT_BLOCK_RESERVED_WEIGHT`, `MINIMUM_BLOCK_RESERVED_WEIGHT` and `DEFAULT_BLOCK_MIN_TX_FEE` there from `policy.h`, since they are distinct from relay policy and not needed by the kernel. But this seems more appropriate for a follow-up and requires additional discussion.
---
I kept variable renaming and other formatting changes to a minimum to ease review with `--color-moved=dimmed-zebra`.
## Commit summary
Tests and test cleanup:
- `test: misc interface_ipc_mining.py improvements`
- `test: add assert_create_fails helper`
- `test: regression test for waitNext mining policy`
- `test: cover IPC blockmaxweight policy`
Refactoring test/bench/fuzz callers:
- `interfaces: make Mining use const NodeContext`
- `mining: use interface for tests, bench and fuzzers`
Moving mining interface types:
- `move-only: add node/mining_types.h`
Separating startup defaults from runtime options:
- `mining: parse block creation args in mining_args`: adds `node/mining_args.{h,cpp}` and moves mining option parsing out of `init.cpp`, without storing the parsed values yet.
- `miner: add block_max_weight to BlockCreateOptions`: moves the runtime maximum block weight setting into `BlockCreateOptions` as an optional value, so it can later be defaulted from startup args when unset.
- `mining: add block create option helpers`: centralizes block template option defaulting and merging, removes `BlockAssembler::Options`, and preserves behavior except for dropping the `Specified ` prefix from startup option error messages.
- `mining: reject invalid block create options`: checks typed `BlockCreateOptions` before block template creation, so invalid runtime options are rejected instead of silently clamped. Startup validation also rejects `-blockmaxweight` values lower than `-blockreservedweight`.
- `mining: store block create options in NodeContext`: stores the startup mining options in `NodeContext` as `BlockCreateOptions`, so startup defaults and runtime overrides can be merged with the same option type.
Include hygiene, CI and release note:
- `refactor: have mining files include what they use`
- `ci: enforce iwyu for touched files`
- `doc: add release note for mining option validation`
ACKs for top commit:
w0xlt:
reACK 1e5d3b4f0d
sedited:
ACK 1e5d3b4f0d
ryanofsky:
Code review ACK 1e5d3b4f0d. Looks good, thanks for the updates!
Tree-SHA512: 28c715023cb78f02775caa787b243c994bd0f8ce4559afc8db9301e93400ebbc74963626a4afe65ae15bcc16b9192d051a745839f4c804848d50746ea5a224b4
fa24693819 test: Allow --usecli in tests that already support it (MarcoFalke)
fa8d4d5c35 test: Catch CalledProcessError to support --usecli in feature_dbcrash.py (MarcoFalke)
faf0f848ef test: use echojson to allow rpc_named_arguments.py --usecli (MarcoFalke)
faf993ee44 test: Stop node before modifying config to support rpc_users.py --usecli (MarcoFalke)
fa4fc8c1d7 test: Set TestNode url field early, so that feature_loadblock.py --usecli works (MarcoFalke)
Pull request description:
Some tests disallow to be run under `--usecli`. This reduces the coverage and risks that bugs in the bitcoin-cli go unnoticed.
The commits should be self-explanatory and can be reviewed and tested one-by-one.
ACKs for top commit:
willcl-ark:
ACK fa24693819
Tree-SHA512: e34077be98f88ad1e8649600a5f43fc8c77e4ebb03bbccd88c33f2d67882ccdd52b5d18bcfbfc611dff3ebf7455f8e624a88d062aa1863c5eb813bbf4f48e58b
7be0d6fa18 test: remove the lazy import of util in authproxy (rkrux)
779f444680 test: move out JSONRPCException from authproxy to util (rkrux)
Pull request description:
I noticed this issue while reviewing #34773 where a lazy import
is added in the __call__ method of the AuthServiceProxy class in
authproxy.py
There's a circular dependency between authproxy.py and util.py
due to which the former can't use the common utility functions
and thus lazy imports are used as a workaround.
This patch set breaks the dependency so that authproxy.py can use
the utility functions from util.py in a standard fashion. Few tests that
explicitly use get_rpc_proxy and JSONRPCException needed to have
their imports updated.
ACKs for top commit:
maflcko:
review ACK 7be0d6fa18 🏽
Tree-SHA512: 56775cb13d989342ba9482edb255170d695ce5c2d5efbbd64586e0d5463af16467dbf9efe8a0411bde4dfb9bb531839284b2d6f5d828080171d847b70570977d
5faf2ad880 doc: add release notes for deprecation of wallet rbf & bip125 fields (rkrux)
aba24a9b62 wallet: remove "RPC Only" from -walletrbf option help description (rkrux)
97f7cc0233 wallet: mark -walletrbf startup option as deprecated (rkrux)
c4a7613e6a wallet: mark `bip125-replaceable` key as deprecated in transaction RPCs (rkrux)
Pull request description:
Partially fixes#32661.
This patch set is in line with the deprecation of outdated
BIP 125 opt-in RBF signalling and fullrbf in wallet transaction
RPCs and startup options.
ACKs for top commit:
achow101:
ACK 5faf2ad880
w0xlt:
ACK 5faf2ad880
polespinasa:
code reviewed ACK 5faf2ad880
Tree-SHA512: fe6e57f49bef7245b2f564ba705647fb49f0bd370da2e9cfdce45c64a2d8b33ea10a8a802c6619c6382a9bbd2b0e2e4792b08077bc4cfa9b03f7916e2185652a
f05b1a3532 rpc: Fix for duplicate external signers case (optout)
Pull request description:
A straightforward fix for a minor bug in a low-probability case; when there are multiple signers with the same fingerprint, the signers following the duplicate are also thrown away (due to a `break` instead of `continue`).
Background. Bitcoin core can work with _external signers_. They can be configured using the `-signer=<cmd>` argument. The `enumeratesigners` RPC can be used to retrieve the available signers.
Precondition A minor bug was found in a special case:
- multiple external signers are detected, and
- at least two of them are duplicate, i.e., have identical fingerprint, and
- the duplicates are followed by at least one additional signer (that is, the last one is not a duplicate),
Current behavior: Only one of the duplicates is kept, however all subsequent signers are also ignored. This last part is probably unintended, thus it's a minor bug.
To put it in another way, the set of fingerprints returned depends on the actual order in which the devices are returned by the external tool, which may be arbitrary.
Expected behavior: De-duplicate the found signers by fingerprint:
- keep one signer for each fingerprint
- keep all non-duplicates.
Fix: The early `break` of the loop of signers upon duplication has been changed to `continue`.
Test added/extended: `RPCSignerTest` in `test/functional/rpc_signer.py` has been extended with a case of multiple duplicates.
ACKs for top commit:
l0rinc:
ACK f05b1a3532
achow101:
ACK f05b1a3532
polespinasa:
code review ACK f05b1a3532
naiyoma:
ACK f05b1a3532
Tree-SHA512: ac685e233192ecaf5bd535dc66badeb84d78212b3242df71f11c682828955a40167438e16fccc667cd8beef574bc4137a586f44b0f119d7574422c70bb078293
So that util is not dependent on authproxy at all and going
forward authproxy can use util methods.
Reviewing with --color-moved=dimmed-zebra option can be helpful.
faf02674b3 refactor: Set TestNode.cli only after RPC is connected (MarcoFalke)
fae376cafb refactor: Inline get_rpc_proxy (MarcoFalke)
fa00c7c7a4 test: Allow rpc_bind.py --usecli (MarcoFalke)
fa8f25118c refactor: Use create_new_rpc_connection in wallet_multiwallet.py (MarcoFalke)
fa3ae6c7d3 test: Allow feature_shutdown.py --usecli (MarcoFalke)
fa2a3683d5 test: Allow mining_getblocktemplate_longpoll.py --usecli (MarcoFalke)
fa37c6a529 test: [move-only] Extract create_new_rpc_connection (MarcoFalke)
Pull request description:
Some tests disallow to be run under `--usecli`. This reduces the coverage and risks that bugs in the bitcoin-cli go unnoticed.
The commits should be self-explanatory and can be reviewed and tested one-by-one.
ACKs for top commit:
willcl-ark:
reACK faf02674b3
Tree-SHA512: 83cd6a696e6dd6efd4f2d295e65bfac51fe26404c37f25936808005cc6136e469a30aebaac547af1c722ed5ac827eaf009a150d82420b6b4e242e89305475abe
7777a92a92 ci: Use path with spaces on windows as well (MarcoFalke)
fac6c4270d ci: Put space and non-ASCII char in scratch dir (MarcoFalke)
fa38759823 ci: Require $FILE_ENV (MarcoFalke)
Pull request description:
It seems unlikely that many users have a space in their paths, but it seems a use-case worth enough to be tested by CI, so that it does not have to be done manually. Ref https://github.com/bitcoin/bitcoin/pull/33929#discussion_r2590523065 / https://mirror.b10c.me/bitcoin-bitcoin/33929/#discussion_r2590523065
So do that here, and also add a non-ASCII char while touching.
Also, fix all tests that are broken and assume no space exists in paths.
ACKs for top commit:
hebasto:
ACK 7777a92a92.
sedited:
ACK 7777a92a92
Tree-SHA512: eceb1f6c932c6966cdca8ca8df750081ec5134db5e5f558f7d955716409117bec7c8585d75865e2c98bc1ae7394f3ce64dff87bcebe1e68591afaeef1831d6dd
fa89098888 test: Document rare failure in test_inv_block better (MarcoFalke)
Pull request description:
This test may fail intermittently, see the tracking issue https://github.com/bitcoin/bitcoin/issues/19732 and https://github.com/bitcoin-core/gui/actions/runs/25819423445/job/75856716144?pr=936#step:10:5303 specifically:
```
2026-05-13T19:37:19.624321Z TestFramework (INFO): Tx should be received at node 1 after 64 seconds
2026-05-13T20:17:19.627627Z TestFramework (ERROR): Unexpected exception:
Traceback (most recent call last):
File "/home/runner/work/_temp/test/functional/test_framework/test_framework.py", line 143, in main
self.run_test()
File "/home/runner/work/_temp/build/test/functional/p2p_tx_download.py", line 413, in run_test
test()
File "/home/runner/work/_temp/build/test/functional/p2p_tx_download.py", line 130, in test_inv_block
self.sync_mempools()
File "/home/runner/work/_temp/test/functional/test_framework/test_framework.py", line 721, in sync_mempools
raise AssertionError("Mempool sync timed out after {}s:{}".format(
AssertionError: Mempool sync timed out after 2400s:
{'2db3a22f9370eb32f110597882acebef1c76cf319837bac69b829917730d2349'}
set()
```
The test assumes that node1 gets the tx from its honest outbound peer (node0). However, the same connection is inbound from the view of node0, so node0 will use an exponentially distributed delay, which may be more than the expected timeout.
Document this clearer, so that the this specific failure can simply be re-run without having to spend a long time thinking and debugging it every time it happens.
ACKs for top commit:
willcl-ark:
ACK fa89098888
Tree-SHA512: b7fa1a584bc61595d7e1b73ed1fbe446604515c24786565e76b385c7be3278cdb8909d8aeabe6d3f05685281851836f2267b97d7f022d634d833d2124ba53c00
d61053d97b build: Drop libevent from bitcoin-cli link libraries (Fabian Jahr)
798d051c80 cli: Remove libevent usage (Fabian Jahr)
376e7ef07c util: Expose IOErrorIsPermanent in sock header (Fabian Jahr)
5d562430de netbase: Add timeout parameter to ConnectDirectly (Fabian Jahr)
a988ac592f cli: Add HTTPResponseHeaders class for parsing response headers (Fabian Jahr)
c471c5085b common: Add unused UrlEncode function (Fabian Jahr)
9687ef1bd9 ci: Tolerate unused free functions in intermediate commits (Fabian Jahr)
Pull request description:
Part of the effort to remove the libevent dependency altogether, see #31194
This takes the parsing logic from the [`HTTPHeaders` class](d549f01caa) from #32061 and puts it into `bitcoin-cli` as a small `HTTPResponseHeaders` class with a comment to revisit potentially sharing this code somehow. This decoupled the two pulls which seems like the most sensible way to deal with this since the actual overlap is very small compared to the impact of each of the pulls which should ideally not block each other.
Otherwise the change itself replaces the libevent-based HTTP client with a simple synchronous implementation which uses the `Sock` class directly.
ACKs for top commit:
hodlinator:
re-ACK d61053d97b
theStack:
re-ACK d61053d97b
w0xlt:
ACK d61053d97b
Tree-SHA512: a3580a45faf540ee844aac8cb1dc056a89e8e11b45781d2807baa4736d5c0934284c6066206101b6984111a48a186d67845545d07639b623cb35ccc2d85d3ab2
Check BlockCreateOptions before block template creation instead of
clamping runtime values. This makes invalid runtime block creation
options, including those passed by IPC mining clients, fail explicitly
instead of silently mining with different values than the caller
requested.
Runtime option validation now uses the same error wording as startup
option validation. Startup validation also rejects -blockmaxweight
values lower than -blockreservedweight instead of allowing them to be
clamped later.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Move block template defaulting into helper functions for
BlockCreateOptions. FlattenMiningOptions() fills hardcoded defaults and
MergeMiningOptions() overlays defaults without replacing caller-provided
values.
Use the shared option type in BlockAssembler so IPC callers and internal
callers can pass through the same options path. This commit does not
change behavior, except for dropping the "Specified " prefix from startup
option error messages.
Keep the -blockmintxfee ParseMoney check in ReadMiningArgs() instead of
CheckMiningOptions(), because CheckMiningOptions() only sees the parsed
CFeeRate value and not the original string.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Verify that -blockmaxweight is honored both when an IPC block template is first created and after waitNext() refreshes the template.
Co-authored-by: w0xlt <94266259+w0xlt@users.noreply.github.com>
Demonstrates that BlockTemplateImpl::waitNext() must respect the
mining policy supplied via -blockmintxfee, not silently fall back
to defaults.
Co-authored-by: Enoch Azariah <enirox001@gmail.com>
The cli can only be called after the RPC is connected, so the cli field
should only be set after that. This is similar to the _rpc field.
This change has also other benefits:
* Manually overwriting the cli with a new rpchost is no longer needed,
so one cli-specific line can be removed from the rpc_bind.py test.
* The datadir and rpc_timeout fields are removed from the TestNodeCLI
struct. They were redundant with the -datadir and -rpcclienttimeout
command line options. Any command line option can be overwritten by
appending it with a new value, if needed.
This allows to run the test under bitcoin-cli, except for one small
part, which is skipped for now.
This also inlines rpc_url directly into create_new_rpc_connection,
because this is the only place where it is needed.
ca5483a662 qa: use NORMAL_GBT_REQUEST_PARAMS consistently (Antoine Poinsot)
Pull request description:
Functional tests have a constant that defines normal parameters to `getblocktemplate` but some tests were still hardcoding its value. This PR updates those tests to use the constant instead, so that if normal parameters to `getblocktemplate` need to be changed, it is only necessary to change them in a single place.
This is preparatory work for the implementation of BIP 54, which introduces a new GBT "forced" rule.
ACKs for top commit:
fanquake:
ACK ca5483a662
sedited:
ACK ca5483a662
Tree-SHA512: 07374e501ea66ba7b62f610797758fd3528950215b987a50cb18aa844bbf5fb5f9f49aafebc1472bf336de99ff96d1e74f550c09866fdf2882c73b87ac2a715e
Since bitcoin/bitcoin#33362, feature_bind_port_externalip.py runs in CI and
auto-detects whether 1.1.1.5 is assigned by starting nodes with -bind=1.1.1.5,
then converting a FailedToStartError containing "Unable to bind to" into a
SkipTest. On Windows CI the address is not configured, so bitcoind fails as
expected — but intermittently, while the process is shutting down, an RPC probe
raises ConnectionAbortedError (WSAECONNABORTED/WinError 10053) before
wait_for_rpc_connection() notices that the process has exited. That error was
not in the suppressed set, so it escaped and the test failed instead of being
skipped.
The intermittency is a race on TCP connection timing. If the probe connects
before the RPC port is listening, connect can fail with ECONNREFUSED. If the
TCP connection is established but is then closed abortively during shutdown,
the probe can see a connection-reset/aborted error instead. On POSIX systems
this commonly shows up as ECONNRESET ("connection reset by peer"). Winsock
also has WSAECONNABORTED for cases where an established connection is aborted
locally or otherwise terminated due to a timeout/protocol failure, in addition
to WSAECONNRESET for a reset by the remote side. This is why the
Windows-specific error may need to be suppressed separately.
Fix by treating ECONNABORTED identically to ECONNRESET, ETIMEDOUT, and
ECONNREFUSED: retry the probe rather than raising.
Fix was suggested by willcl-ark in
https://github.com/bitcoin/bitcoin/issues/35343#issuecomment-4507329622Fixes#35343
b63ef20d54 test: add fuzz harness for CDBWrapper (Andrew Toth)
32169c3855 dbwrapper: accept optional testing leveldb::Env in DBParams (Andrew Toth)
8d390c93fc dbwrapper: make max_file_size a configurable DBParams field (Andrew Toth)
Pull request description:
Inspired by https://github.com/bitcoin/bitcoin/pull/34866#issuecomment-4090291488.
We currently don't have a dedicated harness targeting `CDBWrapper`. OSS-Fuzz has a [rudimentary harness](https://github.com/google/oss-fuzz/blob/master/projects/leveldb/fuzz_db.cc) for levelDB [which fails](https://issues.oss-fuzz.com/issues/447252244), so doesn't appear maintained.
This PR adds a harness targeting `CDBWrapper` against an in-memory oracle to verify correctness.
A `DeterministicEnv` wraps levelDB's `memenv` to eliminate non-determinism by capturing background compaction and running it at fuzzer-chosen points.
The fuzzer also controls the cache_bytes and max_file_size sizes so that small values trigger memtable flushes and compaction.
ACKs for top commit:
l0rinc:
code review ACK b63ef20d54
marcofleon:
ACK b63ef20d54
dergoegge:
utACK b63ef20d54
sedited:
ACK b63ef20d54
Tree-SHA512: da1f738ec90c49830a05b8990bdaa474299b573e966e60f4febef1292d9682f2e50f0016831f26bf4677e5afdaa142dc8766d871c6bce90d35f1695d480ac8c1
All transactions are by default replaceable since v28, the wallet need not have
a configuration option to opt into RBF signalling because it seems redundant
now. Emit a warning if this option is used.
Transactions are replaceable by default since v28 and the corresponding
tweaking argument has been removed since v29. The related key from the
mempool RPC has been marked deprecated as well since v29, this patch
does the same for the wallet transaction RPCs such as listtransactions,
listsinceblock, and gettransaction.
Users can pass the -deprecatedrpc=bip125 startup argument to retrieve
this key in the responses of above mentioned RPCs.
In case of multiple external signers, with some duplicates,
de-duplicate them: keep one signer per fingerprint, and
keep all non-duplicates as well. Add a new test check.
Introduces a libFuzzer harness that exercises CDBWrapper operations
against a std::map oracle, with a DeterministicEnv that captures LevelDB
background compaction for single-threaded determinism.
A sibling dbwrapper_threaded target uses a bare memenv so LevelDB's real
background thread runs, exercising force_compact and threaded compaction
paths that the deterministic variant cannot reach.
Adds an implicit-integer-sign-change suppression for
BytewiseComparatorImpl::FindShortSuccessor (leveldb/util/comparator.cc:58)
to the test ubsan suppressions list. LevelDB's bytewise comparator
implicitly converts a signed `char` byte to `uint8_t` there. The path
is only reached when compaction picks an SST boundary key, so it
requires a small enough max_file_size for compaction to fire during
the fuzz run.
Co-authored-by: l0rinc <pap.lorinc@gmail.com>
Re-using the same rpc connection on multiple threads is obviously
unsafe, so this helper can be used to create one connection per thread.
This refactor does not change any behavior and can be reviewed with the
git options:
--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
75cf9708a0 ci: add one more routable address to the VMs (docker containers) (Vasil Dimov)
1b93983bf5 test: make feature_bind_port_(discover|externalip).py auto-detect the skip condition (Vasil Dimov)
Pull request description:
`feature_bind_port_discover.py` and `feature_bind_port_externalip.py` require a routable address on the machine to run. Since that was not predictably available on CI, those tests required a manual setting up of IP addresses (e.g. using `ifconfig`) and then running the tests with a command line option telling them that the addresses are set up. The tests were not run in CI and [got rot](https://github.com/bitcoin/bitcoin/issues/31293#issuecomment-2497792487).
Change that to auto-detect, from the tests, whether the needed IP addresses are present and if yes, run the test, otherwise skip it. Also change the CI to configure the needed addresses when running the functional tests. This way the tests will be run regularly on CI.
Fixes: https://github.com/bitcoin/bitcoin/issues/31336
ACKs for top commit:
willcl-ark:
ACK 75cf9708a0
frankomosh:
Tested ACK 75cf9708a0. Built from source.
ryanofsky:
Code review ACK 75cf9708a0. Tested locally with and without the special addresses, and the detection seems to work well.
Tree-SHA512: 252911a37a06764f644a1a83c808f5255ac3bc74919426afa5d082c59e1ea924196354735f229d381cb5aff2340e001c2240bbadc8b5f27e5321fb4cfaef0fdb
81348576cc psbt, test: remove address type restrictions in test (rkrux)
Pull request description:
Because the corresponding Taproot fields were added in PSBT in #22558,
so these restrictions are no longer necessary.
ACKs for top commit:
kevkevinpal:
ACK [8134857](81348576cc)
polespinasa:
lgtm ACK 81348576cc
furszy:
utACK 81348576cc
Tree-SHA512: 5621509e103674ee5f39454871ca5acb2bf4c16b85dbbdc38abac22fbaea44cafe4bbd91dc3a5c6435b3859a4dc58f12bf7aeec2952c9f80fbbdad2adb52847c
1c500b1709 test: avoid non-loopback network traffic from node_init_tests/init_test (Vasil Dimov)
Pull request description:
The test `node_init_tests/init_test` calls:
`AppInitMain()` -> `StartMapPort()` -> `StartThreadMapPort()` -> `ThreadMapPort()` -> `ProcessPCP()` -> `PCPRequestPortMap()` -> `CreateSock()` and on the returned value from `CreateSock()` it calls the `Connect()` method.
Thus, change `BasicTestingSetup::BasicTestingSetup()` to set `-natpmp` to 0. This way `node_init_tests/init_test` or other tests will not do network activity due to `ThreadMapPort()`.
Also add a comment about `natpmp=0` in
`test/functional/test_framework/util.py`.
Also set `-dnsseed=0` in `BasicTestingSetup::BasicTestingSetup()` to
avoid DNS queries.
ACKs for top commit:
fjahr:
re-ACK 1c500b1709
ryanofsky:
Code review ACK 1c500b1709, just disabling -dnsseed since previous review, which makes sense.
Tree-SHA512: 3b275d91361804da6d1dc109dffe741ea4b3dd3be916eb12fa63efa233e13ea4dab5a9f8448bd8bf99bc41817f3b7a768abe91531a3f63eae8b4c912bfcbd13e
89af67d79f tests: Add some fuzz test coverage for command-specific args (Anthony Towns)
92df785859 tests: Add some test coverage for ArgsManager::AddCommand (Anthony Towns)
33c8090be9 ArgsManager: automate checking for correct command options (Anthony Towns)
186354a0d8 bitcoin-wallet: use command-specific options (Anthony Towns)
d21e82b7d6 ArgsManager: support command-specific options (Anthony Towns)
Pull request description:
Adds the ability to link particular options to one or more (`OptionsCategory::COMMANDS`) commands, and uses this feature
in `bitcoin-wallet`. Separates out the help information for these command-specific options (duplicating it if an option applies to multiple commands), and provides a function for checking at runtime if some options have been specified by the user that only apply to other commands.
#### Motivation
Currently, `ArgsManager` supports commands like `bitcoin-wallet dump` but while some of the options are command-specific (like `-dumpfile`), `ArgsManager` itself doesn't know that. As a result, `-dumpfile` is listed in the global help rather than under the relevant commands, and if you use `-dumpfile` with a different command that doesn't support it, `ArgsManager` cannot automatically report that as an error, resulting in the commands that don't support the option having to have error-handling specific to all the options they don't support.
#### Changes
Help output moves command-specific options under their associated commands:
Before:
```
Options:
-dumpfile=<file name>
When used with 'dump', writes out the records to this file. When used
with 'createfromdump', loads the records into a new wallet.
...
Commands:
createfromdump
Create new wallet file from dumped records
dump
Print out all of the wallet key-value records
```
After:
```
Commands:
createfromdump
Create new wallet file from dumped records
-dumpfile=<file name>
When used with 'dump', writes out the records to this file. When
used with 'createfromdump', loads the records into a new wallet.
dump
Print out all of the wallet key-value records
-dumpfile=<file name>
When used with 'dump', writes out the records to this file. When
used with 'createfromdump', loads the records into a new wallet.
```
Error messages are now generated automatically by `ArgsManager` rather than ad-hoc wallet code for each option:
Before:
```c++
if (args.IsArgSet("-dumpfile") && command != "dump" && command != "createfromdump") {
tfm::format(std::cerr, "The -dumpfile option can only be used with the \"dump\" and \"createfromdump\" commands.\n");
return false;
}
```
After:
```c++
std::vector<std::string> details;
if (!args.CheckCommandOptions(command, &details)) {
tfm::format(std::cerr, "Error: Invalid arguments provided:\n%s\n", util::MakeUnorderedList(details));
return false;
}
```
#### Limitations
- If an option applies to multiple commands, it shares the same help text. There's no way to provide per-command descriptions.
- Option parsing rules are unchanged — options still cannot appear after the command.
ACKs for top commit:
achow101:
ACK 89af67d79f
sedited:
Re-ACK 89af67d79f
ryanofsky:
Code review ACK 89af67d79f. Since last review: rebase, integration with ClearArgs and fuzz test, and Assume -> Assert switch
Tree-SHA512: 7ae7c3b74d0c8c4db8459e9f0b9c7498b2fa4758954ec49983decbba177877b039779f0f7b55e60c3a0ed74c5e9e4ac4734ba9e049bf3a7743280ef8300869fa
da769855d0 test: add PSBT proprietary merge regression coverage (w0xlt)
3f5b3c7a80 psbt: preserve proprietary fields when combining PSBTs (w0xlt)
Pull request description:
BIP 174 proprietary fields are currently parsed, serialized, and exposed by `decodepsbt`, but they are not preserved by `combinepsbt`.
The reason is that the merge paths in `PartiallySignedTransaction::Merge()`, `PSBTInput::Merge()`, and `PSBTOutput::Merge()` union `unknown`, but never union `m_proprietary`.
This means application-specific PSBT metadata can be lost during combination, even though BIP 174 treats proprietary records as normal PSBT key-value pairs for private or application-specific use.
This PR fixes that by preserving proprietary fields in all three merge paths.
ACKs for top commit:
nervana21:
re-ACK da769855d0
Bicaru20:
re-ACK da769855d0
achow101:
ACK da769855d0
theStack:
Code-review ACK da769855d0
Tree-SHA512: 4474674ac00c3155fd7d3d777bdeb70d4ca2006c8fe62eb84a888ef2f7aa02739bf31d8e9f2cb59e324be1085c77897f0bbf6673254c12408ab09a1582e28b83
8544537f41 mining: drop unused include_dummy_extranonce option (Sjors Provoost)
58eeab790d mining: only pad with OP_0 at heights <= 16 (Sjors Provoost)
00d22328b0 mining: pad coinbase to fix createNewBlock at heights <=16 (Sjors Provoost)
605ff37403 test: bad-cb-length for createNewBlock() at low heights (Sjors Provoost)
1966621b76 test: refactor IPC mining test to use script_BIP34_coinbase_height (Sjors Provoost)
Pull request description:
Blocks 0-16 on any new chain require mining code to be careful not to violate the `bad-cb-length` rule, which states the coinbase transaction scriptSig must be at least 2 bytes.
Our mining code deals with that by padding the `scriptSig` with a 0 `extraNonce`. It does this for every height. As a result IPC clients would get an unnecessary `0` in the `scriptSigPrefix` field of `CoinbaseTx`. #32420 fixed that by introducing a `include_dummy_extranonce` option in `BlockCreateOptions` and turning that off for IPC clients.
A minor issue was missed though: `createNewBlock()` now fails with `bad-cb-length`. An easy workaround is to use the `generate` RPC for the first 16 blocks, as demonstrated in the 2nd commit.
The real fix is to have the miner code always pad the `scriptSig` at lower heights, but to _not_ include that in the `scriptSigPrefix` field of `CoinbaseTx` (introduced in #33819). This is what the 3rd commit implements.
Now that we set `scriptSigPrefix` independent of what our internal miner code does - to get past `CheckBlock()` - the original motivation for `include_dummy_extranonce` goes away and we can just drop it entirely. The last commit drops it, while the 4th commit adjusts the tests and hardcoded block and assume utxo hashes.
This last change does not break IPC clients, because `include_dummy_extranonce` was never exposed in `mining.capnp`.
Instead of adjusting the hardcoded hashes, an alternative approach would be to just always pad the `scriptSig` internally, since we exclude the padding from `scriptSigPrefix` anyway. However, IPC clients can also call `getBlock()` to get the raw block and might be confused about the difference. The miner code is also easier to understand if we limit the exception (`coinbase_tx.script_sig_prefix != coinbaseTx.vin[0].scriptSig`) to `nHeight <= 16`, where the explanation is based purely on consensus rules rather than historical test suite reasons.
The first two commits are preperation test changes:
- extract `assert_capnp_failed` helper for macOS (also part of #34727)
- use `script_BIP34_coinbase_height` in IPC mining test (existing code in `interface_ipc_mining.py` was incorrect for low height
Fixes#35126
ACKs for top commit:
ryanofsky:
Code review ACK 8544537f41. Just rebased to fix silent conflict and applied some minor suggestions since last review. As part of rereviewing I left some more minor suggestions that are fine to ignore.
sedited:
Re-ACK 8544537f41
Tree-SHA512: a01d48842bf4bcc1a9c51a89ef9d750766db7d04edb4dcd6b3a8bf195c6b4fa07445256a49367ff0db00ab489a52a3d7ff6a5c3ab9290ecb1fcb82f532552e9b
The test calls:
`AppInitMain()` -> `StartMapPort()` -> `StartThreadMapPort()` ->
`ThreadMapPort()` -> `ProcessPCP()` -> `PCPRequestPortMap()` ->
`CreateSock()` and on the returned value from `CreateSock()` it calls
the `Connect()` method.
Thus, change `BasicTestingSetup::BasicTestingSetup()` to set `-natpmp`
to 0. This way `node_init_tests/init_test` or other tests will not do
network activity due to `ThreadMapPort()`.
Also add a comment about `natpmp=0` in
`test/functional/test_framework/util.py`.
Also set `-dnsseed=0` in `BasicTestingSetup::BasicTestingSetup()` to
avoid DNS queries.
Also, add a missing quote around -DCMAKE_INSTALL_PREFIX to avoid word
splitting.
Otherwise, cmake would warn:
```
+ cmake -S /home/runner/work/_temp -B /home/runner/work/_temp/build -DCMAKE_COMPILE_WARNING_AS_ERROR=ON -DBUILD_BENCH=ON -DBUILD_FUZZ_BINARY=ON -DCMAKE_INSTALL_PREFIX=/home/runner/work/_temp/ci/scratch_ ₿🧪_/out -Werror=dev --preset=dev-mode -DSANITIZERS=address,float-divide-by-zero,integer,undefined -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_FLAGS=-ftrivial-auto-var-init=pattern -DCMAKE_CXX_FLAGS=-ftrivial-auto-var-init=pattern -DCMAKE_EXE_LINKER_FLAGS=-fuse-ld=mold -DAPPEND_CXXFLAGS=-std=c++23 '-DAPPEND_CPPFLAGS=-DARENA_DEBUG -DDEBUG_LOCKORDER'
CMake Warning:
Ignoring extra path from command line:
"₿🧪_/out"
```
Also, allow spaces in the debug log file regex in a test.
Otherwise, the test would fail:
```
TestFramework (ERROR): Unexpected exception:
Traceback (most recent call last):
File "./test/functional/test_framework/test_framework.py", line 142, in main
self.run_test()
~~~~~~~~~~~~~^^
File "./test/functional/feature_logging.py", line 40, in run_test
self.nodes[0].assert_start_raises_init_error([f"-debuglogfile={invalidname}"], exp_stderr, match=ErrorMatch.FULL_REGEX)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "./test/functional/test_framework/test_node.py", line 743, in assert_start_raises_init_error
self._raise_assertion_error(
~~~~~~~~~~~~~~~~~~~~~~~~~~~^
'Expected message "{}" does not fully match stderr:\n"{}"'.format(expected_msg, stderr))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "./test/functional/test_framework/test_node.py", line 229, in _raise_assertion_error
raise AssertionError(self._node_msg(msg))
AssertionError: [node 0] Expected message "Error: Could not open debug log file \S+$" does not fully match stderr:
"Error: Could not open debug log file /Users/runner/work/bitcoin-core-with-ci/bitcoin-core-with-ci/repo_archive/ci/scratch_ ₿🧪_/test_runner/test_runner_₿_🏃_20260218_095938/feature_logging_31/node0/regtest/foo/foo.log"
```
Also, add missing quotes in a test. Otherwise, the test would fail:
```
455/455 - feature_notifications.py failed, Duration: 402 s
stdout:
TestFramework (INFO): Initializing test directory /home/runner/work/_temp/ci/scratch_ ₿🧪_/test_runner/test_runner_₿_🏃_20260218_113529/feature_notifications_128
TestFramework (INFO): test -blocknotify
TestFramework.utils (ERROR): wait_until() failed. Predicate: ''''
self.wait_until(lambda: len(os.listdir(self.blocknotify_dir)) == block_count, timeout=10)
'''
a39cc16b43 doc: Release note for addhdkey (Ava Chow)
89b9a01b4e wallet, rpc: Disallow importing unused() to wallets without privkeys (Ava Chow)
35bbee6374 wallet, rpc: Disallow import of unused() if key already exists (Ava Chow)
f3f8bcbd1d wallet: Add addhdkey RPC (Ava Chow)
82bc280de4 test: Simple test for importing unused(KEY) (Ava Chow)
80c29bc6f1 descriptor: Add unused(KEY) descriptor (Ava Chow)
Pull request description:
It is sometimes useful for the wallet to have keys that it can sign with but are not (initially) involved in any scripts, e.g. for setting up a multisig. Ryanofsky [suggested](https://github.com/bitcoin/bitcoin/pull/26728#issuecomment-1867721948) A `unused(KEY)` descriptor which allows for a key to be specified, but produces no scripts. These can be imported into the wallet, and subsequently retrieved with `gethdkeys`. Additionally, `listdescriptors` will output these descriptors so that they can be easily backed up.
In order to make it easier for people to add HD keys to their wallet, and to generate a new one if they want to rotate their descriptors, an `addhdkey` RPC is also added. Without arguments, it will generate a new HD key and add it to the wallet via a `unused(KEY)` descriptor. If provided a private key, it will construct the descriptor and add it to the wallet.
See also: https://github.com/bitcoin/bitcoin/pull/26728#issuecomment-1866961865
Based on #29130 as `gethdkeys` is useful for testing this.
ACKs for top commit:
Sjors:
utACK a39cc16
rkrux:
lgtm ACK a39cc16b43
Tree-SHA512: c1288c792ab01ca2eaddd24b0e7d11c259cd59e79042465d0d1eb656fd559c1200dc19750b4d84acc762b5b599935a06df214c18226e662087842ea91ec3011b
