There could be a race with outstanding TxAddedToMempool notifications
being applied to the soon-to-be created wallet.
Fixes an intermittent timeout reproducable by adding a sleep to
AddToWallet.
a58cb3b1c1 qa: sanity check mined block have their coinbase timelocked to height (Antoine Poinsot)
8f2078af6a miner: timelock coinbase transactions (Antoine Poinsot)
788aeebf34 qa: use prev height as nLockTime for coinbase txs created in unit tests (Antoine Poinsot)
c76dbe9b8b qa: timelock coinbase transactions created in fuzz targets (Antoine Poinsot)
9c94069d8b contrib: timelock coinbase transactions in signet miner (Antoine Poinsot)
a5f52cfcc4 qa: timelock coinbase transactions created in functional tests (Antoine Poinsot)
Pull request description:
The Consensus Cleanup soft fork proposal includes enforcing that coinbase transactions set their
nLockTime field to the block height minus 1, as well as their nSequence such as to not disable the
timelock. If such a fork were to be activated by Bitcoin users, miners need to be ready to produce
compliant blocks at the risk of losing substantial amounts mining would-be invalid blocks. As miners
are unfamously slow to upgrade, it's good to make this change as early as possible.
Although Bitcoin Core's GBT implementation does not provide the `coinbasetxn` field, and mining
pool software crafts the coinbase on its own, updating the Bitcoin Core mining code is a first step
toward convincing pools to update their (often closed source) code. A possible followup is also to
introduce new fields to GBT. In addition, this first step also makes it possible to test future
Consensus Cleanup changes.
The commit making the change also updates a bunch of seemingly-unrelated tests. This is because those tests were asserting error messages based on the txid of transactions involved, and changing the coinbase transaction structure necessarily changes the txid of all tests' transactions.
ACKs for top commit:
Sjors:
Code review ACK a58cb3b1c1
achow101:
ACK a58cb3b1c1
TheCharlatan:
Re-ACK a58cb3b1c1
Tree-SHA512: a2aae009a187eb760d34435f518a895ee76c6b02a667eb030ddf6bd584da6e8eae2737d974dbf81a928d60c07bcb4820f055adc067e18d8819640db0240bb513
10845cd7cc qa: Add feature_framework_startup_failures.py (Hodlinator)
28e282ef9a qa: assert_raises_message() - Stop assuming certain structure for exceptions (Hodlinator)
1f639efca5 qa: Work around Python socket timeout issue (Hodlinator)
9b24a403fa qa: Only allow calling TestNode.stop() after connecting (Hodlinator)
6ad21b4c01 qa: Include ignored errors in RPC connection timeout (Hodlinator)
879243e81f qa refactor: wait_for_rpc_connection - Treat OSErrors the same (Hodlinator)
Pull request description:
Improves handling of startup errors in functional tests and puts tests in place to ensure knock-on errors don't creep in.
- `wait_for_rpc_connection()` now appends specific failures leading up to the `Unable to connect to bitcoind` error to that error message:
`[node 0] Unable to connect to bitcoind after 60s (ignored errors: {'missing_credentials': 1, 'OSError.ECONNREFUSED': 239}, latest error: ConnectionRefusedError(111, 'Connection refused'))`
- Fixes Windows Python issue where `socket.timeout` exceptions end up with unset `errno`-fields.
- Also adds comments, refactors code, improves logging.
The underlying purpose is to ensure developer efficiency in finding root causes of test failures.
Prior iterations of the PR partially focused on fixing the same issue as #31620.
Originally inspired by #30390.
### Testing
Can be tested by reverting either faf2f2c654 or fae3bf6b87 from #31620, or the "qa: Avoid calling stop-RPC if not connected" from this PR, and running *feature_framework_startup_failures.py*.
ACKs for top commit:
l0rinc:
ACK 10845cd7cc
ryanofsky:
Code review ACK 10845cd7cc. Only changes since last review were adding a new commit tweaking assert_raises_message(), extending the new test to have a self-check, and to pass through all options to child tests instead of a hardcoded list of options. I left some cleanup suggestions below but they are not important.
Tree-SHA512: f0235c5cbb6d1bb85d8dc5de492a08a34f6edc83499cbf0a5f9a3824809ff84635888c62c9c01101e3cc9ef9f1cdee2c9ab6537fea6feeb005b29f428caf8b22
de054df6dc contrib: Remove legacy wallet RPCs from bash completions (Ava Chow)
5dff04a1bb legacy spkm: Make IsMine() and CanProvide() private and migration only (Ava Chow)
c0f3f3264f wallet: Remove unused db functions (Ava Chow)
83af1a3cca wallet: Delete LegacySPKM (Ava Chow)
8ede6dea0c wallet, rpc: Remove legacy wallet only RPCs (Ava Chow)
4de3cec28d test: rpcs disabled for descriptor wallets will be removed (Ava Chow)
84f671b01d test: Run multisig script limit test (Ava Chow)
810476f31e test: Remove unused options and variables, correct comments (Ava Chow)
04a7a7a28c build, wallet, doc: Remove BDB (Ava Chow)
Pull request description:
The final step of #20160.
A bare minimum of legacy wallet code is kept in order to perform wallet migration. Migration of legacy wallets uses the independent BDB parser and a minimal `LegacyDataSPKM` that allows the legacy data to be loaded so that the migration can be completed.
BDB has been removed as a dependency and documentation have been updated to reflect that.
ACKs for top commit:
Sjors:
re-ACK de054df6dc
maflcko:
re-ACK de054df6dc🔗
w0xlt:
reACK de054df6dc
rkrux:
Concept ACK de054df6dc
Tree-SHA512: 16a6c265bc1ada5e7a5ef9b95f0ff65015672ca46d9a43b7e10d60e9e085052e9bbfe01ac3e494cc606afb652a1b476b10e434d13e9877b67d2cb0196a9bd190
In order to remove potential confusion, this commit adapts all script
error constant names in the functional tests (currently only in
feature_taproot.py) to the ones used in our C++ codebase. This also
makes checking whether we have test coverage for a certain script error
easier.
-BEGIN VERIFY SCRIPT-
ren() { sed -i "s|$1|$2|g" $( git grep -l "$1" -- "./test" ) ; }
ren ERR_SIG_SIZE ERR_SCHNORR_SIG_SIZE
ren ERR_SIG_HASHTYPE ERR_SCHNORR_SIG_HASHTYPE
ren ERR_SIG_SCHNORR ERR_SCHNORR_SIG
ren ERR_CONTROLBLOCK_SIZE ERR_TAPROOT_WRONG_CONTROL_SIZE
ren ERR_PUSH_LIMIT ERR_PUSH_SIZE
ren ERR_MINIMALIF ERR_TAPSCRIPT_MINIMALIF
ren ERR_UNKNOWN_PUBKEY ERR_PUBKEYTYPE
ren ERR_STACK_EMPTY ERR_INVALID_STACK_OPERATION
ren ERR_SIGOPS_RATIO ERR_TAPSCRIPT_VALIDATION_WEIGHT
ren ERR_UNDECODABLE ERR_BAD_OPCODE
ren ERR_NO_SUCCESS ERR_EVAL_FALSE
ren ERR_EMPTY_WITNESS ERR_WITNESS_PROGRAM_WITNESS_EMPTY
-END VERIFY SCRIPT-
c7e2b9e264 tests: Test migration cleans up bad inactive chain derivation path (Ava Chow)
Pull request description:
A bug in 0.21.x and 22.x resulted in some wallets having invalid derivation paths that are the concatenation of two derivation paths. These appear only when inactive hd chains are topped up.
Since key metadata is a legacy wallet only record, migrating legacy wallets to descriptor wallets will fix this issue as all key metadata records are deleted. The derivation path information is derived on-the-fly from the descriptor that is produced for the inactive hd chain.
Thus we only need a test to verify that the derivation paths are good, and that all key metadata records are deleted from the migrated wallet.
ACKs for top commit:
murchandamus:
re-ACK c7e2b9e264 via range-diff:
rkrux:
re-ACK c7e2b9e264
furszy:
utACK c7e2b9e264
Tree-SHA512: 3117c4a43798972109fe2d3539341a8b69db70c6457fcabdd019e6044834dc4b17212abbc006d7b8008f560dce4b7856142b057981b9404f406d58fa0955cbd9
fa58f40b89 test: Slim down previous releases bdb check (MarcoFalke)
Pull request description:
The check iterates over several previous BDB-only releases to check that descriptor wallets are considered "corrupt" when loading. It is unclear why this needs to be done for more than one release.
Avoid the confusion by removing the unused releases from the test and from the download script.
ACKs for top commit:
achow101:
ACK fa58f40b89
rkrux:
ACK fa58f40b89
Tree-SHA512: 8084392481bfe1fba9b80bb865ffbdfa454e9e6e14e02c39fa3f61c1a596b1def2c531c5da1c7566e5fddb77ac7e56f19feabaaf9b5af043fa6c230d9e2370b5
The Consensus Cleanup soft fork proposal includes enforcing that coinbase transactions set their
locktime field to the block height, minus 1 (as well as their nSequence such as to not disable the
timelock). If such a fork were to be activated by Bitcoin users, miners need to be ready to produce
compliant blocks at the risk of losing substantial amounts mining would-be invalid blocks. As miners
are unfamously slow to upgrade, it's good to make this change as early as possible.
Although Bitcoin Core's GBT implementation does not provide the "coinbasetxn" field, and mining
pool software crafts the coinbase on its own, updating the Bitcoin Core mining code is a first step
toward convincing pools to update their (often closed source) code. A possible followup is also to
introduce new fields to GBT. In addition, this first step also makes it possible to test future
Consensus Cleanup changes.
The changes to the seemingly-unrelated RBF tests is because these tests assert an error message
which may vary depending on the txid of the transactions used in the test. This commit changes the
coinbase transaction structure and therefore impact the txid of transactions in all tests.
The change to the "Bad snapshot" error message in the assumeutxo functional test is because this
specific test case reads into the txid of the next transaction in the snapshot and asserts the error
message based it gets on deserializing this txid as a coin for the previous transaction. As this
commit changes this txid it impacts the deserialization error raised.
A bug in 0.21.x and 22.x resulted in some wallets having invalid
derivation paths that are the concatenation of two derivation paths.
These appear only when inactive hd chains are topped up.
Since key metadata is a legacy wallet only record, migrating legacy
wallets to descriptor wallets will fix this issue as all key metadata
records are deleted. The derivation path information is derived
on-the-fly from the descriptor that is produced for the inactive hd
chain.
Thus we only need a test to verify that the derivation paths are good,
and that all key metadata records are deleted from the migrated wallet.
Legacy wallets do not have the descriptors flag set. Don't load wallets
without the descriptors flag.
At the same time, we will no longer load BDB databases since they are
only used for legacy wallets.
Removes all legacy wallet specific functional tests.
Also removes the --descriptor and --legacy-wallet options as these are
no longer necessary with the legacy wallet removed.
fa86190e6e rpc: Allow fullrbf fee bump (MarcoFalke)
Pull request description:
The RPCs (psbt)bumpfee, and the GUI, reject fee bumps when BIP 125 signalling is absent in the transaction even when the mempool and other RPCs allow them. Fix the confusion by allowing the fee bump.
This is done after fullrbf is always on (https://github.com/bitcoin/bitcoin/pull/30592)
ACKs for top commit:
1440000bytes:
reACK fa86190e6e
achow101:
ACK fa86190e6e
w0xlt:
ACK fa86190e6e
rkrux:
reACK fa86190e6e
glozow:
ACK fa86190e6e
Tree-SHA512: b2ffe8dcadbe71e9be767a16cf8aa0bf383c2de7aa1aee9438d125f444e24f3f7e4f02ddb28981bd3b8b645b6a24a407b4ad6bb0b21946ae637e78f6386e05bf
Observed on local machine running Windows / Python v3.13.1 when overriding rpc_timeout to small values (5- seconds). Next commit performs such overrides.
(Still tolerate calling it on a no longer (self.)running node, as in a node that has been queried for is_node_stopped() and modified state before returning True).
Tests should not attempt to use the non-functioning RPC interface to call stop() unless wait_for_connections() has succeeded.
No longer log and suppress http.client.CannotSendRequest as a consequence of stop()-RPC, as error conditions causing this knock-on issue are now guarded against before the call.
When an RPC connection attempt with bitcoind times out, include which ignored errors occurred in the exception message.
May provide clues of what has gone wrong.
ConnectionResetError is an OSError as well (ECONNRESET), no reason to have a separate except-block for it.
Also improves comments for other exceptions and make condition above more Pythonic.
e261eb8d50 tests: Add BIP 373 test vectors (Ava Chow)
26370c68d0 rpc: Include MuSig2 fields in decodepsbt (Ava Chow)
ff3d460898 psbt: Implement un/ser of musig2 fields (Ava Chow)
Pull request description:
Implements un/serialization of MuSig2 PSBT fields and prepares PSBT to be able to sign for MuSig2 inputs.
Split from #29675
ACKs for top commit:
fjahr:
re-ACK e261eb8d50
theStack:
re-ACK e261eb8d50
rkrux:
tACK e261eb8d50
Tree-SHA512: bb852ad074978847ac4dc656332025e2d4d1025d4283537b89618c7cadd61a8ecd2eff24779b8a014bc8d7b431125060449768192fa05ad0577f29e3c64b2374
Also, fix the incorrect documention of the 'replaceable' RPC argument
with respect to sequence number handling. The docs were incorrect
before, so the fix could be extracted, but it seems fine to include here
as well.
fac978fb21 test: Remove fragile and ancient release 0.17 wallet test (MarcoFalke)
Pull request description:
The test checks that the 0.17 wallet rejects wallet files created in "the future".
This is nice, and good to know. However,
* The 0.17 release is ancient and should be unused outside of tests, especially to load future wallets.
* The test intermittently fails, due to ancient RPC server bugs, that were fixed in the meantime. [1]
* Albeit they are not identical, the 0.18 release is still checked in this test, so any theoretical bug that would be caught by 0.17 is hopefully still caught by 0.18 as well.
So fix all issues by removing the test case.
[1] For example from https://api.cirrus-ci.com/v1/task/6161588714995712/logs/ci.log:
```
190/321 - [1mwallet_backwards_compatibility.py --descriptors[0m failed, Duration: 23 s
[17:21:40.700]
[17:21:40.700] [1mstdout:
[17:21:40.700] [0m2025-04-02T21:21:16.575000Z TestFramework (INFO): PRNG seed is: 5772716217847090743
[17:21:40.700] 2025-04-02T21:21:16.580000Z TestFramework (INFO): Initializing test directory /ci_container_base/ci/scratch/test_runner/test_runner_₿_🏃_20250402_210134/wallet_backwards_compatibility_134
[17:21:40.700] 2025-04-02T21:21:26.378000Z TestFramework (INFO): Test wallet backwards compatibility...
[17:21:40.700] 2025-04-02T21:21:33.191000Z TestFramework (INFO): Testing 0.19 addmultisigaddress case (#18075)
[17:21:40.700] 2025-04-02T21:21:33.637000Z TestFramework (INFO): Test that a wallet made on master can be opened on:
[17:21:40.700] 2025-04-02T21:21:33.637000Z TestFramework (INFO): - 250000
[17:21:40.700] 2025-04-02T21:21:34.055000Z TestFramework (INFO): - 240001
[17:21:40.700] 2025-04-02T21:21:34.435000Z TestFramework (INFO): - 230000
[17:21:40.700] 2025-04-02T21:21:34.858000Z TestFramework (INFO): - 220000
[17:21:40.700] 2025-04-02T21:21:35.614000Z TestFramework (INFO): - 210000
[17:21:40.700] 2025-04-02T21:21:35.707000Z TestFramework (INFO): Test descriptor wallet incompatibility on:
[17:21:40.700] 2025-04-02T21:21:35.707000Z TestFramework (INFO): - 200100
[17:21:40.700] 2025-04-02T21:21:35.878000Z TestFramework (INFO): - 190100
[17:21:40.700] 2025-04-02T21:21:36.021000Z TestFramework (INFO): - 180100
[17:21:40.700] 2025-04-02T21:21:36.319000Z TestFramework (INFO): Test descriptor wallet incompatibility with 0.17
[17:21:40.700] 2025-04-02T21:21:37.328000Z TestFramework (INFO): Test that 0.21 cannot open wallet containing tr() descriptors
[17:21:40.700] 2025-04-02T21:21:37.356000Z TestFramework (INFO): Test that a wallet can upgrade to and downgrade from master, from:
[17:21:40.700] 2025-04-02T21:21:37.361000Z TestFramework (INFO): - 250000
[17:21:40.700] 2025-04-02T21:21:37.665000Z TestFramework (INFO): - 240001
[17:21:40.700] 2025-04-02T21:21:37.970000Z TestFramework (INFO): - 230000
[17:21:40.700] 2025-04-02T21:21:38.439000Z TestFramework (INFO): - 220000
[17:21:40.700] 2025-04-02T21:21:38.793000Z TestFramework (INFO): - 210000
[17:21:40.700] 2025-04-02T21:21:39.470000Z TestFramework (INFO): Stopping nodes
[17:21:40.700]
[17:21:40.700]
[17:21:40.700] [1mstderr:
[17:21:40.700] [0mTraceback (most recent call last):
[17:21:40.700] File "/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/test/functional/wallet_backwards_compatibility.py", line 389, in <module>
[17:21:40.700] BackwardsCompatibilityTest(__file__).main()
[17:21:40.700] File "/ci_container_base/test/functional/test_framework/test_framework.py", line 206, in main
[17:21:40.700] exit_code = self.shutdown()
[17:21:40.700] File "/ci_container_base/test/functional/test_framework/test_framework.py", line 379, in shutdown
[17:21:40.700] self.stop_nodes()
[17:21:40.700] File "/ci_container_base/test/functional/test_framework/test_framework.py", line 643, in stop_nodes
[17:21:40.700] node.stop_node(wait=wait, wait_until_stopped=False)
[17:21:40.700] File "/ci_container_base/test/functional/test_framework/test_node.py", line 397, in stop_node
[17:21:40.700] self.stop()
[17:21:40.700] File "/ci_container_base/test/functional/test_framework/coverage.py", line 50, in __call__
[17:21:40.700] return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
[17:21:40.700] File "/ci_container_base/test/functional/test_framework/authproxy.py", line 132, in __call__
[17:21:40.700] response, status = self._request('POST', self.__url.path, postdata.encode('utf-8'))
[17:21:40.700] File "/ci_container_base/test/functional/test_framework/authproxy.py", line 106, in _request
[17:21:40.700] return self._get_response()
[17:21:40.700] File "/ci_container_base/test/functional/test_framework/authproxy.py", line 169, in _get_response
[17:21:40.700] http_response = self.__conn.getresponse()
[17:21:40.700] File "/usr/lib/python3.10/http/client.py", line 1375, in getresponse
[17:21:40.700] response.begin()
[17:21:40.700] File "/usr/lib/python3.10/http/client.py", line 318, in begin
[17:21:40.700] version, status, reason = self._read_status()
[17:21:40.700] File "/usr/lib/python3.10/http/client.py", line 287, in _read_status
[17:21:40.700] raise RemoteDisconnected("Remote end closed connection without"
[17:21:40.700] http.client.RemoteDisconnected: Remote end closed connection without response
[17:21:40.700] [node 10] Cleaning up leftover process
[17:21:40.700] [node 9] Cleaning up leftover process
[17:21:40.700] [node 8] Cleaning up leftover process
[17:21:40.700] [node 7] Cleaning up leftover process
[17:21:40.700] [node 6] Cleaning up leftover process
[17:21:40.700] [node 5] Cleaning up leftover process
[17:21:40.700] [node 4] Cleaning up leftover process
[17:21:40.700] [node 3] Cleaning up leftover process
[17:21:40.700] [node 2] Cleaning up leftover process
[17:21:40.700] [node 1] Cleaning up leftover process
[17:21:40.700] [node 0] Cleaning up leftover process
ACKs for top commit:
laanwj:
Code review ACK fac978fb21
janb84:
Re ACK [fac978f](fac978fb21)
pablomartin4btc:
re ACK fac978fb21
BrandonOdiwuor:
Code Review ACK fac978fb21
Tree-SHA512: 13acdfc6be4293a0ff45ae20b26ba60636e130097da380b7b51716faaa950320462399bef55e74b3cedc82944586dcc1bfd078babb96edb03c4efdb8f40af5a4
459807d566 test: remove strict restrictions on rpc_deprecated (Pol Espinasa)
Pull request description:
Removed the wallet restrictions for `rpc_deprecated.py` and added specific test case for the current deprecated rpc.
`skip_test_if_missing_module` will skip the whole test when the wallet is missing, even if a part of the test is non-wallet related. This PR ensures that other tests not related to wallet can be ran and only this specific test will be skipped if there's no wallet
For more context check https://github.com/bitcoin/bitcoin/pull/31278#discussion_r2011661090
ACKs for top commit:
maflcko:
lgtm ACK 459807d566
rkrux:
ACK 459807d
Tree-SHA512: 922b0fafe8fb5bd88a677ce8be5c3fe2fdd4d0aadcd32cc11738a714cd6f765f07e7e7158c829f8338db0d46a15c030437a1ea09a3187c072bebebb4ca53ad85
f974359e21 test: Add encodable PUSHDATA1 examples to feature_taproot (Greg Sanders)
Pull request description:
Inspired by discussion in https://github.com/bitcoin/bitcoin/pull/31640#issuecomment-2743492906 I made an example adding coverage I think is missing, with some extra commentary that might help future contributors (including myself when I forget how it works again).
Open for suggestions how we can make it more welcoming beyond this.
cc darosior EthanHeilman sipa
ACKs for top commit:
janb84:
Re-ACK [f974359](f974359e21)
rkrux:
ACK f974359e21
Tree-SHA512: 7544d41c39c13d245a8a33522e53f22b4dd7593c069631978303e5a349cd12cf9d45bed648c391618c4732831232c4b82b8de2bf6cba5bf5e1232501db926122
