cbdb891de2 test: Wait for txospender index to be synced in rpc_gettxspendingprevout (Ava Chow)
2db5c049bb test: Sync mempools after tx creation in rpc_gettxspendingprevout (Ava Chow)
Pull request description:
On slower runs, the txospender index may not be synced yet when the tests of its behavior begin, causing intermittent failures. Wait for them to be synced before starting the tests.
The tests also query mempool txs from other nodes, make sure mempools are synced before doing so.
The first commit with the following diff reproduces #34735:
```
diff --git a/src/index/txospenderindex.cpp b/src/index/txospenderindex.cpp
index d451bb1e0a4..e786f05a98c 100644
--- a/src/index/txospenderindex.cpp
+++ b/src/index/txospenderindex.cpp
@@ -129,6 +129,7 @@ static std::vector<std::pair<COutPoint, CDiskTxPos>> BuildSpenderPositions(const
bool TxoSpenderIndex::CustomAppend(const interfaces::BlockInfo& block)
{
+ UninterruptibleSleep(100ms);
WriteSpenderInfos(BuildSpenderPositions(block));
return true;
}
```
Fixes#34735
ACKs for top commit:
furszy:
ACK cbdb891de2
andrewtoth:
ACK cbdb891de2
sedited:
ACK cbdb891de2
rkrux:
crACK cbdb891de2
Tree-SHA512: ba1ab6216ac3b647a5f9e20f4899e51e643bf20829e698aa63c127c88ed4e33afa222bebeae1210fc869c46288e99c0b344330197913e244ffe1a6aca943568d
15c4889497 index: document TxoSpenderIndex::FindSpender (furszy)
f8b9595aaa test: Add missing txospenderindex coverage in feature_init (Fabian Jahr)
a1074d852a index, rpc, test: Misc formatting fixes (Fabian Jahr)
Pull request description:
This addresses my own comments in the last review of #24539: https://github.com/bitcoin/bitcoin/pull/24539#pullrequestreview-3829110465
The first commit fixes three small formatting errors.
The second commit adds some missing coverage in `feature_init` and refactors the code a bit as well so these misses don't happen so easily in the future.
The third commit is by furzy:
> TxoSpenderIndex::FindSpender returns an Expected<optional<TxoSpender>> but
the two levels of the return type were undocumented, making it unclear what a returned
nullopt means. So added doc clarifying each return case.
ACKs for top commit:
furszy:
ACK 15c4889497
sedited:
ACK 15c4889497
rkrux:
crACK 15c4889497
Tree-SHA512: 2e0f060a54b558d2967ebae0835cf81bd86c2d8d983d670a48d1bee7d347f186623e75db7ae311ca1566807f715c1b3fa67cf734c9467d35e13b84d082f28253
37d49f5de6 doc: mention Miniscript expressions inside reference (Pieter Wuille)
771f7642bc doc: fix typo in descriptors.md (Pieter Wuille)
708b84999b doc: reference descriptor BIPs in descriptors.md (Pieter Wuille)
8f2a869a19 doc: do not list descriptor RPCs or history (Pieter Wuille)
65a8b6c2ef doc: mention musig() in descriptors.md (Pieter Wuille)
Pull request description:
This brings doc/descriptors.md up to date:
* Stop trying to exhaustively list all RPCs that involve descriptors. They're used everywhere.
* Stop trying to give the history of descriptor support, we have release notes for that.
* Mention that wallets are now built around descriptors (especially with legacy wallets gone).
* Mention `musig()` KEY expressions in the specification part.
* Mention the miniscript expressions in the specification part.
* Reference the relevant output descriptor BIPs in the text.
ACKs for top commit:
achow101:
ACK 37d49f5de6
darosior:
utACK 37d49f5de6
Tree-SHA512: 2581be9b5d1a7099806d6f830b3a452505f8493d0e493a80b8a50e383f93f3e2c8a2d72a64fdae0adfe63d3c2eeb54a61a059108cd861e58c3d85f2bc576364b
2a7a4f608a depends: Allow building Qt packages after interruption (Hennadii Stepanov)
Pull request description:
If a build is interrupted, a standard `mkdir` command may fail if the directory already exists. Switching to `mkdir -p` ensures the build can resume gracefully.
Fix#34712.
ACKs for top commit:
sedited:
ACK 2a7a4f608a
Tree-SHA512: fad2dce89a7cb68a8a539924d98698fe650802d19c84f216fa65e3c23c1a33ab6acf9f4c98c27381194c2958efa92e9dd8fb5e6bd40098efbcc60f156fd45370
bff8a7a80d subprocess: replace __USING_WINDOWS__ with WIN32 (kevkevinpal)
Pull request description:
## Summary
Motivated by https://github.com/bitcoin/bitcoin/pull/34385#pullrequestreview-3826616188
In `subprocess.h` we are now renaming `__USING_WINDOWS__` with `WIN32`
In the rest of the codebase, we are using `WIN32`, so it makes sense to update `subprocess.h` to match that.
---
Use the following `grep` to assert there is no `__USING_WINDOWS__` in the codebase
```
grep -nri --exclude-dir=build "WIN32" ./ -I
rep -nri --exclude-dir=build "__USING_WINDOWS__" ./ -I
```
ACKs for top commit:
sedited:
ACK bff8a7a80d
hebasto:
ACK bff8a7a80d, I have reviewed the code and it looks OK.
Tree-SHA512: 18c3c8b87cf880053bbf69f837a0a135c5da51cfb15ab1d9fd554d8f931b2ea8202cf0f4d5e6f317d6234480128c2f41a7a1a9d9bd0504697a3c4c6a21797762
5c005363a8 test: improve `wallet_backup` test (rkrux)
04d9515748 test: improve `wallet_assumeutxo` func test (rkrux)
Pull request description:
Relates to #34354
While the actual fix of the issue is in another PR, this one improves the
affected tests by trying to reduce the chain notifications that
need to be processed while simulating erroneous wallet restoration
scenarios.
ACKs for top commit:
maflcko:
lgtm ACK 5c005363a8
furszy:
ACK 5c005363a8
w0xlt:
ACK 5c005363a8
brunoerg:
code review ACK 5c005363a8
Tree-SHA512: 176e3ea8275c7aa082af695f5b76d82c079ff9a7178855b4ce95504edb8ce89b59a772e2d38dd43e997a5bea3d64be700b74cfec7bbc6992538f837877ab7222
d76ec4de14 fuzz: make sure PSBT serialization roundtrips (Antoine Poinsot)
Pull request description:
~~Invalid public keys were accepted in Musig2 partial signatures. Because we serialize invalid keys as the empty byte string, this would lead us to creating an invalid PSBT serializations.~~
~~This can be checked by reverting the first commit with the fix and simply running the target against the existing qa-assets corpus for the `psbt` harness.~~
This patch found the issue fixed in #34219 with a single run against the existing qa-assets corpus. It is useful to make sure there are no similar bugs, and we don't introduce roundtrip regressions outside of the specifc instance of accepting invalid public keys in Musig2 fields.
*(Edited on March 4 to only contain the fuzz harness patch)*
ACKs for top commit:
davidgumberg:
crACK d76ec4de14
achow101:
ACK d76ec4de14
dergoegge:
utACK d76ec4de14
brunoerg:
code review ACK d76ec4de14
Tree-SHA512: ab5f8d4e6a1781ecdef825e1a0e2793a6b553f36c923a4a35cb1af4070eead9d9780f6cc9a76235aa03462e52a129d15e61f631490b43651dc4395f3f1c005f3
faa68ed4bd test: Fix intermittent issue in wallet_assumeutxo.py (MarcoFalke)
Pull request description:
The test has many issues:
* It fails intermittently, due to the use of `-stopatheight` (https://github.com/bitcoin/bitcoin/issues/34710)
* Using `-stopatheight` is expensive, because it requires two restarts, making the test slow
* The overwritten `def setup_network` does not store the extra args via the `add_nodes` call, so if code is added in the future to restart a node, it may not pick up its global extra args.
Fix all issues by:
* Adding and using a fast `dumb_sync_blocks` util helper to achieve what `-stopatheight` doesn't achieve
* Calling `self.add_nodes(self.num_nodes, self.extra_args)` in the overridden `setup_network`
Can be tested via this diff:
```diff
diff --git a/src/node/kernel_notifications.cpp b/src/node/kernel_notifications.cpp
index ab0e5ccb69..49384c10b8 100644
--- a/src/node/kernel_notifications.cpp
+++ b/src/node/kernel_notifications.cpp
@@ -61,2 +61,3 @@ kernel::InterruptResult KernelNotifications::blockTip(SynchronizationState state
if (m_stop_at_height && index.nHeight >= m_stop_at_height) {
+ LogInfo("Send shutdown signal after reaching stop height");
if (!m_shutdown_request()) {
diff --git a/src/util/tokenpipe.cpp b/src/util/tokenpipe.cpp
index c982fa6fc4..a5565ebd36 100644
--- a/src/util/tokenpipe.cpp
+++ b/src/util/tokenpipe.cpp
@@ -4,2 +4,3 @@
#include <util/tokenpipe.h>
+#include <util/time.h>
@@ -60,2 +61,3 @@ int TokenPipeEnd::TokenRead()
ssize_t result = read(m_fd, &token, 1);
+ UninterruptibleSleep(500ms);
if (result < 0) {
```
On master: The test fails
On this pull: The test passes
Fixes https://github.com/bitcoin/bitcoin/issues/34710
ACKs for top commit:
kevkevinpal:
ACK [faa68ed](faa68ed4bd)
achow101:
ACK faa68ed4bd
w0xlt:
ACK faa68ed4bd
Tree-SHA512: 6fcd52b6f6a5fa5a5e41e7b3cf5c733af62af9c60271e7d22c266aca90f2af67f91ffe80a3ed8b8d1a91d001700870f493207998bac988c4e3671a3a0dba7ba7
f51665bee7 psbt: validate pubkeys in MuSig2 pubnonce/partial sig deserialization (tboy1337)
Pull request description:
The previous fix for invalid MuSig2 pubkeys (bitcoin/bitcoin#34010) only
addressed the PSBT_IN_MUSIG2_PARTICIPANT_PUBKEYS field. However, the
PSBT_IN_MUSIG2_PUB_NONCE and PSBT_IN_MUSIG2_PARTIAL_SIG fields also
deserialize pubkeys without validation, which could lead to crashes when
invalid pubkeys are processed.
This commit adds validation to the DeserializeMuSig2ParticipantDataIdentifier
function to ensure all pubkeys in MuSig2 pubnonce and partial signature
fields are fully valid elliptic curve points.
The fix:
- Validates both aggregate and participant pubkeys in MuSig2 pubnonce and
partial signature deserialization
- Throws std::ios_base::failure with descriptive error messages for invalid
pubkeys
- Prevents potential crashes from invalid elliptic curve points
- Maintains backward compatibility for valid PSBTs
This completes the fix for issues [#33999](https://github.com/bitcoin/bitcoin/issues/33999) and [#34201](https://github.com/bitcoin/bitcoin/issues/34201).
ACKs for top commit:
rkrux:
lgtm ACK f51665bee7
w0xlt:
ACK f51665bee7
darosior:
utACK f51665bee7
Tree-SHA512: 8454d77a05aa003a3121b0a5975e8a000125ee0d62343bfa625a75db113358ba7a210ae0376ca1666957b7de7005e06e5a54c95170430ee5e9e1416719b8af53
1c1de334e9 test: avoid interface_ipc.py race and null pointer dereference (Ryan Ofsky)
Pull request description:
Avoid race condition in `run_deprecated_mining_test` caused by creating and immediately destroying an unused worker thread. This is leading to test failures reported by maflcko in #34711
ACKs for top commit:
optout21:
utACK 1c1de334e9
l0rinc:
Tested ACK 1c1de334e9
w0xlt:
ACK 1c1de334e9
enirox001:
ACK 1c1de33
Tree-SHA512: d0af9676a46e991a3f4fda3795c02d1998d30de24991436b8ada425585c6699ff32a7057ca7a0ef2925f782fd3bf73e0381f5d4325e4f1c09f487fce1de49e45
b87a1c27c9 doc: Improve dependencies.md IPC documentation (Ryan Ofsky)
Pull request description:
Improve dependencies.md to document IPC dependencies better ([preview](https://github.com/ryanofsky/bitcoin/blob/pr/ipc-depdoc/doc/dependencies.md#build-1)). Specific changes are listed in the commit message.
This PR is based on #33623 by willcl-ark which made similar changes in the 29.x branch. This PR could also be backported to 30.x (it merges cleanly, and master and 30.x both have the same version requirements).
ACKs for top commit:
l0rinc:
ACK b87a1c27c9
sedited:
ACK b87a1c27c9
Tree-SHA512: 566b5372d189f0ad04f978ddefbd8c200dcc19b25af02c73275c5faf7529943680ec59703bda11640cf7920466b4cdd46305cac4d3770e2303de37694ae78909
fa7612f253 ci: Download script_assets_test.json for Windows CI (MarcoFalke)
7777a13306 test: Move Fetching-print to download_from_url util (MarcoFalke)
faf96286ce test: move-only download_from_url to stand-alone util file (MarcoFalke)
fa0cc1c5a4 test: [doc] Remove outdated comment (MarcoFalke)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/34670 by adding a new `download_script_assets` Python helper and calling it.
ACKs for top commit:
hebasto:
re-ACK fa7612f253.
janb84:
re ACK fa7612f253
hodlinator:
utACK fa7612f253
Tree-SHA512: 73c2cb3a31f231174566fb880b82de92734b1679ef000f8d793d774b7e5f5a7b8c7994a3998ca78821115bdc80c16aada69cf596e92c083cf9b9a95c7cee16ea
5e35a9069d interpreter: remove clang-tidy suppression (fanquake)
4089682f5c ci: use Clang 22 in tidy task (fanquake)
7ea076f996 tidy: remove deprecated header (fanquake)
eb17f29aa5 tidy: clang-tidy is required (fanquake)
Pull request description:
Changes needed for moving to Clang 22 in the tidy job.
ACKs for top commit:
maflcko:
lgtm ACK 5e35a9069d
hebasto:
ACK 5e35a9069d, I have reviewed the code and it looks OK.
Tree-SHA512: 9ca6e841f7480b8abd78d5621d08a5bf80c2ff4facd7a0d76038ac1771bbf3d37dc2df19fa27583679177e4618db6294e2f2bb2129d9c25a53338b49ed71aac2
```bash
15 | #warning The ClangTidyModuleRegistry.h header is deprecated and will be removed in LLVM 24. All of the symbols it used to define have been moved into ClangTidyModule.h.
| ^~~~~~~
[100%] Linking CXX shared module libbitcoin-tidy.so
```
98e8af4bb9 wallet: Drain validation interface queue after notifications disconnect (Ava Chow)
52992ebe1c interfaces: Add waitForNotifications() to call SyncWithValidationInterfaceQueue() (Ava Chow)
Pull request description:
When the wallet disconnects chain notifications, it is expecting no further notifications to execute, but this is not the case. This results in test failures such as in #34354. Instead of disconnecting the notifications and continuing shutdown, we should wait for the validation interface queue to be drained before the rest of wallet shutdown. This is achieved by adding an `interfaces::Chain::waitForNotifications()` function which calls `SyncWithValidationInterfaceQueue()`.
Fixes#34354
ACKs for top commit:
stickies-v:
utACK 98e8af4bb9
furszy:
ACK 98e8af4bb9
rkrux:
crACK 98e8af4bb9
sedited:
ACK 98e8af4bb9
Tree-SHA512: 263628556f740cb633d3970c22a0dfdb52a644bd1d0cd5a69c2970524edbb0e25d592cb39fc9bf1d0c281eebce09578526e2958dffee9026fc7473db35bd0dec
Avoid race condition in run_deprecated_mining_test caused by creating and
immediately destroying an unused worker thread. This leads to test failures
reported by maflcko in #34711
da7f70a532 test: use port 0 for I2P addresses in p2p_private_broadcast.py (Vasil Dimov)
a8ebcfd34c test: let connections happen in any order in p2p_private_broadcast.py (Vasil Dimov)
67696b207f net: extend log message to include attempted connection type (Vasil Dimov)
Pull request description:
If the following two events happen:
* (likely) the automatic 10 initial connections are not made to all
networks
* (unlikely) the network-specific logic kicks in almost immediately.
It is using exponential distribution with a mean of 5 minutes
(`rng.rand_exp_duration(EXTRA_NETWORK_PEER_INTERVAL)`).
So if both happen, then the 11th connection may not be the expected
private broadcast, but a network-specific connection.
Fix this by retrieving the connection type from
`destinations_factory()`. This is more flexible because it allows
connections to happen in any order and does not break if e.g. the 11th
connection is not the expected first private broadcast.
This also makes the test run faster:
before: 19-44 sec
now: 10-25 sec
because for example there is no need to wait for the initial 10
automatic outbound connections to be made in order to proceed.
Fixes: https://github.com/bitcoin/bitcoin/issues/34387
ACKs for top commit:
achow101:
ACK da7f70a532
andrewtoth:
ACK da7f70a532
mzumsande:
Code Review ACK da7f70a532
Tree-SHA512: 7c293e59c15c148a438e0119343b05eb278205640658c99336d4caf4848c5bae92b48e15f325fa616cbc9d5f394649abfa02406a76e802cffbd3d312a22a6885
Reduce the number of blocks that need to be generated before pruning
the blockchain.
Unload the wallet that was restored in a prior test because it is not
needed anymore after the test.
Both the above steps should reduce the number of chain notifications
that need to be processed by the wallet(s) when an erroneous scenario
of restoring wallet is checked.
Improve dependencies.md to document IPC dependencies better:
- Link to native_capnp.mk file not capnp.mk file so it's possible to see what
version of Cap'n Proto is being used in release binaries. This is important
since #31895 dropped the "Version Used" column and the capnp.mk file does not
include version number.
- Indicate Capn"Proto is used for IPC and link to multiprocess.md documenting
the feature.
- Link to correct PR requiring Cap'n Proto 0.7.1. Previous link was
pointing at PR which required 0.7.0.
- Mention libmultiprocess as a dependency even though it is included as a git
subtree and can be built as a cmake subproject. Libmultiprocess still needs
to be built separately when cross compiling, and is useful to build separately
when developing, and is still a depends package.
Based on 2cf352fd8e from #33623 by willcl-ark
which made similar changes in the 29.x branch.
fa48f8c865 test: Add missing timeout_factor to zmq socket (MarcoFalke)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/34189
Otherwise, the test may intermittently fail on slow CI systems that have `--timeout-factor=` properly set.
It can be tested by running `./bld-cmake/test/functional/interface_zmq.py --timeout-factor=10` with this diff:
```diff
diff --git a/src/validationinterface.cpp b/src/validationinterface.cpp
index c7be6abc3a..b14cf2aee6 100644
--- a/src/validationinterface.cpp
+++ b/src/validationinterface.cpp
@@ -166,2 +166,3 @@ void ValidationSignals::SyncWithValidationInterfaceQueue()
LOG_EVENT(fmt, local_name, __VA_ARGS__); \
+ UninterruptibleSleep(45ms); \
event(); \
diff --git a/test/functional/interface_zmq.py b/test/functional/interface_zmq.py
index 6717007626..eee377daea 100755
--- a/test/functional/interface_zmq.py
+++ b/test/functional/interface_zmq.py
@@ -176,3 +176,3 @@ class ZMQTest (BitcoinTestFramework):
for sub in subscribers:
- sub.socket.set(zmq.RCVTIMEO, recv_timeout*1000)
+ sub.socket.set(zmq.RCVTIMEO, int(recv_timeout * 1000))
@@ -271,3 +271,3 @@ class ZMQTest (BitcoinTestFramework):
[(topic, address) for topic in ["hashblock", "hashtx"]],
- recv_timeout=2) # 2 second timeout to check end of notifications
+ recv_timeout=0.2) # 2 second timeout to check end of notifications
self.disconnect_nodes(0, 1)
```
Before this pull: Test fails with `zmq.error.Again: Resource temporarily unavailable`
After this pull: Test passes
ACKs for top commit:
l0rinc:
code review ACK fa48f8c865
achow101:
ACK fa48f8c865
Tree-SHA512: 5ff8bdd807ff4b644daa634bb7b469da3da3915f58afba63a90e662df99cbebc86636e34e2b1b313c8629773aef2a239fb3025226a84d2ec22f6ecd4cea666c4
bbc8f1e0a7 ipc mining: Prevent ``Assertion `m_node.chainman' failed`` errors on early startup (Ryan Ofsky)
a7cabf92e4 init refactor: Only initialize node.notifications one time (Ryan Ofsky)
c8e332cb33 init refactor: Remove node.init accesss in AppInitInterfaces (Ryan Ofsky)
Pull request description:
This fixes ``Assertion `m_node.chainman' failed`` errors first reported https://github.com/bitcoin/bitcoin/issues/33994#issuecomment-3602551596 when IPC mining methods are called before ChainstateManager is loaded.
The fix works by making the `Init.makeMining` method wait until chainstate data is loaded. It's probably the simplest possible fix but other alternatives like moving the wait to `Mining.createNewBlock` were discussed in the thread https://github.com/bitcoin/bitcoin/pull/34661#discussion_r2848176298 and could be implemented later without changes to clients.
ACKs for top commit:
Sjors:
utACK bbc8f1e0a7
ismaelsadeeq:
ACK bbc8f1e0a7
achow101:
ACK bbc8f1e0a7
Tree-SHA512: 3e2e4e28ccff364b2303efd06ce337a229c28609076638500acb29559f716a15ad99409c6970ce9ad91776d53e3f9d959f1bbbd144ea9a4a2fb578ddbf2da267
When unloading a wallet, there may be unexecuted callbacks in the
validation interface queue that can still execute after we have
completed all of the other wallet shutdown tasks. Instead of letting
these run in the background, once the notifications are disconnected,
wait for the queue to drain before continuing with wallet shutdown.
fa36adeb71 ci: [refactor] Drop last use of pwsh (MarcoFalke)
fae31b1e2f ci: [refactor] Move github_import_vs_env to python script (MarcoFalke)
Pull request description:
The use of pwsh was a bit confusing and inconsistent around the exit code. See also https://github.com/bitcoin/bitcoin/pull/32672
I think it is fine to drop it and purely use Bash/Python.
This also moves a bit of code from the github yaml to the python script.
ACKs for top commit:
m3dwards:
Looks good! re-ACK fa36adeb71
hodlinator:
re-ACK fa36adeb71
Tree-SHA512: 78edffc60c58c476b0acca5224150169d154b0b818114844a04295af9ba19b7cdf1fb2afb738f6cafd6172f0f477d546018ebf95061eb5bd8bbb35e065a129d4
fab51e470e test: Move valgrind.supp to the other sanitizer_suppressions files (MarcoFalke)
fa9cf81d39 test: Add missing resolve() to valgrind.supp file (MarcoFalke)
Pull request description:
(see commit message for details)
Can be tested via:
```
./bld-cmake/test/functional/feature_framework_testshell.py --valgrind
```
Before:
```
bitcoind exited with status 1 during initialization. ==1735813== FATAL: can't open suppressions file "/b-c/b-c-2/bld-cmake/contrib/valgrind.supp"
```
After: (passes)
Also, move the suppression file to all the other suppression files.
ACKs for top commit:
frankomosh:
Re-ACK fab51e470e
Tree-SHA512: d3e3e130c0e2292ca3ab9e80d2ebec6b4edc7914280ed90fb4af8a65cd1c9edd19064d86375a6787b864925fe0e47bab2321f89b9be8d1613a0aebf4ec2443fe
fa18be2f2b test: Fix typo (MarcoFalke)
fac932698f ci: Set TEST_RUNNER_PORT_MIN in test-each after cirrus runner switch (MarcoFalke)
Pull request description:
This is needed after the recent switch to cirrus runners in the task in commit c8c9c1e617.
Otherwise, the CI will fail:
```
node1 stderr Error: Unable to bind to 127.0.0.1:12321 on this computer. Bitcoin Core is probably already running.
```
(https://github.com/bitcoin/bitcoin/actions/runs/22398358349/job/64837827234?pr=31723#step:9:2605)
Also, include a random second commit, so that the CI task is run in this pull.
ACKs for top commit:
l0rinc:
ACK fa18be2f2b
willcl-ark:
ACK fa18be2f2b
Tree-SHA512: 6b63f645bf62d3e951ca155cddf3dc562b7ce675ccae4f9179e2202679685b5c147844eb350bd219b173fe2bb976376d0caa073d3e827a48c13aa015f4745b2c
