Implements a child class of DynSock which is used as the mock
socket for HTTPServer unit tests. The ErrorSock::Send() method
raises a non-permanent error on the first HTTPRequest::WriteReply()
and then succeeds after the second.
In httpserver_tests.cpp use this mechanism to ensure that the
server retries a send operation if such an error is encountered,
and cover both optimistic (worker thread WriteReply()) and
non-optimistic (I/O thread SocketHandlerConnected()) send paths.
This prevents a losing race condition that could prevent the server
from reading requests from an HTTP client.
A connected socket can either be written to or read from based on the
result of GenerateWaitSockets(). That method checks the HTTPRemoteClient
flag m_send_ready. If it's `true` the implication is that there is
data in the client's send buffer ready to go. Once that data is sent
and the buffer is empty, MaybeSendBytesFromBuffer() sets it `false` again.
The sad case was when a worker thread calling WriteReply() adds
data to the send buffer, but before it sets m_send_ready to `true`,
the I/O thread sends that data and empties the buffer. With the
buffer unexpectedly empty, WriteReply() sets m_send_ready to `true`.
The effect of this is that the socket will stay in "write" mode
with nothing to write. With nothing to write, MaybeSendBytesFromBuffer()
never sets it back to `false` and the socket is stuck forever.
bbbbab86a8 ci: Bump tsan config to ubuntu:26.04 with -U_FORTIFY_SOURCE (MarcoFalke)
Pull request description:
The `-U_FORTIFY_SOURCE` should be harmless in the Tsan CI, and is required to work around https://github.com/bitcoin/bitcoin/issues/30586
ACKs for top commit:
fanquake:
ACK bbbbab86a8
Tree-SHA512: 7c4b434342861fabc52eda7b551ebe08fcde6079cd39d54a21938c88dcfd80cdb876dd7de859bac1201a2b928163c19704c24d5b5a5e76f4935e33cb3e6ae2bd
The `ERR` macro is defined on illumos-based systems in the `regset.h`
header included by the Boost.Test framework, which may cause a
compilation error.
-BEGIN VERIFY SCRIPT-
ren() { sed -i "s/\<$1\>/$2/g" $( git grep -l "$1" ./src/qt/psbtoperationsdialog.* ) ; }
ren StatusLevel::INFO StatusLevel::Info
ren INFO Info
ren StatusLevel::WARN StatusLevel::Warn
ren WARN Warn
ren StatusLevel::ERR StatusLevel::Error
ren ERR Error
-END VERIFY SCRIPT-
6fa4132298 fuzz: share a single mocked steady clock across FuzzedSock instances (Hao Xu)
Pull request description:
This is a follow-up of https://github.com/bitcoin/bitcoin/pull/35478#issuecomment-4667842057, inspired by maflcko .
Each FuzzedSock used to own its mocked steady clock and call MockableSteadyClock::SetMockTime() directly. Hold the clock by reference to an externally provided SteadyClockContext instead, so that several FuzzedSock instances sharing a test case (e.g. one per peer, or one created via Accept()) advance a single mocked clock, and the mocking goes through the SteadyClockContext RAII helper that resets mocktime on destruction.
SteadyClockContext is a LimitOne type, so each fuzz target constructs one instance per iteration and passes it to ConsumeSock / ConsumeNode / the FuzzedSock constructor.
ACKs for top commit:
maflcko:
review ACK 6fa4132298🌕
marcofleon:
crACK 6fa4132298
Tree-SHA512: 3c773b5c0c3ba42a8245c9ea6042b0bc767df4fad506305f3c200310616b48a59deb1542086eb4ce3e8a1407c4d6b42cef3b37cd84bfe80d4821972b8d3b4286
8791c4764c test: use ExtendedPrivateKey in wallet_taproot.py (rkrux)
89ceafafb9 test: use ExtendedPrivateKey in wallet_listdescriptors.py (rkrux)
bbfffcab58 test: use ExtendedPrivateKey in wallet_send.py (rkrux)
2ab6e590f7 test: use ExtendedPrivateKey in wallet_keypool.py (rkrux)
9e20118720 test: use ExtendedPrivateKey in wallet_fundrawtransaction.py (rkrux)
06af0cddbb test: use ExtendedPrivateKey in wallet_descriptor.py (rkrux)
4100fac20e test: use ExtendedPrivateKey in wallet_createwallet.py (rkrux)
ff3f6def9a test: use ExtendedPrivateKey in wallet_bumpfee.py (rkrux)
003f2a01f6 test: use ExtendedPrivateKey in feature_notifications.py (rkrux)
f988e6d6e6 test: use ExtendedPrivateKey in wallet_importdescriptors.py (rkrux)
d2a03d50ac test: add extendedkey.py unit tests by using BIP32 test vectors (rkrux)
afdb378082 test: introduce ExtendedPrivateKey and ExtendedPublicKey classes (rkrux)
4dbaa7cc65 test: generalise byte_to_base58 utility function to allow more version types (rkrux)
Pull request description:
Many a times there has been a need to come up with dynamic xprvs and xpubs
in the functional tests, but the lack of code that creates them dynamically has
led to the presence of several hardcoded keys in the testing framework. This
is not developer friendly and not self-documenting, clutters the testing code,
and makes it difficult to update the tests in the future.
This PR introduces two utility classes ExtendedPrivateKey and
ExtendedPublicKey that allows the developer to create them on the fly
to be used in the tests. I have intentionally not introduced any library for this
purpose and have reused the existing libraries and functions in the framework.
The implementation is supposed to provide basic functionality for creating
xprv randomly or from a fixed seed, creating corresponding xpub, and
deriving child xprvs and xpubs at custom derivation paths.
I've updated many tests to show how these can be used, there are more
tests as well that can be updated in the future to completely remove such
non-deterministic hardcoded keys.
ACKs for top commit:
achow101:
ACK 8791c4764c
w0xlt:
ACK 8791c4764c
Tree-SHA512: f8ec4e09eaa6cc44b0f1c9a91337e570b12fb882c258be89b470de1a8cecf9d2fd40d9f02ee739dcbf639462ea7710aa145a3726f0f537f5a1f1e7772e5b019d
The local `static constexpr auto ERR` shadowed the `Sock::ERR` static
data member. Rename it to `accept_error`, per the Developer Notes'
shadowing guidance.
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
1a3cfdf1b7 fuzz: connman: cover AddLocalServices/RemoveLocalServices (Bruno Garcia)
c507fb3063 fuzz: connman: add outbound-bytes invariants (Bruno Garcia)
4a6fce43ea fuzz: connman: add AddNode/RemoveAddedNode invariants (Bruno Garcia)
a5859edef4 fuzz: connman: set m_local_services/m_use_addrman_outgoing/m_max_automatic_connections (Bruno Garcia)
4b84c9125a fuzz: connman: add network activity invariants (Bruno Garcia)
Pull request description:
This PR improves the `connman` fuzz target by replacing some "`(void)`" calls with actual invariant checks, adding coverage for previously uncovered methods, and exercising more initialization states.
- Set `m_local_services`, `m_use_addrman_outgoing`, and
`m_max_automatic_connections` via fuzzed values before `Init()` to
explore more startup configurations.
- Add network activity and outbound-bytes invariants.
- Add `AddNode`/`RemoveAddedNode` invariants: e.g. a successful `AddNode`
increases `GetAddedNodeInfo()` by one; adding the same node again
must fail; a subsequent `RemoveAddedNode` must succeed and restore
the original count.
- Add coverage for `AddLocalServices`/`RemoveLocalServices`.
ACKs for top commit:
nervana21:
re-ACK 1a3cfdf1b7
frankomosh:
reACK 1a3cfdf1b7 . Change from the diff is the restoring `(void)connman.RemoveAddedNode(random_string)` arm
sedited:
ACK 1a3cfdf1b7
Tree-SHA512: c7b6799ca65d2e639d8ab9ab0cc77bae663f24fbda934446a8ee2e8ce9e8e36624d16b4f492b1714e2d67375edd35907cb9392d21f368d3d5298275ff1d05c72
8ebfff0f88 doc: add send RPC release note (Sjors Provoost)
5884f5a4fa wallet: remove experimental warning from send RPCs (Sjors Provoost)
Pull request description:
The `send` RPC was introduced in v21 an initially marked experimental. The `sendall` RPC was added in v24, based on `send` and also marked experimental.
I'm not aware of any proposed breaking changes, except #35433 which follows the regular deprecation flow.
Time to mark them as no longer experimental.
ACKs for top commit:
w0xlt:
ACK 8ebfff0f88
achow101:
ACK 8ebfff0f88
polespinasa:
ACK 8ebfff0f88
pablomartin4btc:
ACK 8ebfff0f88
Tree-SHA512: beb5321adaf871157bda396c8e5740daff95ffe342416914340ae4197accebe60236032d1329876b42405437b99f59079a56ec1e5ac592b753031ba2ebd36cfb
58560c281d ci: remove some packages from Chimera job (fanquake)
Pull request description:
This job doesn't build QT, so drop `ninja` and `xz`.
With libevent removed, and no Qt build, we can also drop `pkgconf`.
ACKs for top commit:
sedited:
ACK 58560c281d
hebasto:
ACK 58560c281d.
Tree-SHA512: 4537c8c77334637ea177321d95ad39ee103ab0080e8331f9fe9efb92a3bb4dc00fb59d2e250fd43c267c05f0a2b98e779949ff40723633c2f2bff345fa554a31
9e6546c517 test: raise reindex mining RPC timeout (Lőrinc)
Pull request description:
**Problem:** I often hit a timeout in `feature_reindex.py` when running functional tests locally in parallel in debug mode (especially on battery or in power-saving mode).
**Fix:** Increase the test-local RPC timeout for the reindex mining setup.
ACKs for top commit:
mercie-ux:
ACK 9e6546c517
sedited:
ACK 9e6546c517
Tree-SHA512: d3541dd6752f943921a030393b00e684b3b5d00b93aa0b2b1f85c017698bdfe2216d7f5441bb7dd5f153453385e4df6f9cc8055589e57bca83fa48f0b7de4252
0cdd817a82 add release note (Pol Espinasa)
517d37ce3e test: tests wallet migration with load_wallet disabled (Pol Espinasa)
b98dd63da7 rpc: Add load_wallet argument to migratewallet RPC (Pol Espinasa)
4acd063ba6 wallet: make loading the wallet after migrating optional (Pol Espinasa)
97d08d62ba refactor: store wallet names to MigrationResult (Pol Espinasa)
Pull request description:
This PR is motivated by this [Stack Exchange question](https://bitcoin.stackexchange.com/questions/130713/bitcoin-core-quickest-method-legacy-descriptor-wallet-migration).
Long story short, someone who has a node pruned before his legacy wallet birthday, is unable to migrate the wallet as it is not possible to load it.
Loading is not necessary for migration, and migrating without wanting to use the wallet in that node is a valid use-case.
This PR adds a new RPC argument to `migratewallet` that allow the user disabling the wallet loading.
Second commits adds tests for it.
Follow-up: Add an option to the GUI to not load the wallet after migrating.
ACKs for top commit:
achow101:
ACK 0cdd817a82
w0xlt:
ACK 0cdd817a82
pablomartin4btc:
ACK 0cdd817a82
Tree-SHA512: 8389599e63603b1a532e1bfba0b6c652653386c001f5a881bd49843302b74ff4dbaa4131b5b377c24f483d42e0e70a92b96f760244e3c2e2b44ce08cd04ca1e0
703a671fbc fuzz: compact coins view db during fuzzing (Lőrinc)
0868c85fd5 refactor: rename async coin compaction (Lőrinc)
Pull request description:
**Problem:** #35465 added async chainstate compaction, but the `coins_view_db` fuzz target did not exercise scheduling compaction alongside ordinary coins view operations.
The async wrapper also shared the `CompactFull()` name with the blocking `CDBWrapper` primitive.
**Fix:** Rename the coins DB wrapper to `CompactFullAsync()` and let `coins_view_db` randomly schedule it under `cs_main` (like in production).
The fuzz operation only starts compaction and any running job is joined by the `CCoinsViewDB` destructor at the end of the fuzz input.
ACKs for top commit:
sedited:
ACK 703a671fbc
andrewtoth:
ACK 703a671fbc
Tree-SHA512: 9854c3acbaace795155e7469cb10938fbd872726cbb8a4b4ef71d6d352d00824498747215aee1b237e64452eae2e690a3f7d7daa6b6b0030e75a6f2ccc0802fb
abc33ff043 test: announce field must be 0 or 1 in sendcmpct (brunoerg)
2d0dce0af5 net_processing: fix BIP152 first integer interpretation (brunoerg)
Pull request description:
Fixes#35542
According to the BIP152, the first integer in `sendcmpct` message shall be interpreted as a boolean (and MUST have a value of either 1 or 0). We currently correctly interpret it as boolean, however, we accept any value >=1 and treat it as `true`, deviating from the specification. This PR fixes it.
ACKs for top commit:
edilmedeiros:
utACK abc33ff043
davidgumberg:
crACK abc33ff043 Seems reasonable to comply with BIP152 strictly, test looks good as well.
Sjors:
ACK abc33ff043
jonatack:
re-ACK abc33ff043
achow101:
ACK abc33ff043
w0xlt:
ACK abc33ff043
Tree-SHA512: 77fed86d4de81f7c35ff002b6e1b2a90882ea55f159075da4d34a619d1075f625fca34f929cdd981f1b2eb06f76b64f42cda46502dcd1b4a634c690b0882ec7c
Exercise `CCoinsViewDB::CompactFullAsync()` from the `coins_view_db` fuzz target so the new chainstate compaction wrapper can run concurrently with ordinary coins view operations.
The fuzz operation only schedules compaction, matching production; outstanding work is waited for by the `CCoinsViewDB` destructor at the end of the fuzz input.
Rename the `CCoinsViewDB` async compaction wrapper to `CompactFullAsync()` so it is distinct from the blocking `CDBWrapper::CompactFull()` primitive it calls.
b847626562 test: refresh MiniWallet after node restart (Sjors Provoost)
f4e643cb15 test: merge mining options in package feerate check (Sjors Provoost)
280ce6a0ae miner: ensure block_max_weight is flattened before limit checks (Sjors Provoost)
65bd3164fb mining: clarify test_block_validity comment (Sjors Provoost)
978e7216e6 test: use shared default_ipc_timeout (Sjors Provoost)
Pull request description:
This implement the suggested followups from #33966. Each commit links to the original comment.
The most important change is the extra asserts added in `miner: ensure block_max_weight is flattened before limit checks`.
ACKs for top commit:
achow101:
ACK b847626562
enirox001:
tACK b847626562
sedited:
ACK b847626562
w0xlt:
ACK b847626562
Tree-SHA512: 47678eaed604228269bd892ccf8ff58804745bbc7675b4a93528da9a9292a2eb1e0562cdb8341edac77178563420885b48282bb9e5c2b997b28f2fc64ceeff3d
2fe34808fa wallet: reject sendtoaddress and sendmany for external signers (Sjors Provoost)
bd5a32f7db doc: add taproot descriptor to getdescriptors example (woltx)
7131c82937 doc: clarify which commands receive --chain, --fingerprint and --stdin (woltx)
4fdd4d8d29 doc: replace stale signtransaction wording with current signtx flow (woltx)
fab92257fe doc, rpc: document enumerate model field and fingerprint deduplication (woltx)
Pull request description:
This PR aligns the external signer documentation with current behavior, and makes one previously implicit behavior explicit.
Per review feedback, each commit fixes a limited set of issues:
* **doc, rpc: document enumerate model field and fingerprint deduplication** — the `enumerate` response uses the optional `model` field, which Bitcoin Core maps to the `name` field of the `enumeratesigners` RPC result. Duplicate fingerprints are skipped, and wallet operations require exactly one connected signer.
* **doc: replace stale signtransaction wording with current signtx flow** — spending from an external signer wallet uses `send`/`sendall` (and `bumpfee` for fee-bumping), which invoke `<cmd> --stdin` and pass the `signtx` subcommand and PSBT over stdin.
* **doc: clarify which commands receive --chain, --fingerprint and --stdin** — mark `--chain` and `--fingerprint` as required except for `enumerate`, keep `--stdin` required for protocol flexibility, and match the order and form of the actual invocations in the usage examples.
* **doc: add taproot descriptor to getdescriptors example** — show the BIP86 `tr()` descriptor alongside the other address types.
* **wallet: reject sendtoaddress and sendmany for external signers** — return a specific error instead of the misleading "Private keys are disabled for this wallet", with functional test coverage. Cherry-picked from #33112 (thanks Sjors).
How the documentation went stale:
* The `enumerate` example has shown a `name` field since external signer support landed in #16546, but the implementation has always read `model`.
* `sendtoaddress`/`sendmany` external signer support was effectively precluded by #21201, which was merged a few days before #16546, so the interaction was missed in review and the documented `signtransaction` flow never existed in this form.
* Fingerprint deduplication was added in #35251.
* The documentation was last updated in #33765.
ACKs for top commit:
Sjors:
ACK 2fe34808fa
optout21:
ACK 2fe34808fa
naiyoma:
ACK 2fe34808fa
Tree-SHA512: 86859d2f81ac337f3b4b6578c6ee0151ffb76b8374dfa58e28e00ce4eb69dc200cd6bd2d0a99f73d0475c3824d6ac1cb9e2542b119ca124dd835132dc95cd023
Each FuzzedSock used to own its mocked steady clock and call
MockableSteadyClock::SetMockTime() directly. Hold the clock by reference
to an externally provided FakeSteadyClock instead, so that several
FuzzedSock instances sharing a test case (e.g. one per peer, or one
created via Accept()) advance a single mocked clock, and the mocking goes
through the FakeSteadyClock RAII helper that resets mocktime on
destruction.
FakeSteadyClock is a LimitOne type, so each fuzz target constructs one
instance per iteration and passes it to ConsumeSock / ConsumeNode / the
FuzzedSock constructor.
This avoids hardcoding xprvs and xpubs in the test.
Also use key's parent fingerprint and derivation path while creating descriptors
so that all of it is self-documenting and not dependent on hardcoded values,
making them easy to read and update.
Also use descsum_create function more to avoid hardcoding the descriptor
checksum.
d6359937bf validation: check invariants when inserting into m_blocks_unlinked (stratospher)
0852925bd8 test/doc: remove misleading comment and improve tests (stratospher)
ca4a380281 test: add coverage for UB caused by FindMostWorkChain (stratospher)
c787b3b99b validation: avoid duplicates in m_blocks_unlinked (stratospher)
Pull request description:
This is joint work with @ mzumsande.
note: this requires a pruned node with deep reorgs to trigger. still it breaks assumptions in the codebase and is good to fix. A similar UB was fixed in https://github.com/bitcoin/bitcoin/pull/34521.
This PR prevents duplicate insertions into `m_blocks_unlinked` in `FindMostWorkChain`. There are 3 ways to insert into `m_blocks_unlinked`:
1. `LoadBlockIndex` - not problematic, as each block index is processed only once.
2. `ReceivedBlockTransactions` - not problematic, as this is usually only called once per block when it is first accepted in `AcceptBlock`. in the rare case it’s triggered again after pruning, the block would have been removed from `m_blocks_unlinked` when it was initially pruned, so duplicates still can’t arise.
3. `FindMostWorkChain` - problematic when multiple candidate tips share common chains of ancestors, traversals from each tip to the fork point may insert duplicate (`pprev`, `pindex`) entries for blocks whose parents have been pruned.
When the missing parent is later received and `ReceivedBlockTransactions` processes `m_blocks_unlinked`, the same entry may be processed multiple times. This can result in the block being re-added to `setBlockIndexCandidates` with a modified `nSequenceId`, violating its ordering invariants and leading to undefined behavior. So avoid duplicate insertions into `m_blocks_unlinked` in `FindMostWorkChain`.
### how to test:
use the updated `feature_pruning.py` which adds coverage for this scenario.
- on master: the test (with the below diff) fails since `nSequenceId` is being modified for an entry in `setBlockIndexCandidates`
- on this branch: the test (with the below diff) passes
```diff --git a/src/validation.cpp b/src/validation.cpp
--- a/src/validation.cpp
+++ b/src/validation.cpp
@@ -3814,6 +3814,12 @@ void ChainstateManager::ReceivedBlockTransactions(const CBlock& block, CBlockInd
pindex->nHeight, pindex->m_chain_tx_count, prev_tx_sum(*pindex), CLIENT_NAME, FormatFullVersion(), CLIENT_BUGREPORT);
}
pindex->m_chain_tx_count = prev_tx_sum(*pindex);
+ for (const auto& c : m_chainstates) {
+ if (c->setBlockIndexCandidates.contains(pindex)) {
+ LogInfo("### pindex UB = %s", pindex);
+ assert(false);
+ }
+ }
pindex->nSequenceId = nBlockSequenceId++;
for (const auto& c : m_chainstates) {
c->TryAddBlockIndexCandidate(pindex);
```
ACKs for top commit:
sedited:
Re-ACK d6359937bf
marcofleon:
crACK d6359937bf
stringintech:
ACK d6359937bf
mzumsande:
Sure - Code Review ACK [d635993](d6359937bf)
Tree-SHA512: bb21adc2d92fe1865bbbcebf775a850ca3eccac6fe83d7bca10b78eee4c0abf782e44fa0ddfec9d9a70f42fa40bdc49b68a1d1b4905cdc371ba29117d3120619
Loading the wallet after migrating is not a necessary step.
By making it optional pruned nodes can also migrate legacy wallets.
Also remove the migrated wallet from the list of wallets to load on startup.
