0aba464ce7 test: switch order of error code and message check (rkrux)
Pull request description:
I feel it'd be easier to debug intermittent test failures if the error message is present in the logs instead of error code. So, switching order of error code and message in the `try_rpc` function to aid error debugging.
Should help in debugging #34354 IMO. It's an intermittent failure on Windows that I can't reproduce and it's more difficult to figure out what could have gone wrong only by seeing the error code like below in the CI logs. Given that the functional tests pass, I don't see a harm in checking for error message first and throwing it in case of a mismatch.
```python
AssertionError: Unexpected JSONRPC error code -1
```
<!--
*** Please remove the following help text before submitting: ***
Pull requests without a rationale and clear improvement may be closed
immediately.
GUI-related pull requests should be opened against
https://github.com/bitcoin-core/gui
first. See CONTRIBUTING.md
-->
<!--
Please provide clear motivation for your patch and explain how it improves
Bitcoin Core user experience or Bitcoin Core developer experience
significantly:
* Any test improvements or new tests that improve coverage are always welcome.
* All other changes should have accompanying unit tests (see `src/test/`) or
functional tests (see `test/`). Contributors should note which tests cover
modified code. If no tests exist for a region of modified code, new tests
should accompany the change.
* Bug fixes are most welcome when they come with steps to reproduce or an
explanation of the potential issue as well as reasoning for the way the bug
was fixed.
* Features are welcome, but might be rejected due to design or scope issues.
If a feature is based on a lot of dependencies, contributors should first
consider building the system outside of Bitcoin Core, if possible.
* Refactoring changes are only accepted if they are required for a feature or
bug fix or otherwise improve developer experience significantly. For example,
most "code style" refactoring changes require a thorough explanation why they
are useful, what downsides they have and why they *significantly* improve
developer experience or avoid serious programming bugs. Note that code style
is often a subjective matter. Unless they are explicitly mentioned to be
preferred in the [developer notes](/doc/developer-notes.md), stylistic code
changes are usually rejected.
-->
<!--
Bitcoin Core has a thorough review process and even the most trivial change
needs to pass a lot of eyes and requires non-zero or even substantial time
effort to review. There is a huge lack of active reviewers on the project, so
patches often sit for a long time.
-->
ACKs for top commit:
maflcko:
lgtm ACK 0aba464ce7
polespinasa:
lgtm ACK 0aba464ce7
fjahr:
utACK 0aba464ce7
brunoerg:
code review ACK 0aba464ce7
sedited:
ACK 0aba464ce7
Tree-SHA512: b09ba4b5d13a2c93a4a28a5c1b06af44a91295974236bb8326b74a988878c431e9ce0e19ec14bb98ac2b002da877abaa7da6a9851424453bcb494c0317b57227
75b704df9d wallettool: Disallow creating new unnamed wallets (Ava Chow)
5875a9c502 wallet: disallow unnamed wallets in createwallet and restorewallet (Ava Chow)
d30ad4a912 wallet, rpc: Use HandleWalletError in createwallet (Ava Chow)
Pull request description:
We've been moving in the direction that all wallets must have a name. Therefore, we shouldn't allow creating new unnamed wallets. `createwallet`, `restorewallet`, and the wallet tool's `create` and `createfromdump` all now require the user to provide a non-empty wallet name when creating/restoring a wallet.
The GUI is already enforcing this, but we were not enforcing it for RPCs or in the underlying `CreateWallet` and `RestoreWallet` functions.
Wallet migration does still need to be able to restore unnamed wallets, so there is a new argument to `RestoreWallet` to explicitly allow that behavior for migration only.
ACKs for top commit:
rkrux:
lgtm ACK 75b704df9d
polespinasa:
re ACK 75b704df9d
Tree-SHA512: 8bde76d0b091e9276788c69412934af3426da2a7a69a00f94072d36c1a075cd41744ecdd5fef2b72870c1351b76aae061f124f716bb23f4839be20c464fc5ebd
fab055c907 test: Scale NetworkThread close timeout with timeout_factor (MarcoFalke)
Pull request description:
Not sure if this fixes https://github.com/bitcoin/bitcoin/issues/34248, but scaling here probably makes sense, considering some CI setups run in nested VMs with a different arch system-qemu.
ACKs for top commit:
hebasto:
ACK fab055c907, the diff looks reasonable.
Tree-SHA512: 98f9b0bdc3b02b692a14129f88c05f2df0d1e11e4167ff5d0cc6a3a6efd8994a743e969e83c71cb534537f134e07ba9a5cba3eb2010a6b6cf69bec959faf2c43
faa18dceba refactor: Use std::bind_front over std::bind (MarcoFalke)
Pull request description:
`std::bind` has many issues:
* It is verbosely listing all placeholders, but in a meaningless way, because it doesn't name the args or their types.
* It silently ignores args passed to it, when one arg is overridden. For example [1] compiles fine on current master.
* Accidentally duplicated placeholders compile fine as well.
* Usually the placeholders aren't even needed.
* This makes it hard to review, understand, and maintain.
Fix all issues by using `std::bind_front` from C++20, which allows to drop the brittle `_1, _2, ...` placeholders. The replacement should be correct, if the trailing placeholders are ordered.
Introducing the same silent bug on top of this pull request [2] will now lead to a compile failure.
----
[1]
```diff
diff --git a/src/qt/walletmodel.cpp b/src/qt/walletmodel.cpp
index 694fb535b5..7661dd361e 100644
--- a/src/qt/walletmodel.cpp
+++ b/src/qt/walletmodel.cpp
@@ -412,3 +412,3 @@ void WalletModel::subscribeToCoreSignals()
m_handler_status_changed = m_wallet->handleStatusChanged(std::bind(&NotifyKeyStoreStatusChanged, this));
- m_handler_address_book_changed = m_wallet->handleAddressBookChanged(std::bind(NotifyAddressBookChanged, this, std::placeholders::_1, std::placeholders::_2, std::placeholders::_3, std::placeholders::_4, std::placeholders::_5));
+ m_handler_address_book_changed = m_wallet->handleAddressBookChanged(std::bind(NotifyAddressBookChanged, this, CTxDestination{}, std::placeholders::_2, std::placeholders::_3, std::placeholders::_4, std::placeholders::_5));
m_handler_transaction_changed = m_wallet->handleTransactionChanged(std::bind(NotifyTransactionChanged, this, std::placeholders::_1, std::placeholders::_2));
```
[2]
```diff
diff --git a/src/qt/walletmodel.cpp b/src/qt/walletmodel.cpp
index 578713c0ab..84cced741c 100644
--- a/src/qt/walletmodel.cpp
+++ b/src/qt/walletmodel.cpp
@@ -412,3 +412,3 @@ void WalletModel::subscribeToCoreSignals()
m_handler_status_changed = m_wallet->handleStatusChanged(std::bind_front(&NotifyKeyStoreStatusChanged, this));
- m_handler_address_book_changed = m_wallet->handleAddressBookChanged(std::bind_front(NotifyAddressBookChanged, this));
+ m_handler_address_book_changed = m_wallet->handleAddressBookChanged(std::bind_front(NotifyAddressBookChanged, this, CTxDestination{}));
m_handler_transaction_changed = m_wallet->handleTransactionChanged(std::bind_front(NotifyTransactionChanged, this));
ACKs for top commit:
janb84:
cr ACK faa18dceba
fjahr:
Code review ACK faa18dceba
hebasto:
ACK faa18dceba, I have reviewed the code and it looks OK.
Tree-SHA512: 9dd13f49527e143a2beafbaae80b1358981f07a2ce20d25cffb1853089a32ff71639e6d718d1d193754522f9ac04e3e168ba017d5fc67a11a5918e79a92b3461
faa59b3679 util: Add Expected::swap() (MarcoFalke)
fabb47e4e3 util: Implement Expected::operator*()&& (MarcoFalke)
fab9721430 util: Implement Expected::value()&& and Expected::error()&& (MarcoFalke)
fac4800959 util: Add Expected<void, E> specialization (MarcoFalke)
fa6575d6c2 util: Make Expected::value() throw (MarcoFalke)
fa1de1103f util: Add Unexpected::error() (MarcoFalke)
faa109f8be test: refactor: Use BOOST_CHECK_EQUAL over BOOST_CHECK == (MarcoFalke)
fad4a9fe2b Set bugprone-unused-return-value.AllowCastToVoid (MarcoFalke)
Pull request description:
Reviewers requested more member functions In https://github.com/bitcoin/bitcoin/pull/34006.
They are currently unused, but bring the port closer to the original `std::expected` implementation:
* Make `Expected::value()` throw when no value exists
* Add `Unexpected::error()` methods
* Add `Expected<void, E>` specialization
* Add `Expected::value()&&` and `Expected::error()&&` methods
* Add `Expected::swap()`
Also, include a tiny tidy fixup:
* tidy: Set `AllowCastToVoid` in the `bugprone-unused-return-value` check
ACKs for top commit:
stickies-v:
re-ACK faa59b3679
ryanofsky:
Code review ACK faa59b3679. Thanks for the update. The commit I objected to is fixed now and the rest of the implementation seems good enough for code that's probably temporary.
hodlinator:
re-ACK faa59b3679
Tree-SHA512: b6ac28c1e7241837d9db83fe7534d713ca1283c20a77d2273743157d329f041ec0b503658d14b2f4425211808b61a88fed115d77149e0546825acd3bd9198edf
I feel it'd be easier to debug intermittent test failures if the
error message is present in the logs instead of error code. So,
switching order of error code and message in the `try_rpc` function
to aid error debugging.
3dd815f048 validation: pre-reserve leaves to prevent reallocs with odd vtx count (Lőrinc)
7fd47e0e56 bench: make `MerkleRoot` benchmark more representative (Lőrinc)
f0a2183108 test: adjust `ComputeMerkleRoot` tests (Lőrinc)
Pull request description:
#### Summary
`ComputeMerkleRoot` [duplicates the last hash](39b6c139bd/src/consensus/merkle.cpp (L54-L56)) when the input size is odd. If the caller provides a `std::vector` whose capacity equals its size, that extra `push_back` forces a reallocation, doubling its capacity (causing peak memory usage of 3x the necessary size).
This affects roughly half of the created blocks (those with odd transaction counts), causing unnecessary memory fragmentation during every block validation.
#### Fix
* Pre-reserves vector capacity to account for the odd-count duplication using `(size + 1) & ~1ULL`.
* This syntax produces [optimal assembly](https://github.com/bitcoin/bitcoin/pull/32497#discussion_r2553107836) across x86/ARM and 32/64-bit platforms for GCC & Clang.
* Eliminates default construction of `uint256` objects that are immediately overwritten by switching from `resize` to `reserve` + `push_back`.
#### Memory Impact
[Memory profiling](https://github.com/bitcoin/bitcoin/pull/32497#issuecomment-3563724551) shows **50% reduction in peak allocation** (576KB → 288KB) and elimination of reallocation overhead.
#### Validation
The benchmark was updated to use an odd leaf count to demonstrate the real-world scenario where the reallocation occurs.
A full `-reindex-chainstate` up to block **896 408** ran without triggering the asserts.
<details>
<summary>Validation asserts</summary>
Temporary asserts (not included in this PR) confirm that `push_back` never reallocates and that the coinbase witness hash remains null:
```cpp
if (hashes.size() & 1) {
assert(hashes.size() < hashes.capacity()); // TODO remove
hashes.push_back(hashes.back());
}
leaves.reserve((block.vtx.size() + 1) & ~1ULL); // capacity rounded up to even
leaves.emplace_back();
assert(leaves.back().IsNull()); // TODO remove
```
</details>
#### Benchmark Performance
While the main purpose is to improve predictability, the reduced memory operations also improve hashing throughput slightly.
ACKs for top commit:
achow101:
ACK 3dd815f048
optout21:
reACK 3dd815f048
hodlinator:
re-ACK 3dd815f048
vasild:
ACK 3dd815f048
w0xlt:
ACK 3dd815f048 with minor nits.
danielabrozzoni:
Code review ACK 3dd815f048
Tree-SHA512: e7b578f9deadc0de7d61c062c7f65c5e1d347548ead4a4bb74b056396ad7df3f1c564327edc219670e6e2b2cb51f4e1ccfd4f58dd414aeadf2008d427065c11f
969c840db5 log,blocks: avoid `ComputeTotalSize` and `GetHash` work when logging is disabled (Lőrinc)
babfda332b log,net: avoid `ComputeTotalSize` when logging is disabled (Lőrinc)
1658b8f82b refactor: rename `CTransaction::GetTotalSize` to signal that it's not cached (Lőrinc)
Pull request description:
### Context
The new accounting options introduced in https://github.com/bitcoin/bitcoin/pull/32582 can be quite heavy, and are not needed when debug logging is disabled.
### Problem
`PartiallyDownloadedBlock::FillBlock()` and `PeerManagerImpl::SendBlockTransactions()` accumulate transaction sizes for debug logging by calling `ComputeTotalSize()` in loops, which invokes expensive `GetSerializeSize()` serializations.
The block header hash is also only computed for the debug log.
### Fixes
Guard the size and hash calculations with `LogAcceptCategory()` checks so the serialization and hashing work only occurs when compact block debug logging is enabled.
Also modernized the surrounding code a bit since the change is quite trivial.
### Reproducer
You can test the change by starting an up-to-date `bitcoind` node with `-debug=cmpctblock` and observing compact block log lines such as:
> [cmpctblock] Successfully reconstructed block 00000000000000000001061eaa6c0fe79258e7f79606e67ac495765cb121a520 with 1 txn prefilled, 3122 txn from mempool (incl at least 3 from extra pool) and 641 txn (352126 bytes) requested
<details>
<summary>Test patch</summary>
```patch
diff --git a/src/blockencodings.cpp b/src/blockencodings.cpp
index 58620c93cc..f16eb38fa5 100644
--- a/src/blockencodings.cpp
+++ b/src/blockencodings.cpp
@@ -186,6 +186,7 @@ bool PartiallyDownloadedBlock::IsTxAvailable(size_t index) const
ReadStatus PartiallyDownloadedBlock::FillBlock(CBlock& block, const std::vector<CTransactionRef>& vtx_missing, bool segwit_active)
{
+ LogInfo("PartiallyDownloadedBlock::FillBlock called");
if (header.IsNull()) return READ_STATUS_INVALID;
block = header;
@@ -218,6 +219,7 @@ ReadStatus PartiallyDownloadedBlock::FillBlock(CBlock& block, const std::vector<
}
if (LogAcceptCategory(BCLog::CMPCTBLOCK, BCLog::Level::Debug)) {
+ LogInfo("debug log enabled");
const uint256 hash{block.GetHash()}; // avoid cleared header
uint32_t tx_missing_size{0};
for (const auto& tx : vtx_missing) tx_missing_size += tx->ComputeTotalSize(); // avoid cleared txn_available
diff --git a/src/net_processing.cpp b/src/net_processing.cpp
index 5600c8d389..c081825f77 100644
--- a/src/net_processing.cpp
+++ b/src/net_processing.cpp
@@ -2470,6 +2470,7 @@ uint32_t PeerManagerImpl::GetFetchFlags(const Peer& peer) const
void PeerManagerImpl::SendBlockTransactions(CNode& pfrom, Peer& peer, const CBlock& block, const BlockTransactionsRequest& req)
{
+ LogInfo("PeerManagerImpl::SendBlockTransactions called");
BlockTransactions resp(req);
for (size_t i = 0; i < req.indexes.size(); i++) {
if (req.indexes[i] >= block.vtx.size()) {
@@ -2480,6 +2481,7 @@ void PeerManagerImpl::SendBlockTransactions(CNode& pfrom, Peer& peer, const CBlo
}
if (LogAcceptCategory(BCLog::CMPCTBLOCK, BCLog::Level::Debug)) {
+ LogInfo("debug log enabled");
uint32_t tx_requested_size{0};
for (const auto i : req.indexes) tx_requested_size += block.vtx[i]->ComputeTotalSize();
LogDebug(BCLog::CMPCTBLOCK, "Peer %d sent us a GETBLOCKTXN for block %s, sending a BLOCKTXN with %u txns. (%u bytes)\n", pfrom.GetId(), block.GetHash().ToString(), resp.txn.size(), tx_requested_size);
```
</details>
ACKs for top commit:
davidgumberg:
reACK 969c840db5
achow101:
ACK 969c840db5
hodlinator:
re-ACK 969c840db5
sedited:
Re-ACK 969c840db5
danielabrozzoni:
reACK 969c840db5
Tree-SHA512: 9780102d29778165144e3602d934ed4cb96660fd7b9ff2581b223c619e419139b8348e60f226af448702ae527736a1806d169b44342c5a82795590f664e16efe
a3c71c7201 [test] Add BIP 328 test vectors for Musig2 (w0xlt)
Pull request description:
Built on https://github.com/bitcoin/bitcoin/pull/31244
This PR adds explicit tests for Bitcoin Core's MuSig2 interface.
Any issues in musig2.{cpp,h} will likely also be caught by the descriptor tests, but having more detailed tests for the MuSig2 class itself improves test reporting/coverage.
It uses BIP 328 test vectors.
ACKs for top commit:
achow101:
ACK a3c71c7201
rkrux:
lgtm ACK a3c71c7
Tree-SHA512: fc13beb5445c292cd7c75a47810fb1c4032ee2e3c1800dc44089b95959ccce8330291084bf788457e1d55c02d706ef04be7044badfee134149e004c44b19ec32
9c7e4771b1 test: Test listdescs with priv works even with missing priv keys (Novo)
ed945a6854 walletrpc: reject listdes with priv key on w-only wallets (Novo)
9e5e9824f1 descriptor: ToPrivateString() pass if at least 1 priv key exists (Novo)
5c4db25b61 descriptor: refactor ToPrivateString for providers (Novo)
2dc74e3f4e wallet/migration: use HavePrivateKeys in place of ToPrivateString (Novo)
e842eb90bb descriptors: add HavePrivateKeys() (Novo)
Pull request description:
_TLDR:
Currently, `listdescriptors [private=true]` will fail for a non-watch-only wallet if any descriptor has a missing private key(e.g `tr()`, `multi()`, etc.). This PR changes that while making sure `listdescriptors [private=true]` still fails if there no private keys. Closes #32078_
In non-watch-only wallets, it's possible to import descriptors as long as at least one private key is included. It's important that users can still view these descriptors when they need to create a backup—even if some private keys are missing ([#32078 (comment)](https://github.com/bitcoin/bitcoin/issues/32078#issuecomment-2781428475)). This change makes it possible to do so.
This change also helps prevent `listdescriptors true` from failing completely, because one descriptor is missing some private keys.
### Notes
- The new behaviour is applied to all descriptors including miniscript descriptors
- `listdescriptors true` still fails for watch-only wallets to preserve existing behaviour https://github.com/bitcoin/bitcoin/pull/24361#discussion_r920801352
- Wallet migration logic previously used `Descriptor::ToPrivateString()` to determine which descriptor was watchonly. This means that modifying the `ToPrivateString()` behaviour caused descriptors that were previously recognized as "watchonly" to be "non-watchonly". **In order to keep the scope of this PR limited to the RPC behaviour, this PR uses a different method to determine `watchonly` descriptors for the purpose of wallet migration.** A follow-up PR can be opened to update migration logic to exclude descriptors with some private keys from the `watchonly` migration wallet.
### Relevant PRs
https://github.com/bitcoin/bitcoin/pull/24361https://github.com/bitcoin/bitcoin/pull/32186
### Testing
Functional tests were added to test the new behaviour
EDIT
**`listdescriptors [private=true]` will still fail when there are no private keys because non-watchonly wallets must have private keys and calling `listdescriptors [private=true]` for watchonly wallet returns an error**
ACKs for top commit:
Sjors:
ACK 9c7e4771b1
achow101:
ACK 9c7e4771b1
w0xlt:
reACK 9c7e4771b1 with minor nits
rkrux:
re-ACK 9c7e4771b1
Tree-SHA512: f9b3b2c3e5425a26e158882e39e82e15b7cb13ffbfb6a5fa2868c79526e9b178fcc3cd88d3e2e286f64819d041f687353780bbcf5a355c63a136fb8179698b60
fabf8d1c5b fuzz: Restore SendMessages coverage in process_message(s) fuzz targets (MarcoFalke)
fac7fed397 refactor: Use std::reference_wrapper<AddrMan> in Connman (MarcoFalke)
Pull request description:
*Found and reported by Crypt-iQ (thanks!)*
Currently the process_message(s) fuzz targets do not have any meaningful `SendMessages` code coverage. This is not ideal.
Fix the problem by adding back the coverage, and by hardening the code here, so that the problem hopefully does not happen again in the future.
### Historic context for this regression
The regression was introduced in commit fa11eea405, which built a new deterministic peerman object. However, the patch was incomplete, because it was missing one hunk to replace `g_setup->m_node.peerman->SendMessages(&p2p_node);` with `peerman->SendMessages(&p2p_node);`.
This means the stale and empty peerman from the node context and not the freshly created and deterministic peerman was used.
A simple fix would be to just submit the missing patch hunk. However, this still leaves the risk that the issue is re-introduced at any time in the future. So instead, I think the stale and empty peerman should be de-constructed, so that any call to it will lead to a hard sanitizer error and fuzz failure.
Doing that also uncovered another issue: The connman was holding on to a reference to a stale and empty addrman.
So fix all issues by:
* Allowing the addrman reference in connman to be re-seatable
* Clearing all stale objects, before creating new objects, and then using references to the new objects in all code
ACKs for top commit:
Crypt-iQ:
crACK fabf8d1c5b
frankomosh:
ACK fabf8d1c5b
marcofleon:
code review ACK fabf8d1c5b
sedited:
ACK fabf8d1c5b
Tree-SHA512: 2e478102b3e928dc7505f00c08d4b9e4f8368407b100bc88f3eb3b82aa6fea5a45bae736c211f5af1551ca0de1a5ffd4a5d196d9473d4c3b87cfed57c9a0b69d
de509c6df9 iwyu: Add missed line to IWYU patch (Hennadii Stepanov)
Pull request description:
This PR makes IWYU suggest `<cassert>` over `<assert.h>`.
Fixes https://github.com/bitcoin/bitcoin/issues/34237.
ACKs for top commit:
maflcko:
lgtm ACK de509c6df9
Tree-SHA512: edba91eaf36992f684be2920f5da8c13a25ba6d79b879b92193e2af106cd454a64d7c4cf9dabc25675490df9edbccff1fd54c9f393e984a3a7a628b1c65f6c53
9482f00df0 chore: Update outdated GitHub Actions versions (Padraic Slattery)
Pull request description:
This PR updates outdated GitHub Action versions to ensure compatibility and improve functionality. The following changes are made to the GitHub Actions:
- `actions/upload-artifact` updated from v4 to v6
- `actions/cache` updated from v4 to v5
- `actions/download-artifact` updated from v5 to v7
The updates are necessary to support newer environments and features, and ensure consistent behavior across different workflows. The changes will be tested in the CI pipeline of the pull request.
ACKs for top commit:
fanquake:
ACK 9482f00df0
Tree-SHA512: 248e79162c5b2748e1a367d87a360d62eb961c24b4f8060bb932ef99a79ef10cab3e65175c092226c90140f31686fb9424911e6609729cb186b304b598a9af44
03f363d378 doc: Document IWYU workaround (Hennadii Stepanov)
Pull request description:
This PR addresses the following comments:
- https://github.com/bitcoin/bitcoin/pull/34079#discussion_r2640003086:
> it would be good to reduce and report this bug upstream. Otherwise, wide-spread use of iwyu in this code-base seems risky.
- https://github.com/bitcoin/bitcoin/pull/34079#discussion_r2640035350:
> Would have been good if it was documented, rather than adding undocumented workarounds for buggy tools.
ACKs for top commit:
maflcko:
lgtm ACK 03f363d378
sedited:
ACK 03f363d378
Tree-SHA512: 160a963c07f853995c8b4741a6ccca1d8431a576c760fca082116cebde4d133f7c8ec51f09e8f85f54428f86bad2635e1bd708177eecf71feb0bf1489f1e2b3e
0dafc0d83c clang-format: use AngleBracket for main includes (stickies-v)
Pull request description:
This project uses angle brackets instead of quotes for project-specific headers. Setting [`MainIncludeChar`](https://clang.llvm.org/docs/ClangFormatStyleOptions.html#mainincludechar) enables `clang-format` to automatically detect the main header, so it can be kept as the top group of includes.
For example, without this change, `clang-format` would demote `<signet.h>` from being the main header in `src/signet.cpp`. With this change, the order is preserved.
On 5e49f5d63c:
```
% clang-format src/signet.cpp | head -n 15
// Copyright (c) 2019-present The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#include <consensus/merkle.h>
#include <consensus/params.h>
#include <consensus/validation.h>
#include <logging.h>
#include <primitives/block.h>
#include <primitives/transaction.h>
#include <script/interpreter.h>
#include <script/script.h>
#include <signet.h>
#include <streams.h>
#include <uint256.h>
```
With this PR:
```
% clang-format src/signet.cpp | head -n 10
// Copyright (c) 2019-present The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#include <signet.h>
#include <consensus/merkle.h>
#include <consensus/params.h>
#include <consensus/validation.h>
#include <logging.h>
```
Note: `AngleBracket` `requires clang-format 19`, and will cause older versions (including our current minimum llvm version `17`) to fail
ACKs for top commit:
maflcko:
review ACK 0dafc0d83c
sedited:
Nice, ACK 0dafc0d83c
hebasto:
ACK 0dafc0d83c, tested on Ubuntu 25.10.
Tree-SHA512: c0876f505ec188f76e435af0731c411c66266b83e4c08528d0637263abcd84b3968ee6fbfa72630192f1a0cd2728af873d3d6c32f93ab8b228222fad16f232be
a7b581423e Fix 11-year-old mis-categorized error code in OP_IF evaluation (Calin Culianu)
Pull request description:
This was introduced by commit ab9edbd6b6.
It appears the original author may have gotten tired and pasted the wrong error code into this 1 place. Every other situation where the value stack lacks the required number of arguments for the op-code, SCRIPT_ERR_INVALID_STACK_OPERATION is reported. Not so here.
This commit fixes the situation.
EDIT: Note this turns out to be a dupe of the abandoned #30359 .
ACKs for top commit:
billymcbip:
tACK a7b581423e
achow101:
ACK a7b581423e
darosior:
utACK a7b581423e
sedited:
ACK a7b581423e
Tree-SHA512: e8c01a3e2448b5d49b76a0cab3f38a2d0249b71beeb7d9d05d5ecc3812bd91d0bd1d0f78b809b6f4ccb73186fa119cb1ed3779a73284b83a67ae219ef378fa6c
`PartiallyDownloadedBlock::FillBlock()` computed the block header hash and summed missing transaction sizes for debug logging unconditionally, including when cmpctblock debug logging is disabled.
Guard the debug-only hash and size computations with `LogAcceptCategory`.
Since `txn_available` is invalidated after the first loop (needed for efficient moving), we compute `tx_missing_size` by iterating `vtx_missing` directly. This is safe because the later `tx_missing_offset` check guarantees `vtx_missing` was fully consumed during reconstruction.
Use `block.GetHash()` instead of `header.GetHash()`, since header is cleared before logging.
No behavior change when debug logging is enabled: the reported counts, hashes, and byte totals remain the same.
`PeerManagerImpl::SendBlockTransactions()` computed the total byte size of requested transactions for a debug log line by calling `ComputeTotalSize()` in a tight loop, triggering serialization even when debug logging is off.
Guard the size accumulation with `LogAcceptCategory` so the serialization work only happens when the log line can be emitted.
No behavior change when debug logging is enabled: the reported block hash, transaction count, and byte totals are the same.
The bounds checks still run unconditionally; the debug-only loop iterates the already-validated response contents.
Separating debug-only work from the critical path reduces risk and favors the performance-critical non-debug case.
This also narrows the racy scope of when logging is toggled from another thread.
Transaction hashes are cached, it may not be intuitive that their sizes are actually recalculated every time.
This is done before the other refactors to clarify why we want to avoid calling this method;
Co-authored-by: maflcko <6399679+maflcko@users.noreply.github.com>
Migration still needs to be able to restore unnamed wallets, so
allow_unnamed is added to RestoreWallet to explicitly allow that
behavior for migration only.
This project uses angle brackets instead of quotes for project-specific
headers. Setting MainIncludeChar enables clang-format to automatically
detect the main header, so it can be kept as the top group of includes.
For example, without this change, the below command would demote
<signet.h> from being the main header. With this change, the order is
preserved.
`clang-format -i src/signet.cpp`
`IWYU pragma: export` enforces the transitive inclusion of the headers,
which undermines the purpose of IWYU.
The remained cases seem useful and could be considered separately:
- `<cassert>` in `util/check.h`
- `<filesystem>` in `util/fs.h`
- `<chrono>` in `util/time.h`
faf07bd1ab doc: Fix typo found by LLM (MarcoFalke)
faf66673ac refactor: [move-only] Merge core_io module (MarcoFalke)
fa6947f491 kernel: Remove unused core_read.cpp from kernel (MarcoFalke)
Pull request description:
Currently the core_io module is split across two translation units. This will confuse code readers and tooling about the real state of the module.
Fix that by merging the module and removing the mapping workarounds.
Also, remove the module from the kernel lib, because it is not used there: The kernel does not use any json or string parsing or formatting.
ACKs for top commit:
hebasto:
re-ACK faf07bd1ab, only rebased since my recent [review](https://github.com/bitcoin/bitcoin/pull/34296#pullrequestreview-3675359502).
sedited:
Re-ACK faf07bd1ab
stickies-v:
ACK faf07bd1ab
Tree-SHA512: 3f5d91f1a4cb86dfe329b28ff31e93d65f2f0659a6f6f2de22ca6fb65056fb818ae369ef0ad773d4f5b92f63891a7a9450246377d8e14c34bc43f3deee0554cb
a5a8c4139c ci, iwyu: Fix warnings in `src/kernel` and treat them as errors (Hennadii Stepanov)
Pull request description:
Now seems like a good time to update the includes in `src/kernel`.
ACKs for top commit:
maflcko:
review ACK a5a8c4139c🍱
purpleKarrot:
ACK a5a8c4139c
sedited:
ACK a5a8c4139c
Tree-SHA512: ba401b27b03dee66d52d0b348972268e162506c4bafa40f408349173b68c40a11f20ca24f46c98945515e1d5c84f740d6e6784f7e4c799df46ab816cf5d11483
fa64d8424b refactor: Enforce readability-avoid-const-params-in-decls (MarcoFalke)
faf0c2d942 refactor: Avoid copies by using const references or by move-construction (MarcoFalke)
Pull request description:
Top level `const` in declarations is problematic for many reasons:
* It is often a typo, where one wanted to denote a const reference. For example `bool PSBTInputSignedAndVerified(const PartiallySignedTransaction psbt, ...` is missing the `&`. This will create a redundant copy of the value.
* In constructors it prevents move construction.
* It can incorrectly imply some data is const, like in an imaginary example `std::span<int> Shuffle(const std::span<int>);`, where the `int`s are *not* const.
* The compiler ignores the `const` from the declaration in the implementation.
* It isn't used consistently anyway, not even on the same line.
Fix some issues by:
* Using a const reference to avoid a copy, where read-only of the value is intended. This is only done for values that may be expensive to copy.
* Using move-construction to avoid a copy
* Applying `readability-avoid-const-params-in-decls` via clang-tidy
ACKs for top commit:
l0rinc:
diff reACK fa64d8424b
hebasto:
ACK fa64d8424b, I have reviewed the code and it looks OK.
sedited:
ACK fa64d8424b
Tree-SHA512: 293c000b4ebf8fdcc75259eb0283a2e4e7892c73facfb5c3182464d6cb6a868b7f4a6682d664426bf2edecd665cf839d790bef0bae43a8c3bf1ddfdd3d068d38
dd904298c1 gui: Show an error message if the restored wallet name is empty (Ava Chow)
Pull request description:
The Restore Wallet dialog rejects wallet names that are empty, but was doing so silently. This is confusing, we should be presenting an error message to the user.
ACKs for top commit:
hebasto:
ACK dd904298c1. Tested on Fedora 43.
Tree-SHA512: f4b60f32d1c2550dbce8613f25d29a92588b1ecfc8e8e5dac691a6bdb21a77508288a904539b68333d96bde5ebb993912253f4a293e4c583891f553d95762e77
fa38ffac6f contrib: [refactor] Use shorter read_text from pathlib (MarcoFalke)
fab8bc0308 contrib: Revert "verify-commits sha1 exceptions" (MarcoFalke)
Pull request description:
This reverts commit 8ac134be5e, because it is no longer needed.
See https://github.com/bitcoin/bitcoin/pull/34245#issuecomment-3759448369
Also, use the shorter pathlib `read_text`, which is available since Python 3.5
ACKs for top commit:
dergoegge:
utACK fa38ffac6f
sedited:
ACK fa38ffac6f
hebasto:
ACK fa38ffac6f.
Tree-SHA512: 83049349d4a5c74ad700c2912d727584b88944a75d572c10661a76b69b08093ef7ebf786b359455e36d7467a708de46a77da41a54512e057d7eed8206984c8fd
979d41bfab qa: Fix Windows logging bug (Hennadii Stepanov)
Pull request description:
The regex `(.*)` was capturing `\r` from subprocess output on Windows, causing the closing parenthesis in logs to wrap to the next line.
For [example](https://github.com/hebasto/bitcoin/actions/runs/20993438084/job/60350204808):
```
208/454 - feature_bip68_sequence.py passed, Duration: 10 s
209/454 - rpc_bind.py --ipv4 skipped (not on a Linux system
)
210/454 - rpc_bind.py --ipv6 skipped (not on a Linux system
)
211/454 - rpc_packages.py passed, Duration: 8 s
212/454 - rpc_bind.py --nonloopback skipped (not on a Linux system
)
213/454 - p2p_feefilter.py passed, Duration: 4 s
```
Stripping whitespace from the regex match fixes the formatting. [See](https://github.com/hebasto/bitcoin/actions/runs/20993564177/job/60350024373):
```
208/454 - feature_bip68_sequence.py passed, Duration: 9 s
209/454 - rpc_bind.py --ipv4 skipped (not on a Linux system)
210/454 - rpc_bind.py --ipv6 skipped (not on a Linux system)
211/454 - rpc_bind.py --nonloopback skipped (not on a Linux system)
212/454 - rpc_packages.py passed, Duration: 7 s
```
ACKs for top commit:
maflcko:
lgtm ACK 979d41bfab
l0rinc:
lightly tested ACK 979d41bfab
Tree-SHA512: bafe1937a519e45e4cab395bae622acf65220f313c773a0729ba7dccc3a0a048602f1c04b3e8cdd80d2cf68ae36cef802a819530485d5a745db8abcadf141f68
This is not expected to be needed in this codebase, but brings the
implementation closer to std::expected::value().
Also, add noexcept, where std::expected has them. This will make
operator-> and operator* terminate, when has_value() is false.
3e340672ec test: use ephemeral ports in p2p_private_broadcast.py (w0xlt)
Pull request description:
The test `p2p_private_broadcast.py` gets some Python P2P nodes to listen and instructs the SOCKS5 proxy to redirect connections to them instead of to the requested addresses. This way the `bitcoind` which uses the proxy is tricked to think it has connected to real routable internet IP addresses or `.onion` addresses.
Picking the ports where to Python P2P nodes to listen however is tricky to be done in a non-conflicting way, given that other tests may run in parallel. https://github.com/bitcoin/bitcoin/pull/34186 made it possible to let the OS select a free port, so use that in
`p2p_private_broadcast.py`.
---
_Suggested in https://github.com/bitcoin/bitcoin/pull/29415#discussion_r2654849875_
ACKs for top commit:
l0rinc:
code review ACK 3e340672ec
polespinasa:
tACK 3e340672ec
mzumsande:
utACK 3e340672ec
Tree-SHA512: e94efd33a1845e1767aaada55f91c60bc5fc1166c281ef578a391e95e2791a922d84aa6ed1ce06e7d6ca1a65f84da52fd79d9b2f40705c1944a53c67b7392e4d
d09a19fd41 test: add coverage for issue 34206 (Greg Sanders)
4c7cfd37ad wallet: remove erroneous-on-reorg Assume() (Greg Sanders)
Pull request description:
Resolves https://github.com/bitcoin/bitcoin/issues/34206
I'm not certain the test is worth keeping, but included it for now to show minimal example that crashes without fix. Can be removed.
ACKs for top commit:
bensig:
ACK d09a19fd41
dergoegge:
utACK d09a19fd41
Tree-SHA512: 7eac19e97be6db8e38af396c406066fdcec532332e685a38bb33f0a988701c7bd5a0967f51426737fd56972847b761a3d873495928ff66efa8512fb267a9622b
The addrman field is already a reference. However, some tests would
benefit from the reference being re-seatable, so that they do not have
to create a full Connman each time.
fac70ea8b5 fuzz: Exclude too expensive inputs in miniscript_string target (MarcoFalke)
fa90786478 iwyu: Fix includes for test/fuzz/util/descriptor module (MarcoFalke)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/30498
Accepting "expensive" fuzz inputs which have no real use-case is problematic, because it prevents the fuzz engine from spending time on the next useful fuzz input.
For example this one will take several seconds (the flamegraph shows the time is spent in minscipt `NoDupCheck`):
```
curl -fLO '41bae50cff'
FUZZ=miniscript_string /usr/bin/time ./bld-cmake/bin/fuzz ./41bae50cffd1741150a1b330d02ab09f46ff8cd1
```
Inspecting the inputs shows that it has many sub frags, so rejecting based on `HasTooManySubFrag` should be sufficient.
ACKs for top commit:
darosior:
ACK fac70ea8b5
brunoerg:
code review ACK fac70ea8b5
dergoegge:
utACK fac70ea8b5
Tree-SHA512: 7f1e0d9ce24d67ec63e5b7c2dd194efa51f38beb013564690afe0f920e5ff1980c85ce344828c0dc3f34b6851db7fe72a76b1a775c6d51c94fb91431834f453b
da56ef239b clusterlin: minimize chunks (feature) (Pieter Wuille)
Pull request description:
Part of #30289.
This was split off from #34023, because it's not really an optimization but a feature. The feature existed pre-SFL, so this brings SFL to parity in terms of functionality with the old code.
The idea is that while optimality - as achieved by SFL before this PR - guarantees a linearization whose feerate diagram is optimal, it may be possible to split chunks into smaller equal-feerate parts. This is desirable because even though it doesn't change the diagram, it provides more flexibility for optimization (binpacking is easier when the pieces are smaller).
Thus, this PR introduces the stronger notion of "minimality": optimal chunks, which are also split into their smallest possible pieces. To accomplish that, an additional step in the SFL algorithm is added which aims to split chunks into minimal equal-feerate parts where possible, without introducing circular dependencies between them. It works based on the observation that if an (already otherwise optimal) chunk has a way of being split into two equal-feerate parts, and T is a given transaction in the chunk, then we can find the split in two steps:
* One time, pretend T has $\epsilon$ higher feerate than it really has. If a split exists with T in the top part, this will find it.
* The other time, pretend T has $\epsilon$ lower feerate than it really has. If a split exists with T in the bottom part, this will find it.
So we try both on each found optimal chunk. If neither works, the chunk is minimal. If one works, recurse into the split chunks to split them further.
ACKs for top commit:
instagibbs:
reACK da56ef239b
marcofleon:
crACK da56ef239b
Tree-SHA512: 2e94d6b78725f5f9470a939dedef46450b85c4e5e6f30cba0b038622ec2b417380747e8df923d1f303706602ab6d834350716df9678de144f857e3a8d163f6c2
The test `p2p_private_broadcast.py` gets some Python P2P nodes to listen
and instructs the SOCKS5 proxy to redirect connections to them instead
of to the requested addresses. This way the `bitcoind` which uses the
proxy is tricked to think it has connected to real routable internet
IP addresses or `.onion` addresses.
Picking the ports where to Python P2P nodes to listen however is tricky
to be done in a non-conflicting way, given that other tests may run in
parallel. https://github.com/bitcoin/bitcoin/pull/34186 made it possible
to let the OS select a free port, so use that in
`p2p_private_broadcast.py`.
fa3df52712 bench: Require semicolon after BENCHMARK(foo) (MarcoFalke)
fa8938f08c bench: Remove incorrect __LINE__ in BENCHMARK macro (MarcoFalke)
fa51a28a94 scripted-diff: Remove priority_level from BENCHMARK macro (MarcoFalke)
fa790c3eea bench: Remove -priority-level= option (MarcoFalke)
Pull request description:
The option was added in https://github.com/bitcoin/bitcoin/pull/26158, when the project was using an autotools-based build system. However, in the meantime this option is unused:
* First, commit 27f11217ca removed the option from one CI task
* Then https://github.com/bitcoin/bitcoin/pull/32310 removed the option from CMakeList.txt, because:
* they only run as a sanity check (fastest version)
* no one otherwise runs them, not even CI
* issues have been missed due to this
Finally, after commit 0ad4376a49, I don't see a single reason to keep this option, so remove it.
Also, there is a commit to turn a silent ignore of duplicate bench names into an error.
ACKs for top commit:
achow101:
ACK fa3df52712
l0rinc:
ACK fa3df52712
hebasto:
re-ACK fa3df52712, only suggested changes since my recent [review](https://github.com/bitcoin/bitcoin/pull/34210#pullrequestreview-3652414135).
Tree-SHA512: 68a314bff551fa878196d5a615d41d71e1c8c504135e6fc555659aa9f0c8786957d49ba038448e933554a8bc54caea2ddd7d628042c5627bf3bf37628210f8fb
ce63d37ebe test: use dynamic port allocation to avoid test conflicts (woltx)
Pull request description:
Use `port=0` for dynamic port allocation in test framework components to avoid intermittent "address already in use" errors when running tests concurrently or when ports are stuck in TIME_WAIT state. Example: https://github.com/bitcoin/bitcoin/pull/29415#discussion_r2634509304
Changes:
- Update `socks5.py` and `p2p.py` to support dynamic port allocation
- Convert `feature_proxy.py` and `feature_anchors.py` to use `port=0`
ACKs for top commit:
achow101:
ACK ce63d37ebe
vasild:
ACK ce63d37ebe
mzumsande:
re-ACK ce63d37ebe
Tree-SHA512: 4efcedca3bde209fbd1bdc2a4ae04b7d53515587d86e421ce61064f78c675c71b45d9782b514c5e7cfc0e92842c947d49f7a3fddb03fe619fcdec9b565f0ecbd
7b5d256af4 test: Add bitcoin-chainstate test for assumeutxo functionality (stringintech)
2bc3265649 Fix `ChainstateManager::AddChainstate()` assertion crash (stringintech)
5f3d6bdb66 Add regtest support to bitcoin-chainstate tool (stringintech)
Pull request description:
This PR adds functional test coverage for the bitcoin-chainstate tool loading a datadir initialized with an assumeutxo snapshot.
The PR also includes:
- Fix for assertion crash in `ChainstateManager::AddChainstate()` when `prev_chainstate` has no initialized mempool (required for the test to pass)
- `-regtest` flag support for bitcoin-chainstate to enable the testing
This work started while experimenting with the bitcoin-chainstate tool and how the kernel API (#30595) behaved when loading a datadir containing assumeutxo data, during the time that PR was still under review. sedited suggested opening a PR to add this test coverage.
ACKs for top commit:
achow101:
ACK 7b5d256af4
theStack:
Concept and code-review ACK 7b5d256af4
sedited:
Re-ACK 7b5d256af4
Tree-SHA512: 5d3b0050cf2d53144b5f65451c991d5e212117b4541ae1368ecf58fde5f3cca4f018aad6ae32257b9ebb1c28b926424fbcff496ba5487cdc4eb456cea6db8b24
792e2edf57 p2p: first addr self-announcement in separate msg (0xb10c)
Pull request description:
This makes sure the initial address self-announcement a node sends to a peer happends in a separate P2P message. This has benefits for both inbound and outbound connections:
For inbound connections from a peer to us, previously, we might send the self-announcement along with our response to a GETADDR request. However, the self-announcement might replace an address from the GETADDR response. This isn't clean.
For outbound connections from us to a peer, previously, it could have happend that we send the self-announcement along with other addresses. Since shortly after connection open, the peer might only have one rate-limiting token for us, and the addresses are shuffeld on arrival, it's possible that the self-announcement gets rate-limited. However, note that these rate-limitings seem to be rare in practice.
This is inspired by and based on https://github.com/bitcoin/bitcoin/pull/33699#issuecomment-3462287763. The discussion there should be helpful for reviewers.
ACKs for top commit:
bensig:
ACK 792e2edf57
achow101:
ACK 792e2edf57
fjahr:
Code review ACK 792e2edf57
frankomosh:
Code Review ACK [792e2ed](792e2edf57)
Tree-SHA512: e3d39b1e3ae6208b54df4b36c624a32d70a442e01681f49e0c8a65076a818b5bf203c2e51011dc32edbbe3637b3c0b5f18de26e3461c288aa3806646a209a260
8fb5e5f41d test: check wallet rescan properly in feature_pruning (brunoerg)
9b57c8d2bd test: fix feature_pruning when built without wallet (brunoerg)
Pull request description:
Fixes#34175
In `feature_pruning`, the`wallet_test` doesn't require any specific wallet functionality and this test is important for one of next ones (`test_scanblocks_pruned`). The reason is that it synchronizes the node 5 and, without this sync, `test_scanblocks_pruned` will fail since we expect `scanblocks` to fail due to `Block not available (pruned data)` and it doesn't happen without this sync.
ACKs for top commit:
achow101:
ACK 8fb5e5f41d
furszy:
utACK 8fb5e5f41d
musaHaruna:
Tested ACK [8fb5e5f](8fb5e5f41d)
w0xlt:
ACK 8fb5e5f41d
Tree-SHA512: 812afbf4343a7493e2169eb6735fce25692d5cb19972abafc772b8c05a64b9c7027f6675cd084f345977e916e62a722d671f90831bbdc51683e0cd253fa482f0
de4242f474 refactor: Use reference for chain_start in HeadersSyncState (Daniela Brozzoni)
e37555e540 refactor: Use initializer list in CompressedHeader (Daniela Brozzoni)
0488bdfefe refactor: Remove unused parameter in ReportHeadersPresync (Daniela Brozzoni)
256246a9fa refactor: Remove redundant parameter from CheckHeadersPoW (Daniela Brozzoni)
ca0243e3a6 refactor: Remove useless CBlock::GetBlockHeader (Pieter Wuille)
4568652222 refactor: Use std::span in HasValidProofOfWork (Daniela Brozzoni)
4066bfe561 refactor: Compute work from headers without CBlockIndex (Daniela Brozzoni)
0bf6139e19 p2p: Avoid an IsAncestorOfBestHeaderOrTip call (Pieter Wuille)
Pull request description:
This is a partial* revival of #25968
It contains a list of most-unrelated simplifications and optimizations to the code merged in #25717:
- Avoid an IsAncestorOfBestHeaderOrTip call: Just don't call this function when it won't have any effect.
- Compute work from headers without CBlockIndex: Avoid the need to construct a CBlockIndex object just to compute work for a header, when its nBits value suffices for that. Also use some Spans where possible.
- Remove useless CBlock::GetBlockHeader: There is no need for a function to convert a CBlock to a CBlockHeader, as it's a child class of it.
It also contains the following code cleanups, which were suggested by reviewers in #25968:
- Remove redundant parameter from CheckHeadersPoW: No need to pass consensusParams, as CheckHeadersPow already has access to m_chainparams.GetConsensus()
- Remove unused parameter in ReportHeadersPresync
- Use initializer list in CompressedHeader, also make GetFullHeader const
- Use reference for chain_start in HeadersSyncState: chain_start can never be null, so it's better to pass it as a reference rather than a raw pointer
*I decided to leave out three commits that were in #25968 (4e7ac7b94d, ab52fb4e95, 7f1cf440ca), since they're a bit more involved, and I'm a new contributor. If this PR gets merged, I'll comment under #25968 to note that these three commits are still up for grabs :)
ACKs for top commit:
l0rinc:
ACK de4242f474
polespinasa:
re-ACK de4242f474
sipa:
ACK de4242f474
achow101:
ACK de4242f474
hodlinator:
re-ACK de4242f474
Tree-SHA512: 1de4f3ce0854a196712505f2b52ccb985856f5133769552bf37375225ea8664a3a7a6a9578c4fd461e935cd94a7cbbb08f15751a1da7651f8962c866146d9d4b
f62568c97c ci: Install `pyzmq` for functional tests on Windows (Hennadii Stepanov)
Pull request description:
This PR enables `interface_zmq.py` on Windows by installing the required `pyzmq` package.
ACKs for top commit:
maflcko:
lgtm ACK f62568c97c
Tree-SHA512: a57728ef82f1ff15a3ed1a659e98ad2d10f69693772649eaa97e1803e18f2f68e99782be3cb149322d1bf72607ec103e1837db1db39e9bd8c2a0a0863463635d
7fc465ece8 doc: fix incorrect description of `PackageMempoolChecks` (ismaelsadeeq)
1412b779ad refactor: execute `PackageMempoolChecks` during package rbf only (ismaelsadeeq)
Pull request description:
This is a simple PR that fixes the incorrect description of what is done in `PackageMempoolChecks`
> // Enforce package mempool ancestor/descendant limits (distinct from individual
> // ancestor/descendant limits done in PreChecks) and run Package RBF checks.
After cluster mempool, we no longer enforce ancestor/descendant limits in both `PreChecks` and `PackageMempoolChecks`; instead, cluster limit is enforced in `PackageMempoolChecks`.
This PR fixes the incorrect comment by;
- Making it clear why it is necessary to have two calls of `CheckMempoolPolicyLimts` in both `PackageMempoolChecks` and after in `AcceptMultipleTransactionsInternal` by executing `PackageMempoolChecks` only during package RBF only. No need to jump into the next subroutine when there is no conflict.
- Renames `PackageMempoolChecks` to `PackageRBFChecks`; the method name is self-explanatory now, hence no need for a description comment.
ACKs for top commit:
yashbhutwala:
ACK 7fc465ece8
instagibbs:
ACK 7fc465ece8
glozow:
utACK 7fc465ece8
Tree-SHA512: 38655f9d05be54cadd224fad376da9871a85efc7801306b58d4f51aee658036cdce2ab406143a3439d7211fc9bb0fc86bd330852e8926d79660944872b8fae8d
fa2959e16d test: Fail on self-check warnings in test_runner.py (MarcoFalke)
Pull request description:
I don't see a reason to start running the tests, if the test_runner detects warnings during the self-check.
Usually, this will just lead to a possibly confusing test failure after some wasted time anyway.
So just fail fast before even running any tests.
If there was a reason to ignore the warnings, a new option could trivially be added:
```py
parser.add_argument("--ignore-self-check-warnings", dest="ignore_warnings", default=False, action="store_true",
help="Ignore test runner warnings about self-checks before running the tests")
```
However, I don't see the need.
ACKs for top commit:
hodlinator:
utACK fa2959e16d
Tree-SHA512: 3f2e1af9ba06cd805c69e8c40149214a8a13af286ba6315c346e23d0f1ddae8e850103b6967d2d9799a095f6a8ec8802c3c773af8d7123598e8887f56c764fb5
The regex `(.*)` was capturing `\r` from subprocess output on Windows,
causing the closing parenthesis in logs to wrap to the next line.
Stripping whitespace from the regex match fixes the formatting.
fa5032f0e4 ci: Run feature_unsupported_utxo_db.py on Windows (MarcoFalke)
Pull request description:
The feature_unsupported_utxo_db.py test is skipped on Windows, because the ancient exe used in the test does not support UTF-8.
One way to fix this could be to rework the test_runner to supply an ASCII temp dir. However, this is tedious and probably not worth it for a test that is close to being deleted: The test only checks that a `-reindex-chainstate` is sufficient, but at this point, it seems fine to accept users having to go through a full `-reindex`, or even full re-download.
I am not deleting it just yet, because it is still minimally useful: E.g. https://github.com/bitcoin/bitcoin/pull/31674#issuecomment-2599106040. Though, I think the next time an issue pops up with this test, it can be considered for deletion.
ACKs for top commit:
hebasto:
ACK fa5032f0e4.
Tree-SHA512: eb3be204a1238caa93c7c2947171ed3c1c2717ccf95e5a58102791dbecfcd8710e0d89f001e266c192734bb4326ec6ca6e0df3128ad0d1e76cfe0f34f486a08e
fa1d17d56c refactor: Use uint64_t over size_t for serialize corruption check in fees.dat (MarcoFalke)
Pull request description:
Serialization should not behave differently on different architectures. See also the related commit 3789215f73.
However, on fees.dat file corruption, 32-bit builds may run into an unsigned integer overflow and report the wrong corruption reason, or may even silently continue after the corruption.
This is a bit hard to reproduce, because 32-bit platforms are rare and most of them don't support running the unsigned integer overflow sanitizer. So the possible options to reproduce are:
* Run on armhf and manually annotate the code to detect the overflow
* Run on i386 with the integer sanitizer (possibly via `podman run -it --rm --platform linux/i386 'debian:trixie'`)
* Run the integer sanitizer on any 64-bit platform and manually replace type in the affected line by `uint32_t`
Afterwards, the steps to reproduce are:
```
export DEBIAN_FRONTEND=noninteractive && apt update && apt install curl wget htop git vim ccache -y && git clone https://github.com/bitcoin/bitcoin.git --depth=1 ./b-c && cd b-c && apt install build-essential cmake pkg-config python3-zmq libzmq3-dev libevent-dev libboost-dev libsqlite3-dev systemtap-sdt-dev libcapnp-dev capnproto libqrencode-dev qt6-tools-dev qt6-l10n-tools qt6-base-dev clang llvm libc++-dev libc++abi-dev -y
cmake -B ./bld-cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_C_COMPILER='clang' -DCMAKE_CXX_COMPILER='clang++' -DSANITIZERS=undefined,integer,float-divide-by-zero --preset=dev-mode
cmake --build ./bld-cmake --parallel $(nproc)
curl -fLO '6074731370'
UBSAN_OPTIONS="suppressions=$(pwd)/test/sanitizer_suppressions/ubsan:print_stacktrace=1:halt_on_error=1:report_error_type=1" FUZZ=policy_estimator_io ./bld-cmake/bin/fuzz ./607473137013139e3676e30ec4b29639e673fa9b
```
The output will be something like:
```
/b-c/src/policy/fees/block_policy_estimator.cpp:448:25: runtime error: unsigned integer overflow: 346685954 * 219 cannot be represented in type 'unsigned int'
#0 0x5b0b1bbe in TxConfirmStats::Read(AutoFile&, unsigned int) /b-c/bld-cmake/src/./policy/fees/block_policy_estimator.cpp:448:25
#1 0x5b0b7d3f in CBlockPolicyEstimator::Read(AutoFile&) /b-c/bld-cmake/src/./policy/fees/block_policy_estimator.cpp:1037:29
#2 0x592a9783 in policy_estimator_io_fuzz_target(std::span<unsigned char const, 4294967295u>) /b-c/bld-cmake/src/test/fuzz/./test/fuzz/policy_estimator_io.cpp:32:32
#3 0x5896ba8e in void std::__invoke_impl<void, void (*&)(std::span<unsigned char const, 4294967295u>), std::span<unsigned char const, 4294967295u>>(std::__invoke_other, void (*&)(std::span<unsigned char const, 4294967295u>), std::span<unsigned char const, 4294967295u>&&) /usr/lib/gcc/i686-linux-gnu/14/../../../../include/c++/14/bits/invoke.h:61:14
#4 0x5896b8eb in std::enable_if<is_invocable_r_v<void, void (*&)(std::span<unsigned char const, 4294967295u>), std::span<unsigned char const, 4294967295u>>, void>::type std::__invoke_r<void, void (*&)(std::span<unsigned char const, 4294967295u>), std::span<unsigned char const, 4294967295u>>(void (*&)(std::span<unsigned char const, 4294967295u>), std::span<unsigned char const, 4294967295u>&&) /usr/lib/gcc/i686-linux-gnu/14/../../../../include/c++/14/bits/invoke.h:111:2
#5 0x5896b44b in std::_Function_handler<void (std::span<unsigned char const, 4294967295u>), void (*)(std::span<unsigned char const, 4294967295u>)>::_M_invoke(std::_Any_data const&, std::span<unsigned char const, 4294967295u>&&) /usr/lib/gcc/i686-linux-gnu/14/../../../../include/c++/14/bits/std_function.h:290:9
#6 0x59845c95 in std::function<void (std::span<unsigned char const, 4294967295u>)>::operator()(std::span<unsigned char const, 4294967295u>) const /usr/lib/gcc/i686-linux-gnu/14/../../../../include/c++/14/bits/std_function.h:591:9
#7 0x5983a0da in test_one_input(std::span<unsigned char const, 4294967295u>) /b-c/bld-cmake/src/test/fuzz/util/./test/fuzz/fuzz.cpp:88:5
#8 0x5983cb80 in main /b-c/bld-cmake/src/test/fuzz/util/./test/fuzz/fuzz.cpp:271:13
#9 0xf75aecc2 (/lib/i386-linux-gnu/libc.so.6+0x24cc2) (BuildId: 2dc5f2945fad35c1b07d1a5a32520b3c41afaa75)
#10 0xf75aed87 in __libc_start_main (/lib/i386-linux-gnu/libc.so.6+0x24d87) (BuildId: 2dc5f2945fad35c1b07d1a5a32520b3c41afaa75)
#11 0x58932db6 in _start (/b-c/bld-cmake/bin/fuzz+0x235ddb6) (BuildId: 7d8d83a77923f14e99c0de64acbc5f5bfc2cce9b)
SUMMARY: UndefinedBehaviorSanitizer: unsigned-integer-overflow /b-c/src/policy/fees/block_policy_estimator.cpp:448:25
```
Note: This is marked a "refactor", because the code change does not affect 64-bit builds, and on the still remaining rare 32-bit builds today it is extremely unlikely to happen in production.
ACKs for top commit:
bensig:
ACK fa1d17d56c
ismaelsadeeq:
utACK fa1d17d56c
luke-jr:
Also, utACK fa1d17d56c as an improvement.
Tree-SHA512: 696bf8e0dbe4777c84cb90e313c7f8f9ee90d4b3e64de1222f8472b2d9d0f3a0f6f027fda743dd6ca8c6aab94f404db7a65bb562a76000d9c33a8a39de28d8d4
2f5b1c5f80 psbt: Fix `PSBTInputSignedAndVerified` bounds `assert` (Lőrinc)
Pull request description:
This PR fixes an off-by-one in a debug assertion in `PSBTInputSignedAndVerified`.
The function indexes `psbt.inputs[input_index]`, so the assertion must not allow indexing at `psbt.inputs.size()`.
Found during review: https://github.com/bitcoin/bitcoin/pull/31650#discussion_r2685892867
ACKs for top commit:
optout21:
utACK 2f5b1c5f80
maflcko:
lgtm ACK 2f5b1c5f80
achow101:
ACK 2f5b1c5f80
Tree-SHA512: cec613a9a38358d5caa243197d746baa129aebfd7fe697689f28e652f94c4683873c4676d5eb2eb909ea19de5e5f6e54ecc5f3162384a48f6f38a59273667689
facaf56214 contrib: Remove unused functions (MarcoFalke)
Pull request description:
* `remove_files` is unused since 5668c6473a
* `download_lines_with_urllib` is unused since it was introduced in 37c9fb7a59
* `determine_wellknown_cmd` is unused since 76c090145e
ACKs for top commit:
fjahr:
ACK facaf56214
bensig:
ACK facaf56214 straightforward
l0rinc:
Reproduced it locally, ACK facaf56214
Tree-SHA512: c7d4880944b57108429e31541f1b97a98101cc06f1be716d6028f4193b6f087d9af06fa87e95ab224e027d574c6bd793cf06e540cdb8997805e1470e0c4e77a2
2a746500fa ci: migrate some jobs to Debian Trixie, use GCC 14 (fanquake)
fb0e6edfe8 guix: Apply SSA generation patch to maintain determinism (Mara van der Laan)
34909799fe guix: use GCC 14.3.0 over 13.3.0 (fanquake)
47be9122a7 guix: disable gprofng in GCC (fanquake)
ea29329eb7 guix: build GCC with --enable-host-bind-now (fanquake)
6f54e267d0 guix: disable libquadmath in GCC (fanquake)
7735901ed2 guix: disable building libgomp in GCC (fanquake)
Pull request description:
Switching to using GCC 14.x for release builds has come up multiple times recently. It will eventually be needed for #25573, and could also be useful for #30210.
ACKs for top commit:
hebasto:
ACK 2a746500fa. I have reviewed the code and it looks OK. The new GCC patch looks reasonable.
theuni:
utACK 2a746500fa
sedited:
ACK 2a746500fa
Tree-SHA512: 56912bed19386f06d52fb94e0ef6d96f5415ab2de8b5e94890806d7cc0b937a3c4b11cc161aa2e06ca2fd3c392ef7501c91688e0897e1c1c51aafa963f3e50d9
ab41492c6b test: Prevent loop from running out of utxos in bip68 test (Fabian Jahr)
Pull request description:
This tries to fix#34205
I stared at the test code quite a bit and initially suspected some `MiniWallet` internals to be the issue but I think that was the wrong direction and there is simply a very small chance that the loop in `test_sequence_lock_confirmed_inputs` runs out of available utxos: We are starting out with 200-250 utxos and run the loop 400 times. If a transaction is accepted it could have up to 10 inputs but it always has only one output, so the pool is depleting in this case. And it's actually even worse because the output produced is not recognized as spendable by the `MiniWallet` because it is not using the correct output script. However, only a small fraction of transactions are actually accepted, which is why this issue almost never occurs. I did some extra printing and usually we end up with >100 utxos still available by the end of the test. But there is a small chance that too many transactions are accepted and then we can run out of utxos.
I considered two fixes: The first was a break at the beginning of the loop `if available_utxos == 0: break`, this would work fine but I went with the second option: Simply creating the output with the correct output script so that `MiniWallet` recognizes it as spendable. This minimal replentishment of available utxos ensures that at worst we should get a few 1 input, 1 ouput transactions by the end but we should never run out of available utxos. I didn't look back in history but I suspect that this is how it was intended before `MiniWallet` introduced.
Also moves the `random` import in the same function to the top of the file.
ACKs for top commit:
maflcko:
lgtm ACK ab41492c6b
bensig:
ACK ab41492c6b
darosior:
ACK ab41492c6b
Tree-SHA512: d3ce56b669d011257a4a6967923f56011dbd03362576f564b29464639391851a09113f84b5ca2902911be7aa0923ccc9f402d13e6d673fd089dfe2b1f113ae4d
fa8d56f9f0 fuzz: Reject too large descriptor leaf sizes in scriptpubkeyman target (MarcoFalke)
fabac1b395 fuzz: Reject some more "expensive" descriptors in the scriptpubkeyman target (MarcoFalke)
333333356f fuzz: [refactor] Use std::span over FuzzBufferType in descriptor utils (MarcoFalke)
Pull request description:
Accepting "expensive" fuzz inputs which have no real use-case is problematic, because it prevents the fuzz engine from spending time on the next useful fuzz input.
Also, this may lead to problems, where the fuzz target can not be run at all on some platforms. See https://github.com/bitcoin/bitcoin/issues/34110.
Fixes https://github.com/bitcoin/bitcoin/issues/34110 by rejecting those useless and expensive inputs (via the third commit)
Can be tested by running the input and checking the time before and after the changes here:
```
curl -fLO '1cf91e0c6b'
FUZZ=scriptpubkeyman time ./bld-cmake/bin/fuzz ./1cf91e0c6bfff9dafcd4db5b0ba36b1e906f4cf5
```
Also, the second commit fixes https://github.com/bitcoin/bitcoin/issues/31066.
ACKs for top commit:
brunoerg:
code review ACK fa8d56f9f0
marcofleon:
ACK fa8d56f9f0
sipa:
ACK fa8d56f9f0
Tree-SHA512: e683cb89c3047358add438508c173f1cf647827bcadc3564ad42c757e4c99b8e9b777213fd38ebeb46f4c89a72363e0642f47435e20df3960eaeb5b8257dbd32
6bb66fcccb test: Improve code coverage for pubkey checks (billymcbip)
Pull request description:
Cover these branches in `IsCompressedOrUncompressedPubKey` and `IsCompressedPubKey`:
- `Non-canonical public key: invalid length for uncompressed key`
- `Non-canonical public key: invalid length for compressed key`
- `Non-canonical public key: invalid prefix for compressed key`
See the missed branches here: https://maflcko.github.io/b-c-cov/total.coverage/src/script/interpreter.cpp.gcov.html
`script_tests` succeed on my end.
ACKs for top commit:
maflcko:
ACK 6bb66fcccb🌑
rkrux:
code review ACK 6bb66fcccb
darosior:
ACK 6bb66fcccb
Tree-SHA512: f9b8acdc8bbe95559d594e74ed721d27be715754717b1557796168a6e81ce56d5bc20c40da4c0906ef9e1edcd88f202f000e34d8331d9be8d2694067a98996c6
997e7b4d7c init: Fix non-zero code on interrupt (sedited)
Pull request description:
Reported by dergoegge on irc.
An interrupt does not create a failure exit code during normal operation. This should also be the case when interrupt is triggered during initialization. However a failure exit code is currently returned if an interrupt occurs during init. Fix this by making `AppInitMain` return true instead of false on interrupt, which further up the call stack currently sets the `EXIT_FAILURE` code. Also add a check for the interrupt condition during GUI startup. Returning `EXIT_SUCCESS` seems to be the usual behaviour for daemons, see the discussion on IRC for this: https://www.erisian.com.au/bitcoin-core-dev/log-2026-01-08.html#l-146 .
Best reviewed with `--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space`.
ACKs for top commit:
maflcko:
review ACK 997e7b4d7c🔺
janb84:
ACK 997e7b4d7c
dergoegge:
utACK 997e7b4d7c
Tree-SHA512: c9542e95d9312567e029426a329144b5bc638d8ebc9c966e0246c1bb728d40f56ca425b00c446f5d238067e629c2337d0fe78bcc5a8760424d2ec38a5578e115
8ac134be5e contrib: verify-commits sha1 exceptions (Ava Chow)
Pull request description:
Allow specific commits to not require the sha1 check.
Fixes the CI for aeaa67a9ea which uses SHA1 somewhere in the key's self signatures.
ACKs for top commit:
dergoegge:
utACK 8ac134be5e
darosior:
utACK 8ac134be5e. Looks good to me provided we want to avoid the temporary red CI on master branch.
Tree-SHA512: cafd080ee35e4f735cbe9281ac2a73488c2f9e4acccd39563e933a3775def35b6e8080a6a772e45e64d90275013e430765a29e87116130ff456257c5ec8a2810
c5825d4b7f qa: Require `--exclude` for each excluded test (Hennadii Stepanov)
Pull request description:
This PR allows a long `--exclude ...` argument in the `test/functional/test_runner.py` invocation to be split across multiple lines, with optional per-line explanatory comments. I found this useful for the CI scripts in https://github.com/hebasto/bitcoin-core-nightly.
ACKs for top commit:
l0rinc:
tested ACK c5825d4b7f
maflcko:
review ACK c5825d4b7f🛄
achow101:
ACK c5825d4b7f
rkrux:
ACK c5825d4b7f
Tree-SHA512: bcf42848516197978b65df8a8bc68e036a62c9afc6158274eac74a325dc01991eb063a042f940c53ea15a7feb18d4bdfc45d8c71f0ef20c76140b12e07ba3ac5
- We no longer enforce ancestor/descendant count limit
in both PreChecks and PackageMempoolChecks.
- This commit fixes the incorrect comment by just renaming
`PackageMempoolChecks` to `PackageRBFChecks`
- The method name is self explanatory now; hence no need
for a description comment.
48f57bb35b mining: add new getCoinbaseTx() returning a struct (Sjors Provoost)
d59b4cdb57 mining: rename getCoinbaseTx() to ..RawTx() (Sjors Provoost)
Pull request description:
The first commit renames `getCoinbaseTx()` to `getCoinbaseRawTx()` to reflect that it returns a serialised transaction. This does not impact IPC clients, because they do not use the function name.
The second commit then introduces a replacement `getCoinbase()` that provides a struct with everything clients need to construct a coinbase. This avoids clients having to parse and manipulate our dummy transaction.
Deprecate but don't remove `getCoinbaseRawTx()`, `getCoinbaseCommitment()` and `getWitnessCommitmentIndex()`.
After this change we can drop these deprecated methods, which in turn would allow us to clear the dummy transaction from the `getBlock()` result. But that is left for a followup to keep this PR focussed. See https://github.com/Sjors/bitcoin/pull/106 for an approach.
Expand the `interface_ipc.py` functional test to document its usage.
Can be tested using:
- https://github.com/stratum-mining/sv2-tp/pull/59
ACKs for top commit:
ryanofsky:
Code review ACK 48f57bb35b. Just rebased and addressed comments and dropped coinbase tx "template" suffix, which is a nice change
ismaelsadeeq:
code review ACK 48f57bb35b
vasild:
ACK 48f57bb35b
Tree-SHA512: c4f1d752777fb3086a1a0b7b8b06e4205dbe2f3adb41f218855ad1dee952adccc263cf82acd3bf9300cc83c2c64cebd2b27f66a69beee32d325b9a85e3643b0d
This makes the code more consistent.
Also, use "using BenchFunction = ..." while touching the header.
Also, fixup the whitespace after and earlier scripted-diff.
Duplicate benchmarks with the same name are not supported. Expanding the
name with __LINE__ is confusing and brittle, because it makes duplication
bugs silent.
Fix this twofold:
* By enforcing unique benchmarks at compile-time and link-time. For
example, a link failure may now look like:
"mold: error: duplicate symbol: bench_runner_AddrManAdd"
* By enforcing unique benchmarks at run-time. This should never happen,
due to the build-failure, but a failure may look like:
"Assertion `benchmarks().try_emplace(std::move(name), std::move(func)).second' failed."
The Restore Wallet dialog rejects wallet names that are empty, but was
doing so silently. This is confusing, we should be presenting an error
message to the user.
After the normal optimization process finishes, and finds an optimal
spanning forest, run a second process (while computation budget remains)
to split chunks into minimal equal-feerate chunks.
8937221304 doc: add release notes for 29415 (Vasil Dimov)
582016fa5f test: add unit test for the private broadcast storage (Vasil Dimov)
e74d54e048 test: add functional test for private broadcast (Vasil Dimov)
818b780a05 rpc: use private broadcast from sendrawtransaction RPC if -privatebroadcast is ON (Vasil Dimov)
eab595f9cf net_processing: retry private broadcast (Vasil Dimov)
37b79f9c39 net_processing: stop private broadcast of a transaction after round-trip (Vasil Dimov)
2de53eee74 net_processing: handle ConnectionType::PRIVATE_BROADCAST connections (Vasil Dimov)
30a9853ad3 net_processing: move a debug check in VERACK processing earlier (Vasil Dimov)
d1092e5d48 net_processing: modernize PushNodeVersion() (Vasil Dimov)
9937a12a2f net_processing: move the debug log about receiving VERSION earlier (Vasil Dimov)
a098f37b9e net_processing: reorder the code that handles the VERSION message (Vasil Dimov)
679ce3a0b8 net_processing: store transactions for private broadcast in PeerManager (Vasil Dimov)
a3faa6f944 node: extend node::TxBroadcast with a 3rd option (Vasil Dimov)
95c051e210 net_processing: rename RelayTransaction() to better describe what it does (Vasil Dimov)
bb49d26032 net: implement opening PRIVATE_BROADCAST connections (Vasil Dimov)
01dad4efe2 net: introduce a new connection type for private broadcast (Vasil Dimov)
94aaa5d31b init: introduce a new option to enable/disable private broadcast (Vasil Dimov)
d6ee490e0a log: introduce a new category for private broadcast (Vasil Dimov)
Pull request description:
_Parts of this PR are isolated in independent smaller PRs to ease review:_
* [x] _https://github.com/bitcoin/bitcoin/pull/29420_
* [x] _https://github.com/bitcoin/bitcoin/pull/33454_
* [x] _https://github.com/bitcoin/bitcoin/pull/33567_
* [x] _https://github.com/bitcoin/bitcoin/pull/33793_
---
To improve privacy, broadcast locally submitted transactions (from the `sendrawtransaction` RPC) to the P2P network only via Tor or I2P short-lived connections, or to IPv4/IPv6 peers but through the Tor network.
* Introduce a new connection type for private broadcast of transactions with the following properties:
* started whenever there are local transactions to be sent
* opened to Tor or I2P peers or IPv4/IPv6 via the Tor proxy
* opened regardless of max connections limits
* after handshake is completed one local transaction is pushed to the peer, `PING` is sent and after receiving `PONG` the connection is closed
* ignore all incoming messages after handshake is completed (except `PONG`)
* Broadcast transactions submitted via `sendrawtransaction` using this new mechanism, to a few peers. Keep doing this until we receive back this transaction from one of our ordinary peers (this takes about 1 second on mainnet).
* The transaction is stored in peerman and does not enter the mempool.
* Once we get an `INV` from one of our ordinary peers, then the normal flow executes: we request the transaction with `GETDATA`, receive it with a `TX` message, put it in our mempool and broadcast it to all our existent connections (as if we see it for the first time).
* After we receive the full transaction as a `TX` message, in reply to our `GETDATA` request, only then consider the transaction has propagated through the network and remove it from the storage in peerman, ending the private broadcast attempts.
The messages exchange should look like this:
```
tx-sender >--- connect -------> tx-recipient
tx-sender >--- VERSION -------> tx-recipient (dummy VERSION with no revealing data)
tx-sender <--- VERSION -------< tx-recipient
tx-sender <--- WTXIDRELAY ----< tx-recipient (maybe)
tx-sender <--- SENDADDRV2 ----< tx-recipient (maybe)
tx-sender <--- SENDTXRCNCL ---< tx-recipient (maybe)
tx-sender <--- VERACK --------< tx-recipient
tx-sender >--- VERACK --------> tx-recipient
tx-sender >--- INV/TX --------> tx-recipient
tx-sender <--- GETDATA/TX ----< tx-recipient
tx-sender >--- TX ------------> tx-recipient
tx-sender >--- PING ----------> tx-recipient
tx-sender <--- PONG ----------< tx-recipient
tx-sender disconnects
```
Whenever a new transaction is received from `sendrawtransaction` RPC, the node will send it to a few (`NUM_PRIVATE_BROADCAST_PER_TX`) recipients right away. If after some time we still have not heard anything about the transaction from the network, then it will be sent to 1 more peer (see `PeerManagerImpl::ReattemptPrivateBroadcast()`).
A few considerations:
* The short-lived private broadcast connections are very cheap and fast wrt network traffic. It is expected that some of those peers could blackhole the transaction. Just one honest/proper peer is enough for successful propagation.
* The peers that receive the transaction could deduce that this is initial transaction broadcast from the transaction originator. This is ok, they can't identify the sender.
---
<details>
<summary>How to test this?</summary>
Thank you, @stratospher and @andrewtoth!
Start `bitcoind` with `-privatebroadcast=1 -debug=privatebroadcast`.
Create a wallet and get a new address, go to the Signet faucet and request some coins to that address:
```bash
build/bin/bitcoin-cli -chain="signet" createwallet test
build/bin/bitcoin-cli -chain="signet" getnewaddress
```
Get a new address for the test transaction recipient:
```bash
build/bin/bitcoin-cli -chain="signet" loadwallet test
new_address=$(build/bin/bitcoin-cli -chain="signet" getnewaddress)
```
Create the transaction:
```bash
# Option 1: `createrawtransaction` and `signrawtransactionwithwallet`:
txid=$(build/bin/bitcoin-cli -chain="signet" listunspent | jq -r '.[0] | .txid')
vout=$(build/bin/bitcoin-cli -chain="signet" listunspent | jq -r '.[0] | .vout')
echo "txid: $txid"
echo "vout: $vout"
tx=$(build/bin/bitcoin-cli -chain="signet" createrawtransaction "[{\"txid\": \"$txid\", \"vout\": $vout}]" "[{\"$new_address\": 0.00001000}]" 0 false)
echo "tx: $tx"
signed_tx=$(build/bin/bitcoin-cli -chain="signet" signrawtransactionwithwallet "$tx" | jq -r '.hex')
echo "signed_tx: $signed_tx"
# OR Option 2: `walletcreatefundedpsbt` and `walletprocesspsbt`:
# This makes it not have to worry about inputs and also automatically sends back change to the wallet.
# Start `bitcoind` with `-fallbackfee=0.00003000` for instance for 3 sat/vbyte fee.
psbt=$(build/bin/bitcoin-cli -chain="signet" walletcreatefundedpsbt "[]" "[{\"$new_address\": 0.00001000}]" | jq -r '.psbt')
echo "psbt: $psbt"
signed_tx=$(build/bin/bitcoin-cli -chain="signet" walletprocesspsbt "$psbt" | jq -r '.hex')
echo "signed_tx: $signed_tx"
```
Finally, send the transaction:
```bash
raw_tx=$(build/bin/bitcoin-cli -chain="signet" sendrawtransaction "$signed_tx")
echo "raw_tx: $raw_tx"
```
</details>
---
<details>
<summary>High-level explanation of the commits</summary>
* New logging category and config option to enable private broadcast
* `log: introduce a new category for private broadcast`
* `init: introduce a new option to enable/disable private broadcast`
* Implement the private broadcast connection handling on the `CConnman` side:
* `net: introduce a new connection type for private broadcast`
* `net: implement opening PRIVATE_BROADCAST connections`
* Prepare `BroadcastTransaction()` for private broadcast requests:
* `net_processing: rename RelayTransaction to better describe what it does`
* `node: extend node::TxBroadcast with a 3rd option`
* `net_processing: store transactions for private broadcast in PeerManager`
* Implement the private broadcast connection handling on the `PeerManager` side:
* `net_processing: reorder the code that handles the VERSION message`
* `net_processing: move the debug log about receiving VERSION earlier`
* `net_processing: modernize PushNodeVersion()`
* `net_processing: move a debug check in VERACK processing earlier`
* `net_processing: handle ConnectionType::PRIVATE_BROADCAST connections`
* `net_processing: stop private broadcast of a transaction after round-trip`
* `net_processing: retry private broadcast`
* Engage the new functionality from `sendrawtransaction`:
* `rpc: use private broadcast from sendrawtransaction RPC if -privatebroadcast is ON`
* New tests:
* `test: add functional test for private broadcast`
* `test: add unit test for the private broadcast storage`
</details>
---
**This PR would resolve the following issues:**
https://github.com/bitcoin/bitcoin/issues/3828 Clients leak IPs if they are recipients of a transaction
https://github.com/bitcoin/bitcoin/issues/14692 Can't configure bitocoind to only send tx via Tor but receive clearnet transactions
https://github.com/bitcoin/bitcoin/issues/19042 Tor-only transaction broadcast onlynet=onion alternative
https://github.com/bitcoin/bitcoin/issues/24557 Option for receive events with all networks, but send transactions and/or blocks only with anonymous network[s]?
https://github.com/bitcoin/bitcoin/issues/25450 Ability to broadcast wallet transactions only via dedicated oneshot Tor connections
https://github.com/bitcoin/bitcoin/issues/32235 Tor: TX circuit isolation
**Issues that are related, but (maybe?) not to be resolved by this PR:**
https://github.com/bitcoin/bitcoin/issues/21876 Broadcast a transaction to specific nodes
https://github.com/bitcoin/bitcoin/issues/28636 new RPC: sendrawtransactiontopeer
---
Further extensions:
* Have the wallet do the private broadcast as well, https://github.com/bitcoin/bitcoin/issues/11887 would have to be resolved.
* Have the `submitpackage` RPC do the private broadcast as well, [draft diff in the comment below](https://github.com/bitcoin/bitcoin/pull/29415#pullrequestreview-2972293733), thanks ismaelsadeeq!
* Add some stats via RPC, so that the user can better monitor what is going on during and after the broadcast. Currently this can be done via the debug log, but that is not convenient.
* Make the private broadcast storage, currently in peerman, persistent over node restarts.
* Add (optional) random delay before starting to broadcast the transaction in order to avoid correlating unrelated transactions based on the time when they were broadcast. Suggested independently of this PR [here](https://github.com/bitcoin/bitcoin/issues/30471).
* Consider periodically sending transactions that did not originate from the node as decoy, discussed [here](https://github.com/bitcoin/bitcoin/pull/29415#discussion_r2035414972).
* Consider waiting for peer's FEEFILTER message and if the transaction that was sent to the peer is below that threshold, then assume the peer is going to drop it. Then use this knowledge to retry more aggressively with another peer, instead of the current 10 min. See [comment below](https://github.com/bitcoin/bitcoin/pull/29415#issuecomment-3258611648).
* It may make sense to be able to override the default policy -- eg so submitrawtransaction can go straight to the mempool and relay, even if txs are normally privately relayed. See [comment below](https://github.com/bitcoin/bitcoin/pull/29415#issuecomment-3427086681).
* As a side effect we have a new metric available - the time it takes for a transaction to reach a random node in the network (from the point of view of the private broadcast recipient the tx originator is a random node somewhere in the network). This can be useful for monitoring, unrelated to privacy characteristics of this feature.
---
_A previous incarnation of this can be found at https://github.com/bitcoin/bitcoin/pull/27509. It puts the transaction in the mempool and (tries to) hide it from the outside observers. This turned out to be too error prone or maybe even impossible._
ACKs for top commit:
l0rinc:
code review diff ACK 8937221304
andrewtoth:
ACK 8937221304
pinheadmz:
ACK 8937221304
w0xlt:
ACK 8937221304 with nit https://github.com/bitcoin/bitcoin/pull/29415#discussion_r2654849875
mzumsande:
re-ACK 8937221304
Tree-SHA512: d51dadc865c2eb080c903cbe2f669e69a967e5f9fc64e9a20a68f39a67bf0db6ac2ad682af7fa24ef9f0942a41c89959341a16ba7b616475e1c5ab8e563b9b96
44b12cdb11 doc: add 433 (Pay to Anchor) to bips.md (Sebastian Falbesoner)
Pull request description:
See PR https://github.com/bitcoin/bips/pull/1982 (merged recently) and #30352 (first released in v28.0).
ACKs for top commit:
instagibbs:
ACK 44b12cdb11
glozow:
ACK 44b12cdb11
Tree-SHA512: 0389bfd9b57a354ef15eff726783c83969586a73de52d27b6d3b042dba14c1361af29303b6ce42330a841310811a81c5e45b5c7d25e307787f1cfbfd75ab3e62
Use port=0 for dynamic port allocation in test framework components
to avoid "address already in use" errors from concurrent tests or
ports stuck in TIME_WAIT state from previous test runs.
Changes:
- socks5.py: Update conf.addr after bind() to reflect actual port
- p2p.py: Retrieve actual port after create_server() when port=0
- feature_proxy.py: Use port=0 for all SOCKS5 proxy servers
- feature_anchors.py: Use port=0 for onion proxy server
- No need to jump into the next subroutine when there is no conflict.
- This makes it clear why it is necessary to have two calls of
CheckMempoolPolicyLimts in both PackageMempoolChecks and after in
AcceptMultipleTransactionsInternal, there is a possibilty that we
we want to accept multiple transaction but they are not conflicting
with any in-mempool transaction, in that case also we want to check
that they do not bust the cluster limits.
2cade5d5d1 [miniminer] stop assuming ancestor fees >= self fees (glozow)
Pull request description:
These assertions exist to detect double-deducting values when we update descendants. However, negative fees are possible with `prioritisetransaction` so it doesn't make sense to check this.
Leave the check for sizes because those are never negative.
Fixes#34234
ACKs for top commit:
instagibbs:
ACK 2cade5d5d1
dergoegge:
utACK 2cade5d5d1
Tree-SHA512: 935bbc8bd9a0d508eea43bb49aa43c22735e3f2c1012598f6843e229c13b76e44f9fd3eb8b61c437fa0b32353b4e7b15afa3e31002bdfa382d3d711d16419fde
eeaf28dbe0 wallet: test: Failed migration cleanup (David Gumberg)
Pull request description:
Prior to https://github.com/bitcoin/bitcoin/pull/34156, an issue existed where if migration of a wallet with a relative pathname failed, the relatively specified path where the legacy wallet is would be deleted. This issue predates #32273, because the relative pathnames get stacked together, e.g. "../../", the copy conflict bug that caused migration to abort early instead of getting far enough to attempt clean-up that was fixed in #32273 is avoided.
This is a functional test demonstrating that we handle failed migration clean-up correctly for relatively-named wallets. To see the issue, you can backport this test onto 29.x: https://github.com/davidgumberg/bitcoin/tree/2026-01-07-rel-migration-test-backport
I've also added an absolute path failed migration cleanup test. WRT this and #34156, absolute paths exhibit similar behavior to unnamed wallets. Because of the name-conflict bug prior to #32273 an absolute-path migration would fail no matter what because migration would attempt to copy a file to a destination that already exists. But after #32273, absolute-path migration gets past there, and if it fails for some other reason, the same behavior that's fixed in #34156 occurs where the directory containing the wallet file is deleted.
ACKs for top commit:
achow101:
ACK eeaf28dbe0
furszy:
ACK eeaf28dbe0
rkrux:
lgtm ACK eeaf28dbe0
Tree-SHA512: ee366fe526d0328654a86c2e9e6f228ca81554c8f8a78c259fa7aab90f024f9e5694ecf3f1d188938355f4e6d351c5a6a8ad236701bdd0ce63005e5d42c15e15
An interrupt does not create a failure exit code during normal
operation. This should also be the case when interrupt is triggered
during initialization. However a failure exit code is currently returned
if an interrupt occurs during init. Fix this by making `AppInitMain` return
true instead of false, which further up the call stack sets the
`EXIT_FAILURE` code. Also add a check for the interrupt condition during
GUI startup.
301d9eea66 qt: Remove "Starting Block" from Peer Detail. Following Deprecation in `bitcoin#34197` (WakeTrainDev)
Pull request description:
the `startingheight` rpc field got deprecated in https://github.com/bitcoin/bitcoin/pull/34197
this pr removes it from peer detail
ACKs for top commit:
maflcko:
review lgtm ACK 301d9eea66
theStack:
ACK 301d9eea66
hebasto:
ACK 301d9eea66, I verified `forms/debugwindow.ui` using Qt Designer.
Tree-SHA512: b870b4cff8ead073a17d171c01c46fc7e750c0343b4578ffb63abc8f40b33abdf08beb6733fead5307ef5d48b078b60d29ac0e0e41190a98f50f92154f0878cf
They are exactly the same, but the descriptor utils should not prescribe
to use the FuzzBufferType. Using a dedicated type for them clarifies
that the utils are not tied to FuzzBufferType.
Also, while touching the lines, use `const` only where it is meaningful.
cbf0bd35bb test: migration, avoid backup name mismatch in default_wallet_failure (furszy)
Pull request description:
This is a possible test failure, pushing it in case the CI starts complaining.
The change affects only test code; no cpp logic is involved.
The `test_default_wallet_failure` migration test calls the function
`migrate_and_get_rpc()`, which sets the mock time internally. But, at the
same time, the test already caches the mock time value, to later use it
in the backup existence check.
Setting the mock time twice can lead to a name mismatch during the
mentioned check (diff timestamp == diff backup names), which could
cause the test to fail.
The fix is very simple, just need to call the migration RPC directly.
Since the test expects the migration to fail, `migrate_and_get_rpc()` is
unnecessary here. I'm surprised the CI hasn't complained about this yet.
ACKs for top commit:
achow101:
ACK cbf0bd35bb
bensig:
ACK cbf0bd35bb
Tree-SHA512: 10b43a491b8ad0c5bf53e423b7d7587fc631551bf5d598e145e1defe9d8e5786c0869a9aee26209e63ccafd828ece34fc40c75abe246c1301b9f17467d64ef28
The test calls migrate_and_get_rpc(), which sets mock time internally.
The caller caches a mock time value and later relies on it to predict the
backup filename, so setting the mock time again could cause a naming
mismatch.
Fix this by calling the migration RPC directly. Since the test expects the
migration to fail, migrate_and_get_rpc() is unnecessary here.
f78f6f1dc8 wallettool: do not use fs::remove_all in createfromdump cleanup (Ava Chow)
Pull request description:
As pointed out in https://github.com/bitcoin/bitcoin/pull/34156#issuecomment-3716728670, it is possible for `createfromdump` to also accidentally delete the entire wallets directory if the wallet name is the empty string and the dumpfile contains a checksum error.
This is also fixed by removing the files created by only removing the directory for named wallets, and avoiding the use of `fs::remove_all`.
ACKs for top commit:
waketraindev:
lgtm ACK f78f6f1dc8
polespinasa:
code review and tACK f78f6f1dc8
rkrux:
Code review and tACK f78f6f1dc8
willcl-ark:
ACK f78f6f1dc8
pablomartin4btc:
ACK f78f6f1dc8
Tree-SHA512: ff1e7668131ec3632c67d990c99e8fddff28605e7e553c7e20695e61017c88476c3636e22f2007e763a00d527e80e4d1d3d45409f6678d28729b8397430bfe7a
b7c34d08dd test: coverage for migration failure when last sync is beyond prune height (furszy)
82caa8193a wallet: migration, fix watch-only and solvables wallets names (furszy)
d70b159c42 wallet: improve post-migration logging (furszy)
f011e0f068 test: restorewallet, coverage for existing dirs, unnamed wallet and prune failure (furszy)
36093bde63 test: add coverage for unnamed wallet migration failure (furszy)
f4c7e28e80 wallet: fix unnamed wallet migration failure (furszy)
4ed0693a3f wallet: RestoreWallet failure, erase only what was created (furszy)
Pull request description:
Minimal fix for #34128.
The issue occurs during the migration of a legacy unnamed wallet
(the legacy "default" wallet). When the migration fails, the cleanup
logic is triggered to roll back the state, which involves erasing the
newly created descriptor wallets directories. Normally, this only
affects the parent directories of named wallets, since they each
reside in their own directories. However, because the unnamed
wallet resides directly in the top-level `/wallets/` folder, this
logic accidentally deletes the main directory.
The fix ensures that only the wallet.dat file of the unnamed wallet
is touched and restored, preserving the wallet in BDB format and
leaving the main `/wallets/` directory intact.
#### Story Line:
#32273 fixed a different set of issues and, in doing so, uncovered
this one.
Before the mentioned PR, backups were stored in the same directory
as the wallet.dat file. On a migration failure, the backup was then
copied to the top-level `/wallets/` directory. For the unnamed legacy
wallet, the wallet directory is the `/wallets/` directory, so the source
and destination paths were identical. As a result, we threw early in the
`fs::copy_file` call ([here](https://github.com/bitcoin/bitcoin/blob/29.x/src/wallet/wallet.cpp#L4572)) because the file already existed, as we
were trying to copy the file onto itself. This caused the cleanup logic
to abort early on and never reach the removal line.
#### Testing Notes:
Cherry-pick the test commit on top of master and run it. You will
see the failure and realize the reason by reading the test code.
ACKs for top commit:
achow101:
ACK b7c34d08dd
davidgumberg:
crACK b7c34d08dd
w0xlt:
ACK b7c34d08dd
willcl-ark:
ACK b7c34d08dd
Tree-SHA512: d0be14c0ed6417f999c3f2f429652c2407097d0cc18453c91653e57ae4b5375b327ad3b2553d9ea6ff46a3ae00cdbd5ab325b94eba763072c4fc5a773b85618b
- Refactor Descriptor::ToPrivateString() to allow descriptors with
missing private keys to be printed. Useful in descriptors with
multiple keys e.g tr() etc.
- The existing behaviour of listdescriptors is preserved as much as
possible, if no private keys are availablle ToPrivateString will
return false
This commit modifies the Pubkey providers to return the public string
if private data is not available.
This is setup for a future commit to make Descriptor::ToPrivateString
return strings with missing private key information.
Co-authored-by: rkrux <rkrux.connect@gmail.com>
ToPrivateString() behaviour will be modified in the following commits.
In order to keep the scope of this PR limited to the RPC behaviour,
this commit updates wallet migration to use 'Descriptor::HavePrivateKeys()'
in place of 'Descriptor::ToPrivateString()' to determine watchonly descriptors.
A follow-up PR can be opened to update migration logic to exclude
descriptors with some private keys from the watchonly migration wallet.
Previously, to determine if a desc is watchonly, `ToPrivateString()`, was used.
It returns `false` if there is at least one pubkey in the descriptor for which
the provider does not have a private key.
ToPrivateString() behaviour will change in the following commits to only
return `false` if no priv keys could be found for the pub keys in the descriptor.
HavePrivateKeys() is added here to replace the use of ToPrivateString() for determining
if a descriptor is 'watchonly'.
Co-authored-by: rkrux <rkrux.connect@gmail.com>
fafbc70d48 rpc: [wallet] Use unsigned type for tx version in sendall (MarcoFalke)
Pull request description:
It is confusing to parse the unsigned tx version as a signed type. Also, it makes it harder to use the integer sanitizer.
Can be tested via:
* Build with the flags `-DCMAKE_C_COMPILER='clang' -DCMAKE_CXX_COMPILER='clang++' -DSANITIZERS=undefined,integer,float-divide-by-zero`
* Set the existing suppressions: `export UBSAN_OPTIONS="suppressions=$(pwd)/test/sanitizer_suppressions/ubsan:print_stacktrace=1:halt_on_error=0:report_error_type=1"`
* Start the RPC server, e.g. `./bld-cmake/bin/bitcoin-qt -datadir=/tmp -regtest -server`
* Call the sendall RPC, e.g. `./bld-cmake/bin/bitcoin-cli -datadir=/tmp -regtest -named sendall '["bcrt1qlrt3xps4wxpfcjmljrayr2ualczmnfvd4vzdq3"]' fee_rate=1.234 version=-1`
Before:
```
src/wallet/rpc/spend.cpp:1470:42: runtime error: implicit conversion from type 'int' of value -1 (32-bit, signed) to type 'uint32_t' (aka 'unsigned int') changed the value to 4294967295 (32-bit, unsigned)
Invalid parameter, version out of range(1~3)
```
After:
```
JSON integer out of range
ACKs for top commit:
bensig:
ACK fafbc70d48
achow101:
ACK fafbc70d48
rkrux:
utACK fafbc70d48
theStack:
ACK fafbc70d48
Tree-SHA512: bb7cf54e9691ad2591646b138ffdfac95bf77c5234d489f4e4f2c60b41bdc14cdc18a030fecb0a6ac64e55e4c69b37835afd334f87d8a44b8df6cda053e8fefb
Because the default wallet has no name, the watch-only and solvables
wallets created during migration end up having no name either.
This fixes it by applying the same prefix name we use for the backup
file for an unnamed default wallet.
Before: watch-only wallet named "_watchonly"
After: watch-only wallet named "default_wallet_watchonly"
Right now, after migration the last message users see is "migration completed",
but the migration isn't actually finished yet. We still need to load the new wallets
to ensure consistency, and if that fails, the migration will be rolled back. This
can be confusing for users.
This change logs the post-migration loading step and if a wallet fails to load and
the migration will be rolled back.
The first test verifies that restoring into an existing empty directory
or a directory with no .dat db files succeeds, while restoring into a
dir with a .dat file fails.
The second test covers restoring into the default unnamed wallet
(wallet.dat), which also implicitly exercises the recovery path used
after a failed migration.
The third test covers failure during restore on a prune node. When
the wallet last sync was beyond the pruning height.
When migrating any legacy unnamed wallet, a failed migration would
cause the cleanup logic to remove its parent directory. Since this
type of legacy wallet lives directly in the main '/wallets/' folder,
this resulted in unintentionally erasing all wallets, including the
backup file.
To be fully safe, we will no longer call `fs::remove_all`. Instead,
we only erase the individual db files we have created, leaving
everything else intact. The created wallets parent directories are
erased only if they are empty.
As part of this last change, `RestoreWallet` was modified to allow
an existing directory as the destination, since we no longer remove
the original wallet directory (we only remove the files we created
inside it). This also fixes the restore of top-level default wallets
during failures, which were failing due to the directory existence
check that always returns true for the /wallets/ directory.
This bug started after:
f6ee59b6e2
Previously, the `fs::copy_file` call was failing for top-level wallets,
which prevented the `fs::remove_all` call from being reached.
Track what RestoreWallet creates so only those files and directories
are removed during a failure and nothing else. Preexisting paths
must be left untouched.
Note:
Using fs::remove_all() instead of fs::remove() in RestoreWallet does
not cause any problems currently, but the change is necessary for the
next commit which extends RestoreWallet to work with existing directories,
which may contain files that must not be deleted.
1808b5aaf7 clusterlin: remove unused FixLinearization (cleanup) (Pieter Wuille)
34a77138b7 txgraph: permit non-topological clusters to defer fixing (optimization) (Pieter Wuille)
3380e0cbb5 txgraph: use PostLinearize less prior to linearizing (Pieter Wuille)
62dd88624a txgraph: drop NEEDS_SPLIT_ACCEPTABLE (simplification) (Pieter Wuille)
01ffcf464a clusterlin: support fixing linearizations (feature) (Pieter Wuille)
Pull request description:
Part of #30289, follow-up to #32545.
This gets rid of `FixLinearization()` by integrating the functionality into `Linearize()`, and makes txgraph exploit that (by delaying fixing of clusters until their first re-linearization). It also reduces (but does not eliminate) the number of calls to `PostLinearize`, as the SFL linearization effectively performs something very similar to postlinearization when loading in an existing linearization already.
ACKs for top commit:
instagibbs:
reACK 1808b5aaf7
marcofleon:
code review ACK 1808b5aaf7
Tree-SHA512: 81cd9549de2968f5126079cbb532e2cb052ea8157c9c9ce37fd39ad2294105d7c79ee8d946c3d8f7af5b2119299a232c448b42a33e1e43ccc778a5b52957e387
5b7bf47f9b doc: p2p: replace last remaining "command" terminology with "message type" (Sebastian Falbesoner)
Pull request description:
This small PR is (presumably) the final one in a long series of replacing the confusing "command" terminology with "message type" when referring to the header field of P2P messages, see #18533, #18937, #24078, #24141 and #31163.
The instances were found manually via `$ git grep -i command`, hope I didn't miss any.
ACKs for top commit:
l0rinc:
ACK 5b7bf47f9b
billymcbip:
ACK 5b7bf47f9b
maflcko:
lgtm ACK 5b7bf47f9b
Tree-SHA512: b895873b82f904c2ee9a81b4a2fbb365b60c57f04587ded5ddc7907d209520acb6073f5dd1a19cb2ae6aadab3c85a5ac751c8c398ce7c0e29314eea54e61295c
31852057ea test: fix intermittent failure in p2p_addr_selfannouncement (0xb10c)
Pull request description:
Due to the mocktime being bumped before the expected time is updated, it could happen that the self-announcement is send with an newer timestamp than what we expect. To fix this, update the expected time before we bump the mocktime.
closes#34159
ACKs for top commit:
bensig:
ACK 31852057ea
maflcko:
lgtm ACK 31852057ea
w0xlt:
ACK 31852057ea
naiyoma:
utACK 31852057ea
Tree-SHA512: 24696f6005c7131d4c9328f6ff43ddded863b8ba6b2cac6f6009bcb4617616c0c35a0b55812d5010f74385d8e6d4ea09dd2b06b5f4ada2bb7e86d7abee764192
fa65bc0e79 test: Run bench sanity checks in parallel with functional tests (MarcoFalke)
fa9fdbce79 test: Pass bench exe into test framework utils (MarcoFalke)
Pull request description:
The ctest target `bench_sanity_check` has many issues:
* With sanitizers enabled, it is one of the slowest targets, often taking several minutes. See https://github.com/bitcoin/bitcoin/issues/32770#issuecomment-2984264066.
* There is no insight from ctest into how long each individual sanity check takes.
* On a timeout, or OOM issue, there is no insight into which sub-bench failed. The failure will generally just look like `75/153 Test #9: bench_sanity_check ...................***Failed 770.84 sec out of memory`
* Places that can't use ctest (like the Windows-cross CI task) have to explicitly run it, or risk forgetting to run it.
* All benchmarks are run sequentially, when they could run in parallel instead.
Both issues can lead to CI timeouts and leave CPU unused during testing.
Fix all issues by running it as part of the functional tests instead. This is similar to the rpcauth tests (https://github.com/bitcoin/bitcoin/pull/32881) and util tests [bitcoin-tx, and bitcoin-util] (https://github.com/bitcoin/bitcoin/pull/32697).
ACKs for top commit:
achow101:
ACK fa65bc0e79
l0rinc:
Tested ACK fa65bc0e79
janb84:
tACK fa65bc0e79
willcl-ark:
ACK fa65bc0e79
Tree-SHA512: d27e363b7896a7543a4ee8df41a56e58b74f07d4f296e2e5ee293fc91817d0be310e26905755fb94d44417d94fa29ad4cc5d4aa19e78d25d41bc2d9e0948c034
4ce3f4a265 rpc, net: deprecate `startingheight` field of `getpeerinfo` RPC (Sebastian Falbesoner)
Pull request description:
This PR deprecates the "startingheight" result field of the `getpeerinfo` RPC, following the discussion in #33990.
Rationale: the reported starting height of a peer in the VERSION message is untrusted, and it doesn't seem to be useful anymore (after #20624), so deprecating the corresponding field seems reasonable. After that, it can be removed, along with the `m_starting_height` field of the Peer / CNodeStats structs, as it is sufficient to show the reported height only once at connection in the debug log.
ACKs for top commit:
optout21:
crACK 4ce3f4a265
achow101:
ACK 4ce3f4a265
fjahr:
utACK 4ce3f4a265
rkrux:
crACK 4ce3f4a265
janb84:
cr ACK 4ce3f4a265
Tree-SHA512: b296a28d30084fd35c67a2162e85576e3365e5d6fffe5b1add500034c1850604ee8c37b61afe812bfab8a7cc20f6a9e22db445e3c371311a5f82a777e5700ebf
5805a8b540 psbt: detect invalid MuSig2 pubkeys in deserialization (rkrux)
Pull request description:
Throw error while deserializing PSBT if invalid pubkeys are passed
as a MuSig2 aggregate or participant.
Should fix#33999 & #34201 by throwing error at the very start while decoding
an invalid PSBT that should subsequently not allow the MuSig2
signing operation to take place, thereby avoiding the crash.
ACKs for top commit:
fjahr:
utACK 5805a8b540
achow101:
ACK 5805a8b540
Tree-SHA512: 4741db96b278e9f3d532e1873af9530a70bbc7a8d3625b9e1c07001acc472fc10cbb79995c16bc4d06cc568ef98fe8d2b8e8d87b617dc05d7554085ffb92dfef
With the new SFL algorithm, the process of loading an existing linearization into the
SFL state is very similar to what PostLinearize does. This means there is little benefit
to performing an explicit PostLinearize step before linearizing inside txgraph. Instead,
it seems better to use our allotted CPU time to perform more SFL optimization steps.
With the SFL algorithm, we will practically be capable of keeping
most if not all clusters optimal. With that, it seems less valuable
to avoid doing work after splitting an acceptable cluster, because by
doing some work we may get it to OPTIMAL.
This reduces the complexity of the code a bit as well.
fac5a1b10a test: Allow mempool_updatefromblock.py to run on 32-bit (MarcoFalke)
Pull request description:
The number of dropped parent transactions in the `test_max_disconnect_pool_bytes` test was hard-coded to `2`.
This happens to work fine on 64-bit for now. However, it seems to fail on 32-bit (https://github.com/bitcoin/bitcoin/issues/34108).
I don't think we care about the exact number, as long as it is at least `1`.
So hard-code `1` for an initial sanity check, and then calculate the exact value at runtime via `len(mempool) // 2`.
Also, enable the functional tests in 32-bit CI, to confirm the regression test.
Fixes https://github.com/bitcoin/bitcoin/issues/34108
ACKs for top commit:
bensig:
ACK fac5a1b10a
instagibbs:
ACK fac5a1b10a
Tree-SHA512: 8d468f306d95e52cbfac1803293e3b8e9575c9010200010c7833382112509e0d51827dc9681b0b68eeae742af2c14d12da5fd4cf0e1d871a02f91fc80e6720d1
This makes sure the initial address self-announcement a node sends to
a peer happends in a separate P2P message. This has benefits for both
inbound and outbound connections:
For inbound connections from a peer to us, previously, we might send
the self-announcement along with our response to a GETADDR request.
However, the self-announcement might replace an address from the
GETADDR response. This isn't clean.
For outbound connections from us to a peer, previously, it could have
happend that we send the self-announcement along with other addresses.
Since shortly after connection open, the peer might only have one
rate-limiting token for us, and the addresses are shuffeld on arrival,
it's possible that the self-announcement gets rate-limited. However,
note that these rate-limitings seem to be rare in practice.
This is inspired by and based on https://github.com/bitcoin/bitcoin/pull/33699#issuecomment-3462287763
Co-Authored-By: Anthony Towns <aj@erisian.com.au>
Due to the mocktime being bumped before the expected time is updated,
it could happen that the self-announcement is send with an newer
timestamp than what we expect. To fix this, update the expected time
before we bump the mocktime.
closes#34159
Introduce a new method intended to replace getCoinbaseRawTx(), which
provides a struct with everything clients need to construct a coinbase.
This is safer than providing a raw dummy coinbase that clients then have
to manipulate.
The CoinbaseTx data is populated during the dummy transaction generation
and stored in struct CBlockTemplate.
Expand the interface_ipc.py functional test to document its usage
and ensure equivalence.
fab300b378 test: Enable ruff E713 lint (MarcoFalke)
Pull request description:
Membership tests of the form `not item in stuff` may be confusing, because they could be read as `(not item) in stuff`, which is different.
So enable the ruff E713 lint, which should also help to avoid having to go through review cycles for this.
ACKs for top commit:
bensig:
ACK fab300b378
l0rinc:
ACK fab300b378
rkrux:
lgtm crACK fab300b378
Tree-SHA512: c3eaf0fbe0dd22d8e04b896e98adaf28162fb748e6f7f5ebfd73b2020da66046bf8f0c1a27db5da05250366b98ded8c4a55d53edd8fa050e80521aee42ba3c5a
The reported starting height of a peer in the VERSION message is
untrusted, and it doesn't seem to be useful anymore (after #20624),
so deprecating the corresponding "startingheight" field seems
reasonable. After that, it can be removed, along with the
`m_starting_height` field of the Peer / CNodeStats structs, as it is
sufficient to show the reported height only once at connection in the
debug log.
6da6f503a6 refactor: Let CCoinsViewCache::BatchWrite return void (TheCharlatan)
Pull request description:
CCoinsViewCache::BatchWrite always returns true if called from a backed cache, so just return void instead. Also return void from ::Sync and ::Flush.
This allows for dropping a FatalError condition and simplifying some dead error handling code a bit.
Since we now no longer exercise the "error path" when returning from `CCoinsView::BatchWrite`, make the method clear the cache instead. This should only be exercised by tests and not change production behaviour. This might slightly improve the coins_view fuzz test's ability to generate better coverage.
ACKs for top commit:
l0rinc:
ACK 6da6f503a6
andrewtoth:
re-ACK 6da6f503a6
achow101:
ACK 6da6f503a6
w0xlt:
ACK 6da6f503a6
Tree-SHA512: dfaa325b0cf8108910aebf1b27434aaddb639d10d860e96797c77ea42eca9035a54a7dc1d6a5d4eae2b75fcc9356206d3d5672243d2c906e80d19024c8b95408
76c092ff80 wallet: warn against accidental unsafe older() import (Sjors Provoost)
592157b759 test: move SEQUENCE_LOCKTIME flags to script (Sjors Provoost)
Pull request description:
[BIP 379](https://github.com/bitcoin/bips/blob/master/bip-0379.md) ([Miniscript](https://bitcoin.sipa.be/miniscript/)) allows relative height and time locks that have no consensus meaning in [BIP 68](https://github.com/bitcoin/bips/blob/master/bip-0068.mediawiki) (relative timelocks) / [BIP 112](https://github.com/bitcoin/bips/blob/master/bip-0112.mediawiki) (`CHECKSEQUENCEVERIFY`). This is (ab)used by some protocols, e.g. [by Lightning to encode extra data](https://delvingbitcoin.org/t/exploring-extended-relative-timelocks/1818/23), but is unsafe when used unintentionally: `older(65536)` is equivalent to `older(1)`.
This PR emits a warning when `importdescriptors` contains such a descriptor.
The first commit makes `SEQUENCE_LOCKTIME` flags reusable by other tests.
The main commit adds the `ForEachNode` helper to `miniscript.h` which is then used in the `MiniscriptDescriptor` constructor to check for `Fragment::OLDER` with unsafe values. These are stored in `m_warnings`, which the RPC code then collects via `Warnings()`.
It adds both a unit and functional test.
---
A previous version of this PR prevented the import, unless the user opted in with an `unsafe` flag. It also used string parsing in the RPC code.
---
Based on:
- [x] https://github.com/bitcoin/bitcoin/pull/33914
ACKs for top commit:
pythcoiner:
reACK 76c092ff80
achow101:
ACK 76c092ff80
rkrux:
lgtm re-ACK 76c092ff80
brunoerg:
reACK 76c092ff80
Tree-SHA512: 8e944e499bd4a43cc27eeb889f262b499b9b07aa07610f4a415ccb4e34a9110f9946646f446a54ac5bf17494d8d96a89e4a1fa278385db9b950468f27283e17a
77c9b3c08f change test_runner.py to be cwd independent by calling subprocess.run with cwd arg. (Robin David)
Pull request description:
Dear Maintainers,
While using `test_runner.py` that runs fuzz tests and produces coverage results I encountered the following error.
If not running the script from the project root directory the `git grep --function-context [...]` does not return the same output which results in the following Python error:
```
../../src/protocol.h-', '../../../src/protocol.h-/** nServices flags */']
Traceback (most recent call last):
File "/path/to/bitcoin/build_libfuzzer/test/fuzz/./test_runner.py", line 405, in <module>
main()
~~~~^^
File "/path/to/bitcoin/build_libfuzzer/test/fuzz/./test_runner.py", line 173, in main
return generate_corpus(
fuzz_pool=fuzz_pool,
...<3 lines>...
targets=test_list_selection,
)
File "/path/to/bitcoin/build_libfuzzer/test/fuzz/./test_runner.py", line 249, in generate_corpus
targets = transform_process_message_target(targets, Path(src_dir))
File "/path/to/build_libfuzzer/test/fuzz/./test_runner.py", line 218, in transform_process_message_target
assert len(lines)
~~~^^^^^^^
AssertionError
```
The script is not able to retrieve lines as the filter applied is:
```python
lines = [l.split("::", 1)[1].split(",")[0].lower() for l in lines if l.startswith("src/protocol.h- NetMsgType::")]
```
Which when running from the root directory returns:
```
[snip]
src/protocol.h- NetMsgType::VERSION,
[snip]
```
but returns a relative path to CWD when run from other directories e.g:
```
../../../src/protocol.h- NetMsgType::VERSION,
```
This is very unfortunate as the script rightfully read the `config.ini` relatively to itself and go fetch `BUILDDIR` and `SRCDIR` variables to obtain absolute paths.
Options are:
* enforce running the script from *bitcoin/* directory (and thus explicitly mentioning it in the doc)
* make the script independent from where it is being run
I chose the second option as it was fairly easy to make the script independent from where it is being run.
ACKs for top commit:
maflcko:
lgtm ACK 77c9b3c08f
dergoegge:
Code review ACK 77c9b3c08f
Tree-SHA512: fbc821c4790dd9ac125046a842498e0d9a48549d1c8ef150bce2193ee62bee9c3bfd4b17ce278411102dd200dc9ad86a176ecae29ca1667bb14d6f90ad67e01d
658d38106a policy: remove constant parameter from `IsWellFormedPackage` (Lőrinc)
Pull request description:
`IsWellFormedPackage()` already claims: "parents must appear before children." In practice the `require_sorted` argument was always passed as `true`, making the false-path dead code. It was introduced that way from the beginning in https://github.com/bitcoin/bitcoin/pull/28758/files#diff-f30090b30c9489972ee3f1181c302cf3a484bb890bade0fd7c9ca92ea8d347f6R79.
Remove the unused parameter, updating callers/tests.
ACKs for top commit:
billymcbip:
tACK 658d38106a
instagibbs:
ACK 658d38106a
Tree-SHA512: 8b86dda7e2e1f0d48947ff258f0a3b6ec60676f54d4b506604d24e15c8b6465358ed2ccf174c7620125f5cad6bfc4df0bc482d920e5fc4cd0e1d72a9b16eafa5
95ef0fc5e7 test: ensure clean orphanage before continuing (Greg Sanders)
25e84d3772 test: change low fee parents to 0-fee (Greg Sanders)
Pull request description:
Resolves https://github.com/bitcoin/bitcoin/issues/33318 in a minimal fashion. Given that the orphan transactions aren't being persisted anymore, I'm not that specific case offers much coverage, but kept it around for now to get rid of the timeouts at least.
ACKs for top commit:
glozow:
utACK 95ef0fc5e7
Tree-SHA512: 4952062cb46b0e9f665de454718d093d3eac17532e4330caf80290f82b130614db3ccc5e5abf06f1e66237b9ba53ecdd0d13e4d5b09812f5c91db00b948ebb6b
08ed802bab doc: fix double-word typos in comments (bensig)
Pull request description:
Spotted a few duplicated words while reading through the code:
1. "the the" in mempool_stress.cpp
2. "to to" in txgraph.cpp
3. "for for" in cluster_linearize.h
4. "that that" in txrequest.h
5. "in in" in test/fuzz/txgraph.cpp
ACKs for top commit:
l0rinc:
ACK 08ed802bab
maflcko:
lgtm ACK 08ed802bab
Tree-SHA512: e4eeb9a95489b4c46fbb7a0dbeb549d70a2b087ab6400cc6ba89cbfc015b40e580fab8a68913499af7c83a988e66642dcc7a222b70d2eda5c57f4a02b5a556ae
The tests were written assuming transaction orphans would
persist for a time beyond the test peer's disconnection.
After #31829 this no longer holds, so as a minimal fix we
modify the test to wait until the orphans are removed before
continuing with the final transaction submissions.
The test is harder to read, and had an explicit 1sat/vbyte
floor assumption in a single place which is incorrect. Using
0-fee makes the test more future proof.
fab1f4b800 rpc: [mempool] Remove erroneous Univalue integral casts (MarcoFalke)
Pull request description:
Casting without reason can only be confusing (because it is not needed), or wrong (because it does the wrong thing).
For example, the added test that adds a positive chunk prioritization will fail:
```
AssertionError: not(-1.94936096 == 41.000312)
```
Fix all issues by removing the erroneous casts, and by adding a test to check against regressions.
ACKs for top commit:
rkrux:
tACK fab1f4b800
pablomartin4btc:
ACK fab1f4b800
glozow:
ACK fab1f4b800
Tree-SHA512: b03c888ec07a8bdff25f7ded67f253b2a8edd83adf08980416e2ac8ac1b36ad952cc5828be833d19f64a55abab62d7a1c6f181bc5f1388ed08cc178b4aaec6ee
337b4a2369 Remove stale rationale paragraph (flack)
Pull request description:
It belonged to the note removed in #33892
ACKs for top commit:
instagibbs:
ACK 337b4a2369
Tree-SHA512: 3cb1d3b87aa42ff92130af10ce2c286c0d83cbfdf17096d47b540ffe8e1a9a4727aedb8d477599fbff0002d7e262a6a52549dcccfa38dbe61281c221cf26cae2
fa66e2d07a refactor: [rpc] Remove confusing and brittle integral casts (MarcoFalke)
Pull request description:
When constructing an UniValue from integral values, historically (long ago), in some cases casts where needed. With the current UniValue constructor, only very few are actually needed.
Keeping the unused casts around is:
* confusing, because code readers do not understand why they are needed
* brittle, because some may copy them into new places, where they will lead to hard-to-find logic bugs, such as the ones fixed in pull https://github.com/bitcoin/bitcoin/pull/34112
So fix all issues by removing them, except for a few cases, where casting was required:
* `ret.pushKV("coinbase", static_cast<bool>(coin->fCoinBase));`, or
* `static_cast<std::underlying_type_t<decltype(info.nServices)>>(info.nServices)`.
This hardening refactor does not fix any bugs and does not change any behavior.
ACKs for top commit:
sedited:
ACK fa66e2d07a
rkrux:
ACK fa66e2d07a
Tree-SHA512: 13c9c59ad021ea03cdabe10d58850cef96d792634c499e62227cc2e7e5cace066ebd9a8ef3f979eaba98cadf8a525c6e6df909a07115559c0450bd9fc3a9763e
44e006d438 [kernel] Expose reusable PrecomputedTransactionData in script valid (Josh Doman)
Pull request description:
This PR exposes a reusable `PrecomputedTransactionData` object in script validation using libkernel.
Currently, libkernel computes `PrecomputedTransactionData` each time `btck_script_pubkey_verify` is called, exposing clients to quadratic hashing when validating a transaction with multiple inputs. By externalizing `PrecomputedTransactionData` and making it reusable, libkernel can eliminate this attack vector.
I discussed this problem in [this issue](https://github.com/TheCharlatan/rust-bitcoinkernel/issues/46). The design of this PR is inspired by @sedited's comments.
The PR introduces three new APIs for managing the `btck_PrecomputedTransactionData` object:
```c
/**
* @brief Create precomputed transaction data for script verification.
*
* @param[in] tx_to Non-null.
* @param[in] spent_outputs Nullable for non-taproot verification. Points to an array of
* outputs spent by the transaction.
* @param[in] spent_outputs_len Length of the spent_outputs array.
* @return The precomputed data, or null on error.
*/
btck_PrecomputedTransactionData* btck_precomputed_transaction_data_create(
const btck_Transaction* tx_to,
const btck_TransactionOutput** spent_outputs, size_t spent_outputs_len) BITCOINKERNEL_ARG_NONNULL(1);
/**
* @brief Copy precomputed transaction data.
*
* @param[in] precomputed_txdata Non-null.
* @return The copied precomputed transaction data.
*/
btck_PrecomputedTransactionData* btck_precomputed_transaction_data_copy(
const btck_PrecomputedTransactionData* precomputed_txdata) BITCOINKERNEL_ARG_NONNULL(1);
/**
* Destroy the precomputed transaction data.
*/
void btck_precomputed_transaction_data_destroy(btck_PrecomputedTransactionData* precomputed_txdata);
```
The PR also modifies `btck_script_pubkey_verify` so that it accepts `precomputed_txdata` instead of `spent_outputs`:
```c
/**
* @brief Verify if the input at input_index of tx_to spends the script pubkey
* under the constraints specified by flags. If the
* `btck_ScriptVerificationFlags_WITNESS` flag is set in the flags bitfield, the
* amount parameter is used. If the taproot flag is set, the precomputed data
* must contain the spent outputs.
*
* @param[in] script_pubkey Non-null, script pubkey to be spent.
* @param[in] amount Amount of the script pubkey's associated output. May be zero if
* the witness flag is not set.
* @param[in] tx_to Non-null, transaction spending the script_pubkey.
* @param[in] precomputed_txdata Nullable if the taproot flag is not set. Otherwise, precomputed data
* for tx_to with the spent outputs must be provided.
* @param[in] input_index Index of the input in tx_to spending the script_pubkey.
* @param[in] flags Bitfield of btck_ScriptVerificationFlags controlling validation constraints.
* @param[out] status Nullable, will be set to an error code if the operation fails, or OK otherwise.
* @return 1 if the script is valid, 0 otherwise.
*/
int btck_script_pubkey_verify(
const btck_ScriptPubkey* script_pubkey,
int64_t amount,
const btck_Transaction* tx_to,
const btck_PrecomputedTransactionData* precomputed_txdata,
unsigned int input_index,
btck_ScriptVerificationFlags flags,
btck_ScriptVerifyStatus* status) BITCOINKERNEL_ARG_NONNULL(1, 3);
```
As before, an error is thrown if the taproot flag is set and `spent_outputs` is not provided in `precomputed_txdata` (or `precomputed_txdata` is null). For simple single-input non-taproot verification, `precomputed_txdata` may be null, and the kernel will construct the precomputed data on-the-fly.
Both the C++ wrapper and the test suite are updated with the new API. Tests cover both `precomputed_txdata` reuse and nullability.
Appreciate feedback on this concept / approach!
ACKs for top commit:
sedited:
Re-ACK 44e006d438
stringintech:
ACK 44e006d
Tree-SHA512: 1ed435173e6ff4ec82bc603194cf182c685cb79f167439a442b9b179a32f6c189c358f04d4cb56d153fab04e3424a11b73c31680e42b87b8a6efcc3ccefc366c
5646e6c0d3 index: restrict index helper function to namespace (Martin Zumsande)
032f3503e3 index, refactor: deduplicate LookUpOne (Martin Zumsande)
a67d3eb91d index: deduplicate Hash / Height handling (Martin Zumsande)
Pull request description:
The logic for `DBHashKey` / `DBHeightKey` handling and lookup of entries is shared by `coinstatsindex` and `blockfilterindex`, leading to many lines of duplicated code. De-duplicate this by moving the logic to `index/db_key.h` (using templates for the index-specific `DBVal`).
ACKs for top commit:
fjahr:
re-ACK 5646e6c0d3
furszy:
utACK 5646e6c0d3
sedited:
ACK 5646e6c0d3
Tree-SHA512: 6f41684d6a9fd9bb01239e9f2e39a12837554f247a677eadcc242f0c1a2d44a79979f87249c4e0305ef1aa708d7056e56dfc40e1509c6d6aec2714f202fd2e09
e44dec027c add release note about supporing non-TRUC <minrelay txns (Greg Sanders)
1488315d76 policy: Allow any transaction version with < minrelay (Greg Sanders)
Pull request description:
Prior to cluster mempool, a policy was in place that
disallowed non-TRUC transactions from being
TX_RECONSIDERABLE in a package setting if it was below
minrelay. This was meant to simplify reasoning about mempool
trimming requirements with non-trivial transaction
topologies in the mempool. This is no longer a concern
post-cluster mempool, so this is relaxed.
In effect, this makes 0-value parent transactions relayable
through the network without the TRUC restrictions and
thus the anti-pinning protections.
ACKs for top commit:
ajtowns:
ACK e44dec027c - lgtm
ismaelsadeeq:
ACK e44dec027c
Tree-SHA512: 6fd1a2429c55ca844d9bd669ea797e29eca3f544f0b5d3484743d3c1cdf4364f7c7a058aaf707bcfd94b84c621bea03228cb39487cbc23912b9e0980a1e5b451
fa727e3ec9 test: Avoid hard time.sleep(1) in feature_init.py (MarcoFalke)
Pull request description:
Using a hard-coded `time.sleep` in the tests is usually confusing and brittle. For example, the one in `break_wait_test`:
* Is confusing, because it does not explain why it is needed.
* On fast hardware will just lead to a useless delay.
* On slow hardware may lead to an intermittent, and confusing test failure.
Fix all issues by replacing it with the proper condition to wait on.
ACKs for top commit:
Sjors:
utACK fa727e3ec9
rkrux:
tACK fa727e3
janb84:
tACK fa727e3ec9
Tree-SHA512: 7b59496a1b9b8044548423ad517ff03e98521685cf65499cd0ef499d6fd3d72ad374c92ca815436675ed6ae7be508a5a1afce699b804a384d7aee6a195d8d972
This function is a duplicate of HasEncryptionKeys().
-BEGIN VERIFY SCRIPT-
sed -i '/bool IsCrypted() const;/d' src/wallet/wallet.h
sed -i '/^bool CWallet::IsCrypted() const$/,/^}$/{/^}$/N;d;}' src/wallet/wallet.cpp
sed -i --regexp-extended 's/IsCrypted\(\)/HasEncryptionKeys()/g' $(git ls-files '*.cpp' '*.h')
-END VERIFY SCRIPT-
217dbbbb5e test: Add musig failure scenarios (Fabian Jahr)
c9519c260b musig: Check session id reuse (Fabian Jahr)
e755614be5 sign: Remove duplicate sigversion check (Fabian Jahr)
0f7f0692ca musig: Move MUSIG_CHAINCODE to musig.cpp (Fabian Jahr)
Pull request description:
This is a follow-up to #29675 and primarily adds test coverage for some of the most prominent failure cases in the last commit.
The following commits address a few left-over nit comments that didn't make it in before merge.
ACKs for top commit:
achow101:
ACK 217dbbbb5e
rkrux:
lgtm ACK 217dbbb
Tree-SHA512: d73807bc31791ef1825c42f127c7ddfbc70b2b7cf782bc11341666e32e86b787ffc7aed64caea992909cef3a85fc6629282d8209c173aadec77f72fd0da96c45
1ed8e76165 rpc, doc: clarify the response of listtransactions RPC (rkrux)
Pull request description:
I noticed this behaviour while perf testing PR #27286 and it was not something that I expected, updating the doc to make it present in the RPCHelp command.
ACKs for top commit:
achow101:
ACK 1ed8e76165
furszy:
ACK 1ed8e76165
musaHaruna:
ACK [1ed8e76](1ed8e76165) since my last review. New changes looks good, it's much easier to understand as well, looking at it from a user's perspective.
Tree-SHA512: 893a8e259201ac2140f46f827d81e681d2ec478c9571cceb10864aaa1b941991ce2263357d7c2b0024c04a9f8fbc372a020104b26e022c96289d271675947033
1841bf9cb6 test: address self-announcement (0xb10c)
Pull request description:
Test that a node sends a self-announcement with its external IP to in- and outbound peers after connection open and again sometime later.
Since the code for the test is mostly the same for addr and addrv2 messages, I opted to add a new test file instead of having duplicate code in `p2p_addr_relay.py` and `p2p_addrv2_relay.py`.
ACKs for top commit:
Bicaru20:
ACK 1841bf9cb6
achow101:
ACK 1841bf9cb6
rkrux:
ACK 1841bf9
fjahr:
Code review ACK 1841bf9cb6
Tree-SHA512: 692a01e9f10eb55ee870de623e85182a10a75225766e0f0251ad5d9e369537ec27ca6e06905374190f3afe00ba6f71ae72f262228baaa535238a87160e1ce4f1
56750c4f87 iwyu, clang-format: Sort includes (Hennadii Stepanov)
2c78814e0e ci: Add IWYU job (Hennadii Stepanov)
94e4f04d7c cmake: Fix target name (Hennadii Stepanov)
0f81e00519 cmake: Make `codegen` target dependent on `generate_build_info` (Hennadii Stepanov)
73f7844cdb iwyu: Add patch to prefer C++ headers over C counterparts (Hennadii Stepanov)
7a65437e23 iwyu: Add patch to prefer angled brackets over quotes for includes (Hennadii Stepanov)
Pull request description:
This PR separates the IWYU checks into its own CI job to provide faster feedback to developers. No other changes are made to the treatment of IWYU warnings. The existing “tidy” CI job will no longer run IWYU.
See also the discussion of https://github.com/bitcoin/bitcoin/pull/33779, specifically this [comment](https://github.com/bitcoin/bitcoin/pull/33779#issuecomment-3491515263):
> Maybe a better approach would be to run the enforced sections in a separate, faster job? Some of the linters are already a bit annoying to invoke locally, so I usually just run the lint job. Doing the same for the includes seems fine to me.
Based on ideas from https://github.com/bitcoin/bitcoin/pull/32953.
ACKs for top commit:
maflcko:
review ACK 56750c4f87🌄
sedited:
ACK 56750c4f87
Tree-SHA512: af15326b6d0c5d1e11346ac64939644936c65eb9466cd1a17ab5da347d39aef10f7ab33b39fbca31ad291b0b4b54639b147b24410f4f86197e4a776049882694
d7de5b109f logs: show reindex progress in `ImportBlocks` (Lőrinc)
Pull request description:
### Summary
When triggering a reindex, users have no indication of progress.
### Fix
This patch precomputes the total number of block files so progress can be shown.
Instead of only displaying which block file is being processed, it now shows the percent complete.
### Reproducer + expected results
```bash
cmake -B build -DCMAKE_BUILD_TYPE=Release && make -C build -j && ./build/bin/bitcoind -datadir=demo -reindex
```
Before, the block files were shown one-by-one, there's no way to see how much work is left:
```
Reindexing block file blk00000.dat...
Loaded 119920 blocks from external file in 1228ms
Reindexing block file blk00001.dat...
Loaded 10671 blocks from external file in 284ms
Reindexing block file blk00002.dat...
Loaded 5459 blocks from external file in 263ms
Reindexing block file blk00003.dat...
Loaded 5595 blocks from external file in 267ms
```
After the change we add a percentage:
```
Reindexing block file blk00000.dat (0% complete)...
Loaded 119920 blocks from external file in 1255ms
Reindexing block file blk00001.dat (1% complete)...
Loaded 10671 blocks from external file in 303ms
Reindexing block file blk00002.dat (2% complete)...
Loaded 5459 blocks from external file in 278ms
Reindexing block file blk00003.dat (3% complete)...
Loaded 5595 blocks from external file in 285ms
```
ACKs for top commit:
enirox001:
Concept ACK d7de5b1
rkrux:
lgtm ACK d7de5b109f
danielabrozzoni:
tACK d7de5b109f - code reviewed and tested on my archival node.
maflcko:
review ACK d7de5b109f💇
Tree-SHA512: 359a539b781ad8b73e2a616c951567062a76be27cf90e5b88bb5309295af9cd7994e327f185bacc1482b43b892b38329593b4043a5e71d8800e3e4b7a3954310
356883f0e4 qa-tests: Log expected output in debug (Hodlinator)
7427a03b5a qa-tests: Add test for timeouts due to missing init errors (Hodlinator)
d7f703c1f1 refactor(qa-tests): Extract InternalDurationTestMixin for use in next commit (Hodlinator)
69bcfcad8c fix(qa-tests): Bring back decoding of exception field (Hodlinator)
fb43b2f8cc qa: Improve assert_start_raises_init_error output (Hodlinator)
Pull request description:
Raising a new exception from within a Python `except`-block, as `assert_start_raises_init_error()` does, causes the interpreter to generate extra error output which is unnecessary in this case.
<details><summary>Example output before & after this PR</summary>
Before:
```
2025-07-08T20:05:48.407001Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
File "/home/hodlinator/bitcoin/test/functional/test_framework/test_node.py", line 686, in assert_start_raises_init_error
ret = self.process.wait(timeout=self.rpc_timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/fqm9bqqlmaqqr02qbalm1bazp810qfiw-python3-3.12.9/lib/python3.12/subprocess.py", line 1266, in wait
return self._wait(timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/fqm9bqqlmaqqr02qbalm1bazp810qfiw-python3-3.12.9/lib/python3.12/subprocess.py", line 2053, in _wait
raise TimeoutExpired(self.args, timeout)
subprocess.TimeoutExpired: Command '['/home/hodlinator/bitcoin/build/bin/bitcoind', '-datadir=/tmp/bitcoin_func_test_v96lkcq8/eb2665c7/node0', '-logtimemicros', '-debug', '-debugexclude=libevent', '-debugexclude=leveldb', '-debugexclude=rand', '-uacomment=testnode0', '-disablewallet', '-logthreadnames', '-logsourcelocations', '-loglevel=trace', '-v2transport=0']' timed out after 3 seconds
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/hodlinator/bitcoin/test/functional/test_framework/test_framework.py", line 186, in main
self.setup()
File "/home/hodlinator/bitcoin/test/functional/test_framework/test_framework.py", line 358, in setup
self.setup_network()
File "/home/hodlinator/bitcoin/build/test/functional/feature_framework_startup_failures.py", line 151, in setup_network
self.nodes[0].assert_start_raises_init_error()
File "/home/hodlinator/bitcoin/test/functional/test_framework/test_node.py", line 716, in assert_start_raises_init_error
self._raise_assertion_error(assert_msg)
File "/home/hodlinator/bitcoin/test/functional/test_framework/test_node.py", line 196, in _raise_assertion_error
raise AssertionError(self._node_msg(msg))
AssertionError: [node 0] bitcoind should have exited within 3s with an error
```
After:
```
2025-07-08T20:09:15.330589Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
File "/home/hodlinator/bitcoin/test/functional/test_framework/test_framework.py", line 186, in main
self.setup()
File "/home/hodlinator/bitcoin/test/functional/test_framework/test_framework.py", line 358, in setup
self.setup_network()
File "/home/hodlinator/bitcoin/build/test/functional/feature_framework_startup_failures.py", line 151, in setup_network
self.nodes[0].assert_start_raises_init_error()
File "/home/hodlinator/bitcoin/test/functional/test_framework/test_node.py", line 720, in assert_start_raises_init_error
self._raise_assertion_error(assert_msg)
File "/home/hodlinator/bitcoin/test/functional/test_framework/test_node.py", line 196, in _raise_assertion_error
raise AssertionError(self._node_msg(msg))
AssertionError: [node 0] bitcoind should have exited within 3s with an error (cmd: ['/home/hodlinator/bitcoin/build/bin/bitcoind', '-datadir=/tmp/bitcoin_func_test_v96lkcq8/eb2665c7/node0', '-logtimemicros', '-debug', '-debugexclude=libevent', '-debugexclude=leveldb', '-debugexclude=rand', '-uacomment=testnode0', '-disablewallet', '-logthreadnames', '-logsourcelocations', '-loglevel=trace', '-v2transport=0'])
```
</details>
---
Can be tested on this PR by:
1. Execute test containing new test case:
```shell
build/test/functional/feature_framework_startup_failures.py -ldebug > after.log
```
2. Drop first commit which contains the fix.
3. Re-run test:
```shell
build/test/functional/feature_framework_startup_failures.py -ldebug > before.log
```
4. Diff logs, focusing on `TestInitErrorTimeout OUTPUT` sections.
---
Found while testing #32835 using the suggested method (https://github.com/bitcoin/bitcoin/pull/32835#issue-3188748624) which triggered expected timeouts, but with the extra error noise.
ACKs for top commit:
l0rinc:
ACK 356883f0e4
ryanofsky:
Code review ACK 356883f0e4. Thanks for the updates! Just rearranged commits and made minor changes in "missing init errors" test since last review
furszy:
Code ACK 356883f0e4
Tree-SHA512: 01f2f1f6a5e79cf83a39a143cfb8b2bb8360e0402e91a97a7df8254309fd4436a55468d11825093c052010bfce57f3461d912a578cd2594114aba435ab48b999
d3a479cb07 kernel: Move BlockInfo to a kernel file (TheCharlatan)
d69a582e72 kernel: Remove some unnecessary non-kernel includes (TheCharlatan)
Pull request description:
Found these while attempting to isolate the kernel library sources into their own repository. There still is no mechanism for preventing including headers into the kernel library that don't belong to kernel modules, but it is also fairly straight forward to correct manually for now. However, the changes here might be incomplete.
ACKs for top commit:
hebasto:
re-ACK d3a479cb07.
maflcko:
review ACK d3a479cb07🦏
janb84:
ACK d3a479cb07
Tree-SHA512: b2a40aa758437a4e72648fe38ca308c0bea3a7d8559c62182cd3daa2858de62b7418afe4b9054ebdb88082036bc1691803c2b3b2dacd0ff2208a9ffdcba0e7e9
This should avoid having to include interfaces/chain.h from a kernel
module. interfaces/chain.h in turn includes a bunch of non-kernel
headers, that break the desired library topology and might introduce
entanglement regressions.
Specifically gets rid of batchpriority, chainparams, script/sign.h and
system includes.
Also take the opportunity of cleaning up the headers for the effected
files and adding them to the iwyu-enforced set.
fa4cb13b52 test: [doc] Manually unify stale headers (MarcoFalke)
fa5f297748 scripted-diff: [doc] Unify stale copyright headers (MarcoFalke)
Pull request description:
Historically, the upper year range in file headers was bumped manually
or with a script.
This has many issues:
* The script is causing churn. See for example commit 306ccd4, or
drive-by first-time contributions bumping them one-by-one. (A few from
this year: https://github.com/bitcoin/bitcoin/pull/32008,
https://github.com/bitcoin/bitcoin/pull/31642,
https://github.com/bitcoin/bitcoin/pull/32963, ...)
* Some, or likely most, upper year values were wrong. Reasons for
incorrect dates could be code moves, cherry-picks, or simply bugs in
the script.
* The upper range is not needed for anything.
* Anyone who wants to find the initial file creation date, or file
history, can use `git log` or `git blame` to get more accurate
results.
* Many places are already using the `-present` suffix, with the meaning
that the upper range is omitted.
To fix all issues, this bumps the upper range of the copyright headers
to `-present`.
Further notes:
* Obviously, the yearly 4-line bump commit for the build system (c.f.
b537a2c02a) is fine and will remain.
* For new code, the date range can be fully omitted, as it is done
already by some developers. Obviously, developers are free to pick
whatever style they want. One can list the commits for each style.
* For example, to list all commits that use `-present`:
`git log --format='%an (%ae) [%h: %s]' -S 'present The Bitcoin'`.
* Alternatively, to list all commits that use no range at all:
`git log --format='%an (%ae) [%h: %s]' -S '(c) The Bitcoin'`.
<!--
* The lower range can be wrong as well, so it could be omitted as well,
but this is left for a follow-up. A previous attempt was in
https://github.com/bitcoin/bitcoin/pull/26817.
ACKs for top commit:
l0rinc:
ACK fa4cb13b52
rkrux:
re-ACK fa4cb13b52
janb84:
ACK fa4cb13b52
Tree-SHA512: e5132781bdc4417d1e2922809b27ef4cf0abb37ffb68c65aab8a5391d3c917b61a18928ec2ec2c75ef5184cb79a5b8c8290d63e949220dbeab3bd2c0dfbdc4c5
1e94e562f7 refactor: enable `readability-container-contains` clang-tidy rule (Lőrinc)
fd9f1accbd Fix compilation for old Boost versions (Lőrinc)
Pull request description:
Replace the last few instances of `.count() != 0` and `.count() == 0` and bare `count()` patterns with the more expressive C++20 `.contains()` method:
* `std::set<std::string>` in `getblocktemplate` RPC;
* `std::map<std::string, ...>` in `transaction_tests`;
* other bare `std::unordered_set` and `std::map` count calls.
Also fixes https://github.com/bitcoin/bitcoin/issues/34101 by reverting `boost::multi_index::contains` calls not available in our minimum supported version.
With no remaining violations, enable the `readability-container-contains` clang-tidy check to prevent future regressions.
Follow-up to https://github.com/bitcoin/bitcoin/pull/33192
ACKs for top commit:
hebasto:
ACK 1e94e562f7.
pablomartin4btc:
re-ACK 1e94e562f7
janb84:
ACK 1e94e562f7
rkrux:
re-ACK 1e94e562f7
Tree-SHA512: d54a7821d319bf0d60b6c3a870917464a7d5b9279c6a86708c03a3516ec23bbf18f0e83de62b3b2b1607de96e1470f1144b4918d69a6c770e6b7e09863e7dbac
fa336053aa Move ci_exec to the Python script (MarcoFalke)
fa83555d16 ci: Require rsync to pass (MarcoFalke)
eeee02ea53 ci: Untangle CI_EXEC bash function (MarcoFalke)
fa21fd1dc2 ci: Move macos snippet under DANGER_RUN_CI_ON_HOST (MarcoFalke)
fa37559ac5 ci: Document the retry script in PATH (MarcoFalke)
666675e95f ci: Move folder creation and docker kill to Python script (MarcoFalke)
Pull request description:
The remaining `ci/test/02_run_container.sh` is fine, but has a bunch of shellcheck SC2086 word splitting violations.
This is fine currently, because the only place that needed them had additional escaping, and all other commands happened to split fine on spaces.
However, this may change in the future. So fix it now, by rewriting it in Python, which is recommended in the dev notes.
ACKs for top commit:
frankomosh:
Code Review ACK [fa33605](fa336053aa)
m3dwards:
ACK fa336053aa
Tree-SHA512: 472decb13edca75566dffe49b9b3f554ab977fa60ec7902d5a060fe53381aee8606a10ff0c990a62ee2454dc6d9430cc064f58320b9043070b7bf08845413bf4
75bdb925f4 clusterlin: drop support for improvable chunking (simplification) (Pieter Wuille)
91399a7912 clusterlin: remove unused MergeLinearizations (cleanup) (Pieter Wuille)
5ce2800745 clusterlin: randomize equal-feerate parts of linearization (privacy) (Pieter Wuille)
13aad26b78 clusterlin: randomize various decisions in SFL (feature) (Pieter Wuille)
ddbfa4dfac clusterlin: keep FIFO queue of improvable chunks (preparation) (Pieter Wuille)
3efc94d656 clusterlin: replace cluster linearization with SFL (feature) (Pieter Wuille)
6a8fa821b8 clusterlin: add support for loading existing linearization (feature) (Pieter Wuille)
da48ed9f34 clusterlin: ReadLinearization for non-topological (tests) (Pieter Wuille)
c461259fb6 clusterlin: add class implementing SFL state (preparation) (Pieter Wuille)
95bfe7d574 clusterlin: replace benchmarks with SFL-hard ones (bench) (Pieter Wuille)
86dd550a9b clusterlin: add known-correct optimal linearization tests (tests) (Pieter Wuille)
Pull request description:
Part of cluster mempool: #30289.
This replaces the cluster linearization algorithm introduced in #30126 and #30286 (a combination of LIMO with candidate-set search), with a completely different algorithm: [spanning-forest linearization](https://delvingbitcoin.org/t/spanning-forest-cluster-linearization/1419/1), which appears to have much better performance for hard clusters. See [this post](https://delvingbitcoin.org/t/how-to-linearize-your-cluster/303/68) for a comparison between various linearization algorithms, and [this post](https://delvingbitcoin.org/t/how-to-linearize-your-cluster/303/73) for benchmarks comparing them. Replaying historical mempool data on it shows that it can effectively linearize every observed cluster up to 64 transactions optimally within tens of microseconds, though pathological examples can be created which take longer.
The algorithm is effectively a very specialized version of the [simplex algorithm](https://en.wikipedia.org/wiki/Simplex_algorithm) to the problem of finding high-feerate topological subsets of clusters, but modified to find all consecutive such subsets concurrently rather than just the first one. See the post above for how it is related.
It represents the cluster as partitioned into a set of chunks, each with a spanning tree of its internal dependencies connecting the transactions. Randomized improvements are made by selecting dependencies to add and remove to these spanning trees, merging and splitting chunks, until no more improvements are possible, or a computation budget is reached. Like simplex, it does not necessarily make progress in every step, and thus has no upper bound on its runtime to find optimal, but randomization makes long runtimes very unlikely, and additionally makes it hard to adversarially construct clusters in which the algorithm reliably makes bad choices.
ACKs for top commit:
instagibbs:
reACK 75bdb925f4
marcofleon:
reACK 75bdb925f4
Tree-SHA512: 189d85b34f0eb847562af7da724c61e39f0a785e24ebe2d4c8ee44698d02bd17842d699987d282a79bd1de30f50de28ec0f11d594ebbfa499f6a9b9ce35aecd8
This frees up the name getCoinbaseTx() for the next commit.
Changing a function name does not impact IPC clients, as they only
consider the function signature and sequence number.
Replace the last few instances of `.count() != 0` and `.count() == 0` and `.count()` patterns with the more expressive C++20 `.contains()` method:
* `std::set<std::string>` in `getblocktemplate` RPC;
* `std::map<std::string, ...>` in `transaction_tests`;
* other bare `std::unordered_set` and `std::map` count calls.
With no remaining violations, enable the `readability-container-contains`
clang-tidy check to prevent future regressions.
With MergeLinearizations() gone and the LIMO-based Linearize() replaced by SFL, we do not
need a class (LinearizationChunking) that can maintain an incrementally-improving chunk
set anymore.
Replace it with a function (ChunkLinearizationInfo) that just computes the chunks as
SetInfos once, and returns them as a vector. This simplifies several call sites too.
This places equal-feerate chunks (with no dependencies between them) in random
order in the linearization output, hiding information about DepGraph insertion
order from the output. Likewise, it randomizes the order of transactions within
chunks for the same reason.
This introduces a local RNG inside the SFL state, which is used to randomize
various decisions inside the algorithm, in order to make it hard to create
pathological clusters which predictably have bad performance.
The decisions being randomized are:
* When deciding what chunk to attempt to split, the queue order is
randomized.
* When deciding which dependency to split on, a uniformly random one is
chosen among those with higher top feerate than bottom feerate within
the chosen chunk.
* When deciding which chunks to merge, a uniformly random one among those
with the higher feerate difference is picked.
* When merging two chunks, a uniformly random dependency between them is
now activated.
* When making the state topological, the queue of chunks to process is
randomized.
This introduces a queue of chunks that still need processing, in both
MakeTopological() and OptimizationStep(). This is simultaneously:
* A preparation for introducing randomization, by allowing permuting the
queue.
* An improvement to the fairness of suboptimal solutions, by distributing
the work more fairly over chunks.
* An optimization, by avoiding retrying chunks over and over again which
are already known to be optimal.
This replaces the existing LIMO linearization algorithm (which internally uses
ancestor set finding and candidate set finding) with the much more performant
spanning-forest linearization algorithm.
This removes the old candidate-set search algorithm, and several of its tests,
benchmarks, and needed utility code.
The worst case time per cost is similar to the previous algorithm, so
ACCEPTABLE_ITERS is unchanged.
Rather than using an ad-hoc no-dependency copy of the graph when a potentially
non-topological linearization is needed in the clusterlin fuzz test, add this
directly as a feature in ReadLinearization().
This is preparation for a later commit where another use for such a function
is added.
This adds a data structure representing the optimization state for the spanning-forest
linearization algorithm (SFL), plus a fuzz test for its correctness.
This is preparation for switching over Linearize() to use this algorithm.
See https://delvingbitcoin.org/t/spanning-forest-cluster-linearization/1419 for
a description of the algorithm.
db2d39f642 fuzz: add subtest for re-downloading a previously pruned block (Eugene Siegel)
45f5b2dac3 fuzz: Add fuzzer for block index (Martin Zumsande)
c011e3aa54 test: Wrap validation functions with TestChainstateManager (Martin Zumsande)
Pull request description:
This adds a fuzz target for the block index and various events in validation that interact with it.
It can create arbitrary tree-like structure of block indexes, simulating (so far) the following events:
- Adding a header
- Receiving the full block (may be valid or not)
- `ActivateBestChain()` - Reorging the chain to a new chain tip (possibly encountering invalid blocks on the way)
- Pruning a block in the best chain
- Receiving a previously pruned block again (`getblockfrompeer`)
It might be interesting / possible to extend this to more events, such as dealing with more than one chainstate (assumeutxo).
The test skips all actual validation of header/ block / transaction data by just simulating the outcome, and also doesn't interact with the data directory.
The main goal is to ensure the integrity of the block index tree in all fuzzed constellations, by calling `CheckBlockIndex()` at the end of each iteration.
Compared to #29158 this approach has a more limited scope (by skipping all actual validation), but it is fast - it doesn't do a full init sequence on each iteration, but "cleans up" after itself by resetting the global validation state after each iteration.
ACKs for top commit:
Crypt-iQ:
reACK db2d39f642
maflcko:
review ACK db2d39f642🍶
sedited:
Re-ACK db2d39f642
Tree-SHA512: 76cd5f8f4d7d7258620b46d7438bad4508c3bdc98825b48b60f694b5a9838e2b2cf4967c0ead181f86f66f4939ddfe552471851b9d18f84f584c03dd7e09fc43
Test that a node sends a self-announcement with its external IP to
in- and outbound peers after connection open and again sometime later.
Since the code for the test is mostly the same for addr and addrv2
messages, I opted to add a new test file instead of having duplicate
code in p2p_addr_relay.py and p2p_addrv2_relay.py.
Co-Authored-By: rkrux <rkrux.connect@gmail.com>
I noticed this behaviour while perf testing PR 27286 and it was not something
that I expected, updating the doc to make it present in the RPCHelp command.
cd98caea43 Update ci.yml (Woolfgm)
Pull request description:
Updated actions/checkout from v5 to v6 in ci.yml workflow
ACKs for top commit:
fanquake:
ACK cd98caea43
Tree-SHA512: c2e8168400e0ef959d9a166070a91196d4b6abefda557b7a455fe4e5e6295d10132fb2c46885072379b844a9a9bd6adb25ac3301461db446c610967ca3363fbf
facd3d56cc log: Use `__func__` for -logsourcelocations (MarcoFalke)
Pull request description:
The `-logsourcelocations` option was recently changed to print the full function signature, as a side-effect of moving toward `std::source_location` internally.
This is fine, but at least for me, it makes debugging functional test failures harder, because the log is just so massively verbose, with questionable benefit.
I think the historically used file name, line number, and plain `__func__` name are more than sufficient for `-logsourcelocations`.
So switch back to using that.
For reference, a verbose log may look like:
```
...
node0 2025-12-17T07:28:37.528146Z [init] [checkqueue.h:147] [CCheckQueue<T, R>::CCheckQueue(unsigned int, int) [with T = CScriptCheck; R = std::pair<ScriptError_t, std::__cxx11::basic_string<char> >]] Script verificatio
n uses 1 additional threads
...
```
I don't think there is value in printing stuff, like the (anon) namespace, the class template args, or the functionn (template) args. The following should be more than sufficient:
```
...
node0 2025-12-17T09:45:57.017122Z [init] [checkqueue.h:147] [CCheckQueue] Script verification uses 1 additional threads
...
ACKs for top commit:
ajtowns:
ACK facd3d56cc -- those long signatures are terrible
stickies-v:
ACK facd3d56cc
Tree-SHA512: 22fd1f0074fc6e85754967f9219659f57c905005a2bea9176f0b439abec324d7e6c2f875c8951934a3b11ef7e9d7e38d5d5d307e2bd1e000bc27ee85635cd668
76e0e6087d qa: Account for errno not always being set for ConnectionResetError (Hodlinator)
Pull request description:
The lack of errno can cause unclear and long log output.
Issue can be triggered by:
```diff
--- a/src/httpserver.cpp
+++ b/src/httpserver.cpp
@@ -263,6 +263,7 @@ std::string RequestMethodString(HTTPRequest::RequestMethod m)
/** HTTP request callback */
static void http_request_cb(struct evhttp_request* req, void* arg)
{
+ throw std::runtime_error{"Hello"};
evhttp_connection* conn{evhttp_request_get_connection(req)};
// Track active requests
{
```
and running a functional test such as *test/functional/feature_abortnode.py*.
`http.client.RemoteDisconnected` not specifying `errno` to `ConnectionResetError`-ctor: ce4b0ede16/Lib/http/client.py (L1556C9-L1556C29)
<details><summary>Before/after log examples</summary>
#### Log before
```
2025-11-14T20:53:05.272804Z TestFramework (ERROR): Unexpected exception
Traceback (most recent call last):
File "/home/hodlinator/bc/3/test/functional/test_framework/test_framework.py", line 138, in main
self.setup()
~~~~~~~~~~^^
File "/home/hodlinator/bc/3/test/functional/test_framework/test_framework.py", line 268, in setup
self.setup_network()
~~~~~~~~~~~~~~~~~~^^
File "/home/hodlinator/bc/3/./build/test/functional/feature_abortnode.py", line 21, in setup_network
self.setup_nodes()
~~~~~~~~~~~~~~~~^^
File "/home/hodlinator/bc/3/test/functional/test_framework/test_framework.py", line 381, in setup_nodes
self.start_nodes()
~~~~~~~~~~~~~~~~^^
File "/home/hodlinator/bc/3/test/functional/test_framework/test_framework.py", line 527, in start_nodes
node.wait_for_rpc_connection()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/home/hodlinator/bc/3/test/functional/test_framework/test_node.py", line 326, in wait_for_rpc_connection
rpc.getblockcount()
~~~~~~~~~~~~~~~~~^^
File "/home/hodlinator/bc/3/test/functional/test_framework/coverage.py", line 50, in __call__
return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
File "/home/hodlinator/bc/3/test/functional/test_framework/authproxy.py", line 137, in __call__
response, status = self._request('POST', self.__url.path, postdata.encode('utf-8'))
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hodlinator/bc/3/test/functional/test_framework/authproxy.py", line 111, in _request
return self._get_response()
~~~~~~~~~~~~~~~~~~^^
File "/home/hodlinator/bc/3/test/functional/test_framework/authproxy.py", line 174, in _get_response
http_response = self.__conn.getresponse()
File "/nix/store/62fdlzq1x1ak2lsxp4ij7ip5k9nia3hc-python3-3.13.7/lib/python3.13/http/client.py", line 1430, in getresponse
response.begin()
~~~~~~~~~~~~~~^^
File "/nix/store/62fdlzq1x1ak2lsxp4ij7ip5k9nia3hc-python3-3.13.7/lib/python3.13/http/client.py", line 331, in begin
version, status, reason = self._read_status()
~~~~~~~~~~~~~~~~~^^
File "/nix/store/62fdlzq1x1ak2lsxp4ij7ip5k9nia3hc-python3-3.13.7/lib/python3.13/http/client.py", line 300, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
" response")
http.client.RemoteDisconnected: Remote end closed connection without response
```
#### Log after
```
2025-11-14T20:48:10.552126Z TestFramework (ERROR): Unexpected exception
Traceback (most recent call last):
File "/home/hodlinator/bc/3/test/functional/test_framework/test_framework.py", line 138, in main
self.setup()
~~~~~~~~~~^^
File "/home/hodlinator/bc/3/test/functional/test_framework/test_framework.py", line 268, in setup
self.setup_network()
~~~~~~~~~~~~~~~~~~^^
File "/home/hodlinator/bc/3/./build/test/functional/feature_abortnode.py", line 21, in setup_network
self.setup_nodes()
~~~~~~~~~~~~~~~~^^
File "/home/hodlinator/bc/3/test/functional/test_framework/test_framework.py", line 381, in setup_nodes
self.start_nodes()
~~~~~~~~~~~~~~~~^^
File "/home/hodlinator/bc/3/test/functional/test_framework/test_framework.py", line 527, in start_nodes
node.wait_for_rpc_connection()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/home/hodlinator/bc/3/test/functional/test_framework/test_node.py", line 316, in wait_for_rpc_connection
raise FailedToStartError(self._node_msg(
f'bitcoind exited with status {self.process.returncode} during initialization. {str_error}'))
test_framework.test_node.FailedToStartError: [node 0] bitcoind exited with status -6 during initialization. terminate called after throwing an instance of 'std::runtime_error'
what(): Hello
************************
```
Note how even the C++ exception message is now included.
</details>
ACKs for top commit:
maflcko:
review ACK 76e0e6087d 🌬
furszy:
Tested ACK 76e0e6087d
l0rinc:
untested code review ACK 76e0e6087d
Tree-SHA512: 55a83d664624932b919ab2a5b6369121db448d27628029f21c5df297892dd56d179d710ad744f6407b51aa576fb6905a38bbc29885c534ec20704c22717a0880
caf4843a59 fuzz: doc: remove any mention to address_deserialize_v2 (brunoerg)
Pull request description:
We don't have `address_deserialize_v2` target anymore since fac81affb5 (we used to have `address_deserialize_v1_notime`, `address_deserialize_v1_withtime` and `address_deserialize_v2` but now we only have a single `address_deserialize` target) so it removes any mention to it.
ACKs for top commit:
maflcko:
review ACK caf4843a59🎾
marcofleon:
ACK caf4843a59
Tree-SHA512: 539d69edbfe4ca11eb0701ed5c789ad81976e3e85e8a229e39e9dc1b1c72264f01d10a1c16d0a3bb4a354794412dc8b625298f4f72430905a00b65faeaa37d6b
d9319b06cf refactor: unify container presence checks - non-trivial counts (Lőrinc)
039307554e refactor: unify container presence checks - trivial counts (Lőrinc)
8bb9219b63 refactor: unify container presence checks - find (Lőrinc)
Pull request description:
### Summary
Instead of counting occurrences in sets and maps, the C++20 `::contains` method expresses the intent unambiguously and can return early on first encounter.
### Context
Applied clang‑tidy's [readability‑container‑contains](https://clang.llvm.org/extra/clang-tidy/checks/readability/container-contains.html) check, though many cases required manual changes since tidy couldn't fix them automatically.
### Changes
The changes made here were:
| From | To |
|------------------------|------------------|
| `m.find(k) == m.end()` | `!m.contains(k)` |
| `m.find(k) != m.end()` | `m.contains(k)` |
| `m.count(k)` | `m.contains(k)` |
| `!m.count(k)` | `!m.contains(k)` |
| `m.count(k) == 0` | `!m.contains(k)` |
| `m.count(k) != 1` | `!m.contains(k)` |
| `m.count(k) == 1` | `m.contains(k)` |
| `m.count(k) < 1` | `!m.contains(k)` |
| `m.count(k) > 0` | `m.contains(k)` |
| `m.count(k) != 0` | `m.contains(k)` |
> Note that `== 1`/`!= 1`/`< 1` only apply to simple [maps](https://en.cppreference.com/w/cpp/container/map/contains)/[sets](https://en.cppreference.com/w/cpp/container/set/contains) and had to be changed manually.
There are many other cases that could have been changed, but we've reverted most of those to reduce conflict with other open PRs.
-----
<details>
<summary>clang-tidy command on Mac</summary>
```bash
rm -rfd build && \
cmake -B build \
-DCMAKE_C_COMPILER="$(brew --prefix llvm)/bin/clang" \
-DCMAKE_CXX_COMPILER="$(brew --prefix llvm)/bin/clang++" \
-DCMAKE_OSX_SYSROOT="$(xcrun --show-sdk-path)" \
-DCMAKE_C_FLAGS="-target arm64-apple-macos11" \
-DCMAKE_CXX_FLAGS="-target arm64-apple-macos11" \
-DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DBUILD_BENCH=ON -DBUILD_FUZZ_BINARY=ON -DBUILD_FOR_FUZZING=ON
"$(brew --prefix llvm)/bin/run-clang-tidy" -quiet -p build -j$(nproc) -checks='-*,readability-container-contains' | grep -v 'clang-tidy'
```
</details>
Note: this is a take 2 of https://github.com/bitcoin/bitcoin/pull/33094 with fewer contentious changes.
ACKs for top commit:
optout21:
reACK d9319b06cf
sedited:
ACK d9319b06cf
janb84:
re ACK d9319b06cf
pablomartin4btc:
re-ACK d9319b06cf
ryanofsky:
Code review ACK d9319b06cf. I manually reviewed the full change, and it seems there are a lot of positive comments about this and no more very significant conflicts, so I will merge it shortly.
Tree-SHA512: e4415221676cfb88413ccc446e5f4369df7a55b6642347277667b973f515c3c8ee5bfa9ee0022479c8de945c89fbc9ff61bd8ba086e70f30298cbc1762610fe1
fa904fc683 lint: Remove confusing, redundant, and brittle lint-spelling (MarcoFalke)
Pull request description:
`codespell` was a fun experiment. However, it has many issues, when used in this project:
* The number of false-positives and true-positives are in the same ballpark. There are also many false-negatives, so the overall net-benefit is questionable.
* There is often confusion around spelling errors leading to a failing CI (they do not, which was intended).
* LLMs released this year are capable to detect typos with less false-positives and less false-negatives, so the `codespell` integration is a bit redundant in that sense.
Fix all issues by removing it.
Going forward, anyone is free to continue to use `codespell`, or any LLM, or any other tool, locally. Also, DrahtBot has the LLM typo linter integrated in the summary comment. I think the options are plenty, and are more than sufficient for now.
ACKs for top commit:
l0rinc:
ACK fa904fc683
rkrux:
ACK fa904fc683
pablomartin4btc:
ACK fa904fc683
Tree-SHA512: 5e2008a77c2c313605f30d73286111eba034a2a6bb2a0a48e2f77ec6ccc7afaa274e00bbfcb727be0ac5e547b8ae9c801d30c43589b0cad2099565e6716b9ec7
5ac3579520 refactor: Add compile-time-checked hex txid (rustaceanrob)
Pull request description:
Suggested by l0rinc as a comment in #34004.
There are tests that utilize `FromHex` that will only fail during runtime if malformed. Adds a compile time constructor that can be caught by LSPs.
ACKs for top commit:
l0rinc:
ACK 5ac3579520
maflcko:
review ACK 5ac3579520🦎
rkrux:
crACK 5ac3579520
Tree-SHA512: b0bae2bf0b8cd8c9a90765a14c46146313cf8b224a29d58a253e65ca95c4205c0beddea9c49ae58901e72c8c5202b91695d074ffb1c48e448d2e5606eb1bd5b4
fa5ed16aa4 move-only: MAX_BLOCK_TIME_GAP to src/qt (MarcoFalke)
Pull request description:
`MAX_BLOCK_TIME_GAP` was used in some incorrect heuristics, which were removed in commit e30b6ea194.
This leaves a single module in src/qt using the constant.
Instead of exposing it in a central kernel header, just move it to the single gui module that uses it.
ACKs for top commit:
sedited:
ACK fa5ed16aa4
hebasto:
ACK fa5ed16aa4, I have reviewed the code and it looks OK.
Tree-SHA512: d0e0e5257f6585d793bfed118d61a3e5d56b2be397fa3b09b34db64e3e018eba9f223cd56541d258b422119fdd7501f07cd3bb8ad5dc28b535922aa21ea76fa6
e7e51952dc contrib: Avoid outputting binary data to TTY (Hodlinator)
Pull request description:
Verify that we wouldn't be writing encoded asmap binary data directly to the TTY since it is the default but makes no sense. (Having stdout as default does make sense when piping to other applications however).
Found while exploring the ASMap data pipeline (https://github.com/asmap/asmap-data/pull/38#pullrequestreview-3547352533) from Kartograf into Bitcoin Core.
ACKs for top commit:
fjahr:
tACK e7e51952dc
sipa:
ACK e7e51952dc
Tree-SHA512: e1ae1ee129715471cbb824268e68cec267d159d4073297af35c06eadfb6b98eeae040beaafeb6489c2853ea9b83cd04471bcd0b27f0ae8fcb377e6e10b4ae6c5
faa8ee62f5 ci: Pin native tests on cross-builds to same commit (MarcoFalke)
Pull request description:
After commit 13809b867a, the native tests may check out a different commit than the cross-build task that produced the artefacts they run on.
Obviously, this may lead to test failures.
Fix it, by first determining a fixed commit, to be used for both the build and the native tests.
An alternative could be to fully or partially revert 13809b867a, but that comes again with the downsides making it harder to detect silent merge conflicts by re-running CI, or clearing unrelated and fixed intermittent test issues by re-running CI. Then, the only alternative would be to close and re-open the pull request.
ACKs for top commit:
janb84:
ACK faa8ee62f5
ryanofsky:
Code review ACK faa8ee62f5. Thanks for the naming & display updates since last review!
hodlinator:
crACK faa8ee62f5
Tree-SHA512: 01391cdfad34e3f2f5b3a6247b1aeb412d023e368bc17572aa66324688439786e79e263288276053ffcfa8521635b07339dbf087b8a30d6670373556a77c22ee
59b93f11e8 rest: print also HTTP response reason in case of an error (Roman Zeyde)
7fe94a0493 rest: add a test for unsuported `/blockpart/` request type (Roman Zeyde)
55d0d19b5c rest: deduplicate `interface_rest.py` negative tests (Roman Zeyde)
89eb531024 rest: update release notes for `/blockpart/` endpoint (Roman Zeyde)
41118e17f8 blockstorage: simplify partial block read validation (Roman Zeyde)
599effdeab rest: reformat `uri_prefixes` initializer list (Roman Zeyde)
Pull request description:
The commits below should resolve a few leftovers from #33657.
ACKs for top commit:
l0rinc:
ACK 59b93f11e8
hodlinator:
re-ACK 59b93f11e8
Tree-SHA512: ae45e08edd315018e11283b354fb32f9658f5829c956554dc662a81c2e16397def7c3700e6354e0a91ff03c850def35638a69ec2668b7c015d25d6fed42b92bb
Helpful when comparing expected/unexpected outputs against each other for working/broken code.
Also account for TimeoutExpired.output being None and halt instead of re-raising.
Re-raising within the except-block would trigger excessive "During handling of the above exception, another exception occurred"-output.
Also changed comment - exceptions are raised in Python, not thrown.
Remove the transaction from the list of transactions to broadcast after
we receive it from the network.
Only remove the transaction if it is the same as the one we sent: has
the same wtxid (and it follows the same txid). Don't remove transactions
that have the same txid and different wtxid. Such transactions show that
some of the private broadcast recipients malleated the witness and the
transaction made it back to us. The witness could be either:
* invalid, in which case the transaction will not be accepted in
anybody's pool; or
* valid, in which case either the original or the malleated transaction
will make it to nodes' mempools and eventually be mined. Our response
is to keep broadcasting the original. If the malleated transaction
wins then we will eventually stop broadcasting the original when it
gets stale and gets removed from the "to broadcast" storage cause it
is not acceptable in our mempool.
For connections of type `ConnectionType::PRIVATE_BROADCAST`:
* After receiving VERACK, send a transaction from the list of
transactions for private broadcast and disconnect
* Don't process any messages after VERACK (modulo `GETDATA` and `PONG`)
* Don't send any messages other than the minimum required for the
transaction send - `INV`, `TX`, `PING`.
The following commit will interrupt the processing of the `VERACK`
message earlier for private broadcast connections. The `Assume()` would
be nice to have for private broadcast as well, so move it earlier.
This is a non-functional change.
Rename some variables in `PeerManagerImpl::PushNodeVersion()` and
use one log call instead of 2 almost identical. Also, assign
variables after they are declared to make it easy to assign them
different values, conditionally.
This is a non-functional change.
Move the debug log message that we have received a `VERSION` message
earlier, before any `MakeAndPushMessage()`. Thus, if the processing is
interrupted before `MakeAndPushMessage()`s, the log would still be
executed.
Change the order in which code snippets are executed as a result of
receiving the `VERSION` message. Move the snippets that do
`MakeAndPushMessage()` near the end. This makes it easier to interrupt
the execution when no messages should be sent as a response to the
`VERSION` messages, in private broadcast connections.
This is a non-functional change.
Extend `PeerManager` with a transaction storage and a new method
`InitiateTxBroadcastPrivate()` which:
* adds a transaction to that storage and
* calls `CConnman::PrivateBroadcast::NumToOpenAdd()` to open dedicated
privacy connections that will pick an entry from the transaction
storage and broadcast it.
Extend `node::TxBroadcast` with a 3rd option to not add the
transaction to the mempool and broadcast privately.
This is a non-functional change - `BroadcastTransaction()` will not
do anything if the 3rd options is passed and is not used by any of
its callers.
Rename `PeerManager::RelayTransaction()` to
`PeerManager::InitiateTxBroadcastToAll()`. The transaction is not
relayed when the method returns. It is only enqueued for a possible
broadcasting at a later time. Also, there will be another method which
only does so to Tor or I2P peers.
Implement opening `ConnectionType::PRIVATE_BROADCAST` connections with
the following properties:
* Only to Tor or I2P (or IPv4/IPv6 through the Tor proxy, if provided)
* Open such connections only when requested and don't maintain N opened
connections of this type.
* Since this is substantially different than what
`OpenNetworkConnection()` does, open the private broadcast connections
from a different thread instead of modifying `OpenNetworkConnection()`
to also open those types of connections.
Co-authored-by: Andrew Toth <andrewstoth@gmail.com>
This imitates the use of the getblockfrompeer rpc.
Note that currently pruning is limited to blocks in the active chain.
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
This fuzz target creates arbitrary tree-like structure of indices,
simulating the following events:
- Adding a header to the block tree db
- Receiving the full block (may be valid or not)
- Reorging to a new chain tip (possibly encountering invalid blocks on
the way)
- pruning
The test skips all actual validation of header/ block / transaction data
by just simulating the outcome, and also doesn't interact with the data directory.
The main goal is to test the integrity of the block index tree in
all fuzzed constellations, by calling CheckBlockIndex()
at the end of each iteration.
7e9de20c0c fuzz: exercise `ComputeMerkleRoot` without mutated parameter (Lőrinc)
Pull request description:
The `mutated` parameter in `ComputeMerkleRoot` unlocks a different path that was always exercised in the fuzz test.
Adjusted to be fuzzer to pass `nullptr` as well to make sure that path is also tested: 24ed820d4f/src/consensus/merkle.cpp (L49-L53)
Follow-up to https://github.com/bitcoin/bitcoin/pull/33805#discussion_r2589073735
ACKs for top commit:
frankomosh:
ACK [7e9de20](7e9de20c0c)
hodlinator:
ACK 7e9de20c0c
sedited:
ACK 7e9de20c0c
Tree-SHA512: bf27029ac04003447b24a95544ec863f9ceca6c28d51ea811dd6ca2b412a2a780bb9fdbcdc82719f39dd710a746eb2446263e8377d67a8be52a1694571d03498
d8fe5f0326 test: improve interface_ipc.py waitNext tests (Ryan Ofsky)
a5e61b1917 test: interface_ipc.py minor fixes and cleanup (Ryan Ofsky)
ded11fb04d test: fix interface_ipc.py template destruction (Ryan Ofsky)
Pull request description:
This PR cleans up the `interface_ipc.py` test, fixing broken checks, fixing missing await calls, removing to_dict calls, renaming variables, reducing `.result` accesses, and giving template objects explicit lifetimes. More details are in the commit messages.
The first commit changes a lot of indentation so is easiest to review ignoring whitespace.
ACKs for top commit:
Sjors:
ACK d8fe5f0326
sedited:
ACK d8fe5f0326
Tree-SHA512: f0de309a15cb23f109cf6909e51ddd132a60bd4d4cb25b20bdc74545516670f1cdb0c9cc98c397c2f24e67e2380c2dac9d00435009618a3c00b6b85cca5c3e2e
82be652e40 doc: Improve ChainstateManager documentation, use consistent terms (Ryan Ofsky)
af455dcb39 refactor: Simplify pruning functions (TheCharlatan)
ae85c495f1 refactor: Delete ChainstateManager::GetAll() method (Ryan Ofsky)
6a572dbda9 refactor: Add ChainstateManager::ActivateBestChains() method (Ryan Ofsky)
491d827d52 refactor: Add ChainstateManager::m_chainstates member (Ryan Ofsky)
e514fe6116 refactor: Delete ChainstateManager::SnapshotBlockhash() method (Ryan Ofsky)
ee35250683 refactor: Delete ChainstateManager::IsSnapshotValidated() method (Ryan Ofsky)
d9e82299fc refactor: Delete ChainstateManager::IsSnapshotActive() method (Ryan Ofsky)
4dfe383912 refactor: Convert ChainstateRole enum to struct (Ryan Ofsky)
352ad27fc1 refactor: Add ChainstateManager::ValidatedChainstate() method (Ryan Ofsky)
a229cb9477 refactor: Add ChainstateManager::CurrentChainstate() method (Ryan Ofsky)
a9b7f5614c refactor: Add Chainstate::StoragePath() method (Ryan Ofsky)
840bd2ef23 refactor: Pass chainstate parameters to MaybeCompleteSnapshotValidation (Ryan Ofsky)
1598a15aed refactor: Deduplicate Chainstate activation code (Ryan Ofsky)
9fe927b6d6 refactor: Add Chainstate m_assumeutxo and m_target_utxohash members (Ryan Ofsky)
6082c84713 refactor: Add Chainstate::m_target_blockhash member (Ryan Ofsky)
de00e87548 test: Fix broken chainstatemanager_snapshot_init check (Ryan Ofsky)
Pull request description:
This PR contains the first part of #28608, which tries to make assumeutxo code more maintainable, and improve it by not locking `cs_main` for a long time when the snapshot block is connected, and by deleting the snapshot validation chainstate when it is no longer used, instead of waiting until the next restart.
The changes in this PR are just refactoring. They make `Chainstate` objects self-contained, so for example, it is possible to determine what blocks to connect to a chainstate without querying `ChainstateManager`, and to determine whether a Chainstate is validated without basing it on inferences like `&cs != &ActiveChainstate()` or `GetAll().size() == 1`.
The PR also tries to make assumeutxo terminology less confusing, using "current chainstate" to refer to the chainstate targeting the current network tip, and "historical chainstate" to refer to the chainstate downloading old blocks and validating the assumeutxo snapshot. It removes uses of the terms "active chainstate," "usable chainstate," "disabled chainstate," "ibd chainstate," and "snapshot chainstate" which are confusing for various reasons.
ACKs for top commit:
maflcko:
re-review ACK 82be652e40🕍
fjahr:
re-ACK 82be652e40
sedited:
Re-ACK 82be652e40
Tree-SHA512: 81c67abba9fc5bb170e32b7bf8a1e4f7b5592315b4ef720be916d5f1f5a7088c0c59cfb697744dd385552f58aa31ee36176bae6a6e465723e65861089a1252e5
09dfa4d3f8 test: fix race condition in p2p_v2_misbehaving.py peerid assertion (stratospher)
Pull request description:
Remove the hard-coded peer id from the debug message in `p2p_v2_misbehaving.py`.
asyncio's non-deterministic task scheduling might cause [peer2](938d7aacab/test/functional/p2p_v2_misbehaving.py (L181))'s connection to happen before [peer1](938d7aacab/test/functional/p2p_v2_misbehaving.py (L179))'s. since we test that peer2 [remains connected](938d7aacab/test/functional/p2p_v2_misbehaving.py (L182)), any disconnection must originate from peer1, making the specific peer id not necessary for test correctness. so we can remove the hard coded peer id from the expected debug log message.
Fixes#34035.
ACKs for top commit:
maflcko:
lgtm ACK 09dfa4d3f8
mzumsande:
Code Review ACK 09dfa4d3f8
Tree-SHA512: 542b08ddae09db7454e8c08b1d26aade50a53c2505683df99556cf071a6a38195b64f8700f6db3f4e1b318497fc4b5232246ad4e9d6f3af45fad83e333fa91fb
14371fd1fc gui: Add a menu item to restore then migrate a wallet file (Ava Chow)
f11a7d248c gui: Add restore_and_migrate function to restore then migrate a wallet (Ava Chow)
16ab6dfc10 gui: Move actual migration part of migrate() to its own function (Ava Chow)
4ec2d18a07 wallet, interfaces, gui: Expose load_after_restore parameter (Ava Chow)
Pull request description:
Some users will have a backup of their legacy wallet. These cannot be restored since the "Restore Wallet" action expects to be able to load the wallet after restoring, and this fails for legacy wallets now that they are deleted. Furthermore, the "Migrate Wallet" action only allows users to migrate wallets that are in the wallets directory, so such backups cannot be migrated from the GUI.
This PR resolves this issue by adding a menu item in the "Migrate Wallet" menu which allows users to select their backup file so that it will first be restored but not loaded, and then migrated.
Depends on https://github.com/bitcoin/bitcoin/pull/32620
ACKs for top commit:
hebasto:
ACK 14371fd1fc.
Tree-SHA512: 2b09c012f4c70d0cb283305bf3d1a18ae5a2bfb80977c91544ac1fbc29d6360df49438cfdc8f66661ddb42ddab728c8ef1f9e0d7031877fbd846f9cea957398e
CCoinsViewCache::BatchWrite always returns true if called from a backed
cache, so just return void instead. Also return void from ::Sync and
::Flush.
This allows for dropping a FatalError condition and simplifying some
dead error handling code a bit.
Since we now no longer exercise the "error path" when returning from
`CCoinsView::BatchWrite`, make the method clear the cache instead. This
should only be exercised by tests and not change production behaviour.
This might slightly improve the coins_view fuzz test's ability to
generate better coverage.
Co-authored-by: l0rinc <pap.lorinc@gmail.com>
fa8a5d215c log: Remove brittle and confusing LogPrintLevel (MarcoFalke)
fac24bbec8 test: Clarify logging_SeverityLevels test (MarcoFalke)
f273167661 ipc: separate log statements per level (stickies-v)
94c51ae540 libevent: separate log statements per level (stickies-v)
Pull request description:
`LogPrintLevel` has many issues:
* It encourages to log several levels in one source location. This is problematic, because all levels (even warnings and errors) will be rate limited equally for the same location.
* Its warning and error logs are specially formatted compared to all other warning and error logs in the codebase, making them harder to spot (both in the debug log and in the code).
* It is verbose to type and read.
* It is confusing, because the majority of code uses the `Log$LEVEL(...)` macros. Having less ways to achieve the same makes the code more consistent and easier to review.
Fix all issues by removing it
ACKs for top commit:
stickies-v:
re-ACK fa8a5d215c
ajtowns:
ACK fa8a5d215c
pablomartin4btc:
re-ACK fa8a5d215c
Tree-SHA512: 9fbb04962d9c26e566338694a7725b3c0e88ef733322d890bcc6aeddb45266c754e7c885c69bbfebd1588cc09912c6784cfc00e69882f1271a8c87d201490478
a70a14a3f4 refactor: Separate out logic for building a tree-shaped dependency graph (marcofleon)
ce29d7d626 fuzz: Fix variable in `clusterlin_postlinearize_tree` check (marcofleon)
876e2849b4 fuzz: Fix incorrect loop bounds in `clusterlin_postlinearize_tree` (marcofleon)
Pull request description:
Addresses two issues in the `clusterlin_postlinearize_tree` target:
1. The loop iteration while creating tree dependency graphs was incorrect.
2. We were accidentally passing in `post_linearization` to `PostLinearize` instead of the copy we just made, resulting in an ineffective check.
ACKs for top commit:
sipa:
ACK a70a14a3f4
instagibbs:
ACK a70a14a3f4
Tree-SHA512: 2cc1b70d572250d8e7b8db8957ae1f3447f8524c09e638ce08af27ff3f6b7aace3cf834c300f2a7947553cc919e2feedfd64355ff94eb2311fb9cd632cb7358a
Suggested by @l0rinc in #34004
Message by @l0rinc:
This adds a consteval constructor to transaction_identifier (Txid/Wtxid) to allow parsing hex strings at compile-time.
This replaces runtime FromHex checks in tests, ensuring that malformed hardcoded hashes cause build failures rather than runtime test failures.
Test variables are explicitly marked constexpr. This is required to workaround a regression in GCC 14 (Bug 117501) where the compiler incorrectly flags consteval initialization of non-constexpr variables as "statements with no effect".
GCC Bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=117501
Reproducer: https://godbolt.org/z/xb5TMaPs6
Co-authored-by: l0rinc <pap.lorinc@gmail.com>
The test was a bit confusing, because it just referred to the "global
log level" without explicitly specifying what it is. The level is set
though the LogSetup constructor. However, it is easier to follow unit
tests, if they are self-contained. So just set the level to Debug
explicitly here.
Also, add a new debug_3 log, to further document the intended behavior
of the unit test.
Also, replace the LogPrintLevel with the shorter and exact replacements
LogTrace and LogDebug.
Avoids ratelimiting unconditional log statements when debug logging
is enabled. Introduces slight behaviour change by removing
the category from unconditional logs, making them more uniform
with the other unconditional logs in the codebase.
Also, in a slight behavior change, prefix the info-level (and higher)
messages with "ipc:".
Avoids ratelimiting unconditional log statements when debug logging
is enabled. Introduces slight behaviour change by removing
the category from unconditional logs, making them more uniform
with the other unconditional logs in the codebase.
Also, in a slight behavior change, prefix the info-level (and higher)
messages with "libevent:".
The test intends to verify that running `PostLinearize` a
second time on a tree-structured graph doesn't change the
result. But `PostLinearize` was being called on the original
variable, not the copy. So the check was comparing the
unmodified copy against itself, which is useless.
Fix by post-linearizing the correct variable.
The dependency graphs generated by this test can have holes
(unused indices) in them. This means some of the transactions
were skipped when using `depgraph_gen.TxCount()` as the upper
bound of the loop. Switch to using `depgraph.Positions()` to
correctly handle sparse graphs.
due to asyncio's non-deterministic task scheduling, peer2's
connection might happen before peer1's, causing peer2 to get
assigned peer_id=1 on bitcoind side and peer1 to get assigned
peer_id=2 on bitcoind side.
since we test that peer2 remains connected, any disconnection
must originate from peer1, making the specific peer id unnecessary
for test correctness. so we can remove the specific peer_id from
the expected debug log.
5f5c1ea019 net: Cache -capturemessages setting (Anthony Towns)
cea443e246 net: Pass time to InactivityChecks fuctions (Anthony Towns)
Pull request description:
Cuts out some wasted time in net socket handling. First, only calculates the current time once every 50ms, rather than once for each peer, which given we only care about second-level precision seems more than adequate. Second, caches the value of the `-capturemessages` setting in `CConnman` rather than re-evaluating it every time we invoke `PushMessaage`.
ACKs for top commit:
maflcko:
review ACK 5f5c1ea019🏣
vasild:
ACK 5f5c1ea019
sedited:
ACK 5f5c1ea019
mzumsande:
ACK 5f5c1ea019
Tree-SHA512: 0194143a3a4481c6355ac9eab27ce6ae4bed5db1d483ba5d06288dd92f195ccb9f0f055a9eb9d7e16e9bbf72f145eca1ff17c6700ee9aa42730103a8f047b32c
Deduplicate code looping over chainstate objects and calling
ActivateBestChain() and avoid need for code outside ChainstateManager to use
the GetAll() method.
Use to replace m_active_chainstate, m_ibd_chainstate, and m_snapshot_chainstate
members. This has several benefits:
- Ensures ChainstateManager treats chainstates instances equally, making
distinctions based on their attributes, not having special cases and making
assumptions based on their identities.
- Normalizes ChainstateManager representation so states that should be
impossible to reach and validation code has no handling for (like
m_snapshot_chainstate being set and m_ibd_chainstate being unset, or both
being set but m_active_chainstate pointing to the m_ibd_chainstate) can no
longer be represented.
- Makes ChainstateManager more extensible so new chainstates can be added for
different purposes, like indexing or generating and validating assumeutxo
snapshots without interrupting regular node operations. With the
m_chainstates member, new chainstates can be added and handled without needing
to make changes all over validation code or to copy/paste/modify the existing
code that's been already been written to handle m_ibd_chainstate and
m_snapshot_chainstate.
- Avoids terms that are confusing and misleading:
- The term "active chainstate" term is confusing because multiple chainstates
will be active and in use at the same time. Before a snapshot is validated,
wallet code will use the snapshot chainstate, while indexes will use the IBD
chainstate, and netorking code will use both chainstates, downloading
snapshot blocks at higher priority, but also IBD blocks simultaneously.
- The term "snapshot chainstate" is ambiguous because it could refer either
to the chainstate originally loaded from a snapshot, or to the chainstate
being used to validate a snapshot that was loaded, or to a chainstate being
used to produce a snapshot, but it is arbitrary used to refer the first
thing. The terms "most-work chainstate" or "assumed-valid chainstate" should
be less ambiguous ways to refer to chainstates loaded from snapshots.
- The term "IBD chainstate" is not just ambiguous but actively confusing
because technically IBD ends and the node is considered synced when the
snapshot chainstate finishes syncing, so in practice the IBD chainstate
will mostly by synced after IBD is complete. The term "fully-validated" is
a better way of describing the characteristics and purpose of this
chainstate.
SnapshotBlockhash() is only called two places outside of tests, and is used
redundantly in some tests, checking the same field as other checks. Simplify by
dropping the method and using the m_from_snapshot_blockhash field directly.
IsSnapshotValidated() is only called one place outside of tests, and is use
redundantly in some tests, asserting that a snapshot is not validated when a
snapshot chainstate does not even exist. Simplify by dropping the method and
checking Chainstate m_assumeutxo field directly.
IsSnapshotActive() method is only called one place outside of tests and
asserts, and is confusing because it returns true even after the snapshot is
fully validated.
The documentation which said this "implies that a background validation
chainstate is also in use" is also incorrect, because after the snapshot is
validated, the background chainstate gets disabled and IsUsable() would return
false.
Change ChainstateRole parameter passed to wallets and indexes. Wallets and
indexes need to know whether chainstate is historical and whether it is fully
validated. They should not be aware of the assumeutxo snapshot validation
process.
CurrentChainstate() is basically the same as ActiveChainstate() except it
requires cs_main to be locked when it is called, instead of locking cs_main
internally.
The name "current" should also be less confusing than "active" because multiple
chainstates can be active, and CurrentChainstate() returns the chainstate
targeting the current network tip, regardless of what chainstates are being
downloaded or how they are used.
Use to simplify code determining the chainstate leveldb paths. New method is
the now the only code that needs to figure out the storage path, so the path
doesn't need to be constructed multiple places and backed out of leveldb.
Remove hardcoded references to m_ibd_chainstate and m_snapshot_chainstate so
MaybeCompleteSnapshotValidation function can be simpler and focus on validating
the snapshot without dealing with internal ChainstateManager states.
This is a step towards being able to validate the snapshot outside of
ActivateBestChain loop so cs_main is not locked for minutes when the snapshot
block is connected.
Move duplicate code from ChainstateManager::ActivateSnapshot and
ChainstateManager::ActivateExistingSnapshot methods to a new
ChainstateManager::AddChainstate method.
The "AddChainstate" method name doesn't mention snapshots even though it is
only used to add snapshot chainstates now, because it becomes more generalized
in a later commit in this PR ("refactor: Add ChainstateManager::m_chainstates
member")
Get rid of m_disabled/IsUsable members. Instead of marking chains disabled for
different reasons, store chainstate assumeutxo status explicitly and use that
information to determine how chains should be treated.
Make Chainstate objects aware of what block they are targeting. This makes
Chainstate objects more self contained, so it's possible for validation code to
look at one Chainstate object and know what blocks to connect to it without
needing to consider global validation state or look at other Chainstate
objects.
The motivation for this change is to make validation and networking code more
readable, so understanding it just requires knowing about chains and blocks,
not reasoning about assumeutxo download states. This change also enables
simplifications to the ChainstateManager interface in subsequent commits, and
could make it easier to implement new features like creating new Chainstate
objects to generate UTXO snapshots or index UTXO data.
Note that behavior of the MaybeCompleteSnapshotValidation function is not
changing here but some checks that were previously impossible to trigger like
the BASE_BLOCKHASH_MISMATCH case have been turned into asserts.
The following test code never checked anything because the if statement was
always false:
if (cs != &chainman_restarted.ActiveChainstate()) {
BOOST_CHECK_EQUAL(cs->m_chain.Height(), 109);
}
Also, the height of the background chainstate it was intending to check is 110,
not 109. Fix both problems by rewriting the check.
Some users will have backups of a legacy wallet which cannot be restored
due to being a legacy wallet, and therefore cannot be migrated from the
GUI. This menu item allows such users to restore and migrate their
wallets in a single action.
restore_and_migrate first restores a wallet file to the wallets
directory in the expected layout, then it performs legacy to descriptor
wallet migration on the restored wallet.
c1e554d3e5 refactor: consolidate 3 separate locks into one block (Andrew Toth)
41479ed1d2 test: add test for periodic flush inside ActivateBestChain (Andrew Toth)
84820561dc validation: periodically flush dbcache during reindex-chainstate (Andrew Toth)
Pull request description:
After #30611 we periodically do a non-erasing flush of the dbcache to disk roughly every hour during IBD.
The intention was to also do this periodic flush during reindex-chainstate, so we would not risk losing progress during a system failure when reindexing with a high dbcache value.
It was discovered that reindex-chainstate does not perform a PERIODIC flush until it has already reached the tip. Since reindexing to tip usually happens within 24 hours, this behaviour was unnoticed with the previous periodic flush interval. Note that reindex-chainstate still does IF_NEEDED flushes during `ConnectBlock`, so this also would not be noticed when running with a lower dbcache value.
This patch moves the PERIODIC flush from after the outer loop in `ActivateBestChain` to inside the outer loop after we release `cs_main`. This will periodically flush during IBD, reindex-chainstate, and steady state.
ACKs for top commit:
l0rinc:
ACK c1e554d3e5
achow101:
ACK c1e554d3e5
sipa:
utACK c1e554d3e5
Tree-SHA512: c447ad03e16c9978b8ed2c285b38e1b4c56e7778ab93b6f64435116f47b8931017f5f56ab53eb61656693146aaced776f666af573a41ab28e8f2b6d8657fa756
It will enable different error handling flows for different error types.
Also, `ReadRawBlockBench` performance has decreased due to no longer reusing a vector
with an unchanging capacity - mirroring our production code behavior.
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
Co-authored-by: Lőrinc <pap.lorinc@gmail.com>
fa89f60e31 scripted-diff: LogPrintLevel(*,BCLog::Level::*,*) -> LogError()/LogWarning() (MarcoFalke)
fa6c7a1954 scripted-diff: LogPrintLevel(*,BCLog::Level::Debug,*) -> LogDebug() (MarcoFalke)
Pull request description:
Errors and warnings should normally not happen. However, if they do happen, it is easier to spot them, if they are all logged in the same format via `LogError` or `LogWarning`.
So do that with a scripted-diff.
This is a minimal behavior change and unifies the log output from:
[net:error] Something bad happened
[net:warning] Something problematic happened
to either
[error] Something bad happened
[warning] Something problematic happened
or, when `-loglevelalways=1` is enabled:
[all:error] Something bad happened
[all:warning] Something problematic happened
Such a behavior change is desired, because all warning and error logs are written in the same style in the source code and they are logged in the same format for log consumers.
Removing the category should be harmless, because warning and error messages should be descriptive and unique anyway.
ACKs for top commit:
ajtowns:
ACK fa89f60e31
stickies-v:
ACK fa89f60e31
rkrux:
lgtm code review ACK fa89f60e31
Tree-SHA512: dafa47ab561609a79005faf008fe188dd714f6e07bb2dfbe4db49290d6636b12eb7ac4a18ed32bcc5526641a9f258dbc37c08e10c223ec068b97976590ff0b52
cdaf25f9c3 test: Log IP of download server in get_previous_releases.py (Ava Chow)
Pull request description:
In order to help debug issues with previous release downloads from our web server, we need to know which IP the downloader connected to.
ACKs for top commit:
fjahr:
utACK cdaf25f9c3
l0rinc:
untested ACK cdaf25f9c3
janb84:
ACK cdaf25f9c3
rkrux:
tACK cdaf25f9c3
glozow:
ACK cdaf25f9c3
Tree-SHA512: 38b1ad5fe91b12fe5c4b71b35e3d66effb327c4515598b721a163f64a8efdd1e6237ff9f86c4897394d2c69c6e3a28ae4ba7ed9567a0e27ab6a6e90df8688b39
Update the integer fuzz test to move the vector into `ComputeMerkleRoot`, matching production usage patterns and avoiding unnecessary copies.
Update `merkle_test_BlockWitness` to use an odd number of transactions to ensure the test covers the scenario where leaf duplication occurs. Also switch to `GetWitnessHash` to match `BlockWitnessMerkleRoot` semantics.
The manual vector setup retains the exact-size `resize` to explicitly verify the behavior against the calculated root.
0ac969cddf validation: don't reallocate cache for short-lived CCoinsViewCache (Lőrinc)
c8f5e446dc coins: reduce lookups in dbcache layer propagation (Lőrinc)
Pull request description:
This change is part of [[IBD] - Tracking PR for speeding up Initial Block Download](https://github.com/bitcoin/bitcoin/pull/32043)
### Summary
Previously, when the parent coins cache had no entry and the child did, `BatchWrite` performed a find followed by `try_emplace`, which resulted in multiple `SipHash` computations and bucket traversals on the common insert path.
On a different path, these caches were recreated needlessly for every block connection.
### Fix for double fetch
This change uses a single leading `try_emplace` and branches on the returned `inserted` flag. In the `FRESH && SPENT` case (not used in production, only exercised by tests), we erase the just-inserted placeholder (which is constant time with no rehash anyway). Semantics are unchanged for all valid parent/child state combinations.
This change is a minimal version of [bitcoin/bitcoin@`723c49b` (#32128)](723c49b63b) and draws simplification ideas [bitcoin/bitcoin@`ae76ec7` (#30673)](ae76ec7bcf) and https://github.com/bitcoin/bitcoin/pull/30326.
### Fix for temporary cache recreation
Related to parent cache propagation, the second commit makes it possible to avoid destructuring-recreating-destructuring of these short-live parent caches created for each new block.
A few temporary `CCoinsViewCache`'s are destructed right after the `Flush()`, therefore it is not necessary to call `ReallocateCache` to recreate them right before they're killed anyway.
This change was based on a subset of https://github.com/bitcoin/bitcoin/pull/28945, the original authors and relevant commenters were added as coauthors to this version.
-----
Reindex-chainstate indicates ~1% speedup.
<details>
<summary>Details</summary>
```python
COMMITS="647cdb4f7e8041affed887e2325ee03a91078bb1 0b0c3293ffd75afb27dadc0b28426b40132a8c6b"; \
STOP=909090; DBCACHE=4500; \
CC=gcc; CXX=g++; \
BASE_DIR="/mnt/my_storage"; DATA_DIR="$BASE_DIR/BitcoinData"; LOG_DIR="$BASE_DIR/logs"; \
(echo ""; for c in $COMMITS; do git fetch -q origin $c && git log -1 --pretty='%h %s' $c || exit 1; done; echo "") && \
hyperfine \
--sort command \
--runs 2 \
--export-json "$BASE_DIR/rdx-$(sed -E 's/(\w{8})\w+ ?/\1-/g;s/-$//'<<<"$COMMITS")-$STOP-$DBCACHE-$CC.json" \
--parameter-list COMMIT ${COMMITS// /,} \
--prepare "killall bitcoind 2>/dev/null; rm -f $DATA_DIR/debug.log; git checkout {COMMIT}; git clean -fxd; git reset --hard && \
cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DENABLE_IPC=OFF && ninja -C build bitcoind && \
./build/bin/bitcoind -datadir=$DATA_DIR -stopatheight=$STOP -dbcache=1000 -printtoconsole=0; sleep 20" \
--cleanup "cp $DATA_DIR/debug.log $LOG_DIR/debug-{COMMIT}-$(date +%s).log" \
"COMPILER=$CC ./build/bin/bitcoind -datadir=$DATA_DIR -stopatheight=$STOP -dbcache=$DBCACHE -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0"
647cdb4f7e Merge bitcoin/bitcoin#33311: net: Quiet down logging when router doesn't support natpmp/pcp
0b0c3293ff validation: don't reallocate cache for short-lived CCoinsViewCache
Benchmark 1: COMPILER=gcc ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=909090 -dbcache=4500 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = 647cdb4f7e)
Time (mean ± σ): 16233.508 s ± 9.501 s [User: 19064.578 s, System: 951.672 s]
Range (min … max): 16226.790 s … 16240.226 s 2 runs
Benchmark 2: COMPILER=gcc ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=909090 -dbcache=4500 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = 0b0c3293ffd75afb27dadc0b28426b40132a8c6b)
Time (mean ± σ): 16039.626 s ± 17.284 s [User: 18870.130 s, System: 950.722 s]
Range (min … max): 16027.405 s … 16051.848 s 2 runs
Relative speed comparison
1.01 ± 0.00 COMPILER=gcc ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=909090 -dbcache=4500 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = 647cdb4f7e)
1.00 COMPILER=gcc ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=909090 -dbcache=4500 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = 0b0c3293ffd75afb27dadc0b28426b40132a8c6b)
```
</details>
ACKs for top commit:
optout21:
utACK 0ac969cddf
achow101:
ACK 0ac969cddf
andrewtoth:
utACK 0ac969cddf
sedited:
ACK 0ac969cddf
Tree-SHA512: 9fcc3f1a8314368576a4fba96ca72665527eaa3a97964ab5b39491757f3527147d134f79a5c3456f76c1330c7ef862989d23f764236c5e2563be89a81c1cee47
fa75480c84 test: Detect truncated download in get_previous_releases.py (MarcoFalke)
Pull request description:
Without this, and end-of-stream is not detected and will just lead to an immediate exit, instead of a re-try.
E.g. https://github.com/bitcoin/bitcoin/actions/runs/20089133013/job/57633839315?pr=34038#step:12:201:
```
...
Downloading: [##--------------------------------------] 5.4%
Downloading: [##--------------------------------------] 5.4%
Downloading: [##--------------------------------------] 5.5%
Downloading: [##--------------------------------------] 5.6%
Checksum dd02eab18f9154604e38135ef3f98fd310ba3c748074aeb83a71118cd2cd1367 did not match
Error: Process completed with exit code 1.
```
Also, remove the `0` fallback value, because if the fallback was ever hit, the program would fail anyway with `division by zero` error.
ACKs for top commit:
Sjors:
utACK fa75480c84
rkrux:
Looks fine, ACK fa75480c84
l0rinc:
code review ACK fa75480c84
Tree-SHA512: 230eaf155701ed833636b401118f11ff5c6521c61bf4f3a01fcf390a71a508ba6a570eea855ef659134e118b74f75e3d5772ec8a261db23ebfe4ac7ec87cab5a
e7ac5a133c doc: add release note for 34031 (fanquake)
c4c70a256e netbase: Remove "tor" as a network specification (Carl Dong)
Pull request description:
"tor" as a network specification was deprecated in 60dc8e4208 in favor of "onion"
and this commit removes it and updates the relevant test.
Previously #16029. This has been warning as being deprecated since `v0.17.0`.
This PR only removes the already deprecated usage of tor as a network specification, the use of tor throughout the codebase, is not deprecated.
ACKs for top commit:
davidgumberg:
crACK e7ac5a133c
laanwj:
Code review ACK e7ac5a133c
janb84:
ACK e7ac5a133c
stickies-v:
ACK e7ac5a133c
Tree-SHA512: f211dec151c21728b4cd2b1716ee68907871beaa85d8c89e2bc17576e701d03c03e5455593de94970d787aa3264fab60d8c6debeeff908e00d8feb48804692e9
a1f7623020 qa: Only complain about expected messages that were not found (Hodlinator)
1e54125e2e refactor(qa): Avoid unnecessary string operations (Hodlinator)
a9021101dc qa: Replace always-escaped regexps with "X in Y" (Hodlinator)
5c16e4631c doc: Remove no longer correct comment (Hodlinator)
Pull request description:
* Remove incorrect docstring in `busy_wait_for_debug_log()`.
* Replace nerfed regex searches with `X in Y` expressions.
* Only compute the log string to be printed on failure *when we actually fail* instead of every 0.05s.
* As we find each needle (expected message) in the haystack (log output), stop searching for it. **If we fail and time out, we will only complain about the needles (expected messages) we didn't find. On master we also include found needles, which is less helpful.**
Found while developing a new test case in https://github.com/bitcoin/bitcoin/pull/33336#discussion_r2351892330
ACKs for top commit:
l0rinc:
Code review ACK a1f7623020
maflcko:
review ACK a1f7623020💨
Tree-SHA512: 191ea7647b0ea8b4220e37c62d176861c2fd0e3737aee3641b262915d9118f48953cf1204767c93a93a8fc78a44c2c29206fb390b44c59d99fc2aa7d12bf4889
d4d184eda9 log: don't rate-limit "new peer" with -debug=net (0xb10c)
Pull request description:
Previously, when `debug=net` is enabled, we log "New [..] peer connected" for new inbound peers with `LogInfo`. However, `LogInfo` will get rate-limited since https://github.com/bitcoin/bitcoin/pull/32604. When we specifically turn on `debug=net`, we don't want these log messages to be rate-limited.
To fix this, use `LogDebug(BCLog::NET, ...)` for potentially high-rate inbound connections. Otherwise use `LogInfo`. This means we don't rate-limit the messages for inbound peers when `debug=net` is turned on but will rate-limit if we created outbound at a high rate as these are logged via `LogInfo`.
The new log messages look similar to:
```
2025-12-08T00:00:00Z [net] New inbound peer connected: transport=v2 version=70016 blocks=0 peer=1
2025-12-08T00:00:00Z New outbound-full-relay peer connected: transport=v2 version=70016 blocks=281738 peer=5
```
--
I ran into this message getting rate-limited on one of my monitoring nodes with `-logsourcelocations=1`: With logsourcelocations, one of these lines is about 338 chars (or 338 bytes) long. We rate-limit after more than 1048576 bytes per hour, which results in about 3100 in- and outbound connections per hour. With evicted and instantly reconnecting connections from an entity like LinkingLion, this can be reached fairly quickly.
ACKs for top commit:
stickies-v:
utACK d4d184eda9
Crypt-iQ:
tACK d4d184eda9
maflcko:
review ACK d4d184eda9🚲
rkrux:
lgtm code review ACK d4d184eda9
glozow:
lgtm ACK d4d184eda9
Tree-SHA512: 14dbf693fa44a74c9822590e7a08167d2deeb1bc6f4b8aeb00c1b035c0df7101087d5c80a3c0d637879d5c52f88b30f0cb4c0577cff6f647d2eb3300f49d8ea3
4b47113698 validation: Reword CheckForkWarningConditions and call it also during IBD and at startup (Martin Zumsande)
2f51951d03 p2p: Add warning message when receiving headers for blocks cached as invalid (Martin Zumsande)
Pull request description:
In case of corruption that leads to a block being marked as invalid that is seen as valid by the rest of the network, the user currently doesn't receive good error messages, but will often be stuck in an endless headers-sync loop with no explanation (#26391).
This PR improves warnings in two ways:
- When we receive a header that is already saved in our disk, but invalid, add a warning. This will happen repeatedly during the headerssync loop (see https://github.com/bitcoin/bitcoin/issues/26391#issuecomment-1291765534 on how to trigger it artificially).
- Removes the IBD check from `CheckForkWarningConditions` and adds a call to the function during init (`LoadChainTip()`). The existing check was added in 55ed3f1475 a long time ago when we had more sophisticated fork detection that could lead to false positives during IBD, but that logic was removed in fa62304c97 so that I don't see a reason to suppress the warning anymore.
Fixes#26391 (We'll still do the endless looping, trying to find a peer with a headers that we can use, but will now repeatedly log warnings while doing so).
ACKs for top commit:
glozow:
ACK `git range-diff 6d2c8ea9dbd77c71051935b5ab59224487509559...4b4711369880369729893ba7baef11ba2a36cf4b`
theStack:
re-ACK 4b47113698
sedited:
ACK 4b47113698
Tree-SHA512: 78bc53606374636d616ee10fdce0324adcc9bcee2806a7e13c9471e4c02ef00925ce6daef303bc153b7fcf5a8528fb4263c875b71d2e965f7c4332304bc4d922
Split the repeated `SipHash` v[0..3] initialization into a small `SipHashState` helper that is used by both `CSipHasher` and `PresaltedSipHasher`.
Added explanatory comments to clarify behavior, documenting the equivalence of `PresaltedSipHasher` `operator()` overloads to `CSipHasher` usage.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Replaces separate `shorttxidk0`/`shorttxidk1` members with a cached `PresaltedSipHasher`, so `GetShortID()` reuses the precomputed `SipHash` state instead of rebuilding it on every call.
`CBlockHeaderAndShortTxIDs` was never intended to be used before `FillShortTxIDSelector()` runs; doing so already relied on indeterminate salt values.
The new `Assert(m_hasher)` just makes this invariant explicit and fails fast if the object is used in an uninitialized state.
Replaces standalone `SipHashUint256` with an `operator()` overload in `PresaltedSipHasher`.
Updates all hasher classes (`SaltedUint256Hasher`, `SaltedTxidHasher`, `SaltedWtxidHasher`) to use `PresaltedSipHasher` internally, enabling the same constant-state caching optimization while keeping behavior unchanged.
Benchmark was also adjusted to cache the salting part.
a7c96f874d tests: Add witness commitment if we have a witness transaction in FullBlockTest.update_block() (Chris Stewart)
Pull request description:
This is useful for test cases where we want to test logic invalid blocks that contain witness transactions. If we don't add the witness commitment as per [BIP141](https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#user-content-Commitment_structure), blocks will be rejected with the error [`Block mutated`](fb0ada982a/src/validation.cpp (L4180)).
This change was needed in https://github.com/ajtowns/bitcoin/pull/13 which is a soft fork proposal to disallow 64 byte transactions. We want to test that 64 byte transactions serialized without the witness are invalid. If we do not have this change, we cannot directly test the logic that rejects 64 byte transactions.
I decided to PR this upstream as many soft fork proposals may not see the light of day, but this functionality seems strictly additive to the test framework.
ACKs for top commit:
theStack:
ACK a7c96f874d
sedited:
ACK a7c96f874d
glozow:
ACK a7c96f874d
Tree-SHA512: 7c185838abaf068bc96b425c3c971b73f75dfcb41dacc8b2f2543c7602f23f19d908633278b93738f18049e6bd8c845c152cfb93b289bef501c7e86ed8dae0ab
Aligns test variable naming with the `k0`/`k1` convention used consistently throughout the codebase for `SipHash` keys.
Also splits the single-param `SipHash` test from the one with extra, for clarity.
9d5021a05b script: add SCRIPT_ERR_TAPSCRIPT_EMPTY_PUBKEY (billymcbip)
Pull request description:
We currently have two callsites for `SCRIPT_ERR_PUBKEYTYPE`:
- A pre-tapscript policy error behind the `SCRIPT_VERIFY_STRICTENC` flag: 4de26b111f/src/script/interpreter.cpp (L220)
- A [consensus error](https://github.com/bitcoin/bips/blob/master/bip-0342.mediawiki?plain=1#L93) in Tapscript: 4de26b111f/src/script/interpreter.cpp (L368)
It would be good for readability and testability to have separate errors for both cases, as they are quite distinct (policy vs. consensus, format vs. emptiness).
**This PR adds `SCRIPT_ERR_TAPSCRIPT_EMPTY_PUBKEY` for the consensus error path.**
This change would make our error handling more consistent. We have more granular errors for other pubkey error paths already: `SCRIPT_ERR_WITNESS_PUBKEYTYPE`, `SCRIPT_ERR_DISCOURAGE_UPGRADABLE_PUBKEYTYPE`. We also have separate errors for MINIMAL_IF: `SCRIPT_ERR_MINIMALIF` for the policy error pre-tapscript, and `SCRIPT_ERR_TAPSCRIPT_MINIMALIF` for the consensus error post-tapscript.
Tests:
Added a test case to `script_tests` and ran `build/bin/test_bitcoin --run_test=script_tests --log_level=success`.
```
test/script_tests.cpp:144: info: check '[["aa","#SCRIPT# 0 CHECKSIG","#CONTROLBLOCK#",0.00000001],"","0x51 0x20 #TAPROOTOUTPUT#","P2SH,WITNESS,TAPROOT","TAPSCRIPT_EMPTY_PUBKEY","TAPSCRIPT: OP_CHECKSIG with empty pubkey must fail"] (with flags 165d5d)' has passed
...
```
Ran `DIR_UNIT_TEST_DATA="$(pwd)/../qa-assets/unit_test_data" build/bin/test_bitcoin --run_test=script_assets_tests --log_level=success`.
Updated `feature_taproot.py` and ran `build/test/functional/feature_taproot.py`.
Looking forward to your feedback.
ACKs for top commit:
sedited:
ACK 9d5021a05b
darosior:
utACK 9d5021a05b
sipa:
ACK 9d5021a05b
Tree-SHA512: bc0b7f64454313fe392ffb2d23aa4eca3deadc5ea1d10b3fba0b3ab4cb0575a5ddcb002dc27b4fa7aa3c180840a83d1b3e5c89351009ce7ffe684d58e1980ace
b5a7a685bb ci: Make the max number of commits tested explicit (Hodlinator)
Pull request description:
Gives less of a false sense of security.
ACKs for top commit:
maflcko:
lgtm ACK b5a7a685bb
rkrux:
crACK b5a7a685bb
janb84:
ACK b5a7a685bb
glozow:
lgtm ACK b5a7a685bb
Tree-SHA512: 9f50a86f440d6a551a0c1ff547e61b61b829e98cd0cd2d5ca65966af0b48d40582f698bcb039a7467c4b71166920413c334eac0e9e4f0141c3e02cd68555865b
ff06e2468a init: point out -stopatheight may be imprecise (brunoerg)
Pull request description:
`-stopatheight` is used to stop running bitcoind after reaching a given height. However, this feature is imprecise since some blocks can still be processed during the shutdown.
There are some previous discussions around it in https://github.com/bitcoin/bitcoin/pull/13713, https://github.com/bitcoin/bitcoin/pull/13490 and https://github.com/bitcoin/bitcoin/issues/13477. However, I'm not sure if it will get fixed since it's undesirable to burden the validation code further with this and we can bypass this behavior by using `invalidateblock` to wind back. Anyway, since at this moment its behavior is imprecise I think worth mentioning it in documentation.
ACKs for top commit:
rkrux:
re-ACK ff06e2468a
stickies-v:
ACK ff06e2468a
pablomartin4btc:
ACK ff06e2468a
jaonoctus:
re-ACK ff06e2468a
Tree-SHA512: 222d5e89021d5f9a7ce0edca44c4ce20b13f71832413dccea78ad40a01f2a615a061f8cf446d7290ed911023922adbc6fa22f0c88cff306dcd8b4ae14194e9b8
Previously, when `debug=net` is enabled, we log "New [..] peer connected"
for new inbound peers with `LogInfo`. However, `LogInfo` will get
rate-limited since https://github.com/bitcoin/bitcoin/pull/32604.
When we specifically turn on `debug=net`, we don't want these log
messages to be rate-limited.
To fix this, use `LogDebug(BCLog::NET, ...)` for potentially high-
rate inbound connections. Otherwise use `LogInfo`. This means we
don't rate-limit the messages for inbound peers when `debug=net`
is turned on but will rate-limit if we created outbound at a high
rate as these are logged via `LogInfo`.
--
I ran into this message getting rate-limited on one of my monitoring
nodes with `-logsourcelocations=1`: With logsourcelocations, one of
these lines is about 338 chars (or 338 bytes) long. We rate-limit
after more than 1048576 bytes per hour, which results in about
3100 in- and outbound connections per hour. With evicted and
instantly reconnecting connections from an entity like LinkingLion,
this can be reached fairly quickly.
Co-Authored-By: Eugene Siegel <elzeigel@gmail.com>
Co-Authored-By: Anthony Towns <aj@erisian.com.au>
"tor" as a network specification was deprecated in 60dc8e4208 in favor
of "onion" and this commit removes it and updates the relevant test.
Co-authored-by: Mara van der Laan <126646+laanwj@users.noreply.github.com>
This is a minimal behavior change and changes log output from:
[net:error] Something bad happened
[net:warning] Something problematic happened
to either
[error] Something bad happened
[warning] Something problematic happened
or, when -loglevelalways=1 is enabled:
[all:error] Something bad happened
[all:warning] Something problematic happened
Such a behavior change is desired, because all warning and error logs
are written in the same style in the source code and they are logged in
the same format for log consumers.
-BEGIN VERIFY SCRIPT-
sed --regexp-extended --in-place \
's/LogPrintLevel\((BCLog::[^,]*), BCLog::Level::(Error|Warning), */Log\2(/g' \
$( git grep -l LogPrintLevel ':(exclude)src/test/logging_tests.cpp' )
-END VERIFY SCRIPT-
As pointed out by Sjors in
https://github.com/bitcoin/bitcoin/pull/34003#discussion_r2598844209 and
https://github.com/bitcoin/bitcoin/pull/34003#discussion_r2598858386 the
original intention of having waitNext and waitTipChanged calls in the test was
to ensure that if new blocks were connected or fees were increased *during* the
waits, that the calls would wake up and return.
But the tests were written incorrectly, to generate blocks and transactions
before the wait calls instead of during the calls. So the tests were less
meaningful then they should be.
There was also a similar problem in the interruptWait test. The test was
intended to test the interruptWait method, but it was never actually calling
the method due to a missing await keyword. Instead it was testing that
miniwallet.send_self_transfer would interrupt the wait.
This commit fixes these issues by introducing a wait_and_do() helper function
to start parallel tasks and trigger an action after a wait call is started.
Co-authored-by: Sjors Provoost <sjors@sprovoost.nl>
There are a few things that are incorrect or messy in the interface_ipc.py test.
This commit tries to clean them up:
- isTestChain and isInitialBlockDownload asserts were not checking the results
of those calls, only that calls were, made because they were not checking the
responses' .result member.
- A lot of result accesses like `template.result` `mining.result` were repeated
unnecessarily because variables like `template` and `mining` were assigned
response objects instead of result objects. These variables are now changed
to point directly to results.
- Some coroutine calls were assigned to temporary `wait` before being awaited.
This was unnecessarily confusing and would make code not run in top-down
order.
- `to_dict` calls were being made to check if result variables were unset. This
was inefficient and indirect because it iterates over all fields in response
structs instead of just checking whether the result field is present. The
to_dict calls are now replaced with more direct `_has('result')` calls.
- The `res` variables used to hold various responses did not have descriptive
names. These are replaced with clearer names.
Co-authored-by: rkrux <rkrux.connect@gmail.com>
faa23738fc refactor: Enable clang-tidy bugprone-unused-return-value (MarcoFalke)
fa114be27b Add util::Expected (std::expected) (MarcoFalke)
Pull request description:
Some low-level code could benefit from being able to use `std::expected` from C++23:
* Currently, some code is using `std::optional<E>` to denote an optional error. This is fine, but a bit confusing, because `std::optional` is normally used for values, not errors. Using `std::expected<void, E>` is clearer.
* Currently, some code is using `std::variant<V, E>` to denote either a value or an error. This is fine, but a bit verbose, because `std::variant` requires a visitor or get_if/holds_alternative instead of a simple call of the `operator bool` for `std::expected`.
In theory, `util::Result` could be taught to behave similar to `std::expected` (see https://github.com/bitcoin/bitcoin/pull/34005). However, it is unclear if this is the right approach:
* `util::Result` is mostly meant for higher level code, where errors come with translated error messages.
* `std::expected` is mostly meant for lower level code, where errors could be an enum, or any other type.
* https://github.com/bitcoin/bitcoin/pull/25665 aims to minimize the memory footprint of the error by wrapping it in a unique_ptr internally. `std::expected` requires the value and error to be "nested within it" (https://cplusplus.github.io/LWG/issue4141). So from a memory-layout perspective, the two are not compatible.
* `std::expected` also comes with `std::unexpected`, which also does not map cleanly to `util::Result`.
So just add a minimal drop-in port of `std::expected`.
ACKs for top commit:
romanz:
tACK faa23738fc
sedited:
Re-ACK faa23738fc
hodlinator:
ACK faa23738fc
rkrux:
light Code Review ACK faa23738fc
ryanofsky:
Code review ACK faa23738fc, only adding `value_or` implementation and `CheckedReturnTypes` clang-tidy commit since last review.
stickies-v:
ACK faa23738fc
Tree-SHA512: fdbd0f6bf439738ffe6a68da5522f1051537f8df9c308eb90bef6bd2e06931d79f1c5da22d5500765e9cb1d801d5be39e11e10d47c9659fec1a8c8804cb7c872
48840bfc2d refactor: Prefer `<=>` over multiple relational operators (Daniel Pfeifer)
5a0f49bd26 refactor: Remove all `operator!=` definitions (Daniel Pfeifer)
Pull request description:
Remove all `operator!=` definitions and provide `operator<=>` as a replacement where all relational comparison operators were defined before.
The compiler is able to deduce missing comparison operators from `operator!=` and `operator<=>`. The compiler provided operators have the following advantages:
1. less code
2. guaranteed consistency
Refactoring that changes the implementation, or replaces it with `= default` is left for a separate PR.
ACKs for top commit:
optout21:
utACK 48840bfc2d
Chand-ra:
tACK [`48840bf`](48840bfc2d). Built the PR and ran unit tests; everything passes.
maflcko:
review ACK 48840bfc2d🌖
stickies-v:
utACK 48840bfc2d. Pretty straightforward cleanup taking advantage of C++20 improvements, nice.
janb84:
ACK 48840bfc2d
sipa:
ACK 48840bfc2d
Tree-SHA512: 7fedc4abc451c7ad611e3a960ff939a35580667222009cb30ca546e564dc9161e3e8d4d1d7d44c538d961cc8f7adba6e6dbcebcd1be370bf33aef294d06f236b
41e657aacf guix: add bitcoin-qt runtime libs doc in symbol-check (fanquake)
ef4ce19a15 depends: freetype 2.11.1 (fanquake)
Pull request description:
Update freetype to `2.11.1`.
Updating fontconfig (currently `2.12.6`) to `2.13.1` requires what looks like a hard dep on gperf; leaving that as-is for now.
Document expectations in `symbol-check.py`.
Closes#29977 (changes are based on discussion there).
ACKs for top commit:
sedited:
ACK 41e657aacf
Tree-SHA512: 71c4ccc442df0b90bebc475003eb325564111b8312c42bc7d7a9c81a2fc166fdc0814c9ddde3cfe562c3c835556e7f97107458b02a07b981b1a199bf65d5ac1d
7b90b4f5bb guix: reduce allowed exported symbols (fanquake)
Pull request description:
Need to double-check, but pretty sure this is atleast partly from #33181.
ACKs for top commit:
sedited:
Nice, ACK 7b90b4f5bb
Tree-SHA512: 538c03dc32aab9b3e18100e8ffa0d664aea5ceba6aafee9e8e0894c2d02eea3b3fb09733cf7b5bd0aefb6b56d0ac3b92f28da932e135b23f55404efd8f43664a
We run InactivityChecks() for each node everytime poll()/select() every
50ms or so. Rather than calculating the current time once for each node,
just calculate it once and reuse it.
fa4395dffd refactor: Remove unused LogPrintf (MarcoFalke)
fa05181d90 scripted-diff: LogPrintf -> LogInfo (MarcoFalke)
Pull request description:
`LogPrintf` has many issues:
* It does not mention the log severity (info).
* It is a deprecated alias for `LogInfo`, according to the dev notes.
* It wastes review cycles, because reviewers sometimes point out that it is deprecated.
* It makes the code inconsistent, when both versions of the alias are used.
Fix all issues by removing the deprecated alias.
ACKs for top commit:
ajtowns:
ACK fa4395dffd
stickies-v:
ACK fa4395dffd
rkrux:
lgtm ACK fa4395dffd
Tree-SHA512: de95d56df27b9ee33548cc7ee7595e2d253474094473089ee67787ddb171384383c683142672c3e2c1984e19eee629b2c469dc85713640a73391610581edbdbe
57b888ce0e fuzz: Add a test case for `ParseByteUnits()` (Chandra Pratap)
Pull request description:
`ParseByteUnits()` is the only parsing function in `strencodings.cpp` lacking a fuzz test. Add a test case to check the function against arbitrary strings and randomized `default_multiplier`.
ACKs for top commit:
maflcko:
lgtm ACK 57b888ce0e
dergoegge:
utACK 57b888ce0e
marcofleon:
crACK 57b888ce0e
Tree-SHA512: c16557442987437e5e0c9d9a8b016df93e513e34acb78242a1f73dabc4482632ec57eb35cb4c84f9a1ea838fa6bda2094f2a8b52ace431f8064a79fad96e9a52
This requires some small refactors to silence false-positive warnings.
Also, expand the bugprone-unused-return-value.CheckedReturnTypes option
to include util::Result, and util::Expected.
710031ebef Revert "guix: sqlite wants tcl" (Hennadii Stepanov)
4cf5ea6c3d depends: Propagate native C compiler to `sqlite` package (Hennadii Stepanov)
Pull request description:
This PR:
1. Ensures that autosetup can build the local bootstrap `jimsh0` when neither `jimsh` nor `tclsh` is available on the system.
2. Removes the `tcl` package from the Guix manifest.
This is an alternative to https://github.com/bitcoin/bitcoin/pull/33975.
ACKs for top commit:
fanquake:
ACK 710031ebef
sedited:
ACK 710031ebef
Tree-SHA512: bdaa29af977799669bfc2aa3a8d0a4a688263b99c5f06b1582fbefb71ef77be0ee6223903e8357e51a9e0a7744807174b94262c2f4a3afd9f39737b61b00863e
fd4ce55121 contrib: Count entry differences in asmap-tool diff summary (Fabian Jahr)
Pull request description:
Currently the output of `asmap-tool.py diff` returns the total number of addresses that has changed at the end of the list.
Example output currently:
```
2602:feda:c0::/48 AS1029 # was AS43126
2604:7c00:100::/40 AS29802 # was AS40244
# 0 IPv4 addresses changed; 79552154633921058212365205504 (2^96.01) IPv6 addresses changed
```
This is good indicator but in case of a longer list I would like the number of changed entries as well, since that is an easier number to parse and for debugging of certain issues also the more relevant value. This PR adds the count of changed entries to this summary output at the end. There as also a bit more structure so it's easier to parse as well.
Example new output:
```
2602:feda:c0::/48 AS1029 # was AS43126
2604:7c00:100::/40 AS29802 # was AS40244
# Summary
IPv4: 0 entries with 0 addresses changed
IPv6: 12 entries with 79552154633921058212365205504 (2^96.01) addresses changed
```
ACKs for top commit:
jurraca:
utACK [`fd4ce55121`](fd4ce55121)
janb84:
utACK fd4ce55121
hodlinator:
ACK fd4ce55121
Tree-SHA512: 97cc543eaba80a33f0291b20630411bda869d3b8d1b35ed7f36792064cb1edccc8fe4740b7229b5451a88b7bd8d68c42f96829ce4255ecac3e29d70b68061608
`ParseByteUnits()` is the only parsing function in `strencodings.cpp`
lacking a fuzz test. Add a test case to check the function against
arbitrary strings and randomized default_multiplier's.
ffcae82a68 test: exercise TransactionMerklePath with empty block; targets the MerkleComputation empty-leaves path that was only reached by fuzz tests (frankomosh)
Pull request description:
As noted in [#32243 (comment)](https://github.com/bitcoin/bitcoin/pull/32243#issuecomment-2988854482), the early return inside `MerkleComputation` when `leaves.size() == 0` was only exercised by fuzz tests.
The existing `merkle_test_empty_block` calls `BlockMerkleRoot`, which uses `ComputeMerkleRoot`, but does not exercise the `TransactionMerklePath` → `ComputeMerklePath` → `MerkleComputation` code path.
Coverage before adding test:
<img width="2459" height="66" alt="before" src="https://github.com/user-attachments/assets/ca94015a-d7c2-4281-ac60-13b22f177b67" />
Coverage after adding test:
<img width="2459" height="66" alt="after" src="https://github.com/user-attachments/assets/b1d4e1bb-af72-46ab-8898-f18db39dd2fb" />
ACKs for top commit:
kevkevinpal:
ACK [ffcae82](ffcae82a68)
maflcko:
lgtm ACK ffcae82a68
brunoerg:
code review ACK ffcae82a68
sedited:
ACK ffcae82a68
Tree-SHA512: d2499d91269c4f4f9a86011f7ad13f675834662a5bd37b0e7cbe887a7d9acf4170e53f0bdc528011fc82866b9c1dec34f4e7e9cd64cc3100591c1580a4df5d00
167df7a98c net: fix use-after-free with v2->v1 reconnection logic (Eugene Siegel)
Pull request description:
`CConnman::Stop()` resets `semOutbound`, yet `m_reconnections` is not cleared in `Stop`. Each `ReconnectionInfo` contains a `grant` member that points to the memory that `semOutbound` pointed to and `~CConnman` will attempt to access the grant field (memory that was already freed) when destroying `m_reconnections`. Fix this by calling `m_reconnections.clear()` in `CConnman::Stop()` and add appropriate annotations.
I was able to reproduce the original issue https://github.com/bitcoin/bitcoin/issues/33615 with the following diff by randomly stopping my node while it was attempting to reconnect (and verified that this patch fixes the issue, at least in my ~40-50 runs):
<details>
<summary> diff </summary>
```diff
diff --git a/src/net.cpp b/src/net.cpp
index ef1c63044a..9c1d161d8b 100644
--- a/src/net.cpp
+++ b/src/net.cpp
@@ -1918,8 +1918,8 @@ void CConnman::DisconnectNodes()
{
LOCK(m_nodes_mutex);
- const bool network_active{fNetworkActive};
- if (!network_active) {
+// const bool network_active{fNetworkActive};
+// if (!network_active) {
// Disconnect any connected nodes
for (CNode* pnode : m_nodes) {
if (!pnode->fDisconnect) {
@@ -1927,7 +1927,7 @@ void CConnman::DisconnectNodes()
pnode->fDisconnect = true;
}
}
- }
+// }
// Disconnect unused nodes
std::vector<CNode*> nodes_copy = m_nodes;
@@ -1941,7 +1941,7 @@ void CConnman::DisconnectNodes()
// Add to reconnection list if appropriate. We don't reconnect right here, because
// the creation of a connection is a blocking operation (up to several seconds),
// and we don't want to hold up the socket handler thread for that long.
- if (network_active && pnode->m_transport->ShouldReconnectV1()) {
+ if (true) {
reconnections_to_add.push_back({
.addr_connect = pnode->addr,
.grant = std::move(pnode->grantOutbound),
```
</details>
I'm curious to see if others can reproduce as well.
ACKs for top commit:
dergoegge:
Code review ACK 167df7a98c
darosior:
utACK 167df7a98c
mzumsande:
ACK 167df7a98c
Tree-SHA512: 33fdfb110a7cdae182b5cd5400eea8a271308a62dd56491e0aef8865eff24a9ea908be74e4e2e2ee00ac1cb698e46f270f56dffffe34cf2cfd79e9b1079d6531
The code was largely duplicated between coinstatsindex
and blockfilterindex.
Deduplicate it by moving it to a shared file.
slight change in behavior: the index name is no longer
part of the error msg in case of (un)serialization errors.
b0c706795c Remove unreliable seed from chainparams.cpp, and the associated README (SatsAndSports)
Pull request description:
The DNS seed `dnsseed.bitcoin.dashjr-list-of-p2p-nodes.us.` is not returning a representative sample of bitcoin nodes. It currently returns nothing later than 28.1.0, breaching the policy.
This PR removes that seed from the list of DNS seeds
### Rationale
The [policy for seeds](https://github.com/bitcoin/bitcoin/blob/master/doc/dnsseed-policy.md) includes this:
> The DNS seed results must consist exclusively of fairly selected and functioning Bitcoin nodes from the public network
A number of comments below, in response to this PR, include apparent breaches of this policy: [1](https://github.com/bitcoin/bitcoin/pull/33723#issuecomment-3458071231) [2](https://github.com/bitcoin/bitcoin/pull/33723#issuecomment-3457655364), [3](https://github.com/bitcoin/bitcoin/pull/33723#issuecomment-3457712557), in particular the first linked comment ([1](https://github.com/bitcoin/bitcoin/pull/33723#issuecomment-3458071231)) comparing the distribution at this seed to other seeds. This seed is not including anything later than 28.2.0, breaching this policy.
To ensure the policy is followed, and the seeds include a representative sample of Bitcoin nodes, this PR removes this seed from the list
### Data
I ran this:
```
# Get some ip address from that seed:
# Repeated multiple times, to get many different IPs:
dig +short dnsseed.bitcoin.dashjr-list-of-p2p-nodes.us >> dnsseed.bitcoin.dashjr-list-of-p2p-nodes.us
# For each distinct ip gathered from the seed, get basic info about the node, including it's User Agent string:
cat dnsseed.bitcoin.dashjr-list-of-p2p-nodes.us | sort -u | while read ip; do echo ===; echo $ip; nmap -p 8333 --script bitcoin-info "$ip"; done > seed_versions.txt
```
and then summarized the agents with `egrep 'User Agent' seed_versions.txt | sort | uniq -c` and got:
```
1 User Agent: /Satoshi:22.0.0/
1 User Agent: /Satoshi:22.1.0/
5 User Agent: /Satoshi:24.0.1/
1 User Agent: /Satoshi:25.1.0/
30 User Agent: /Satoshi:27.0.0/
1 User Agent: /Satoshi:27.1.0/
1 User Agent: /Satoshi:27.1.0/Knots:20240801/
1 User Agent: /Satoshi:28.0.0/
7 User Agent: /Satoshi:28.1.0/
2 User Agent: /Satoshi:28.1.0/Knots:20250305/
```
ACKs for top commit:
l0rinc:
reACK b0c706795c
delta1:
reACK b0c706795c
Crypt-iQ:
crACK b0c706795c
laanwj:
ACK b0c706795c
murchandamus:
ACK b0c706795c
RandyMcMillan:
ACK b0c7067
wiz:
ACK b0c706795c
dergoegge:
ACK b0c706795c
stickies-v:
re-ACK b0c706795c
mzumsande:
ACK b0c706795c
instagibbs:
ACK b0c706795c
Tree-SHA512: 7230b8dd24560ce6f8247e2e82ae7846ded8b91e230c59cc3643da3f5b9c12b5f025c1bb14490c19ca55f3794e81ce08106b31b3bf883d5c2dced05017123ac4
866bbb98fd cmake, test: Improve locality of `bitcoin_ipc_test` library description (Hennadii Stepanov)
ae2e438b25 cmake: Move IPC tests to `ipc/test` (Hennadii Stepanov)
Pull request description:
This PR follows up on https://github.com/bitcoin/bitcoin/pull/33445 and:
1. Organizes the IPC tests in the same way as the wallet tests.
2. Removes no longer needed `src/test/.clang-tidy.in`.
See the previous discussion:
- https://github.com/bitcoin/bitcoin/pull/33445#discussion_r2379651340
- https://github.com/bitcoin/bitcoin/pull/33445#pullrequestreview-3411868329
Additionally, the locality of the `bitcoin_ipc_test` build target description has been improved.
ACKs for top commit:
Sjors:
ACK 866bbb98fd
janb84:
ACK 866bbb98fd
ryanofsky:
Code review ACK 866bbb98fd, just adding back the suggested comment, and also fixing bad include arguments passed to target_capnp_sources. It would probably be a little better if the include fix was done in an earlier commit, since it's not really related to the other changes in the last commit, but would also be ok to make both changes at the same time.
Tree-SHA512: ed7cc817ccb88595d8516978bff0ea2560048d35b3f548e7913aec7d58b8d6ac550e230e992c527fb747bef175580be92dc4df6342e4485f3a9870dba0a25cba
dcd42d6d8f [test] wallet send 3 generation TRUC (glozow)
e753fadfd0 [wallet] never try to spend from unconfirmed TRUC that already has ancestors (glozow)
Pull request description:
Addresses https://github.com/bitcoin/bitcoin/issues/33368#issuecomment-3319935660
There is not an explicit check that the to-be-created wallet transaction would be within the {TRUC, normal} ancestor limits. This means that the wallet may create a transaction that violates these limits, but fail to broadcast it in `CommitTransaction`.
This appears to be expected behavior for the normal ancestor limits (and any other situation in which the wallet creates a tx that was rejected by mempool) and AFAIK the transaction will be rebroadcast at some point after the ancestors confirm.
1ed00a0d39/test/functional/wallet_basic.py (L502-L506)
It's a bit complex to address this for the normal ancestor limit, and probably unrealistic for the wallet to check all possible mempool policies in coin selection, but it's quite trivial for TRUC: just skip any unconfirmed UTXOs that have any ancestors. I think it would be much more helpful to the user to say there are insufficient funds.
ACKs for top commit:
achow101:
ACK dcd42d6d8f
monlovesmango:
ACK dcd42d6d8f
rkrux:
lgtm ACK dcd42d6d8f
Tree-SHA512: b4cf9685bf0593c356dc0d6644835d53e3d7089f42b65f647795257dc7f5dac90c5ee493b41ee30a1c1beb880a859db8e049d3c64a43d5ca9b3e6482ff6bddd5
Use context managers to destroy block templates. Previously, block templates
were not being destroyed before disconnecting because the destroy coroutines
were called but never awaited. It's not necessary to explicitly destroy the
templates since they will get garbage collected asynchronously, but it's good
to destroy them to make the test more predictable, and to make the destroy
calls that are present actually do something.
This change also removes `await waitnext` expressions without changing
behavior, because the previous code was misleading about what order waitNext
calls were executed.
This change is easiest to review ignoring whitespace.
Co-authored-by: Sjors Provoost <sjors@sprovoost.nl>
The changes made here were:
| From | To |
|-------------------|------------------|
| `m.count(k) == 1` | `m.contains(k)` |
| `m.count(k) == 0` | `!m.contains(k)` |
| `m.count(k) != 1` | `!m.contains(k)` |
| `m.count(k) < 1` | `!m.contains(k)` |
* `mapInfo` is instance of `std::unordered_map` and can only contain 0 or 1 value for a given key;
* similarly, `g_enabled_filter_types` and `setClientRules` are both `std::set` instances;
* lastly, while `mapTxSpends` is `std::unordered_multimap` that could potentially hold multiple values, having a size less than 1 means that the value is missing.
`QMap<WalletModel*, WalletView*> mapWalletViews` values were also migrated manually.
Co-authored-by: pablomartin4btc <pablomartin4btc@gmail.com>
Co-authored-by: fanquake <fanquake@gmail.com>
The changes made here were:
| From | To |
|-------------------|------------------|
| `m.count(k)` | `m.contains(k)` |
| `!m.count(k)` | `!m.contains(k)` |
| `m.count(k) == 0` | `!m.contains(k)` |
| `m.count(k) != 0` | `m.contains(k)` |
| `m.count(k) > 0` | `m.contains(k)` |
The commit contains the trivial, mechanical refactors where it doesn't matter if the container can have multiple elements or not
Co-authored-by: Jan B <608446+janb84@users.noreply.github.com>
The changes made here were:
| From | To |
|------------------------|------------------|
| `m.find(k) == m.end()` | `!m.contains(k)` |
| `m.find(k) != m.end()` | `m.contains(k)` |
e9536faaee contrib: fix manpage generation (fanquake)
Pull request description:
0972f55040 from #33229 broke manpage generation, because the assumption that the last word in the line containing the version number, was the version number, no-longer holds for some binaries. i.e `bitcoind`.
ACKs for top commit:
janb84:
re ACK e9536faaee
rkrux:
re-ACK e9536faaee
Tree-SHA512: 471b1800beeec3ea70d722ac2dcc26073805c8fcdf0418ceb79728cc001eb7c2f11a3d832b54a7ae68d26fe5c97934a9c87eedae7601515857e660fac7532c0a
fa6db79302 test: Avoid shutdown race in NetworkThread (MarcoFalke)
Pull request description:
Locally, I am seeing rare intermittent exceptions in the network thread:
```
stderr:
Exception in thread NetworkThread:
Traceback (most recent call last):
File "/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "./test/functional/test_framework/p2p.py", line 744, in run
self.network_event_loop.run_forever()
File "/python3.10/asyncio/base_events.py", line 603, in run_forever
self._run_once()
File "/python3.10/asyncio/base_events.py", line 1871, in _run_once
event_list = self._selector.select(timeout)
AttributeError: 'NoneType' object has no attribute 'select'
```
I can reproduce this intermittently via `while ./bld-cmake/test/functional/test_runner.py $(for i in {1..400}; do echo -n "tool_rpcauth "; done) -j 400 ; do true ; done`.
I suspect this is a race where the shutdown starts the close of the network thread while it is starting.
A different exception showing this race can be reproduced via:
```diff
diff --git a/test/functional/test_framework/p2p.py b/test/functional/test_framework/p2p.py
index 610aa4ccca..64561e157c 100755
--- a/test/functional/test_framework/p2p.py
+++ b/test/functional/test_framework/p2p.py
@@ -741,6 +741,7 @@ class NetworkThread(threading.Thread):
def run(self):
"""Start the network thread."""
+ import time;time.sleep(.1)
self.network_event_loop.run_forever()
def close(self, *, timeout=10):
```
It is trivial to reproduce via any test (e.g. `./bld-cmake/test/functional/tool_rpcauth.py`) and shows a similar traceback to the one above:
```
Exception in thread NetworkThread:
Traceback (most recent call last):
File "/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "./test/functional/test_framework/p2p.py", line 745, in run
self.network_event_loop.run_forever()
File "/python3.10/asyncio/base_events.py", line 591, in run_forever
self._check_closed()
File "/python3.10/asyncio/base_events.py", line 515, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
```
So fix the second runtime error in hope of fixing the first one as well.
ACKs for top commit:
brunoerg:
code review ACK fa6db79302
Tree-SHA512: ca352ebf7929456ea2bbfcfe4f726adcbfcfb3dc0edeaddae7f6926f998888f0bd8b987ddef60308266eeab6bffa7ebdc32f5908db9de5404df95635dae4a8f6
0972f55040 from #33229 broke manpage
generation, because the assumption that the last word in the line
containing the version number, was the version number, no-longer holds
for some binaries. i.e bitcoind.
cb7d5bfe4a test, assumeutxo: loading a wallet (backup) on a pruned node (Alfonso Roman Zubeldia)
7a365244f8 test, refactor snapshot import and background validation (Alfonso Roman Zubeldia)
Pull request description:
Adding tests in `./test/functional/wallet_assumeutxo.py` to cover the following scenario:
- test loading a wallet (backup) on a pruned node
ACKs for top commit:
fjahr:
re-ACK cb7d5bfe4a
theStack:
re-ACK cb7d5bfe4a
Tree-SHA512: 88cc419f340d31e80120e0c6cafe567efc678df27576db6e08aeab62d2b50ed1153d56f3f3343e9bae49262e38f9fb81db7769f02a4a01e4ef25c5d029c12323
fad6118586 test: Fix "typo" in written invalid content (MarcoFalke)
fab085c15f contrib: Use text=True in subprocess over manual encoding handling (MarcoFalke)
fa71c15f86 scripted-diff: Bump copyright headers after encoding changes (MarcoFalke)
fae612424b contrib: Remove confusing and redundant encoding from IO (MarcoFalke)
fa7d72bd1b lint: Drop check to enforce encoding to be specified in Python scripts (MarcoFalke)
faf39d8539 test: Clarify that Python UTF-8 mode is the default today for most systems (MarcoFalke)
fa83e3a81d lint: Do not allow locale dependent shell scripts (MarcoFalke)
Pull request description:
Historically, there was an attempt via `test/lint/lint-python-utf8-encoding.py` to enforce explicit UTF8 in every Python IO statement (`open`, `subprocess`, ...). However, the lint check has many problems:
* The check is incomplete and many IO statements lack the explicit UTF8 specification.
* It was added at a time when some systems were not UTF8 by default.
* The check is brittle, as it depends on a fragile regex.
In theory, now that the minimum Python version is 3.10 (since commit 2123c94448), the check could be replaced by `PYTHONWARNDEFAULTENCODING=1` from https://docs.python.org/3/whatsnew/3.10.html#optional-encodingwarning-and-encoding-locale-option. However, this comes with many other problems:
* All our Python scripts already assume and require UTF8 to be set externally. On almost all modern systems, this is already the default. Some Windows versions do not have UTF8 by default and require `PYTHONUTF8=1` to be set for the tests to run already today (with or without the changes in this pull). Also, the CI and many other Bash scripts force UTF8 via `LC_ALL`. Finally, Python 3.15 will likely enable UTF8 on *all* systems by default, per https://peps.python.org/pep-0686/#abstract.
* So adding UTF8 to every single IO call is redundant, verbose, and confusing, given that it is the expected default.
So fix all issues, by:
* Removing the `test/lint/lint-python-utf8-encoding.py` check.
* Removing the encoding on the individual IO calls.
* Clarifying the existing docs around the existing UTF8 requirement and assumption.
Obviously, every IO call is still free to specify UTF8 or any other encoding explicitly, if there is a documented need for it in the future.
ACKs for top commit:
theStack:
re-ACK fad6118586
laanwj:
Re-ACK fad6118586
Tree-SHA512: 78025ea3508597d2299490347614f0ee3e4c66e3ba559ff50e498045a9c8bbd92f3a5ced18719d8fcebbd1e47bdbb56a0c85a5b73b425adb0ea4f02fe69c3149
The existing IBD disable was added at a time when CheckForkWarningConditions
did also sophisticated fork detection that could lead to false positives
during IBD (55ed3f1475).
The fork detection logic doesn't exist anymore
(since fa62304c97), so the IBD check is no
longer necessary.
Displaying the log at startup will help node operators diagnose the
problem better.
Also unify log message and alert warning text, since a long invalid chain
could be due to chainstate corruption or an actual consensus incompatibility
with peers. Previously the log assumed the former and the alert the latter.
Currently, if database corruption leads to a block being marked as
invalid incorrectly, we can get stuck in an infinite headerssync
loop with no indication what went wrong or how to fix it.
With the added log message, users will receive an explicit warning after each
failed headerssync attempt with an outbound peer.
804329400a fuzz: gate mempool entry based on weight (Greg Sanders)
Pull request description:
The mempool implementation now uses TxGraph with entries using FeePerWeight, not vsize. This means our package_rbf harness will erroneously add more transaction weight than we can support inside of FeeFrac. Gate more aggressively using WITNESS_SCALE_FACTOR.
Fixes https://github.com/bitcoin/bitcoin/issues/33981
ACKs for top commit:
sdaftuar:
ACK 804329400a
ismaelsadeeq:
utACK 804329400a
dergoegge:
utACK 804329400a
Tree-SHA512: e78d0f73f9b9cbb8c0db1e8e91dbffeb4110cf8113e90f34af5c132acf0819c54254891a4dd5da63016e4edf9d8e886f469f959bd3504b7deb66989d96fe4cf1
fa45a1503e log: Use LogWarning for non-critical logs (MarcoFalke)
fa0018d011 log: Use LogError for fatal errors (MarcoFalke)
22229de728 doc: Fix typo in init log (MarcoFalke)
Pull request description:
Logging supports severity levels above info via the legacy `LogPrintf`. So use the more appropriate `LogError` or `LogWarning`, where it applies.
This has a few small benefits:
* It often allows to remove the manual and literal "error: ", "Warning:", ... prefixes. Instead the uniform log level formatting is used.
* It is easier to grep or glance for more severe logs, which indicate some kind of alert.
* `LogPrintf` didn't indicate any severity level, but it is an alias for `LogInfo`. So having the log level explicitly spelled out makes it easier to read the code.
* Also, remove the redundant trailing `\n` newline, while touching.
* Also, remove the `__func__` formatting in the log string, which is redundant with `-logsourcelocations`. Instead, use a unique log string for each location.
ACKs for top commit:
l0rinc:
Code review ACK fa45a1503e
stickies-v:
ACK fa45a1503e
rkrux:
crACK fa45a1503e
Tree-SHA512: 516d439c36716f969c6e82d00bcda03c92c8765a9e41593b90052c86f8fa3a3dacbb2c3dc98bfc862cefa54cae34842b488671a20dd86cf1d15fb94aa5563406
BIP 379 allows height and time locks that have no consensus meaning in BIP 68 / BIP 112.
This is used by some protocols like Lightning to encode extra data, but is unsafe when
used unintentionally. E.g. older(65536) is equivalent to older(1).
This commit emits a warning when importing such a descriptor.
It introduces a helper ForEachNode to traverse all miniscript nodes.
b8d279a81c doc: add comment to explain correctness of GatherClusters() (Suhas Daftuar)
aba7500a30 Fix parameter name in getmempoolcluster rpc (Suhas Daftuar)
6c1325a091 Rename weight -> clusterweight in RPC output, and add doc explaining mempool terminology (Suhas Daftuar)
bc2eb931da Require mempool lock to be held when invoking TRUC checks (Suhas Daftuar)
957ae23241 Improve comments for getTransactionAncestry to reference cluster counts instead of descendants (Suhas Daftuar)
d97d6199ce Fix comment to reference cluster limits, not chain limits (Suhas Daftuar)
a1b341ef98 Sanity check feerate diagram in CTxMemPool::check() (Suhas Daftuar)
23d6f457c4 rpc: improve getmempoolcluster output (Suhas Daftuar)
d2dcd37aac Avoid using mapTx.modify() to update modified fees (Suhas Daftuar)
d84ffc24d2 doc: add release notes snippet for cluster mempool (Suhas Daftuar)
b0417ba944 doc: Add design notes for cluster mempool and explain new mempool limits (Suhas Daftuar)
2d88966e43 miner: replace "package" with "chunk" (Suhas Daftuar)
6f3e8eb300 Add a GetFeePerVSize() accessor to CFeeRate, and use it in the BlockAssembler (Suhas Daftuar)
b5f245f6f2 Remove unused DEFAULT_ANCESTOR_SIZE_LIMIT_KVB and DEFAULT_DESCENDANT_SIZE_LIMIT_KVB (Suhas Daftuar)
1dac54d506 Use cluster size limit instead of ancestor size limit in txpackage unit test (Suhas Daftuar)
04f65488ca Use cluster size limit instead of ancestor/descendant size limits when sanity checking TRUC policy limits (Suhas Daftuar)
634291a7dc Use cluster limits instead of ancestor/descendant limits when sanity checking package policy limits (Suhas Daftuar)
fc18ef1f3f Remove ancestor and descendant vsize limits from MemPoolLimits (Suhas Daftuar)
ed8e819121 Warn user if using -limitancestorsize/-limitdescendantsize that the options have no effect (Suhas Daftuar)
80d8df2d47 Invoke removeUnchecked() directly in removeForBlock() (Suhas Daftuar)
9292570f4c Rewrite GetChildren without sets (Suhas Daftuar)
3e39ea8c30 Rewrite removeForReorg to avoid using sets (Suhas Daftuar)
a3c31dfd71 scripted-diff: rename AddToMempool -> TryAddToMempool (Suhas Daftuar)
a5a7905d83 Simplify removeRecursive (Suhas Daftuar)
01d8520038 Remove unused argument to RemoveStaged (Suhas Daftuar)
bc64013e6f Remove unused variable (cacheMap) in mempool (Suhas Daftuar)
Pull request description:
As suggested in the main cluster mempool PR (https://github.com/bitcoin/bitcoin/pull/28676#pullrequestreview-3177119367), I've pulled out some of the non-essential optimizations and cleanups into this separate PR.
Will continue to add more commits here to address non-blocking suggestions/improvements as they come up.
ACKs for top commit:
instagibbs:
ACK b8d279a81c
sipa:
ACK b8d279a81c
Tree-SHA512: 1a05e99eaf8db2e274a1801307fed5d82f8f917e75ccb9ab0e1b0eb2f9672b13c79d691d78ea7cd96900d0e7d5031a3dd582ebcccc9b1d66eb7455b1d3642235
The mempool implementation now uses TxGraph with entries
using FeePerWeight, not vsize. This means our package_rbf
harness will erroneously add more transaction weight than we
can support inside of FeeFrac. Gate more aggressively using
WITNESS_SCALE_FACTOR.
fe1815d48f cmake: Make `BUILD_KERNEL_TEST` depend on `BUILD_KERNEL_LIB` (Hennadii Stepanov)
Pull request description:
The CMake script in the `test/kernel` subdirectory is already gated by `BUILD_KERNEL_LIB`:f6acbef108/src/CMakeLists.txt (L405-L409)
As a result, the following configuration summary is misleading:
```
$ cmake -B build -DBUILD_KERNEL_LIB=OFF -DBUILD_KERNEL_TEST=ON
<snip>
bitcoin-chainstate (experimental) ... OFF
libbitcoinkernel (experimental) ..... OFF
kernel-test (experimental) .......... ON
<snip>
```
This PR fixes the behaviour by making the `BUILD_KERNEL_TEST` option explicitly depend on `BUILD_KERNEL_LIB`.
ACKs for top commit:
maflcko:
lgtm ACK fe1815d48f
sedited:
ACK fe1815d48f
Tree-SHA512: 24524d43b195b0e3907f3257ef907c5ead8e9921b888bc82765f4dbbe44728b92956233c8fe624e8509bf8146a41cf8c1ac26f6043b8a21f681ad2ae19bebc5d
49c6728535 cmake: Set `WITH_ZMQ` to `ON` in Windows presets (Hennadii Stepanov)
Pull request description:
The "zeromq" feature is already enabled by default in `vcpkg.json`, and there appears to be no reason to omit this configuration option when building on Windows.
ACKs for top commit:
maflcko:
lgtm ACK 49c6728535
Tree-SHA512: acaef1eba56e75f9979db1809c6ebac59b2ed49002ae557fcb172f6119f6b8927580583616628f24d71ccbb32544f4d96317ff1d3125a9b5446ae89d1d318de0
Also update the help text for -limitancestorcount/-limitdescendantcount to
explain they no longer affect the mempool, and are only used by the wallet for
coin selection.
Also improve test coverage for removeForReorg by creating a scenario where
there are in-mempool descendants that are only invalidated due to an in-mempool
parent no longer spending a mature coin.
The "zeromq" feature is already enabled by default in `vcpkg.json`, and
there appears to be no reason to omit this configuration option when
building on Windows.
2e27bd9c3a ci: Add Windows + UCRT jobs for cross-compiling and native testing (Hennadii Stepanov)
bd130db994 ci: Rename items specific to Windows + MSVCRT (Hennadii Stepanov)
Pull request description:
This PR is part of the ongoing effort to migrate to the modern UCRT runtime for cross-compiled Windows binaries, including release builds.
For more details about this migration, see:
- https://github.com/bitcoin/bitcoin/issues/30210
- https://github.com/bitcoin/bitcoin/pull/33593
MSVCRT-related CI jobs should be removed from the CI framework once the migration to UCRT is complete.
ACKs for top commit:
maflcko:
review ACK 2e27bd9c3a 🖊
fanquake:
ACK 2e27bd9c3a
Tree-SHA512: 222ca5e54646bcce9db6e20191d5891e988274e18b2f30085de6435a3b288a9d0fc414e8f76342e275ae58ee6603f751933d1faa8bdff446edf2695091f8ca4c
Empty public keys in tapscript are rejected by consensus rules, independent of SCRIPT_VERIFY_STRICTENC. Add SCRIPT_ERR_TAPSCRIPT_EMPTY_PUBKEY to distinguish this from STRICTENC policy failures currently reported as SCRIPT_ERR_PUBKEYTYPE.
libfreetype and libfontconfig are our two remaining runtime libs for
bitcoin-qt. According to #29977 Ubuntu 22.04 should be considered the
baseline for what is supported. Document that.
Closes#29977.
3e01b5d0e7 contrib: rename gen-sdk to gen-sdk.py (fanquake)
c1213a35ab macdeploy: disable compression in macOS gen-sdk script (fanquake)
a33d034545 contrib: more selectively pick files for macOS SDK (fanquake)
Pull request description:
This includes three changes. The first is to more selectively pick files for inclusion into our macOS SDK tarball (skip manpages, binaries etc), which is nice because it redues the size of the tarball (from ~80mb to 20mb), and makes the size increase that happens with the next commit, less-bad.
The second change removes compression of the tarball. Starting with Python 3.11, Pythons gzip might delegate to zlib. Depending on the OS, i.e Ubuntu vs Fedora, the underlying zlib implementation might differ, resulting in different output.
For now, or until a better solution exists, remove compression. This results in the SDK increasing in size to ~157mb. Which is not unreasonable, to regain determinism (and would be significantly worse without the previous commit).
See: https://docs.python.org/3/library/gzip.html#gzip.compress
The third renames `gen-sdk` to `gen-sdk.py`, so that it will be linted, along with the rest of our Python files.
Fixes#31873. We could probably also put this into 30.x.
ACKs for top commit:
stickies-v:
ACK 3e01b5d0e7 modulo the new .tar SDK being uploaded
davidgumberg:
Tested ACK 3e01b5d0e7
Tree-SHA512: 272164a98e0e6f10822870162c1b3a405693c2f64d3ed085a2d2243a48641d940704b5ef6022256915ac9cf383e87a4f8d4dc2ec4eaa9d25e2abd30f5498778b
e07e57368e ci: clear out space on centos job (will)
Pull request description:
Fixes#33293
Clear out space on jobs running on GHA by deleteing unnecessary files.
Raised in #33293 which pointed to a solution like b7f04d7822 which is adapted slightly here.
Only runs when cache provider (runner) is `gha`.
A run on my fork can be seen here: https://github.com/willcl-ark/bitcoin/actions/runs/19703413734/job/56444984809
ACKs for top commit:
maflcko:
lgtm ACK e07e57368e
m3dwards:
ACK e07e57368e
janb84:
ACK e07e57368e
Tree-SHA512: 723589df4c434dd3eaed43acefe25f1788837743882e910e79eceee25e2bd98990cd01b8b80a46ba82418867b32c5ee1b96341223696244504e118eae6ad4a16
c0bfe72f6e Change Parse descriptor argument to string_view (Sjors Provoost)
Pull request description:
While investigating a silent merge conflict in #33135 I noticed that #32983 changed the descriptor `Parse` function signature from `const std::string& descriptor` to `std::span<const char> descriptor`.
Calling that new version of `Parse` with a string literal will trigger a confusing "Invalid characters in payload" due to the trailing "\0".
It can be worked around by having (the test) wrap string literals in `std::string()`, but that's easy to forget.
Using `string_view` is easier and more compact than (as a previous version of this PR did) checking for trailing `\0`.
Also add a test.
ACKs for top commit:
maflcko:
review ACK c0bfe72f6e🍨
enirox001:
tACK c0bfe72
stickies-v:
ACK c0bfe72f6e
rkrux:
crACK c0bfe72f6e
Tree-SHA512: 6b20307f834dae66826c8763f6c2ba0071f4e369375184cb5ff8543b85220fcaf33a47ddb065e418d1af3ed9a3fac401a7854f8924f52aab2b000b1f65328f2c
52230a7f69 test: check for output to stdout in `TestShell` test (Sebastian Falbesoner)
Pull request description:
This is a small follow-up PR to the recently added `TestShell` test (#33546), verifying the stdout message "TestShell is already running!" when trying to instantiate a second instance.
ACKs for top commit:
maflcko:
lgtm ACK 52230a7f69
rkrux:
crACK 52230a7f69
Tree-SHA512: 096d70e1bd0f09c1b389e58fa4b880442406c56f0c8ef8b8fbd0627081bc390b1ce5d6032bcca19b03206b7a444d9c523f9b62078b5ca5b7f1ae3c57bb4129c9
CConnman::Stop() resets semOutbound, yet m_reconnections is not
cleared in Stop. Each ReconnectionInfo contains a grant member
that points to the memory that semOutbound pointed to and ~CConnman
will attempt to access the grant field (memory that was already
freed) when destroying m_reconnections. Fix this by calling
m_reconnections.clear() in CConnman::Stop() and add appropriate
annotations.
2909655fba fix: remove redundant mempool lock in ChainImpl::isInMempool() (Fibonacci747)
Pull request description:
This PR removes an unnecessary `LOCK(mempool->cs)` in `ChainImpl::isInMempool()`. The method calls `CTxMemPool::exists()`, which already locks `mempool->cs` internally. Because the mempool mutex is a RecursiveMutex, double-locking was safe but redundant. Dropping the outer lock matches patterns used elsewhere in ChainImpl (e.g. `hasDescendantsInMempool()` and `GetTransactionAncestry()` callers) where mempool read APIs are invoked without an additional lock and rely on the callee’s internal locking. `isRBFOptIn()` remains unchanged since `IsRBFOptIn(tx, pool)` explicitly requires the caller to hold `pool.cs` as indicated by its thread-safety annotation.
ACKs for top commit:
maflcko:
lgtm ACK 2909655fba
instagibbs:
utACK 2909655fba
stickies-v:
ACK 2909655fba
Tree-SHA512: 4dfd88e01d8c7a4b6ceb3c736243fb22bfee5ccfc422d134acb633b908ca14c807637a2aa20de89e86e583b23ec70a1d121d77e35af60e114d93971b2a4bfd3b
Prior to cluster mempool, a policy was in place that
disallowed non-TRUC transactions from being
TX_RECONSIDERABLE in a package setting if it was below
minrelay. This was meant to simplify reasoning about mempool
trimming requirements with non-trivial transaction
topologies in the mempool. This is no longer a concern
post-cluster mempool, so this is relaxed.
In effect, this makes 0-value parent transactions relayable
through the network without the TRUC restrictions and
thus the anti-pinning protections.
Clear out space on the centos job be deleteing unnecessary files.
Raised by #33293 which pointed to a solution like b7f04d7822
Only runs when cache provider (runner) is `gha`, and on the CentOS job.
70d9e8f0a1 fix: reorg behaviour in mempool tests to match real one (yuvicc)
540ed333f6 Move the create_empty_fork method to the test framework's blocktools.py module to enable reuse across multiple tests. (yuvicc)
Pull request description:
Updated functional tests to replace direct use of `invalidateblock` with proper fork-based reorg behaviour. The direct invalidation approach bypasses important validation checks and has depth limitations(10 block) that don't match real-world reorg scenarios. For more details see #32531.
Fixes#32531
ACKs for top commit:
instagibbs:
reACK 70d9e8f0a1
theStack:
re-ACK 70d9e8f0a1
Tree-SHA512: 8aae298bfa295b4e0e4627b522e9eac549399008fd8e336a66f8c9950c886917da0b3f0bdc62d0c8ea2b8082f36639300cac4070986a7766398e15bc1f666da5
3e4355314b depends: latest config.sub (fanquake)
04eb84fe3f depends: latest config.guess (fanquake)
Pull request description:
It's been about a year since these were last updated.
Pull in the latest versions.
ACKs for top commit:
hebasto:
ACK 3e4355314b, I have reviewed the code and it looks OK.
Tree-SHA512: f18a0b95e71588e9f1ea55efb6379664aa6e9154801448e9425362414c3f3c4dab29dbe0e3ab02c46ac1f2e2ad1d067bc6feb8c550ccde37cabd1c0bd9d1b87c
Starting with Python 3.11, Pythons gzip might delegate to zlib.
Depending on the OS, i.e Ubuntu vs Fedora, the underlying zlib
implementation might differ, resulting in different output.
For now, or until a better solution exists, disable compression. This
results in the SDK increasing in size to ~157mb. Which is not
unreasonable, to regain determinism (and would be significantly worse
without the previous commit).
See: https://docs.python.org/3/library/gzip.html#gzip.compress
Co-authored-by: stickies-v <stickies-v@protonmail.com>
All touched Python scripts already assume and require UTF8, so manually
specifying encoding or decoding for functions in the subprocess module
is redundant to just using text=True, which exists since Python 3.7
Historically, the headers have been bumped some time after a file has
been touched. Do it now to avoid having to touch them again in the
future for that reason.
-BEGIN VERIFY SCRIPT-
sed -i --regexp-extended 's;( 20[0-2][0-9])(-20[0-2][0-9])? The Bitcoin Core developers;\1-present The Bitcoin Core developers;g' $( git show --pretty="" --name-only HEAD~0 )
-END VERIFY SCRIPT-
The encoding arg is confusing, because it is not applied consistently
for all IO.
Also, it is useless, as the majority of files are ASCII encoded, which
are fine to encode and decode with any mode.
Moreover, UTF-8 is already required for most scripts to work properly,
so setting the encoding twice is redundant.
So remove the encoding from most IO. It would be fine to remove from all
IO, however I kept it for two files:
* contrib/asmap/asmap-tool.py: This specifically looks for utf-8
encoding errors, so it makes sense to sepecify the utf-8 encoding
explicitly.
* test/functional/test_framework/test_node.py: Reading the debug log in
text mode specifically counts the utf-8 characters (not bytes), so it
makes sense to specify the utf-8 encoding explicitly.
The check was incomplete and brittle. A better check would be to enable
`PYTHONWARNDEFAULTENCODING=1`
https://docs.python.org/3/whatsnew/3.10.html#optional-encodingwarning-and-encoding-locale-option
However, it is unclear what the goal of adding explicit encodings
everywhere is, given that:
* Most modern systems already have UTF-8 enabled by default, except for
Windows.
* Python 3.15 will likely enable it globally by default, according to
https://peps.python.org/pep-0686/#abstract
* Adding the explicit encodings will bloat all code for no benefit.
So remove the lint check and drop all redundant encoding= kwargs.
All encoding= that are set for a reason, are kept.
It will likely be the default for all systems, starting with Python
3.15, according to https://peps.python.org/pep-0686/#abstract.
It is hard to find a system other than Windows that has it not enabled
today. Nonetheless, Bitcoin Core requires UTF-8 in scripts and normally
enforces it via LC_ALL=C.UTF-8 or PYTHONUTF8=1.
Bash is discouraged, and there was never a need to write locale
dependent Bash.
So remove the option and clarify that the LC_ALL settings enable UTF-8
mode in Python.
Also changes the the non-constant variable NUM_WALLETS to lower case and
refactors the success case scenarios to reuse existing code.
Co-authored-by: rkrux <rkrux.connect@gmail.com>
The Bash script was acceptable, but CI_EXEC_CMD_PREFIX was a single
string, relying on brittle word splitting that the shellcheck SC2086
would warn about.
So just fix that by moving everything to the Python script and deleting
the Bash script.
This also removes the need to export the CI_CONTAINER_ID env var.
In theory one could run the CI without the rsync package installed, and
with DANGER_RUN_CI_ON_HOST=1. However, this seems to be an edge case.
Simply requiring rsync to be installed is less code and avoids brittle
edge cases around rsync failures.
It contains a large `bash -c` string, which is hard to parse. So pull
out components:
* CI_EXEC is only called with absolute folders as args, so the `cd` is
not needed in CI_EXEC. It is only needed to specify the working dir of
running the tests in 03_test_script.sh, so move it there.
* The PATH modification is only needed after commit
4756114e50 to check that depends does
work properly, even when the PATH contains a space.
* This allows to also drop the `bash -c` and use the proper and safer
"$@" to forward args without the risk of word splitting.
This move-only refactor clarifies that macos assumes and requires
DANGER_RUN_CI_ON_HOST.
So move the snippet under the condition for self-documenting code.
Can be reviewed with the git options:
--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
The `retry` script is required for CI_RETRY_EXE and there are two ways
to put it into PATH:
* When running in a container engine, by copying it into /usr/bin
* When running without a container engine, by prepending its location to PATH
fa0fee44a8 ci: Remove redundant busybox option (MarcoFalke)
fa102ec69f doc: Shorten ci name (MarcoFalke)
2222223780 doc: Remove bash -c wrapper (MarcoFalke)
Pull request description:
The option was fine, but now that there is a dedicated Alpine Linux task, which uses BusyBox, it seems redundant.
(See: `ci/test/00_setup_env_native_alpine_musl.sh`)
So remove the `USE_BUSY_BOX` option, along with the `BINS_SCRATCH_DIR` env var.
Also includes two small ci-doc fixups.
ACKs for top commit:
fanquake:
ACK fa0fee44a8
Tree-SHA512: abad7888cbf5833cc322d133187f74653c232c03b8949ad06dd1ea5ace14127871322fe682268ebec7720963615c3f113c7660a2055bf48bbb9ff9a730221d7e
c34bc01b2f doc: clarify and cleanup macOS fuzzing notes (dergoegge)
Pull request description:
* Remove or consolidate macOS notes sprinkled throughout the doc into dedicated section
* Note that support for fuzzing on macOS is not maintained
Closes#33731
ACKs for top commit:
frankomosh:
ACK c34bc01
janb84:
Concept ACK c34bc01b2f
darosior:
reACK c34bc01b2f
brunoerg:
ACK c34bc01b2f
rkrux:
ACK c34bc01b2f
ismaelsadeeq:
reACK c34bc01b2f
Tree-SHA512: 76a79a102b4941b31bd05b65aa37beb0f48d8bb7b07f4ec2fa55be7e6b4880ce7031fe9a7764984dbd879bf6238dc61689006a9225deee39b8b759722078b808
ade0397f59 txgraph: drop move assignment operator (Anthony Towns)
Pull request description:
This removes the only place where move-assignment of `TxGraph::Ref` is used (in tests), and drops supports for it.
Suggested in https://github.com/bitcoin/bitcoin/pull/33629#discussion_r2518940184
ACKs for top commit:
l0rinc:
reACK ade0397f59
instagibbs:
ACK ade0397f59
Tree-SHA512: 0f49e454d0d44817278cbd3fbb8fce89c64c6f6c6d852bea26c728b9f1b6827a0f2b8731ac2031150af92b0ec479c2fe4ece01256fb3b6b2bf941f16c0e2c541
096924d39d kernel: add btck_block_tree_entry_equals (stickies-v)
Pull request description:
`BlockTreeEntry` objects are often compared. This happens frequently in our own codebase and seems likely to be the case for clients, too. Users can already work around this by comparing based on block hash (and optionally height as belt-and-suspenders), but I think this should be part of the interface for performance and consistency reasons.
Note: perhaps this is too ad-hoc, and we should extend this PR to add the operator for more types? `BlockTreeEntry` is the main one I've needed this for in developing `py-bitcoinkernel`, though.
ACKs for top commit:
maflcko:
review ACK 096924d39d📓
TheCharlatan:
ACK 096924d39d
yuvicc:
Code Review ACK 096924d39d
Tree-SHA512: a0c08c01ab6c855aec4e2b2b898e9550493cd4cf8c6e1fe9e4fe5039d0d9ef3bffb2f2ab0454c7cc449b9deedd7889f5fd7b5f100fa706a855023af4adb803c6
fa7e222a23 clang-format: Set Bitcoin Core IncludeCategories (MarcoFalke)
Pull request description:
Replace the default llvm include categories with the ones specific to Bitcoin Core.
Ref: https://releases.llvm.org/17.0.1/tools/clang/docs/ClangFormatStyleOptions.html#includecategories
Also, format a file as example. To test this, the diff in src/test needs
to be reverted. Also `IncludeBlocks: Regroup` needs to be set. Then
`clang-format -i src/test/blockchain_tests.cpp` should recreate the
diff.
```diff
diff --git a/src/.clang-format b/src/.clang-format
index 15335fe9ae..579079095f 100644
--- a/src/.clang-format
+++ b/src/.clang-format
@@ -99,3 +99,3 @@ IfMacros:
- KJ_IF_MAYBE
-IncludeBlocks: Preserve
+IncludeBlocks: Regroup
IncludeCategories:
```
ACKs for top commit:
l0rinc:
ACK fa7e222a23
Tree-SHA512: f500060858898c48f709e8fecfe4c41fc05645e2ff3b1504cc82ce354129642dd658a2311fd038363068ed682d016c32b56d935783c8084591ac5152b413fa2c
fa7ea497c3 ci: Run GUI unit tests in cross-Windows task (MarcoFalke)
Pull request description:
Most users of the cross-compiled releases for Windows will most likely pick the GUI, so running the cross-compiled GUI unit tests on a real Windows seems desirable.
ACKs for top commit:
fanquake:
ACK fa7ea497c3 - didn't test.
Tree-SHA512: 6826474295ecf0170e45a644100887d5e53899b4a3eb49635e5cc62c774b512b383bc0b2de15ac854a527da0779b5ab8f52aae9f817c7f2971a0cbfc68c027b7
8343a9ffcc test: add `-alertnotify` test for large work invalid chain warning (Sebastian Falbesoner)
Pull request description:
This PR adds missing test coverage for the `LARGE_WORK_INVALID_CHAIN` fork warning, checked with the `-alertnotify` option:
ead849c9f1/src/validation.cpp (L2033-L2040)
Found that this is missing during review of #32587. The test works by first creating a bunch of invalid blocks, that are first announced by headers and then submitted fully in reverse (invalid tip first), in order to set `m_best_invalid` to that value, finally leading to the best chain / invalid chain gap of >= 6 blocks. I'd be curious if there are other (more realistic?) ways to test this. One simple alternative is just to call `invalidateblock` twice (once at the tip, once at the base of the invalid chain).
Note that the written warning doesn't include the exclamation mark, as it is removed via `SanitizeString` in the `AlertNotify` function.
ACKs for top commit:
brunoerg:
reACK 8343a9ffcc
mzumsande:
re-ACK 8343a9ffcc
Tree-SHA512: d81e9ce7622026498cad5cdcdb867a22068670983737502888c72c72209ca6ff183e77d7429f758765a42c25cda439e01f795884864ac6fe6ff258a98d0bbcbc
fa9537cde1 ci: Use latest Xcode that the minimum macOS version allows (MarcoFalke)
Pull request description:
Changing the CI policy to use the *latest* Xcode (instead of the *earliest*), allowed by the Bitcoin Core minimum supported macOS version, makes sense: While this may require the developer or user to install a later security point-release on macOS, this should generally be fine and it is even expected that users run the latest supported security release of their operating system. Also, in practise, this often doesn't result in a visible change anyway: This specific change from Xcode 16.0 to 16.2 does not change any behavior of the Bitcoin Core CI, because there are no C++-related changes in those point releases.
ACKs for top commit:
l0rinc:
ACK fa9537cde1
vasild:
ACK fa9537cde1
fanquake:
ACK fa9537cde1 - seems fine.
Tree-SHA512: ca0e6c4fc396cad7569d04d7a648bc5318cce171f2fa941d8b00a30cf49b054e70a0133738533c744bccd65baf9129ba527b1950b15e10c3f7953a24ef359977
17cf9ff7ef Use cluster size limit for -maxmempool bound, and allow -maxmempool=0 in general (Suhas Daftuar)
315e43e5d8 Sanity check `GetFeerateDiagram()` in CTxMemPool::check() (Suhas Daftuar)
de2e9a24c4 test: extend package rbf functional test to larger clusters (Suhas Daftuar)
4ef4ddb504 doc: update policy/packages.md for new package acceptance logic (Suhas Daftuar)
79f73ad713 Add check that GetSortedScoreWithTopology() agrees with CompareMiningScoreWithTopology() (Suhas Daftuar)
a86ac11768 Update comments for CTxMemPool class (Suhas Daftuar)
9567eaa66d Invoke TxGraph::DoWork() at appropriate times (Suhas Daftuar)
6c5c44f774 test: add functional test for new cluster mempool RPCs (Suhas Daftuar)
72f60c877e doc: Update mempool_replacements.md to reflect feerate diagram checks (Suhas Daftuar)
21693f031a Expose cluster information via rpc (Suhas Daftuar)
72e74e0d42 fuzz: try to add more code coverage for mempool fuzzing (Suhas Daftuar)
f107417490 bench: add more mempool benchmarks (Suhas Daftuar)
7976eb1ae7 Avoid violating mempool policy limits in tests (Suhas Daftuar)
84de685cf7 Stop tracking parents/children outside of txgraph (Suhas Daftuar)
88672e205b Rewrite GatherClusters to use the txgraph implementation (Suhas Daftuar)
1ca4f01090 Fix miniminer_tests to work with cluster limits (Suhas Daftuar)
1902111e0f Eliminate CheckPackageLimits, which no longer does anything (Suhas Daftuar)
3a646ec462 Rework RBF and TRUC validation (Suhas Daftuar)
19b8479868 Make getting parents/children a function of the mempool, not a mempool entry (Suhas Daftuar)
5560913e51 Rework truc_policy to use descendants, not children (Suhas Daftuar)
a4458d6c40 Use txgraph to calculate descendants (Suhas Daftuar)
c8b6f70d64 Use txgraph to calculate ancestors (Suhas Daftuar)
241a3e666b Simplify ancestor calculation functions (Suhas Daftuar)
b9cec7f0a1 Make removeConflicts private (Suhas Daftuar)
0402e6c780 Remove unused limits from CalculateMemPoolAncestors (Suhas Daftuar)
08be765ac2 Remove mempool logic designed to maintain ancestor/descendant state (Suhas Daftuar)
fc4e3e6bc1 Remove unused members from CTxMemPoolEntry (Suhas Daftuar)
ff3b398d12 mempool: eliminate accessors to mempool entry ancestor/descendant cached state (Suhas Daftuar)
b9a2039f51 Eliminate use of cached ancestor data in miniminer_tests and truc_policy (Suhas Daftuar)
ba09fc9774 mempool: Remove unused function CalculateDescendantMaximum (Suhas Daftuar)
8e49477e86 wallet: Replace max descendant count with cluster_count (Suhas Daftuar)
e031085fd4 Eliminate Single-Conflict RBF Carve Out (Suhas Daftuar)
cf3ab8e1d0 Stop enforcing descendant size/count limits (Suhas Daftuar)
89ae38f489 test: remove rbf carveout test from mempool_limit.py (Suhas Daftuar)
c0bd04d18f Calculate descendant information for mempool RPC output on-the-fly (Suhas Daftuar)
bdcefb8a8b Use mempool/txgraph to determine if a tx has descendants (Suhas Daftuar)
69e1eaa6ed Add test case for cluster size limits to TRUC logic (Suhas Daftuar)
9cda64b86c Stop enforcing ancestor size/count limits (Suhas Daftuar)
1f93227a84 Remove dependency on cached ancestor data in mini-miner (Suhas Daftuar)
9fbe0a4ac2 rpc: Calculate ancestor data from scratch for mempool rpc calls (Suhas Daftuar)
7961496dda Reimplement GetTransactionAncestry() to not rely on cached data (Suhas Daftuar)
feceaa42e8 Remove CTxMemPool::GetSortedDepthAndScore (Suhas Daftuar)
21b5cea588 Use cluster linearization for transaction relay sort order (Suhas Daftuar)
6445aa7d97 Remove the ancestor and descendant indices from the mempool (Suhas Daftuar)
216e693729 Implement new RBF logic for cluster mempool (Suhas Daftuar)
ff8f115dec policy: Remove CPFP carveout rule (Suhas Daftuar)
c3f1afc934 test: rewrite PopulateMempool to not violate mempool policy (cluster size) limits (Suhas Daftuar)
47ab32fdb1 Select transactions for blocks based on chunk feerate (Suhas Daftuar)
dec138d1dd fuzz: remove comparison between mini_miner block construction and miner (Suhas Daftuar)
6c2bceb200 bench: rewrite ComplexMemPool to not create oversized clusters (Suhas Daftuar)
1ad4590f63 Limit mempool size based on chunk feerate (Suhas Daftuar)
b11c89cab2 Rework miner_tests to not require large cluster limit (Suhas Daftuar)
95a8297d48 Check cluster limits when using -walletrejectlongchains (Suhas Daftuar)
95762e6759 Do not allow mempool clusters to exceed configured limits (Suhas Daftuar)
edb3e7cdf6 [test] rework/delete feature_rbf tests requiring large clusters (glozow)
435fd56711 test: update feature_rbf.py replacement test (Suhas Daftuar)
34e32985e8 Add new (unused) limits for cluster size/count (Suhas Daftuar)
838d7e3553 Add transactions to txgraph, but without cluster dependencies (Suhas Daftuar)
d5ed9cb3eb Add accessor for sigops-adjusted weight (Suhas Daftuar)
1bf3b51396 Add sigops adjusted weight calculator (Suhas Daftuar)
c18c68a950 Create a txgraph inside CTxMemPool (Suhas Daftuar)
29a94d5b2f Make CTxMemPoolEntry derive from TxGraph::Ref (Suhas Daftuar)
92b0079fe3 Allow moving CTxMemPoolEntry objects, disallow copying (Suhas Daftuar)
6c73e47448 mempool: Store iterators into mapTx in mapNextTx (Suhas Daftuar)
51430680ec Allow moving an Epoch::Marker (Suhas Daftuar)
Pull request description:
[Reopening #28676 here as a new PR, because GitHub is slow to load the page making it hard to scroll through and see comments. Also, that PR was originally opened with a prototype implementation which has changed significantly with the introduction of `TxGraph`.]
This is an implementation of the [cluster mempool proposal](https://delvingbitcoin.org/t/an-overview-of-the-cluster-mempool-proposal/393).
This branch implements the following observable behavior changes:
- Maintains a partitioning of the mempool into connected clusters (via the `txgraph` class), which are limited in vsize to 101 kvB by default, and limited in count to 64 by default.
- Each cluster is sorted ("linearized") to try to optimize for selecting highest-feerate-subsets of a cluster first
- Transaction selection for mining is updated to use the cluster linearizations, selecting highest feerate "chunks" first for inclusion in a block template.
- Mempool eviction is updated to use the cluster linearizations, selecting lowest feerate "chunks" first for removal.
- The RBF rules are updated to: (a) drop the requirement that no new inputs are introduced; (b) change the feerate requirement to instead check that the feerate diagram of the mempool will strictly improve; (c) replace the direct conflicts limit with a directly-conflicting-clusters limit.
- The CPFP carveout rule is eliminated (it doesn't make sense in a cluster-limited mempool)
- The ancestor and descendant limits are no longer enforced.
- New cluster count/cluster vsize limits are now enforced instead.
- Transaction relay now uses chunk feerate comparisons to determine the order that newly received transactions are announced to peers.
Additionally, the cached ancestor and descendant data are dropped from the mempool, along with the multi_index indices that were maintained to sort the mempool by ancestor and descendant feerates. For compatibility (eg with wallet behavior or RPCs exposing this), this information is now calculated dynamically instead.
ACKs for top commit:
instagibbs:
reACK 17cf9ff7ef
glozow:
reACK 17cf9ff7ef
sipa:
ACK 17cf9ff7ef
Tree-SHA512: bbde46d913d56f8d9c0426cb0a6c4fa80b01b0a4c2299500769921f886082fb4f51f1694e0ee1bc318c52e1976d7ebed8134a64eda0b8044f3a708c04938eee7
Previously we would sanity check the -maxmempool configuration based on a
multiple of the descendant size limit, but with cluster mempool the maximum
evicted size is now the cluster size limit, so use that instead.
Also allow -maxmempool=0 in general (and not just if
-limitdescendantsize/-limitclustersize is set to 0).
We use CompareMiningScoreWithTopology() for sorting transaction announcements
during tx relay, and we use GetSortedScoreWithTopology() in
CTxMemPool::check().
288b8c30be doc: Drop (default: none) from -i2psam description (Ryan Ofsky)
f6ec3519a3 init: Require explicit -asmap filename (Ryan Ofsky)
Pull request description:
Currently, if `-asmap` is specified without a filename bitcoind tries to load `ip_asn.map` data file.
This change now requires `-asmap=ip_asn.map` or another filename to be specified explicitly.
The change is intended to make behavior of the option explicit and avoid confusion reported https://github.com/bitcoin/bitcoin/issues/33386 where documentation specifies a default file which is not actually loaded by default. It was originally implemented in
https://github.com/bitcoin/bitcoin/pull/33631#issuecomment-3410302383 and various alternatives are discussed there.
ACKs for top commit:
brunoerg:
reACK 288b8c30be
fjahr:
re-ACK 288b8c30be
vostrnad:
utACK 288b8c30be
achow101:
ACK 288b8c30be
Tree-SHA512: 11a38a03892a58d6ccc1505cfbf915f58a86df9891761d89dc54b92d40593ee3cbb2d7c7bdbb922b871b3529072ef7f34cc98393aff6e8f0633b56352315b27c
de7c3587cd doc: Update add checksum instructions in tutorial (Ben Westgate)
2a46e94a16 doc: Update multisig-tutorial.md to use multipath descriptors (Ben Westgate)
Pull request description:
### Summary
Update `doc/multisig-tutorial.md` to use multipath descriptor format
instead of separate external/internal descriptors. The tutorial now:
- extracts a single `xpub_n` per participant
- constructs a multipath `wsh(sortedmulti(...))` descriptor with `<0;1>`
change index semantics
- uses `getdescriptorinfo` to compute descriptor checksum
- explains that `importdescriptors` expands the multipath descriptor
into internal and external descriptors
- update `/test/functional/wallet_multisig_descriptor_psbt.py` functional test / documentation to use multi-path descriptors
---
### Motivation
A single multipath descriptor is the most convenient pattern for multisig; our documentation should use it.
---
### What changed
- replaced extraction of `external_xpub_n` and `internal_xpub_n` with
extraction of a single `xpub_n`
- removed instructions to create and import separate external/internal
descriptors
- added instructions to build a multipath `wsh(sortedmulti(...))`
descriptor and derive checksum with `getdescriptorinfo`
- checksum field is parsed and appended as the multipath descriptor is not the canonical "desc" output
- clarified that `importdescriptors` automatically expands multipath
descriptors into internal and external forms
- similar changes to the functional test: wallet_multisig_descriptor_psbt.
---
### Testing
I have run the updated shell snippets and confirmed the multipath descriptor produces the same `listdescriptors` output after importing as the two descriptor method in bitcoin:master.
---
### Related issues / PRs
This tutorial change references the multipath descriptor
consolidation (see commit / PR referenced in the change). The commit
message points to bitcoin#22838 as the upstream change that enables
this behavior.
---
### Release note (for changelog)
Documentation: update multisig tutorial and multisig functional test to use multipath descriptors
ACKs for top commit:
Sjors:
utACK de7c3587cd
kannapoix:
Core review ACK: de7c3587cd
achow101:
ACK de7c3587cd
rkrux:
crACK de7c3587cd
Tree-SHA512: ca7275d8ad04922b3fa8d2a3084ff96aa2104265f1fc2f749814dd16776351ab692d67e7e643d08052f7492e3eaa1a9a4dfe3470163e57939a49f782d3df511a
fad06f3bb4 test: retry download in get_previous_releases.py (MarcoFalke)
Pull request description:
Hopefully fixes https://github.com/bitcoin/bitcoin/issues/33913 (intermittent download issues)
If not, the diff there to cache the bins can be considered.
ACKs for top commit:
davidgumberg:
ACK fad06f3bb4
achow101:
ACK fad06f3bb4
enirox001:
ACK fad06f3
TheCharlatan:
ACK fad06f3bb4
janb84:
ACK fad06f3bb4
Tree-SHA512: 6597e993cc64259e361f65045a938e693b47ff73a7dc22f5a70b6c9890986817849915f90096e3b87684f25152a6d8ae4ec5480c326c32d36273cdb0a33ceb0c
fad0c76d0a clang-format: Set PackConstructorInitializers: CurrentLine (MarcoFalke)
Pull request description:
Now that the minimum supported clang version is larger than 14, the `PackConstructorInitializers` setting can be set to `CurrentLine` in the clang-format file. (This option was added in clang 14. Ref: https://releases.llvm.org/17.0.1/tools/clang/docs/ClangFormatStyleOptions.html#packconstructorinitializers)
The `CurrentLine` option will either put all constructor initializers on the current line if they fit. Otherwise, it will put each one on its own line.
The `CurrentLine` option is desirable over the current `BinPack` option, because:
* It is what the majority of the codebase is currently using.
* It makes it easier to skim the lines to ensure all fields are properly initialized, without having to parse bin-packed constructor initializers, possibly with nested initializer lists, function calls, or ternary operators.
* It makes diffs smaller when an initializer is added or removed, because only a single line is touched. Otherwise, the whole bin-packed block could re-flow, making the diff harder to parse.
Note: The previous `BinPack` option allows any formatting, due to the current `ColumnLimit: 0`. I presume developers manually formatted most constructor initializers to be on separate lines? With the new `CurrentLine` setting, one has to only put the first initializer on a separate line, and clang-format will take care of the rest.
For example:
```sh
echo 'A::A(O o)
: m_first{o.a, o.b},
m_second{fun(o)}, m_third{o.c?o.d:o.e} {}' | clang-format --style=file:./src/.clang-format
```
Will put each on a separate line. Previously, it was left as-is.
ACKs for top commit:
l0rinc:
ACK fad0c76d0a
TheCharlatan:
ACK fad0c76d0a
hebasto:
ACK fad0c76d0a.
Tree-SHA512: f26a0980ecfa01b2a5279561e3df316c10241f8e67830034d493d70a6d0baae8831498233e8986cfa8f3b434cb9bc1e7e525b3d4587dca66b2d609ddae522a88
The option was fine, but now that there is a dedicated Alpine Linux
task, which uses BusyBox, it seems redundant.
(See: ci/test/00_setup_env_native_alpine_musl.sh)
So remove the USE_BUSY_BOX option, along with the BINS_SCRATCH_DIR env
var.
Also, enable pipefail in the ci/test/00_setup_env.sh script, while
touching it.
55555db055 doc: Add missing --platform=linux to docker build command (MarcoFalke)
fa0ce4c148 ci: Re-enable LINT_CI_SANITY_CHECK_COMMIT_SIG (MarcoFalke)
faa0973de2 ci: [refactor] Rename CIRRUS_PR env var to LINT_CI_IS_PR (MarcoFalke)
fa1dacaebe ci: Move lint exec snippet to stand-alone py file (MarcoFalke)
Pull request description:
The sanity check to check the last few merge commit signatures on the main branch was accidentally and silently disabled while moving from the `cirrus-ci.com` platform to the GHA platform.
So fix that by re-enabling it.
Also, contains a few other lint cleanup commits.
ACKs for top commit:
janb84:
re ACK 55555db055
willcl-ark:
ACK 55555db055
Tree-SHA512: e623dc88035ee4d1c6a8efa5fad33c35cface87f54e78c7ebfe5d468d28d8d8097150344d276f90f8ed52a89e61609ce95380476ea0151b50f73ad5919233933
99d012ec80 refactor: return reference instead of pointer (Andrew Toth)
f743e6c5dd refactor: add missing LIFETIMEBOUND annotation for parameter (Andrew Toth)
141117f5e8 refactor: remove incorrect LIFETIMEBOUND annotations (Andrew Toth)
Pull request description:
The [developer-notes say](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#lifetimebound):
> You can use the attribute by adding a `LIFETIMEBOUND`
annotation defined in `src/attributes.h`; please grep the codebase for examples.
While grepping, I found an incorrect usage of the `LIFETIMEBOUND` annotation on `BlockManager::CheckBlockDataAvailability`. This could be misleading about usage for other greppers. As I was looking, I also noticed a missing `LIFETIMEBOUND` on `BlockManager::GetFirstBlock`. While looking more closely at that method, it should return a reference instead of a pointer. The only reason to return a pointer is if it can be null.
ACKs for top commit:
maflcko:
review ACK 99d012ec80💧
l0rinc:
ACK 99d012ec80
stickies-v:
ACK 99d012ec80
optout21:
ACK 99d012ec80
vasild:
ACK 99d012ec80
Tree-SHA512: d6c56ee223d6614d52ee6cf5cd66e787125c98c6ae37705a17e51a6e15774e260ac55b3d60f2fc818132e766ad98dd94232d6c8829119f628498e9d0d2bd977f
552eb90071 doc: CI - Describe qemu-user-static usage (Hodlinator)
2afbbddee5 doc: CI - Clarify how important `env -i` is and why (Hodlinator)
Pull request description:
Should at least partially fix#31199
ACKs for top commit:
maflcko:
lgtm ACK 552eb90071
janb84:
ACK 552eb90071
Tree-SHA512: 45807a61d805646384c8162501f432537b7e655aa01434766ffb90ea47da9532387a76fcccac7fe208ad77f4ea5573f60b9be09e1235b9493eaa8795e1d7fbdd
2578e6fc0f test: Fix race condition in IPC interface block propagation test (Fabian Jahr)
Pull request description:
CI failed on this condition here: https://github.com/bitcoin/bitcoin/actions/runs/19395398994/job/55494696022?pr=33878#step:9:3983
The check was added not too long ago in https://github.com/bitcoin/bitcoin/pull/33745 and the fix here switches the check to the node which actually produces the block. There are also some comments added to make the checks easier so understand.
Closes#33884
ACKs for top commit:
Sjors:
re-utACK 2578e6fc0f
maflcko:
lgtm ACK 2578e6fc0f
Tree-SHA512: bfb7ae44aede50a00d4096e1a9922f9b8df31ce4242e12863e329d0d1e714d8cb46c852f694c32314e4bd26b524535e3a6967b7c57861a9b00cf09831a950b99
fa411f938e ci: Consistenly only cache on the default branch (MarcoFalke)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/33685
The general idea for caches is to only save them on pushes to the default branch, because the cache is limited in size and time that the only benefit of the cache can be to speed up pull requests against the default branch.
Backport pull requests to older branches don't benefit from caches, because usually they will be running into a cache miss anyway. Also, they would cause the cache size to overflow and lead to cache misses down the line.
So fix it by consistently applying cache saves only on the default branch.
For reference, the same is already done for the composite action in this repo: 2444488f6a/.github/actions/save-caches/action.yml (L15)
ACKs for top commit:
hebasto:
ACK fa411f938e.
willcl-ark:
ACK fa411f938e
Tree-SHA512: 59d3de4201b596e5f7eb1915c4bd5ded80bcd4df217f8f5d62d92fa8977a77e2c4c8602b17755b33ff0dfa87f2896e7c1c8f4da8e6a77c111f2a42ec9cf78ffd
fae83611b8 ci: [refactor] Use --preset=dev-mode in mac_native task (MarcoFalke)
fadb67b4b4 ci: [refactor] Base nowallet task on --preset=dev-mode (MarcoFalke)
6666980e86 ci: Enable bitcoin-chainstate and test_bitcoin-qt in win64 task (MarcoFalke)
faff7b2312 ci: Enable experimental kernel stuff in i686 task (MarcoFalke)
fa1632eecf ci: Enable experimental kernel stuff in mac-cross tasks (MarcoFalke)
fad10ff7c9 ci: Enable experimental kernel stuff in armhf task (MarcoFalke)
fa9d67c13d ci: Enable experimental kernel stuff in Alpine task (MarcoFalke)
fab3fb8302 ci: Enable experimental kernel stuff in s390x task (MarcoFalke)
fa7da8a646 ci: Enable experimental kernel stuff in valgrind task (MarcoFalke)
fa9c2973d6 ci: Enable experimental kernel stuff in TSan task (MarcoFalke)
fad30d4395 ci: Enable experimental kernel stuff in MSan task (MarcoFalke)
Pull request description:
Most of the CI tasks have a long list of stuff that they enable. This makes it hard to see what each CI task is actually running.
Also, most of the CI tasks should probably mimic the `dev-mode` CMake preset and run on as much stuff as possible. Usually, changing the `dev-mode` comes with changing those CI tasks as well in the same commit, which is verbose.
Fix both issues, by basing most CI tasks on the `dev-mode`. In the future, this makes it easier to change the `dev-mode` in a single place. If CI tasks explicitly disable something, it will be listed explicitly in them.
As a side-effect this will enable the kernel stuff for some CI task that did not have it enabled, which seems desirable.
ACKs for top commit:
TheCharlatan:
Nice, ACK fae83611b8
janb84:
ACK fae83611b8
hebasto:
ACK fae83611b8, I have reviewed the code and it looks OK.
Tree-SHA512: 58d9d553437b57362e9ec0766bd202482435f263d3f4c6ee7020c5e1e5ba69f8c064630423424f9d754254a66981e670b964a5aee58ef87f30b7d775642255be
Commit b3bf18f0ba changed the function
signature from Parse(const std::string& descriptor,...) to
Parse(std::span<const char> descriptor,...).
Calling this new version of Parse with a string literal will trigger
a confusing "Invalid characters in payload" due to the trailing "\0".
Switch to string_view and add a test.
Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
chain_start can never be null, so it's better to pass it as a reference
rather than a raw pointer
Also slightly reformat HeaderSyncState constructor to make clang-format
happy
Lastly, remove `const` from `chain_start` declaration in
headers_sync_chainwork_tests, to work aroud a false-positive
dangling-reference warning in gcc 13.0
Co-Authored-By: maflcko <6399679+maflcko@users.noreply.github.com>
fa1bf6818f clang-format: Set InsertNewlineAtEOF: true (MarcoFalke)
Pull request description:
Now that the minimum supported clang version is 17, the `InsertNewlineAtEOF` setting can be set to `true` in the clang-format file. (https://releases.llvm.org/16.0.0/tools/clang/docs/ClangFormatStyleOptions.html#insertnewlineateof)
This is in line with the already existing newline linter. Can be tested via:
```
truncate --size=-1 src/init.cpp
git diff
# Should fail:
cargo run --manifest-path ./test/lint/test_runner/Cargo.toml -- --lint=trailing_newline
# Restore newline:
git diff -U0 | ./contrib/devtools/clang-format-diff.py -p1 -i -v
```
ACKs for top commit:
l0rinc:
ACK fa1bf6818f
achow101:
ACK fa1bf6818f
hodlinator:
ACK fa1bf6818f
stickies-v:
ACK fa1bf6818f
hebasto:
ACK fa1bf6818f.
janb84:
ACK fa1bf6818f
Tree-SHA512: 6443f0f9e2710fb31c85e4ece5deb9fa25bf22ef2d5cadad2d54a239194860c3a785ffa5dd97be942e42139336575425499f085aa2e3f29003e529fdc20fce20
6657bcbdb4 kernel: allow null data_directory (stickies-v)
Pull request description:
An empty path may be represented with a `nullptr`. For example, `std::string_view{}.data()` may return nullptr.
Removes the `BITCOINKERNEL_ARG_NONNULL` attribute for `btck_chainstate_manager_options_create` 's `data_directory` parameter, and instead handles such null arguments in the implementation. [Because an empty path is meaningless](https://github.com/bitcoin/bitcoin/pull/33867#discussion_r2523930442), `btck_chainstate_manager_options_create` now treats both null and empty directories as invalid, tightening the interface.
Also documents how `BITCOINKERNEL_ARG_NONNULL` should be used.
Follow-up to https://github.com/bitcoin/bitcoin/pull/33853#pullrequestreview-3454620265
ACKs for top commit:
stringintech:
ACK 6657bcb
maflcko:
review ACK 6657bcbdb4🐪
achow101:
ACK 6657bcbdb4
TheCharlatan:
ACK 6657bcbdb4
janb84:
ACK 6657bcbdb4
Tree-SHA512: 11c02b221ff19a5357e94355808e3b503b3a336c16fc5186c9c9137931709e880383ed1f4990fc4cc6b0e23961e2e1e03fc90154a3b546b9490ef66bd63688b7
7f318e1dd0 test: Add better coverage for Autofile size() (Fabian Jahr)
b7af960eb8 refactor: Add AutoFile::size (Fabian Jahr)
ec0f75862e refactor: Modernize logging in util/asmap.cpp (Fabian Jahr)
606a251e0a tests: add unit test vectors for asmap interpreter (Pieter Wuille)
Pull request description:
This contains some commits from #28792 that can be easily reviewed and merged independently. I hope splitting this change off can make this part move a bit faster and reduce frequency of needed rebases for #28792.
The commits in order:
- Add additional unit test vectors to the asmap interpreter (written by sipa). This helps to ensure that the further refactors in #28792 don't change behavior.
- Modernizes the logging in `util/asmap.cpp`, I added this while touching the rest of the file all over anyway.
- Adds an `AutoFile::size` helper function with some additional test coverage in a separate commit
ACKs for top commit:
maflcko:
review ACK 7f318e1dd0🏀
hodlinator:
tACK 7f318e1dd0
laanwj:
Code review ACK 7f318e1dd0
Tree-SHA512: 45156b74e4bd9278a7ec24521dfdafe4dab1ba3384243c7d589ef17e16ca374ee2af7178c86b7229e80ca262dbe78c4d456d80a6ee742ec31d2ab5243dac8b57
310e4979b3 qt: Added test coverage for qt gui#901 console history filter (WakeTrainDev)
Pull request description:
Add test coverage for the QT rpc console updated filtered commands in gui#901
ACKs for top commit:
pablomartin4btc:
ACK 310e4979b3
hebasto:
ACK 310e4979b3, tested on Fedora 42 by reverting 4e352efa2c.
Tree-SHA512: 45bb8583311f145353d8265d28f220d2a318c701346f147979c5d33b27811276d5e18586bf58f35e455701495d2cb87ec54dd78f4ca8631a0c7bd2c1d7fe640c
929f69d0ff qt: Remove HD seed reference from blank wallet tooltip (John Moffett)
Pull request description:
Blank descriptor wallets currently do not have HD seeds and none can be added (or 'set') by the user, so remove the reference in the tooltip.
As I understand it, descriptor wallets don't have a global HD seed and don't store the HD seeds for keys they generate. Currently, no new HD seeds can be added by the user (even for old wallets since `sethdseed` was removed), though it may be possible in the future, eg - https://github.com/bitcoin/bitcoin/pull/33043
ACKs for top commit:
maflcko:
lgtm ACK 929f69d0ff
Tree-SHA512: 85e9c8e18a92b3da6fd62b70570cef58e03843633300b65aee5789d38c7bcaa46738970f0aea63f4e9b3e8814abb5bf1e1aa93f568a875ad1e0443d4dafb0aab
28a4fcb03c test: check listdescriptors do not return a mix of hardened derivation marker (pythcoiner)
975783cb79 descriptor: account for all StringType in MiniscriptDescriptor::ToStringHelper() (pythcoiner)
Pull request description:
In `MiniscriptDescriptor::ToStringHelper()` only the `StringType::Private` variant of the `type` argument was handled. This PR implements serializing w/ all variants of `StringType` & add a functional test for the descriptor triggering the related issue.
Closes#31694: previously when calling `listdescriptors` RPC on a wallet containing a taproot descriptor w/ a (miniscript) taptree, origins of internal key & taptree were serialized w/ differents hardened derivation markers:
- origin of the internal key were serialized w/ `StringType::Normalized` type (using `h` as marker)
- origins of taptree keys were serialized w/ `StringType::Private` type (using `'` as marker)
Note: Origins in segwit (`wsh()`) miniscript descriptors were also serialized w/ `StringType::Private` type (`'` marker) and are now serialized w/ `StringType::Normalized` type (`h` marker).
ACKs for top commit:
sipa:
Code review ACK 28a4fcb03c
achow101:
ACK 28a4fcb03c
rkrux:
Concept ACK 28a4fcb03c
Tree-SHA512: 15d14000b5951ca69a64a05b9a0b138c48a07b81eaf2fa86b91ac20cc8735533355a787363c64ba88403dd8a56ef5232cba57d34bea80835a0f40774d62fbc2b
f53dbbc505 test: Add functional tests for named argument parsing (zaidmstrr)
694f04e2bd rpc: Handle -named argument parsing where '=' character is used (zaidmstrr)
Pull request description:
Addresses [comment](https://github.com/bitcoin/bitcoin/pull/31375#discussion_r2091886628) and [this](https://github.com/bitcoin/bitcoin/pull/31375#discussion_r2092039999).
The [PR #31375](https://github.com/bitcoin/bitcoin/pull/31375) got merged and enables `-named` by default in the `bitcoin rpc` interface; `bitcoin rpc` corresponds to `bitcoin-cli -named` as it's just a wrapper. Now, the problem arises when we try to parse the positional paramater which might contain "=" character. This splits the parameter into two parts first, before the "=" character, which treats this as the parameter name, but the other half is mostly passed as an empty string. Here, the first part of the string is an unknown parameter name; thus, an error is thrown. These types of errors are only applicable to those RPCs which might contain the `=` character as a parameter. Some examples are `finalizepsbt`, `decodepsbt`, `verifymessage` etc.
This is the one example of the error in `finalizepsbt` RPC:
```
./bitcoin-cli -named -regtest finalizepsbt cHNidP8BAJoCAAAAAqvNEjSrzRI0q80SNKvNEjSrzRI0q80SNKvNEjSrzRI0AAAAAAD9////NBLNqzQSzas0Es2rNBLNqzQSzas0Es2rNBLNqzQSzasBAAAAAP3///8CoIYBAAAAAAAWABQVQBGVs/sqFAmC8HZ8O+g1htqivkANAwAAAAAAFgAUir7MzgyzDnRMjdkVa7d+Dwr07jsAAAAAAAAAAAA=
error code: -8
error message:
Unknown named parameter cHNidP8BAJoCAAAAAqvNEjSrzRI0q80SNKvNEjSrzRI0q80SNKvNEjSrzRI0AAAAAAD9////NBLNqzQSzas0Es2rNBLNqzQSzas0Es2rNBLNqzQSzasBAAAAAP3///8CoIYBAAAAAAAWABQVQBGVs/sqFAmC8HZ8O+g1htqivkANAwAAAAAAFgAUir7MzgyzDnRMjdkVa7d+Dwr07jsAAAAAAAAAAAA
```
This PR fixes this by updating the `vRPCConvertParams` table that identifies parameters that need special handling in `-named` parameter mode. The parser now recognises these parameters and handles strings with "=" char correctly, preventing them from being incorrectly split as parameter assignments.
ACKs for top commit:
ryanofsky:
Code review ACK f53dbbc505. Just applied comment & test suggestions since last review
kannapoix:
Code review ACK: f53dbbc505
achow101:
ACK f53dbbc505
Tree-SHA512: 1b517144efeff45a4c4256c27a39ddf187f1d6189d133402a45171678214a10ff2925c31edcfd556d67f85bd26d42f63c528b941b68c9880eab443f2c883e681
With the move from cirrus-ci to GHA, the CIRRUS_REPO_FULL_NAME env var
was always unset, never triggering the sanity check.
Fix this by introducing a new vendor-agnostic env var and setting it
properly.
The CIRRUS_PR env var was cirrus-specific and using a provider-agnostic
name makes more sense.
Also, enable pipefail, while touching this file.
This refactor is needed for the next commit.
Changes AddToMempool() helper to only apply changes if the mempool limits are
respected.
Fix package_rbf fuzz target to handle mempool policy violations
Calculating mempool ancestors for a new transaction should not be done until
after cluster size limits have been enforced, to limit CPU DoS potential.
Achieve this by reworking TRUC and RBF validation logic:
- TRUC policy enforcement is now done using only mempool parents of
new transactions, not all mempool ancestors (note that it's fine to calculate
ancestors of in-mempool transactions, if the number of such calls is
reasonably bounded).
- RBF replacement checks are performed earlier (which allows for checking
cluster size limits earlier, because cluster size checks cannot happen until
after all conflicts are staged for removal).
- Verifying that a new transaction doesn't conflict with an ancestor now
happens later, in AcceptSingleTransaction() rather than in PreChecks(). This
means that the test is not performed at all in AcceptMultipleTransactions(),
but in package acceptance we already disallow RBF in situations where a
package transaction has in-mempool parents.
Also to ensure that all RBF validation logic is applied in both the single
transaction and multiple transaction cases, remove the optimization that skips
the PackageMempoolChecks() in the case of a single transaction being validated
in AcceptMultipleTransactions().
Now that ancestor calculation never fails (due to ancestor/descendant limits
being eliminated), we can eliminate the error handling from
CalculateMemPoolAncestors.
With the descendant size limits removed, replace the concept of "max number of
descendants of any ancestor of a given tx" with the cluster count of the cluster
that the transaction belongs to.
The mempool clusters and linearization permit sorting the mempool topologically
without making use of ancestor counts (as long as the graph is not oversized).
Co-authored-by: Pieter Wuille <pieter@wuille.net>
Previously, transaction batches were first sorted by ancestor count and then
feerate, to ensure transactions are announced in a topologically valid order,
while prioritizing higher feerate transactions. Ancestor count is a crude
topological sort criteria, so replace this with linearization order so that the
highest feerate transactions (as would be observed by the mining algorithm) are
relayed before lower feerate ones, in a topologically valid way.
This also fixes a test that only worked due to the ancestor-count-based sort
order.
With a total ordering on mempool transactions, we are now able to calculate a
transaction's mining score at all times. Use this to improve the RBF logic:
- we no longer enforce a "no new unconfirmed parents" rule
- we now require that the mempool's feerate diagram must improve in order
to accept a replacement
- the topology restrictions for conflicts in the package rbf setting have been
eliminated
Revert the temporary change to mempool_ephemeral_dust.py that were previously
made due to RBF validation checks being reordered.
Co-authored-by: Gregory Sanders <gsanders87@gmail.com>, glozow <gloriajzhao@gmail.com>
The addition of a cluster size limit makes the CPFP carveout rule useless,
because carveout cannot be used to bypass the cluster size limit. Remove this
policy rule and update tests to no longer rely on the behavior.
After cluster mempool, the mini_miner will no longer match the miner's block
construction. Eventually mini_miner should be reworked to directly use
linearizations done in the mempool.
Rather than evicting the transactions with the lowest descendant feerate,
instead evict transactions that have the lowest chunk feerate.
Once mining is implemented based on choosing transactions with highest chunk
feerate (see next commit), mining and eviction will be opposites, so that we
will evict the transactions that would be mined last.
Include an adjustment to mempool_tests.cpp due to the additional memory used by
txgraph.
Includes a temporary change to the mempool_ephemeral_dust.py functional test,
due to validation checks being reordered. This change will revert once the RBF
rules are changed in a later commit.
4d893c0f46 net: Remove unused `local_socket_bytes` variable in `CConnman::GetAddresses()` (WakeTrainDev)
Pull request description:
The local_socket_bytes variable was never used. Removed it to clean up dead code.
ACKs for top commit:
mzumsande:
ACK 4d893c0f46
theStack:
ACK 4d893c0f46
Tree-SHA512: f423bcf975aa2602464fcb96db323cbd6007a7491ddbe119f1d20e890c883dd351a55976151c5d25f5d26267b0efe1f0836fbd65e540c920dac931ed8d67846a
0aebdac95d init: completely remove `-maxorphantx` option (Sebastian Falbesoner)
Pull request description:
This is a small follow-up for #32941 (commit 1384dbaf6d), removing the `-maxorphantx` option completely, now that v30 has been released. If removing it for v31 is seen as controversial/premature (I personally don't think it is), the merge can be delayed for a future release.
ACKs for top commit:
maflcko:
lgtm ACK 0aebdac95d
achow101:
ACK 0aebdac95d
w0xlt:
ACK 0aebdac95d
rkrux:
lgtm ACK 0aebdac95d
stickies-v:
ACK 0aebdac95d
Tree-SHA512: 818633b903174387ae259acb1d1e8ce07f78e158de2c150742ef0950b0f5d62af553e4e35ab962432306e04e07c45b1be11dbae459a8b62c4b9a6b5ef1746d26
d31158d364 psbt: clarify PSBT, PSBTInput, PSBTOutput unserialization flows (rkrux)
Pull request description:
The unserialization flows of the PSBT types work based on few underlying assumptions of functions from `serialize.h` & `stream.h` that takes some to understand when read the first time.
Add few comments that highlight these assumptions hopefully making it easier to grasp. Also, mention key/value format types as per BIP 174.
ACKs for top commit:
achow101:
ACK d31158d364
theStack:
ACK d31158d364
Tree-SHA512: 45111ef7f0258ebbc41d058b3ef2a72472774ab2878caf2d71d7b57b27549c46a51ccbeda5fe164bcf4f7ec10627bbae6e7763aa80b1e66912703a2088682817
Moving the python code out of the yaml string makes it easier to lint,
format, and edit.
This can be reviewed with the git options:
--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
fa95353902 ci: Run macos tasks in a git archive, not git checkout (MarcoFalke)
faf99ae379 refactor: Avoid -W*-whitespace in git archive (MarcoFalke)
Pull request description:
Otherwise, compilation with GCC-15+ will warn about it:
```
src/clientversion.cpp:33:79: error: trailing whitespace [-Werror=trailing-whitespace=]
33 | //! git will put "#define GIT_COMMIT_ID ..." on the next line inside archives.
```
Follow-up to https://github.com/bitcoin/bitcoin/pull/32482#issuecomment-3522280482
Can be tested via `git archive --output=/tmp/a.tar HEAD`
ACKs for top commit:
fanquake:
ACK fa95353902
Tree-SHA512: 73940ffc0fd83db557275bd5e993a3c47c5397682a1188447c48e077ead597ba0fc3e5ef9da7b746746ff04a26022ce35ac10768888bbd4707f25b799af43e45
2594d5a189 build: Remove CMAKE_SKIP_BUILD_RPATH and SKIP_BUILD_RPATH settings (Henry Romp)
Pull request description:
Remove CMAKE_SKIP_BUILD_RPATH and SKIP_BUILD_RPATH settings that are no longer needed after reordering the Guix build script to perform binary checks after installation.
This PR also removes the unused CMake maintenance targets (`check-security` and `check-symbols`) and updates the Guix security checks to include binaries in the `libexec/` directory (added in PR #31679).
ACKs for top commit:
purpleKarrot:
ACK 2594d5a189
hebasto:
ACK 2594d5a189.
Tree-SHA512: ed451a298f5aae05c177b0033b092faaa7536caeaa3d84da9b8b611e2aa905e1dd337e57aef0efd69ce6ce6ac0cf77dc57adf175079b95bf53dd96d5d0c8118b
c29eaeeaf9 doc: Update NetBSD Build Guide (Hennadii Stepanov)
Pull request description:
The `py310-zmq` binary package is not available by default on NetBSD 10.1. It has been updated to `py313-zmq`, and the `python310` package is updated accordingly.
See: https://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/index-all.html.
ACKs for top commit:
fanquake:
ACK c29eaeeaf9
Tree-SHA512: 6924a974d6ed494609c789cc3f28cf173af3a37b940520ad7b169eff87e30af8346fec07e46f8bffe14a329060c41ac46d46b91910a00994cdf8a7ace8391d1c
0dd8d5c237 cmake: Specify Windows plugin path in `test_bitcoin-qt` property (Hennadii Stepanov)
Pull request description:
This PR simplifies testing on Windows by removing the need to set the `QT_PLUGIN_PATH` environment variable for different build configurations. For example, the paths might otherwise be:
- `C:/Users/hebasto/dev/bitcoin/build/vcpkg_installed/x64-windows/Qt6/plugins/` for "Release"
- `C:/Users/hebasto/dev/bitcoin/build/vcpkg_installed/x64-windows/debug/Qt6/plugins/` for "Debug"
ACKs for top commit:
purpleKarrot:
ACK 0dd8d5c237
Tree-SHA512: 0418b8fa4d74ca500aae9e36e56ebcefb566d2ac04a9d22e17d309400ad38dd5a6e75f0195c680796b761fb145444c33336b64180f15c6b1125fe190d58396b6
3e9aca6f1b depends: drop qtbase-moc-ignore-gcc-macro.patch qt patch (fanquake)
0da5a82700 depends: drop unused qt patch (fanquake)
Pull request description:
Drop one patch that was already unused, and one that compilation succeeds without.
ACKs for top commit:
TheCharlatan:
ACK 3e9aca6f1b
Tree-SHA512: 4416348c80d8af8530d46d4f5a02a1170f7a4e2fc8ef88cffb8888fa913ed86d1bef10efb437434ebcdac1b1ed23a3669c1ba654cf6f4395dc0a73192fe0024f
An empty path may be represented with a nullptr. For example,
std::string_view::data() may return nullptr.
Removes the BITCOINKERNEL_ARG_NONNULL attribute for data_directory,
and instead handles such null arguments in the implementation.
Also documents how BITCOINKERNEL_ARG_NONNULL should be used.
The return value of BlockManager::GetFirstBlock must always be non-null. This
can be inferred by the implementation, which has an assertion that the return
value is not null. A raw pointer should only be returned if the result may be
null. In this case a reference is more appropriate.
The BlockManager::GetFirstBlock lower_block parameter can have its lifetime
extended by the return parameter. In the case where lower_block is returned,
its lifetime will be bound to the return value. A LIFETIMEBOUND annotation is
appropriate here.
The return value of CheckBlockDataAvailability does not extend the lifetime of
the input parameters, nor does BlockManager instance retain references to the
parameters. The LIFETIMEBOUND annotations are misleading here since the lifetime
of the parameters are not extended past the method call.
The same was done for the other CI tasks in commit fa6aa9f42f. This may
guard against intermittent network issues to download the base image or
packages ...
b0a3887154 scripted-diff: fix leftover references to `policy/fees.h` (ismaelsadeeq)
Pull request description:
Fixes#33863
ryanofsky wrote
> I still see some references to the src/policy/fees.h file removed by this PR:
```
$ git grep -n policy/fees.h
src/wallet/rpc/spend.cpp:206: * @param[in] conf_target UniValue integer; confirmation target in blocks, values between 1 and 1008 are valid per policy/fees.h;
test/functional/rpc_estimatefee.py:39: # max value of 1008 per src/policy/fees.h
test/functional/rpc_psbt.py:604: assert_raises_rpc_error(-8, "Invalid conf_target, must be between 1 and 1008", # max value of 1008 per src/policy/fees.h
test/functional/wallet_basic.py:337: assert_raises_rpc_error(-8, "Invalid conf_target, must be between 1 and 1008", # max value of 1008 per src/policy/fees.h
test/functional/wallet_fundrawtransaction.py:851: assert_raises_rpc_error(-8, "Invalid conf_target, must be between 1 and 1008", # max value of 1008 per src/policy/fees.h
test/functional/wallet_send.py:315: expect_error=(-8, "Invalid conf_target, must be between 1 and 1008")) # max value of 1008 per src/policy/fees.h
```
This is fixed in this PR by running a script that searches for what he greps and replaces it with the right reference.
```
git grep -l "policy\/fees\.h" | xargs sed -i "s/policy\/fees.h/policy\/fees\/block_policy_estimator.h/g"
```
ACKs for top commit:
kevkevinpal:
ACK [b0a3887](b0a3887154)
janb84:
ACK b0a3887154
rkrux:
lgtm ACK b0a3887154
Tree-SHA512: e24f2aaf18fcfb0ae047a53ed209135a644ff08f5a8bc162c1522be3f99d7d01d550fc2e73d8db5fec7b748902daf68e61e7a5624f5913b9824feba5641fc78c
Remove CMake settings that are no longer needed after reordering Guix build script to perform binary checks after installation.
Also removes unused CMake maintenance targets (check-security and check-symbols) and updates security checks to include libexec/ directory binaries (see PR #31679).
c25a5e670b init: Signal m_tip_block_cv on Ctrl-C (Ryan Ofsky)
6a29f79006 test: Test SIGTERM handling during waitforblockheight call (Ryan Ofsky)
Pull request description:
Signal `m_tip_block_cv` when Ctrl-C is pressed or `SIGTERM` is received, the same way it is currently signaled when the `stop` RPC is called. This lets RPC calls like `waitforblockheight` and IPC calls like `waitTipChanged` be interrupted, instead of waiting for their original timeouts and delaying shutdown.
This issue was reported by plebhash in #33463. These hangs have been present since #30409. A similar bug was also fixed previously in Qt in #18452 and this PR simplifies that fix.
ACKs for top commit:
Sjors:
tACK c25a5e670b
TheCharlatan:
ACK c25a5e670b
enirox001:
Concept ACK c25a5e6
Tree-SHA512: 320aaa74fd308e826521c48c9a8aca4bd5f5530064cda2303d251d8e93e50c474bcd0db760ce04921928e73abefe4847aff797ac9ca7c89e74e5051bbed061cd
6eaa00fe20 test: clarify submitBlock() mutates the template (Sjors Provoost)
862bd43283 mining: ensure witness commitment check in submitBlock (Sjors Provoost)
00d1b6ef4b doc: clarify UpdateUncommittedBlockStructures (Sjors Provoost)
Pull request description:
When an IPC client requests a new block template via the Mining interface, we hold on to its `CBlock`. That way when they call `submitSolution()` we can modify it in place, rather than having to reconstruct the full block like the `submitblock` RPC does.
Before this commit however we forgot to invalidate `m_checked_witness_commitment`, which we should since the client brings a new coinbase.
This would cause us to accept an invalid chaintip.
Fix this and add a test to confirm that we now reject such a block. As a sanity check, we add a second node to the test and confirm that will accept our mined block.
As first noticed in #33374 the IPC code takes the coinbase as provided, unlike the `submitblock` RPC which calls `UpdateUncommittedBlockStructures()` and adds witness commitment to the coinbase if it was missing.
Although that could have been an alternative fix, we instead document that IPC clients are expected to provide the full coinbase including witness commitment.
Patch to produce the original issue:
```diff
diff --git a/src/node/miner.cpp b/src/node/miner.cpp
index b988e28a3f..28e9048a4d 100644
--- a/src/node/miner.cpp
+++ b/src/node/miner.cpp
@@ -450,15 +450,10 @@ void AddMerkleRootAndCoinbase(CBlock& block, CTransactionRef coinbase, uint32_t
}
block.nVersion = version;
block.nTime = timestamp;
block.nNonce = nonce;
block.hashMerkleRoot = BlockMerkleRoot(block);
-
- // Reset cached checks
- block.m_checked_witness_commitment = false;
- block.m_checked_merkle_root = false;
- block.fChecked = false;
}
std::unique_ptr<CBlockTemplate> WaitAndCreateNewBlock(ChainstateManager& chainman,
KernelNotifications& kernel_notifications,
CTxMemPool* mempool,
diff --git a/test/functional/interface_ipc.py b/test/functional/interface_ipc.py
index cce56e3294..bf1b7048ab 100755
--- a/test/functional/interface_ipc.py
+++ b/test/functional/interface_ipc.py
@@ -216,22 +216,22 @@ class IPCInterfaceTest(BitcoinTestFramework):
assert_equal(res.result, True)
# The remote template block will be mutated, capture the original:
remote_block_before = await self.parse_and_deserialize_block(template, ctx)
- self.log.debug("Submitted coinbase must include witness")
+ self.log.debug("Submitted coinbase with missing witness is accepted")
assert_not_equal(coinbase.serialize_without_witness().hex(), coinbase.serialize().hex())
res = await template.result.submitSolution(ctx, block.nVersion, block.nTime, block.nNonce, coinbase.serialize_without_witness())
- assert_equal(res.result, False)
+ assert_equal(res.result, True)
self.log.debug("Even a rejected submitBlock() mutates the template's block")
# Can be used by clients to download and inspect the (rejected)
# reconstructed block.
remote_block_after = await self.parse_and_deserialize_block(template, ctx)
assert_not_equal(remote_block_before.serialize().hex(), remote_block_after.serialize().hex())
- self.log.debug("Submit again, with the witness")
+ self.log.debug("Submit again, with the witness - does not replace the invalid block")
res = await template.result.submitSolution(ctx, block.nVersion, block.nTime, block.nNonce, coinbase.serialize())
assert_equal(res.result, True)
self.log.debug("Block should propagate")
assert_equal(self.nodes[1].getchaintips()[0]["height"], current_block_height + 1)
```
ACKs for top commit:
ryanofsky:
Code review ACK 6eaa00fe20. Just documentation updates and test clarifications since last review, also splitting up a commit.
TheCharlatan:
Re-ACK 6eaa00fe20
ismaelsadeeq:
Code review and tested ACK 6eaa00fe20
Tree-SHA512: 3a6280345b0290fe8300ebc63c13ad4058d24ceb35b7d7a784b974d5f04f420860ac03a9bf2fc6a799ef3fc55552ce033e879fa369298f976b9a01d72bd55d9e
8810642b57 test: add option to skip large re-org test in feature_block (brunoerg)
Pull request description:
Fixes#32877
This PR adds a config flag `--skipreorg` which is used to skip the large re-org test. According to corecheck, `feature_block` is our slowest functional test and primarily because of this large re-org test. However, this test might not be useful for the mutation analysis of some files and could be skipped to save a huge amount of time.
```
time ./build/test/functional/feature_block.py --skipreorg
./build/test/functional/feature_block.py --skipreorg 11.38s user 0.33s system 37% cpu 31.422 total
time ./build/test/functional/feature_block.py
./build/test/functional/feature_block.py 25.87s user 3.53s system 56% cpu 52.317 total
```
ACKs for top commit:
maflcko:
review ACK 8810642b57🥁
enirox001:
tACK 8810642 – Ran tests with/without --skipreorg; saw ~40 % speedup; no regressions.
theStack:
Concept and code-review ACK 8810642b57
glozow:
lgtm ACK 8810642b57
Tree-SHA512: 4ef38bd32b8ad8ec2b7f30c96d2fe545d920759645ff52f632699f829b64f8d26fe878f3fdd255142235edd0a740a7feb64da8f5a10d0d740ebfa46c43ae60eb
fa6c0bedd3 refactor: Return uint64_t from GetSerializeSize (MarcoFalke)
fad0c8680e refactor: Use uint64_t over size_t for serialized-size values (MarcoFalke)
fa4f388fc9 refactor: Use fixed size ints over (un)signed ints for serialized values (MarcoFalke)
fa01f38e53 move-only: Move CBlockFileInfo to kernel namespace (MarcoFalke)
fa2bbc9e4c refactor: [rpc] Remove cast when reporting serialized size (MarcoFalke)
fa364af89b test: Remove outdated comment (MarcoFalke)
Pull request description:
Consensus code should arrive at the same conclusion, regardless of the architecture it runs on. Using architecture-specific types such as `size_t` can lead to issues, such as the low-severity [CVE-2025-46597](https://bitcoincore.org/en/2025/10/24/disclose-cve-2025-46597/).
The CVE was already worked around, but it may be good to still fix the underlying issue.
Fixes https://github.com/bitcoin/bitcoin/issues/33709 with a few refactors to use explicit fixed-sized integer types in serialization-size related code and concluding with a refactor to return `uint64_t` from `GetSerializeSize`. The refactors should not change any behavior, because the CVE was already worked around.
ACKs for top commit:
Crypt-iQ:
crACK fa6c0bedd3
l0rinc:
ACK fa6c0bedd3
laanwj:
Code review ACK fa6c0bedd3
Tree-SHA512: f45057bd86fb46011e4cb3edf0dc607057d72ed869fd6ad636562111ae80fea233b2fc45c34b02256331028359a9c3f4fa73e9b882b225bdc089d00becd0195e
e346ecae83 Add eclipse, partitioning, and fingerprinting note to i2p.md (da1sychain)
19a6a3e75e Add eclipse, partitioning, and fingerprinting note in tor.md (da1sychain)
Pull request description:
Operating a Bitcoin node across multiple networks poses some fingerprinting risk. [0] Currently, this is not clear from the documentation and may be causing direct harm to users who are unaware of this.
The included documentation change indicates this risk factor but also notes that operating a node across multiple networks does provide an important benefit (increases the cost of eclipse and partitioning attacks) and is thus not discouraged outright.
The i2p documentation did not include a privacy recommendations section, so that is added as well.
[0] https://delvingbitcoin.org/t/fingerprinting-nodes-via-addr-requests/1786
ACKs for top commit:
danielabrozzoni:
ACK e346ecae83
rkrux:
crACK e346ecae83
mzumsande:
ACK e346ecae83
glozow:
lgtm ACK e346ecae83
Tree-SHA512: d35a00f604ed70bb9d2339066612414f590d8cfe4d02eb0f586364b32016c7259aaaf059bc5162779f36b06fb06508ff7162022bdf65aa22a840bc34f69b7b50
a3ac59a431 ci: Enable experimental kernel stuff in ASan task (MarcoFalke)
5b89956eeb kernel: Allow null arguments for serialized data (TheCharlatan)
Pull request description:
An empty span constructed from an empty vector may have a null data pointer depending on the implementation. Remove the BITCOINKERNEL_ARG_NONNULL requirement for these arguments and instead handle such null arguments in the implementation.
Also cherry-picked from #33845 to show that CI task passing now.
ACKs for top commit:
yuvicc:
Code review ACK a3ac59a431
maflcko:
review ACK a3ac59a431🥈
laanwj:
code review ACK a3ac59a431
Tree-SHA512: 629e463796f2f057df5be8e8981a45751c578ed0021be731c1d57fe849a539fe38b0a445914b0fc48f32f0408ad6d566984bd7f3a68797fcfdf1c6889e316a08
The Bash snippet was shorter, but relying on implicit word splitting
(see the shellcheck SC2086 warning).
For example, the DOCKER_BUILD_CACHE_ARG shlex.split is now done
identical to how ci/test/02_run_container.py does it.
Moreover, the Python will hopefully be easier to modify in the future,
as the dev notes recommend Python over Bash.
Base the task on --preset=dev-mode to ensure maximal coverage and add
the following:
bitcoin-chainstate (experimental) ... ON
test_bitcoin-qt ..................... ON
IPC and USDT remain explicitly disabled.
BlockTreeEntry objects are often compared. By exposing an equality
function, clients don't have to implement more expensive
comparisons based on height and block hash.
40dcbf580d build: add -Wtrailing-whitespace=any (fanquake)
d7659cd7e6 build: add -Wleading-whitespace=spaces (fanquake)
d86650220a cmake: Disable `-Wtrailing-whitespace` warnings for RCC-generated files (Hennadii Stepanov)
aabc5ca6ed cmake: Switch from AUTORCC to `qt6_add_resources` (Hennadii Stepanov)
25ae14c339 subprocess: replace tab with space (fanquake)
0c2b9dadd5 scripted-diff: remove whitespace in sha256_sse4.cpp (fanquake)
4da084fbc9 scripted-diff: change whitespace to spaces in univalue (fanquake)
e6caf150b3 ci: add moreutils to lint job (fanquake)
Pull request description:
GCC 15 now has options to turn leading & trailing whitespace into compile failures: https://gcc.gnu.org/gcc-15/changes.html#c-family. Fix the few cases of leading tabs, and trailing whitespace, and then enable `-Wleading-whitespace` and `-Wtrailing-whitespace`.
We currently get PRs that are opened with various whitespace, i.e #33822, so turning that into compile-time failure where possible, seems useful, to avoid a CI roundtrip.
ACKs for top commit:
ajtowns:
utACK 40dcbf580d
hebasto:
re-ACK 40dcbf580d.
Tree-SHA512: a128001ab2abb41cd6d249dcf46be4167ebd608d6b0f1452212a3ec9a383747bea623ab0382ec7bc0ac7a232a47cca5174e1cd73d4eda6751aa3cb2365ad2ede
fa9f29a4a7 doc: Recommend latest Debian stable or Ubuntu LTS (MarcoFalke)
fa1711ee0d doc: Add GCC-12 min release notes (MarcoFalke)
faa8be75c9 ci: Enable experimental kernel stuff in G++-12 task (previous releases) (MarcoFalke)
fabce97b30 test: Remove gccbug_90348 test case (MarcoFalke)
fa3854e432 test: Remove unused fs::create_directories test (MarcoFalke)
fa9dacdbde util: [refactor] Remove unused create_directories workaround (MarcoFalke)
fa807f78ae build: Bump g++ minimum supported version to 12 (MarcoFalke)
Pull request description:
All supported operating systems that previously came with at least g++-11, also come with at least g++-12, so bumping the minimum should be fine.
For reference:
* https://packages.ubuntu.com/jammy/g++-12
* https://packages.ubuntu.com/noble/g++ (g++-13)
* https://packages.debian.org/bookworm/g++ (g++-12)
* FreeBSD Ports ship a recent GCC
* RHEL-based 8, and 9 ship with g++-14 via appstream (`dnf install gcc-toolset-14` -> `/opt/rh/gcc-toolset-14/`)
* RHEL-based 10 ships with g++ (14 by default)
* OpenSuse Leap and Tumbleweed ship with g++ 15 https://software.opensuse.org/package/gcc15-c++
Obviously, downloading pre-compiled releases or compiling previous release branches is unaffected by this change.
ACKs for top commit:
janb84:
re-ACK fa9f29a4a7
TheCharlatan:
Re-ACK fa9f29a4a7
hebasto:
ACK fa9f29a4a7.
Tree-SHA512: ce14ecf78ccfe4f221dcbc9147dcfc00c0512b23a6fcda5ba71b62b4f5d39a5139f083d035113f189bfbd396d485e1ebc626a9a16b6fa0b74fd95aed2041c841
Base the task on --preset=dev-mode to ensure maximal coverage and add
the following:
bitcoin-chainstate (experimental) ... ON
libbitcoinkernel (experimental) ..... ON
kernel-test (experimental) .......... ON
IPC remains explicitly disabled.
Base the task on --preset=dev-mode to ensure maximal coverage and add
the following:
bitcoin-chainstate (experimental) ... ON
libbitcoinkernel (experimental) ..... ON
kernel-test (experimental) .......... ON
USDT remains explicitly disabled.
Base the task on --preset=dev-mode to ensure maximal coverage and add
the following:
bitcoin-chainstate (experimental) ... ON
libbitcoinkernel (experimental) ..... ON
kernel-test (experimental) .......... ON
Base the task on --preset=dev-mode to ensure maximal coverage and add
the following:
bitcoin-chainstate (experimental) ... ON
libbitcoinkernel (experimental) ..... ON
kernel-test (experimental) .......... ON
Base the task on --preset=dev-mode to ensure maximal coverage and add
the following:
bitcoin-chainstate (experimental) ... ON
libbitcoinkernel (experimental) ..... ON
kernel-test (experimental) .......... ON
Base the task on --preset=dev-mode to ensure maximal coverage and add
the following:
bitcoin-chainstate (experimental) ... ON
libbitcoinkernel (experimental) ..... ON
kernel-test (experimental) .......... ON
The GUI and USDT remain disabled explicitly.
Base the task on --preset=dev-mode to ensure maximal coverage and add
the following:
bitcoin-chainstate (experimental) ... ON
libbitcoinkernel (experimental) ..... ON
kernel-test (experimental) .......... ON
The GUI remains disabled explicitly.
Base the task on --preset=dev-mode to ensure maximal coverage and add
the following:
bitcoin-chainstate (experimental) ... ON
libbitcoinkernel (experimental) ..... ON
kernel-test (experimental) .......... ON
The GUI remains disabled explicitly.
Base the task on --preset=dev-mode to ensure maximal coverage and add
the following:
bitcoin-chainstate (experimental) ... ON
libbitcoinkernel (experimental) ..... ON
kernel-test (experimental) .......... ON
Also, shorten the name, for a less cluttered web view.
The test case no longer detects this specific issue for GCC versions
12.1+, as explained in the
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90348 thread and in this
compiler-explorer playground:
https://godbolt.org/z/Y48osrjM8
So remove the test case and update the -fstack-reuse=none cmake
docstring with the underlying affected GCC versions, and the bug URL.
The test was added in commit ddb75c2e87.
After the create_directories wrapper removal, the test is redundant with
the unit test in the upstream stdlib. Also, there is a Bitcoin Core
functional test that covers this behavior in
test/functional/feature_dirsymlinks.py
So remove this unit test.
Finally, I could not find a real system that still ships a buggy stdlib
(v11.2) in their package manager. A stand-alone test is also available
in compiler-explorer under https://godbolt.org/z/aeMKraYrT.
169f93d2ac depends: drop qtbase_avoid_native_float16 qt patch (fanquake)
Pull request description:
There is no-longer a minimum required / max supported libgcc version, after https://github.com/bitcoin/bitcoin/pull/33181.
ACKs for top commit:
laanwj:
Code review ACK 169f93d2ac
hebasto:
ACK 169f93d2ac.
Tree-SHA512: 1cb3639742d1466ae4355f99bea08afd1dab89a03b10aa7c0e04c8ec18e7654913028155badbfa67fdfa39764f6e04e7a0b5d007b0b3af8606425641db01f1e3
7a4901c902 test, refactor: Fix `-Warray-bounds` warning (Hennadii Stepanov)
faf2759c8c test: [refactor] Use reference over ptr to chainman (MarcoFalke)
Pull request description:
Just some minor test-only refactor commits to fix GCC false positive warnings, along with making the test code easier to read and understand:
* First change requested in https://github.com/bitcoin/bitcoin/pull/33785#discussion_r2510727269
* Second change requested in commit 3b135a8fc4451c93b3ea50b3f4621e0d19f35daf
Those changes are required in a bunch of pulls touching the CI system, so merging them allows to drop them in all pulls.
ACKs for top commit:
l0rinc:
ACK 7a4901c902
hebasto:
ACK 7a4901c902, I have reviewed the code and it looks OK.
Tree-SHA512: 64dca52ec7b25078bf489e2d8b43e449f4968fbac14a09c66a60cdc75b513588403665f248368820694a6f72c4f7f465589d9306355239cffe35c38111929eff
Base the task on --preset=dev-mode to ensure maximal coverage and add
the following:
bitcoin-chainstate (experimental) ... ON
libbitcoinkernel (experimental) ..... ON
kernel-test (experimental) .......... ON
An empty span constructed from an empty vector may have a null data
pointer depending on the implementation. Remove the
BITCOINKERNEL_ARG_NONNULL requirement for these arguments and instead
handle such null arguments in the implementation.
f06c6e1898 guix: build for Linux HOSTS with -static-libgcc (fanquake)
1bdf4695b0 guix: patch store paths out of libunwind (fanquake)
078a72c35f guix: move static-libc++ into CMAKE_EXE_LINKER_FLAGS flags (fanquake)
Pull request description:
Build release binaries with `-static-libgcc`.
Would avoid future issues like https://github.com/bitcoin/bitcoin/pull/33178.
ACKs for top commit:
willcl-ark:
ACK f06c6e1898
hebasto:
ACK f06c6e1898.
janb84:
Concept ACK f06c6e1898
Tree-SHA512: 79409d9044fe7a339ea8090ca0e70e1305816aa3225b41ca6e4f2fec37650206ab5a78c1b2495a27a0c6c0dd6d5f86bd696101d2d1c5ecc72c630dc34e55f7dc
66978a1a95 kernel: remove btck_chain_get_tip (stickies-v)
4dd7e6dc48 kernel: remove btck_chain_get_genesis (stickies-v)
Pull request description:
Removes `btck_chain_get_genesis` and `btck_chain_get_tip`.
They are trivially replaced with `btck_chain_get_by_height` (as indicated in the updated `bitcoinkernel_wrapper.h`), so I think it makes sense to trim the interface.
For `btck_chain_get_tip`: on `master` we don't provide any guarantees that the returned block index still corresponds to the actual tip, so the extra call doesn't seem like a regression to me.
ACKs for top commit:
TheCharlatan:
ACK 66978a1a95
janb84:
ACK 66978a1a95
Tree-SHA512: f583fbb7f2e3f8f23afb57732b2cbe9e1d550bfc43c9a2619895ee30c27f5f3c5cd9e4ecb7e05b1f6ab9e11c368596ec9b733d67e06cfafb12326d88e8e4dd7d
743abbcbde refactor: inline constant return value of `BlockTreeDB::WriteBatchSync` and `BlockManager::WriteBlockIndexDB` and `BlockTreeDB::WriteFlag` (Lőrinc)
e030240e90 refactor: inline constant return value of `CDBWrapper::Erase` and `BlockTreeDB::WriteReindexing` (Lőrinc)
cdab9480e9 refactor: inline constant return value of `CDBWrapper::Write` (Lőrinc)
d1847cf5b5 refactor: inline constant return value of `TxIndex::DB::WriteTxs` (Lőrinc)
50b63a5698 refactor: inline constant return value of `CDBWrapper::WriteBatch` (Lőrinc)
Pull request description:
Related to https://github.com/bitcoin/bitcoin/pull/31144#discussion_r2223587480
### Summary
`WriteBatch` always returns `true` - the errors are handled by throwing `dbwrapper_error` instead.
### Context
This boolean return value of the `Write` methods is confusing because it's inconsistent with `CDBWrapper::Read`, which catches exceptions and returns a boolean to indicate success/failure. It's bad that `Read` returns and `Write` throws - but it's a lot worse that `Write` advertises a return value when it actually communicates errors through exceptions.
### Solution
This PR removes the constant return values from write methods and inlines `true` at their call sites. Many upstream methods had boolean return values only because they were propagating these constants - those have been cleaned up as well.
Methods that returned a constant `true` value that now return `void`:
- `CDBWrapper::WriteBatch`, `CDBWrapper::Write`, `CDBWrapper::Erase`
- `TxIndex::DB::WriteTxs`
- `BlockTreeDB::WriteReindexing`, `BlockTreeDB::WriteBatchSync`, `BlockTreeDB::WriteFlag`
- `BlockManager::WriteBlockIndexDB`
### Note
`CCoinsView::BatchWrite` (and transitively `CCoinsViewCache::Flush` & `CCoinsViewCache::Sync`) were intentionally not changed here. While all implementations return `true`, the base `CCoinsView::BatchWrite` returns `false`. Changing this would cause `coins_view` tests to fail with:
> terminating due to uncaught exception of type std::logic_error: Not all unspent flagged entries were cleared
We can fix that in a follow-up PR.
ACKs for top commit:
achow101:
ACK 743abbcbde
janb84:
ACK 743abbcbde
TheCharlatan:
ACK 743abbcbde
sipa:
ACK 743abbcbde
Tree-SHA512: b2a550bff066216f1958d2dd9a7ef6a9949de518cc636f8ab9c670e0b7a330c1eb8c838e458a8629acb8ac980cea6616955cd84436a7b8ab9096f6d648073b1e
060bb55508 rpc: add decoded tx details to gettransaction with extra wallet fields (Matthew Zipkin)
ad1c3bdba5 [move only] move DecodeTxDoc() to a common util file for sharing (Matthew Zipkin)
d633db5416 rpc: add "ischange: true" in wallet gettransaction decoded tx output (Matthew Zipkin)
Pull request description:
This change is motivated by external RBF clients like https://github.com/CardCoins/additive-rbf-batcher/. It saves the user a redundant re-looping of tx outputs, calling `getaddressinfo` on each one, looking for the change output in order to adjust the fee.
The field `"ischange"` only appears when `gettransaction` is called on a wallet, and is either `true` or not present at all. I chose not to include `ischange: false` because it is confusing to see that on *received* transactions.
Example of the new field:
```
"vout": [
{
"value": 1.00000000,
"n": 0,
"scriptPubKey": {
"asm": "0 5483235e05c76273b3b50af62519738781aff021",
"desc": "addr(bcrt1q2jpjxhs9ca388va4ptmz2xtns7q6lupppkw7wu)#d42g84j6",
"hex": "00145483235e05c76273b3b50af62519738781aff021",
"address": "bcrt1q2jpjxhs9ca388va4ptmz2xtns7q6lupppkw7wu",
"type": "witness_v0_keyhash"
}
},
{
"value": 198.99859000,
"n": 1,
"scriptPubKey": {
"asm": "0 870ab1ab58632b05a417d5295f4038500e407592",
"desc": "addr(bcrt1qsu9tr26cvv4stfqh65547spc2q8yqavj7fnlju)#tgapemkv",
"hex": "0014870ab1ab58632b05a417d5295f4038500e407592",
"address": "bcrt1qsu9tr26cvv4stfqh65547spc2q8yqavj7fnlju",
"type": "witness_v0_keyhash"
},
"ischange": true
}
]
```
ACKs for top commit:
furszy:
ACK [060bb55](060bb55508)
maflcko:
review ACK 060bb55508🌛
achow101:
ACK 060bb55508
rkrux:
lgtm ACK 060bb55508
Tree-SHA512: aae4854d2bb4e9a7bc1152691ea90e594e8da8a63c9c7fda72a504fb6a7e54ae274ed5fa98d35d270e0829cc8f8d2fd35a5fc9735c252a10aa42cc22828930e7
01cc20f330 test: improve coverage for a resolved stalling situation (Martin Zumsande)
9af6daf07e test: remove magic number when checking for blocks that have arrived (Martin Zumsande)
3069d66dca p2p: During block download, adjust pindexLastCommonBlock better (Martin Zumsande)
Pull request description:
As described in #32179, `pindexLastCommonBlock` is updated later than necessary
in master.
In case of a linear chain with no forks, it can be moved forward at the beginning of
`FindNextBlocksToDownload`, so that the updated value can be used to better estimate `nWindowEnd`.
This helps the node to request all blocks from peers within the correct 1024-block-window and avoids peers being incorrectly marked as stallers.
I also changed `p2p_ibd_stalling.py` to cover the situation after a resolved situation, making sure that no additional peers are marked for stalling.
Fixes#32179
ACKs for top commit:
Crypt-iQ:
crACK 01cc20f330
stringintech:
re-ACK 01cc20f
achow101:
ACK 01cc20f330
sipa:
utACK 01cc20f330
Tree-SHA512: a97f7a7ef5ded538ee35576e04b3fbcdd46a6d0189c7ba3abacc6e0d81e800aac5b0c2d2565d0462ef6fd4acc751989f577fd6adfd450171a7d6ab26f437df32
1fc7a81f1f log: reduce excessive messages during block replay (Lőrinc)
Pull request description:
### Summary
After an incomplete reindex the blocks will need to be replayed.
This results in excessive `Rolling back` and `Rolling forward` messages which quickly triggers the recently introduced log rate limiter.
Change the logging strategy to:
- Add single `LogInfo` messages showing the full range being replayed for both rollback and roll forward;
- Log progress at `LogInfo` level only every 10,000 blocks to track the long operations.
### Reproducer:
* Start a normal ibd, stop after some progress
* Do a reindex, stop before it finishes
* Restart the node normally without specifying the reindex parameter
It should start rolling the blocks forward.
Before this change the excessive logging would show:
```
[*] Rolling forward 000000002f4f55aecfccc911076dc3f73ac0288c83dc1d79db0a026441031d40 (46245)
[*] Rolling forward 0000000017ffcf34c8eac010c529670ba6745ea59cf1edf7b820928e3b40acf6 (46246)
```
After the change it shows:
```
Replaying blocks
Rolling forward to 00000000000000001034012d7e4facaf16ca747ea94b8ea66743086cfe298ef8 (326223 to 340991)
Rolling forward 00000000000000000faabab19f17c0178c754dbed023e6c871dcaf74159c5f02 (330000)
Rolling forward 00000000000000000d9b2508615d569e18f00c034d71474fc44a43af8d4a5003 (340000)
...
Rolled forward to 00000000000000001034012d7e4facaf16ca747ea94b8ea66743086cfe298ef8
```
(similarly to rolling back)
ACKs for top commit:
Crypt-iQ:
crACK 1fc7a81f1f
stickies-v:
ACK 1fc7a81f1f
achow101:
ACK 1fc7a81f1f
vasild:
ACK 1fc7a81f1f
hodlinator:
Concept ACK 1fc7a81f1f
Tree-SHA512: 44ed1da8336de5a3d937e11a13e6f1789064e23eb70640a1c406fbb0074255344268f6eb6b06f036ca8d22bfeb4bdea319c3085a2139d848f6d36a4f8352b76a
79b4c276e7 Bugfix: QA: rpc_bind: Skip nonloopback test if no such address is found (Luke Dashjr)
Pull request description:
Without this, I get:
```
2025-09-19T03:14:05.157000Z TestFramework (INFO): PRNG seed is: 3218602557639511064
2025-09-19T03:14:05.158000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin-test/a
2025-09-19T03:14:05.158000Z TestFramework (INFO): Check for ipv6
2025-09-19T03:14:05.158000Z TestFramework (INFO): Check for non-loopback interface
2025-09-19T03:14:05.158000Z TestFramework (INFO): Bind test for []
2025-09-19T03:14:05.516000Z TestFramework (INFO): Bind test for []
2025-09-19T03:14:05.871000Z TestFramework (INFO): Bind test for ['[::1]']
2025-09-19T03:14:06.227000Z TestFramework (INFO): Bind test for ['127.0.0.1', '[::1]']
2025-09-19T03:14:06.583000Z TestFramework (INFO): Using interface None for testing
2025-09-19T03:14:06.583000Z TestFramework (INFO): Bind test for [None]
2025-09-19T03:14:06.583000Z TestFramework (ERROR): Unexpected exception
Traceback (most recent call last):
File "/Bitcoin/bitcoin/workingtree/test/functional/test_framework/test_framework.py", line 135, in main
self.run_test()
~~~~~~~~~~~~~^^
File "/Bitcoin/bitcoin/workingtree/test/functional/rpc_bind.py", line 126, in run_test
self._run_nonloopback_tests()
~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/Bitcoin/bitcoin/workingtree/test/functional/rpc_bind.py", line 157, in _run_nonloopback_tests
self.run_bind_test([self.non_loopback_ip], self.non_loopback_ip, [self.non_loopback_ip],
~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[(self.non_loopback_ip, self.defaultport)])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Bitcoin/bitcoin/workingtree/test/functional/rpc_bind.py", line 38, in run_bind_test
expected = [(addr_to_hex(addr), port) for (addr, port) in expected]
~~~~~~~~~~~^^^^^^
File "/Bitcoin/bitcoin/workingtree/test/functional/test_framework/netutil.py", line 132, in addr_to_hex
if '.' in addr: # IPv4
^^^^^^^^^^^
TypeError: argument of type 'NoneType' is not iterable
```
ACKs for top commit:
maflcko:
review ACK 79b4c276e7🏑
theStack:
Tested ACK 79b4c276e7
Tree-SHA512: 2a723d9bc5d1d50a8321a4f8a8cac3da3125d373ea71e6cc9d03de07307008f58970e361490d4c34530a6a976cb078b62d0ef09b7fb321ca1cfb9249a70d99a5
4543a3bde2 Squashed 'src/minisketch/' changes from ea8f66b1ea..d1bd01e189 (Hennadii Stepanov)
Pull request description:
This PR updates the `minisketch` subtree to latest upstream, which includes:
- https://github.com/bitcoin-core/minisketch/pull/75
- https://github.com/bitcoin-core/minisketch/pull/98
ACKs for top commit:
fanquake:
ACK c235aa468b
Tree-SHA512: 856fb8b7dc2e743c9c67164023bf53faf8766079aeccc82a30c8b90c85920b31977b6a8b26e51e5485b20e445a3ca6ff806e701a53e95f70181ea30055e3528c
It is equivalent to calling btck_chain_get_by_height with the
height obtained from btck_chain_get_height. In neither case do we
provide guarantees that the returned block index still corresponds
to the actual tip.
It does not make sense to use a pointer, when a reference is more
appropriate, especially given that nullptr has been ruled out.
This is also allows to remove the CI workaround to avoid warnings:
```
C++ compiler .......................... GNU 13.0.0, /bin/x86_64-w64-mingw32-g++-posix
...
/ci_container_base/src/test/blockmanager_tests.cpp: In member function ‘void blockmanager_tests::blockmanager_scan_unlink_already_pruned_files::test_method()’:
/ci_container_base/src/test/blockmanager_tests.cpp:63:17: error: possibly dangling reference to a temporary [-Werror=dangling-reference]
63 | const auto& chainman = Assert(m_node.chainman);
| ^~~~~~~~
In file included from /ci_container_base/src/streams.h:13,
from /ci_container_base/src/dbwrapper.h:11,
from /ci_container_base/src/node/blockstorage.h:10,
from /ci_container_base/src/test/blockmanager_tests.cpp:8:
/ci_container_base/src/util/check.h:116:49: note: the temporary was destroyed at the end of the full expression ‘inline_assertion_check<true, std::unique_ptr<ChainstateManager>&>(((blockmanager_tests::blockmanager_scan_unlink_already_pruned_files*)this)->blockmanager_tests::blockmanager_scan_unlink_already_pruned_files::<anonymous>.TestChain100Setup::<anonymous>.TestingSetup::<anonymous>.ChainTestingSetup::<anonymous>.BasicTestingSetup::m_node.node::NodeContext::chainman, std::source_location{(& *.Lsrc_loc27)}, std::basic_string_view<char>(((const char*)"m_node.chainman")))’
116 | #define Assert(val) inline_assertion_check<true>(val, std::source_location::current(), #val)
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/ci_container_base/src/test/blockmanager_tests.cpp:63:28: note: in expansion of macro ‘Assert’
63 | const auto& chainman = Assert(m_node.chainman);
| ^~~~~~
cc1plus: all warnings being treated as errors
gmake[2]: Leaving directory '/ci_container_base/ci/scratch/build-x86_64-w64-mingw32'
gmake[2]: *** [src/test/CMakeFiles/test_bitcoin.dir/build.make:382: src/test/CMakeFiles/test_bitcoin.dir/blockmanager_tests.cpp.obj] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:1810: src/test/CMakeFiles/test_bitcoin.dir/all] Error 2
gmake[1]: Leaving directory '/ci_container_base/ci/scratch/build-x86_64-w64-mingw32'
gmake: *** [Makefile:146: all] Error 2
```
This false-positive warning is also fixed in later GCC versions.
See also https://godbolt.org/z/fjc6be65M
fad6efd3be refactor: Use STR_INTERNAL_BUG macro where possible (MarcoFalke)
fada379589 doc: Remove unused bugprone-lambda-function-name suppression (MarcoFalke)
fae1d99651 refactor: Use const reference to std::source_location (MarcoFalke)
fa5fbcd615 util: Allow Assert() in contexts without __func__ (MarcoFalke)
Pull request description:
Without this, compile warnings could be hit about `__func__` being only valid inside functions.
```
warning: predefined identifier is only valid inside function [-Wpredefined-identifier-outside-function] note: expanded from macro Assert
115 | #define Assert(val) inline_assertion_check<true>(val, __FILE__, __LINE__, __func__, #val)
| ^
```
Ref https://github.com/bitcoin/bitcoin/pull/32740#discussion_r2486258473
This also introduces a slight behaviour change, because `std::source_location::function_name` usually includes the entire function signature instead of just the name.
ACKs for top commit:
l0rinc:
Code review ACK fad6efd3be
stickies-v:
ACK fad6efd3be
hodlinator:
re-ACK fad6efd3be
Tree-SHA512: e78a2d812d5ae22e45c93db1661dafbcd22ef209b3d8d8d5f2ac514e92fd19a17c3f0a5db2ef5e7748aa2083b10c0465326eb36812e6a80e238972facd2c7e98
0698c6b494 doc: Correct `pkgin` command usage on NetBSD (Hennadii Stepanov)
Pull request description:
When using `pkgin` on NetBSD, the `install` command must be specified.
ACKs for top commit:
fanquake:
ACK 0698c6b494
Tree-SHA512: 840fc1621d6fa9ad43501a3691a31cffd66c1ac8d34167f7ab0fe33e1a395198c241b3c31f3d0ebc314e28c0edb6055cc2ca3deba6408dcbd14390fd679a4803
dee7eec643 doc: mention coverage build in quickstart section (frankomosh)
Pull request description:
Adds a single comment in the libFuzzer quick-start that links to the Developer Notes coverage section. No build flags are changed or shown.
ACKs for top commit:
janb84:
ACK dee7eec643
dergoegge:
ACK dee7eec643
Tree-SHA512: 2fe5ffb6c3d06f75694646473c29b4cc9fe571f4659631ec174d444a14716771308eedeb7acab3bef7f62e9bfa8ed0462da0163b214cccdc6a9ad63bbf66d2a0
fa6db67369 ci: [refactor] Extract build_dir constant in ci-test-each-commit-exec.py (MarcoFalke)
fa95e6cdc1 ci: Use cmake --preset=dev-mode in test-each-commit task (MarcoFalke)
Pull request description:
Using the preset should reduce the bloat and need to maintain several places to list the same cmake cache variables.
The only difference should be that `bitcoin-chainstate (experimental)` will be enabled, which seems fast and in line with the goal of the CI task.
* Before: https://github.com/bitcoin/bitcoin/actions/runs/19174075826/job/54814118651#step:8:315
* After: (this pull) https://github.com/bitcoin/bitcoin/actions/runs/19190748069/job/54864837086#step:7:324
```diff
bitcoin-tx .......................... ON
bitcoin-util ........................ ON
bitcoin-wallet ...................... ON
- bitcoin-chainstate (experimental) ... OFF
+ bitcoin-chainstate (experimental) ... ON
libbitcoinkernel (experimental) ..... ON
kernel-test (experimental) .......... ON
Optional features:
ACKs for top commit:
hebasto:
ACK fa6db67369, I have reviewed the code and it looks OK.
Tree-SHA512: 61a78de7bcbf42bd266cb035f354862f5d1e1235acd2a81041e3a68a4d3ab4703fa2cfc993f28e4dacaa74e3cccc9ef568d5d4526605ce5a00bcd7c347b97121
dcb56fd4cb interfaces: add interruptWait method (ismaelsadeeq)
Pull request description:
This is an attempt to fix#33575 see the issue for background and the usefulness of this feature.
This PR uses one of the suggested approaches: adding a new `interruptWaitNext()` method to the mining interface.
It introduces a new boolean variable, `m_interrupt_wait`, which is set to `false` when the thread starts waiting. The `interruptWaitNext()` method wakes the thread and sets `m_interrupt_wait` to `true`.
Whenever the thread wakes up, it checks whether the wait was aborted; if so, it simply set ` m_interrupt_wait ` to false and return`nullptr`.
This PR also adds a functional test for the new method. The test uses `asyncio` to spawn two tasks and attempts to ensure that the wait is executed before the interrupt by using an event monitor. It adds a 0.1-second buffer to ensure the wait has started executing.
If that buffer elapses without `waitNext` executing, the test will fail because a transaction is created after the buffer.
ACKs for top commit:
furszy:
Code ACK dcb56fd4cb
ryanofsky:
Code review ACK dcb56fd4cb, just tweaking semantics slightly since last review so if an `interruptWait` call is made shortly after a `waitNext` call it will reliably cause the `waitNext` call to return right away without blocking, even if the `waitNext` call had not begun to execute or wait yet.
Sjors:
tACK dcb56fd4cb
TheCharlatan:
ACK dcb56fd4cb
Tree-SHA512: a03f049e1f303b174a9e5d125733b6583dfd8effa12e7b6c37bd9b2cff9541100f5f4514e80f89005c44a57d7e47804afe87aa5fdb6831f3b0cd9b01d83e42be
The removed comment become obsolete after bitcoin/bitcoin#32697 and
bitcoin/bitcoin#32881.
-BEGIN VERIFY SCRIPT-
sed -i "s/ Some tests are disabled if Python 3 is not available.//g" \
$( git grep -l " Some tests are disabled if Python 3 is not available." ./doc/ )
-END VERIFY SCRIPT-
fa1e8d8bad refactor: Add missing include in bitcoinkernel_wrapper.h (MarcoFalke)
Pull request description:
Otherwise, the compilation may fail with:
```
/home/admin/actions-runner/_work/_temp/src/kernel/bitcoinkernel_wrapper.h:271:14: error: no type named 'exception_ptr' in namespace 'std'; did you mean 'exception'?
271 | std::exception_ptr exception;
| ~~~~~^~~~~~~~~~~~~
| exception
/cxx_build/include/c++/v1/__exception/exception.h:72:33: note: 'exception' declared here
72 | class _LIBCPP_EXPORTED_FROM_ABI exception {
| ^
In file included from /home/admin/actions-runner/_work/_temp/src/bitcoin-chainstate.cpp:1:
/home/admin/actions-runner/_work/_temp/src/kernel/bitcoinkernel_wrapper.h:284:35: error: no member named 'current_exception' in namespace 'std'
284 | data.exception = std::current_exception();
| ^~~~~~~~~~~~~~~~~
/home/admin/actions-runner/_work/_temp/src/kernel/bitcoinkernel_wrapper.h:290:14: error: no member named 'rethrow_exception' in namespace 'std'
290 | std::rethrow_exception(user_data.exception);
| ^~~~~~~~~~~~~~~~~
/home/admin/actions-runner/_work/_temp/src/kernel/bitcoinkernel_wrapper.h:273:65: error: no viable conversion from 'std::nullptr_t' to 'std::exception'
273 | UserData user_data = UserData{.bytes = &bytes, .exception = nullptr};
| ^~~~~~~
/home/admin/actions-runner/_work/_temp/src/kernel/bitcoinkernel_wrapper.h:733:16: note: in instantiation of function template specialization 'btck::write_bytes<btck_Block>' requested here
733 | return write_bytes(get(), btck_block_to_bytes);
| ^
/cxx_build/include/c++/v1/__exception/exception.h:75:25: note: candidate constructor not viable: no known conversion from 'std::nullptr_t' to 'const exception &' for 1st argument
75 | _LIBCPP_HIDE_FROM_ABI exception(const exception&) _NOEXCEPT = default;
| ^ ~~~~~~~~~~~~~~~~
4 errors generated.
ACKs for top commit:
TheCharlatan:
ACK fa1e8d8bad
hebasto:
ACK fa1e8d8bad.
yuvicc:
ACK fa1e8d8bad
Tree-SHA512: c0127678db5913402c92b7602d159faae26539dc33f6159abd909b33746dd4626b8cbb6a86d8ccd3c9c83e06956fe55fb721a034480498d0cd87349aceea51f9
24bcad3d4d refactor: remove dead code in `CountWitnessSigOps` (Lőrinc)
Pull request description:
Found while reviewing #32840
The `nullptr` witness path was dead in normal code paths: replacing it with reference enables us deleting unreachable logic.
Code coverage proof:
https://maflcko.github.io/b-c-cov/total.coverage/src/script/interpreter.cpp.gcov.html#L2135
ACKs for top commit:
kevkevinpal:
ACK [24bcad3](24bcad3d4d)
maflcko:
review ACK 24bcad3d4d🐏
darosior:
Neat. utACK 24bcad3d4d.
stickies-v:
ACK 24bcad3d4d
Tree-SHA512: 92c87e431f06a15d8eeb02e20e9154b272c4586ddacf77c8d83783091485fb82c24ecbd711db7043a92cf6169746db24ad46a5904d694aea9d3c3aa96da725f0
ec8516ceb7 test: remove obsolete `get_{key,multisig}` helpers from wallet_util.py (Sebastian Falbesoner)
Pull request description:
This small cleanup PR is a late follow-up to #31250 (commit c847dee148). These helpers are unused and wouldn't work anymore, as they call a legacy wallet RPC (`dumpprivkey`). They were only ever used for testing the `importmulti` RPC, which also doesn't exist anymore. Functional tests that need to create key pairs and derive various output script types from them can use `get_generate_key` (introduced in #16528, commit f193ea889d) instead, without involving the node.
ACKs for top commit:
rkrux:
crACK ec8516ceb7
brunoerg:
code review ACK ec8516ceb7
Tree-SHA512: cab3701f1a8fbcff0eecea4cfdc632ffac226afd2eefe3c9274a84ee1bb71fb231a57cd0876025c714be257a249157b048b67e309b3734442c425d85cf481cf6
2bd155e6ee test: move create_malleated_version() to messages.py for reuse (Vasil Dimov)
Pull request description:
Move `create_malleated_version()` from `p2p_orphan_handling.py` to `test_framework/messages.py` so that it can be reused by other tests.
---
This is part of [#29415 Broadcast own transactions only via short-lived Tor or I2P connections](https://github.com/bitcoin/bitcoin/pull/29415). Putting it in its own PR to reduce the size of #29415 and because it does not depend on the other commits from there.
ACKs for top commit:
maflcko:
review ACK 2bd155e6ee🍨
l0rinc:
ACK 2bd155e6ee
brunoerg:
ACK 2bd155e6ee
w0xlt:
Code Review ACK 2bd155e6ee
pablomartin4btc:
cr ACK 2bd155e6ee
rkrux:
crACK 2bd155e6ee
Tree-SHA512: 566bd204ebf8b7e1b06513fd86fd5a3bb8097c5e875e50758f886abdc405c02717554334b10eb77e72400d0361924824b655e558b1f06e3064d1c837252e04af
9577daa3b8 doc: Add cmake help option in Windows build instructions (frankomosh)
Pull request description:
Follow-up to #33088.
Adds `cmake -B build -LH` documentation to Windows build guides, similar to Unix build documentation.
Based on the suggestion and example provided by stickies-v in #33088, with minor adjustment to match existing indented code block format in `build-windows.md`.
Tested for:
- WSL Ubuntu with mingw-w64 cross-compilation
- Windows 11 with Visual Studio 2022 (MSVC)
ACKs for top commit:
waketraindev:
ACK 9577daa
stickies-v:
ACK 9577daa3b8
hebasto:
ACK 9577daa3b8.
pablomartin4btc:
ACK 9577daa3b8
Tree-SHA512: a9c08944aff962a61d0111317b79cbb44368e85d0255daf8d3c229c6532cf8c11046fe6bba81025ef994d92b4324247ec75f5bdaf4a89b3ef403c5be37e63bd8
Now that the __func__ is no longer used, the
NOLINTBEGIN(bugprone-lambda-function-name) can be removed.
Also, re-format the NONFATAL_UNREACHABLE macro, while touching the
adjacent line.
The `mutated` parameter is never used at any call site - all callers pass `nullptr`.
The explicit comment in `validation.cpp` explains the reason:
// The malleation check is ignored; as the transaction tree itself
// already does not permit it, it is impossible to trigger in the
// witness tree.
There's a single call to the methods from `ComputeMerklePath` where the last parameter is always provided.
This simplifies the implementation by not having to check for missing parameter.
ed5720509f kernel: Use enumeration type for flags argument (TheCharlatan)
Pull request description:
Just a small followup from https://github.com/bitcoin/bitcoin/pull/30595#issuecomment-3485634089.
ACKs for top commit:
alexanderwiederin:
ACK ed5720509f
rkrux:
lgtm ACK ed5720509f as per the mentioned review comment of the previous PR.
stickies-v:
ACK ed5720509f
Tree-SHA512: f365d86c76b88b7730c4182192f8fbacc536121de367d03f27450087b39d13bb0cc21ca5ede9428077ccf5be90e959e892d7d383c8a2900b7bfd2864dde37466
038849e2e0 clang-tidy: Remove no longer needed NOLINT (Hennadii Stepanov)
Pull request description:
From https://github.com/bitcoin/bitcoin/pull/33714/files#r2491476516:
> Actually, the `NOLINT` was fixed and can be removed? You've confirmed that it is undeclared on the listed platforms, so it can't be hit by `readability-redundant-declaration`
ACKs for top commit:
maflcko:
lgtm ACK 038849e2e0
l0rinc:
I wanted to ask the same on the original PR but forgot - ACK 038849e2e0
Tree-SHA512: c0b24235425e80baeac3158c7169122364f31140367bc289430d34f01cd38f9f6a3931319f6fe4e1dc86bc4d87e21a5b4b8a2263c199e8083593f89ce592a177
5c41fa2918 guix: disable libsanitizer in Linux GCC build (fanquake)
Pull request description:
This causes issues when building against newer glibcs (i.e 2.42), and isn't needed in any case.
```bash
../../../../gcc-14.3.0/libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cpp:483:31: error: invalid application of ‘sizeof’ to incomplete type ‘__sanitizer::termio’
483 | unsigned struct_termio_sz = sizeof(struct termio);
| ^~~~~~~~~~~~~~~~~~~~~
```
Extracted from #25573.
ACKs for top commit:
maflcko:
lgtm ACK 5c41fa2918
hebasto:
ACK 5c41fa2918.
willcl-ark:
utACK 5c41fa2918
Tree-SHA512: a2a7b1a72155d47a1f2a1f3270d7a8255fad432c2d5d77f139e489634a3abb0ce1152c3c19fd7df629fa17c53ddb80eb1807dd195b7b7455d12d49a09c7b92dc
Used to display available configuration options, for consistency with recent changes in Unix build docs
Co-authored-by: stickies-v <stickies-v@users.noreply.github.com>
Performance likely does not matter here, but from a perspective of
code-readablilty, a const reference should be preferred for read-only
access.
So use it here.
This requires to set -Wno-error=dangling-reference for GCC 13.1
compilations, but this false-positive is fixed in later GCC versions.
See also https://godbolt.org/z/fjc6be65M
Without this, compile warnings could be hit about __func__ being only
valid inside functions.
warning: predefined identifier is only valid inside function [-Wpredefined-identifier-outside-function]
note: expanded from macro Assert
115 | #define Assert(val) inline_assertion_check<true>(val, __FILE__, __LINE__, __func__, #val)
| ^
Ref https://github.com/bitcoin/bitcoin/pull/32740#discussion_r2486258473
These helpers use a legacy wallet RPC (`dumpprivkey`) and thus don't
work anymore. They were only ever used for testing the `importmulti`
RPC, which also doesn't exist anymore.
This causes issues when building against newer glibcs (i.e 2.42), and isn't needed
in any case.
```bash
../../../../gcc-14.3.0/libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cpp:483:31: error: invalid application of ‘sizeof’ to incomplete type ‘__sanitizer::termio’
483 | unsigned struct_termio_sz = sizeof(struct termio);
| ^~~~~~~~~~~~~~~~~~~~~
```
Extracted from #25573.
6c7a34f3b0 kernel: Add Purpose section to header documentation (TheCharlatan)
7e9f00bcc1 kernel: Allowing reducing exports (TheCharlatan)
7990463b10 kernel: Add pure kernel bitcoin-chainstate (TheCharlatan)
36ec9a3ea2 Kernel: Add functions for working with outpoints (TheCharlatan)
5eec7fa96a kernel: Add block hash type and block tree utility functions to C header (TheCharlatan)
f5d5d1213c kernel: Add function to read block undo data from disk to C header (TheCharlatan)
09d0f62638 kernel: Add functions to read block from disk to C header (TheCharlatan)
a263a4caf2 kernel: Add function for copying block data to C header (TheCharlatan)
b30e15f432 kernel: Add functions for the block validation state to C header (TheCharlatan)
aa262da7bc kernel: Add validation interface to C header (TheCharlatan)
d27e27758d kernel: Add interrupt function to C header (TheCharlatan)
1976b13be9 kernel: Add import blocks function to C header (TheCharlatan)
a747ca1f51 kernel: Add chainstate load options for in-memory dbs in C header (TheCharlatan)
070e77732c kernel: Add options for reindexing in C header (TheCharlatan)
ad80abc73d kernel: Add block validation to C header (TheCharlatan)
cb1590b05e kernel: Add chainstate loading when instantiating a ChainstateManager (TheCharlatan)
e2c1bd3d71 kernel: Add chainstate manager option for setting worker threads (TheCharlatan)
65571c36a2 kernel: Add chainstate manager object to C header (TheCharlatan)
c62f657ba3 kernel: Add notifications context option to C header (TheCharlatan)
9e1bac4585 kernel: Add chain params context option to C header (TheCharlatan)
337ea860df kernel: Add kernel library context object (TheCharlatan)
28d679bad9 kernel: Add logging to kernel library C header (TheCharlatan)
2cf136dec4 kernel: Introduce initial kernel C header API (TheCharlatan)
Pull request description:
This is a first attempt at introducing a C header for the libbitcoinkernel library that may be used by external applications for interfacing with Bitcoin Core's validation logic. It currently is limited to operations on blocks. This is a conscious choice, since it already offers a lot of powerful functionality, but sits just on the cusp of still being reviewable scope-wise while giving some pointers on how the rest of the API could look like.
The current design was informed by the development of some tools using the C header:
* A re-implementation (part of this pull request) of [bitcoin-chainstate](https://github.com/bitcoin/bitcoin/blob/master/src/bitcoin-chainstate.cpp).
* A re-implementation of the python [block linearize](https://github.com/bitcoin/bitcoin/tree/master/contrib/linearize) scripts: https://github.com/TheCharlatan/bitcoin/tree/kernelLinearize
* A silent payment scanner: https://github.com/josibake/silent-payments-scanner
* An electrs index builder: https://github.com/josibake/electrs/commits/electrs-kernel-integration
* A rust bitcoin node: https://github.com/TheCharlatan/kernel-node
* A reindexer: https://github.com/TheCharlatan/bitcoin/tree/kernelApi_Reindexer
The library has also been used by other developers already:
* A historical block analysis tool: https://github.com/ismaelsadeeq/mining-analysis
* A swiftsync hints generator: https://github.com/theStack/swiftsync-hints-gen
* Fast script validation in floresta: https://github.com/vinteumorg/Floresta/pull/456
* A swiftsync node implementation: https://github.com/2140-dev/swiftsync/tree/master/node
Next to the C++ header also made available in this pull request, bindings for other languages are available here:
* Rust: https://github.com/TheCharlatan/rust-bitcoinkernel
* Python: https://github.com/stickies-v/py-bitcoinkernel
* Go: https://github.com/stringintech/go-bitcoinkernel
* Java: https://github.com/yuvicc/java-bitcoinkernel
The rust bindings include unit and fuzz tests for the API.
The header currently exposes logic for enabling the following functionality:
* Feature-parity with the now deprecated libbitcoin-consensus
* Optimized sha256 implementations that were not available to previous users of libbitcoin-consensus thanks to a static kernel context
* Full support for logging as well as control over categories and severity
* Feature parity with the existing experimental bitcoin-chainstate
* Traversing the block index as well as using block index entries for reading block and undo data.
* Running the chainstate in memory
* Reindexing (both full and chainstate-only)
* Interrupting long-running functions
The pull request introduces a new kernel-only test binary that purely relies on the kernel C header and the C++ standard library. This is intentionally done to show its capabilities without relying on other code inside the project. This may be relaxed to include some of the existing utilities, or even be merged into the existing test suite.
The complete docs for the API as well as some usage examples are hosted on [thecharlatan.ch/kernel-docs](https://thecharlatan.ch/kernel-docs/index.html). The docs are generated from the following repository (which also holds the examples): [github.com/TheCharlatan/kernel-docs](https://github.com/TheCharlatan/kernel-docs).
#### How can I review this PR?
Scrutinize the commit messages, run the tests, write your own little applications using the library, let your favorite code sanitizer loose on it, hook it up to your fuzzing infrastructure, profile the difference between the existing bitcoin-chainstate and the bitcoin-chainstate introduced here, be nitty on the documentation, police the C interface, opine on your own API design philosophy.
To get a feeling for the API, read through the tests, or one of the examples.
To configure this PR for making the shared library and the bitcoin-chainstate and test_kernel utilities available:
```
cmake -B build -DBUILD_KERNEL_LIB=ON -DBUILD_UTIL_CHAINSTATE=ON
```
Once compiled the library is part of the build artifacts that can be installed with:
```
cmake --install build
```
#### Why a C header (and not a C++ header)
* Shipping a shared library with a C++ header is hard, because of name mangling and an unstable ABI.
* Mature and well-supported tooling for integrating C exists for nearly every popular language.
* C offers a reasonably stable ABI
Also see https://github.com/bitcoin/bitcoin/pull/30595#issuecomment-2285719575.
#### What about versioning?
The header and library are still experimental and I would expect this to remain so for some time, so best not to worry about versioning yet.
#### Potential future additions
In future, the C header could be expanded to support (some of these have been roughly implemented):
* Handling transactions, block headers, coins cache, utxo set, meta data, and the mempool
* Adapters for an abstract coins store
* Adapters for an abstract block store
* Adapters for an abstract block tree store
* Allocators and buffers for more efficient memory usage
* An "[io-less](https://sans-io.readthedocs.io/how-to-sans-io.html)" interface
* Hooks for an external mempool, or external policy rules
#### Current drawbacks
* For external applications to read the block index of an existing Bitcoin Core node, Bitcoin Core needs to shut down first, since leveldb does not support reading across multiple processes. Other than migrating away from leveldb, there does not seem to be a solution for this problem. Such a migration is implemented in #32427.
* The fatal error handling through the notifications is awkward. This is partly improved through #29642.
* Handling shared pointers in the interfaces is unfortunate. They make ownership and freeing of the resources fuzzy and poison the interfaces with additional types and complexity. However, they seem to be an artifact of the current code that interfaces with the validation engine. The validation engine itself does not seem to make extensive use of these shared pointers.
* If multiple instances of the same type of objects are used, there is no mechanism for distinguishing the log messages produced by each of them. A potential solution is #30342.
* The background leveldb compaction thread may not finish in time leading to a non-clean exit. There seems to be nothing we can do about this, outside of patching leveldb.
ACKs for top commit:
alexanderwiederin:
re-ACK 6c7a34f3b0
stringintech:
re-ACK 6c7a34f
laanwj:
Code review ACK 6c7a34f3b0
ismaelsadeeq:
reACK 6c7a34f3b0👾
fanquake:
ACK 6c7a34f3b0 - soon we'll be running bitcoin (kernel)
Tree-SHA512: ffe7d4581facb7017d06da8b685b81f4b5e4840576e878bb6845595021730eab808d8f9780ed0eb0d2b57f2647c85dcb36b6325180caaac469eaf339f7258030
79d6f458e2 random: scope environ extern to macOS, BSDs and Illumos (fanquake)
Pull request description:
These platforms fail to compile with it removed.
macOS: #33675
BSDs / Illumos: https://github.com/hebasto/bitcoin-core-nightly/pull/79.
ACKs for top commit:
l0rinc:
ACK 79d6f458e2
hebasto:
re-ACK 79d6f458e2.
Tree-SHA512: dcaa15f0939d65a804107ceb110037f44d0ff70759f4d42fcc497a9c173ac28b1287b867f01732224788d1c1f9c883565bafc3abed3ccf28f1b67f23997ce3cf
52b1595850 depends: disable builtin variables (fanquake)
8b5a28fa78 depends: disable builtin rules and suffixes. (Dmitry Goncharov)
Pull request description:
This picks up #22126. Previously, this was more complicated to do, as depends packages (upnp, natpmp) used the rules being disabled. Those packages have since been removed.
When there is no rule to build a target in the makefile, make looks for a builtin rule. When `-r` is specified make no longer performs this lookup.
E.g. the following in an excerpt from `make -d` output. Here, make looks for a rule to build `all`.
```bash
Considering target file 'all'.
File 'all' does not exist.
Looking for an implicit rule for 'all'.
Trying pattern rule with stem 'all'.
Trying implicit prerequisite 'all.o'.
Trying pattern rule with stem 'all'.
Trying implicit prerequisite 'all.c'.
Trying pattern rule with stem 'all'.
Trying implicit prerequisite 'all.cc'.
Trying pattern rule with stem 'all'.
Trying implicit prerequisite 'all.C'.
Trying pattern rule with stem 'all'.
Trying implicit prerequisite 'all.cpp'.
Trying pattern rule with stem 'all'.
Trying implicit prerequisite 'all.p'.
Trying pattern rule with stem 'all'.
Trying implicit prerequisite 'all.f'.
...
```
Many more lines like this are omitted.
Because this build system does not use make builtin rules or suffixes, there is no benefit in having builtin rules enabled.
There are 2 benefits in having builtin rules disabled.
1. Improves performance by eliminating redundant lookups.
2. Simplifies troubleshooting by reducing the output of make `-d` or make `-p`.
Also see: https://www.gnu.org/software/make/manual/make.html#index-_002d_002dno_002dbuiltin_002drules.
ACKs for top commit:
hebasto:
re-ACK 52b1595850.
Tree-SHA512: 8b5fab5f8b01096942801d34edc47511e794d7eaf628ea78f8d2e020b83c91ed430f397dfb2b735d5fe957cdf5f10b0942fde6396f031f3ed9a773e0e6a6fabc
1e6e32fa8a ci: run native fuzz with MSAN job (fanquake)
3784d15bcd ci: use LLVM libcxx 21.1.5 (fanquake)
Pull request description:
I think this job should exist in this repo (not just qa-assets), if the alternative is double-handling changes to the interpreter. #32998 made changes which were then re-changed in #33600, to work around a false positive.
The unchached runtime of this job with `-lg` is `~32m`, with `-md` it's `~43m`.
Timeout is set to 150m, as the slow GHA runners were close to hitting a 120m limit.
ACKs for top commit:
maflcko:
lgtm ACK 1e6e32fa8a
dergoegge:
utACK 1e6e32fa8a
Tree-SHA512: afd4cb0039f4f49ddc23f5553a5bf6d5ceffbc12d91acd6890d5cc40c30b7421b23d04f305983d94c862daa6fc07535b1331d7fa2a8ebfe9f19c20d83d95c692
7632e0ba31 ci: fix configure docker action inputs (will)
0b3b8a3be1 ci: fix lint docker caching (will)
Pull request description:
Fixes: #33735
Correct runner type selection for the lint job.
This was erroneously left-out during refactor of the runner selection mechanism in #33302 causing the lint job to run on GH hosts (and therefore not be able to acces local cirrus caches).
ACKs for top commit:
maflcko:
re-ACK 7632e0ba31📞
hebasto:
ACK 7632e0ba31.
Tree-SHA512: b228a79d13ed80c75fc5e51c4fb93c7fad1cb33c00a659afe65033ce09d95e6ac84e01627f2e58e640ff483d798ac1b9e23f14d31a9c045fd99367059ceef5b4
b4d0288c46 doc: update Guix INSTALL.md (fanquake)
Pull request description:
It's somewhat annoying that Guix is falling out of being packaged by distros. For some more context, see https://lwn.net/Articles/1035491/.
> However, it is likely that the [Guix](https://guix.gnu.org/en/) package manager will soon be removed from the repositories for Debian 13 and Debian 12 ("bookworm", also called oldstable).
This seems to be happening. You can't `apt install guix` using the current release of Debian. https://packages.debian.org/search?keywords=guix. Guix is not going to be included in next release of Ubuntu (`25.10`): https://packages.ubuntu.com/search?keywords=guix.
Looking at https://aur.archlinux.org/packages/guix, comments over the last few months seem to indicate that the build is broken.
A 1.5.0 release is planned for sometime in January 2026: https://codeberg.org/guix/release-planning/wiki/release-1.5.0-project/. So hopefully the situation is going to improve in future.
ACKs for top commit:
willcl-ark:
ACK b4d0288c46
hebasto:
ACK b4d0288c46.
Tree-SHA512: 545f3529af82c18556ddfe104c01f77e28da31018a44047812450565a9b3bad3afa60f714b375c06c3a10aed722d54500846aa70a8069c5fe2d96b26d426b6c1
5d784bebaf clang-tidy: Disable `ArrayBound` check in src/ipc and src/test (Hennadii Stepanov)
5efdb0ef30 ci: Update Clang in "tidy" job (Hennadii Stepanov)
Pull request description:
This PR:
1. Updates to [IWYU 0.25](https://github.com/include-what-you-use/include-what-you-use/releases/tag/0.25), which is compatible with Clang 21.
2. Fixes new "modernize-use-default-member-init" warnings. The warning in `interpreter.cpp` is a [false positive](https://github.com/llvm/llvm-project/issues/160394), so it has been suppressed.
ACKs for top commit:
maflcko:
review ACK 5d784bebaf🎒
ryanofsky:
Code review ACK 5d784bebaf, just adding clang version comment since last review.
Tree-SHA512: a1d853675ec064170ee0f1cd16be6a900676588d4a1e7b5def8733933b140ba1a9520ec6f6a42bf7638b2ff7cf2fe4d5866d407f68b677b49d2bd68ff345f735
81e5c8385b test: cover invalid codesep positions for signature in taproot (Greg Sanders)
Pull request description:
There is some basic coverage, but I felt like adding some boundary conditions where the only issue is the codesep value would be nice.
ACKs for top commit:
ajtowns:
ACK 81e5c8385b
TheCharlatan:
ACK 81e5c8385b
Tree-SHA512: de74895c3bb49854987654720ebcefea2f47c4a55ba6ab4a52878f6a9a0bd8b3085afa3485101610327fa8d35c3d074542f58540e126460bd4bea918cb0054ee
Now that an API has been defined, remove the override for symbol
visibility of the library. It is now possible to build the library with
reduced exports.
This introduces the transaction outpoint, input and id types. This now
allows a user to retrieve a transaction output from a prior transaction
that a transaction outpoint is pointing to by either scanning through
all available transactions, or maintaining a data structure for lookups.
This is exercised in the tests by verifying the script of every
transaction in the test chain.
Introduce btck_BlockHash as a type-safe identifier for a block. Adds
functions to retrieve block tree entries by hash or height, get block
hashes and heights from entries. access the genesis block, and check if
blocks are in the active chain.
This adds functions for reading the undo data from disk with a retrieved
block tree entry. The undo data of a block contains all the spent
script pubkeys of all the transactions in a block. For ease of
understanding the undo data is renamed to spent outputs with seperate
data structures exposed for a block's and a transaction's spent outputs.
In normal operations undo data is used during re-orgs. This data might
also be useful for building external indexes, or to scan for silent
payment transactions.
Internally the block undo data contains a vector of transaction undo
data which contains a vector of the coins consumed. The coins are all
int the order of the transaction inputs of the consuming transactions.
Each coin can be used to retrieve a transaction output and in turn a
script pubkey and amount.
This translates to the three-level hierarchy the api provides: Block
spent outputs contain transaction spent outputs, which contain
individual coins. Each coin includes the associated output, the height
of the block is contained in, and whether it is from a coinbase
transaction.
This adds functions for reading a block from disk with a retrieved block
tree entry. External services that wish to build their own index, or
analyze blocks can use this to retrieve block data.
The block tree can now be traversed from the tip backwards. This is
guaranteed to work, since the chainstate maintains an internal block
tree index in memory and every block (besides the genesis) has an
ancestor.
The user can use this function to iterate through all blocks in the
chain (starting from the tip). The tip is retrieved from a separate
`Chain` object, which allows distinguishing whether entries are
currently in the best chain. Once the block tree entry for the genesis
block is reached a nullptr is returned if the user attempts to get the
previous entry.
This adds a function for streaming bytes into a user-owned data
structure.
Use it in the tests for verifying the implementation of the validation
interface's `BlockChecked` method.
These allow for the interpretation of the data in a `BlockChecked`
validation interface callback. The validation state passed through
`BlockChecked` is the source of truth for the validity of a block (the
mode). It is
also useful to get richer information in case a block failed to
validate (the result).
This adds the infrastructure required to process validation events. For
now the external validation interface only has support for the
`BlockChecked` , `NewPoWValidBlock`, `BlockConnected`, and
`BlockDisconnected` callback. Support for the other internal
validation interface methods can be added in the future.
The validation interface follows an architecture for defining its
callbacks and ownership that is similar to the notifications.
The task runner is created internally with a context, which itself
internally creates a unique ValidationSignals object. When the user
creates a new chainstate manager the validation signals are internally
passed to the chainstate manager through the context.
A validation interface can register for validation events with a
context. Internally the passed in validation interface is registerd with
the validation signals of a context.
The callbacks block any further validation execution when they are
called. It is up to the user to either multiplex them, or use them
otherwise in a multithreaded mechanism to make processing the validation
events non-blocking.
I.e. for a synchronous mechanism, the user executes instructions
directly at the end of the callback function:
```mermaid
sequenceDiagram
participant V as Validation
participant C as Callback
V->>C: Call callback
Note over C: Process event (blocks)
C-->>V: Return
Note over V: Validation resumes
```
To avoid blocking, the user can submit the data to e.g. a worker thread
or event manager, so processing happens asynchronously:
```mermaid
sequenceDiagram
participant V as Validation
participant C as Callback
participant W as Worker Thread
V->>C: Call callback
C->>W: Submit to worker thread
C-->>V: Return immediately
Note over V: Validation continues
Note over W: Process event async
```
Add `btck_import_blocks` to import block data and rebuild indexes. The
function can either reindex all existing block files if the indexes were
previously wiped through the chainstate manager options, or import
blocks from specified file paths.
This allows a user to run the kernel without creating on-disk files for
the block tree and chainstate indexes. This is potentially useful in
scenarios where the user needs to do some ephemeral validation
operations.
One specific use case is when linearizing the blocks on disk. The block
files store blocks out of order, so a program may utilize the library
and its header to read the blocks with one chainstate manager, and then
write them back in order, and without orphans, with another chainstate
maanger. To save disk resources and if the indexes are not required once
done, it may be beneficial to keep the indexes in memory for the
chainstate manager that writes the blocks back again.
Adds options for wiping the chainstate and block tree indexes to the
chainstate manager options. In combination and once the
`*_import_blocks(...)` function is added in a later commit, this
triggers a reindex. For now, it just wipes the existing data.
The added function allows the user process and validate a given block
with the chainstate manager. The *_process_block(...) function does some
preliminary checks on the block before passing it to
`ProcessNewBlock(...)`. These are similar to the checks in the
`submitblock()` rpc.
Richer processing of the block validation result will be made available
in the following commits through the validation interface.
The commits also adds a utility for deserializing a `CBlock`
(`kernel_block_create()`) that may then be passed to the library for
processing.
The tests exercise the function for both mainnet and regtest. The
commit also adds the data of 206 regtest blocks (some blocks also
contain transactions).
The library will now internally load the chainstate when a new
ChainstateManager is instantiated.
Options for controlling details of loading the chainstate will be added
over the next few commits.
This is the main driver class for anything validation related, so expose
it here.
Creating the chainstate manager options will currently also trigger the
creation of their respectively configured directories.
The chainstate manager and block manager options are consolidated into a
single object. The kernel might eventually introduce a separate block
manager object for the purposes of being a light-weight block store
reader.
The chainstate manager will associate with the context with which it was
created for the duration of its lifetime and it keeps it in memory with
a shared pointer.
The tests now also create dedicated temporary directories. This is
similar to the behaviour in the existing unit test framework.
Co-authored-by: stickies-v <stickies-v@protonmail.com>
The notifications are used for notifying on connected blocks and on
warning and fatal error conditions.
The user of the C header may define callbacks that gets passed to the
internal notification object in the
`kernel_NotificationInterfaceCallbacks` struct.
Each of the callbacks take a `user_data` argument that gets populated
from the `user_data` value in the struct. It can be used to recreate the
structure containing the callbacks on the user's side, or to give the
callbacks additional contextual information.
As a first option, add the chainparams. For now these can only be
instantiated with default values. In future they may be expanded to take
their own options for regtest and signet configurations.
This commit also introduces a unique pattern for setting the option
values when calling the `*_set(...)` function.
The context introduced here holds the objects that will be required for
running validation tasks, such as the chosen chain parameters, callbacks
for validation events, and interrupt handling. These will be used by the
chainstate manager introduced in subsequent commits.
This commit also introduces conventions for defining option objects. A
common pattern throughout the C header will be:
```
options = object_option_create();
object = object_create(options);
```
This allows for more consistent usage of a "builder pattern" for
objects where options can be configured independently from
instantiation.
Exposing logging in the kernel library allows users to follow
operations. Users of the C header can use
`kernel_logging_connection_create(...)` to pass a callback function to
Bitcoin Core's internal logger. Additionally the level and category can
be globally configured.
By default, the logger buffers messages until
`kernel_loggin_connection_create(...)` is called. If the user does not
want any logging messages, it is recommended that
`kernel_disable_logging()` is called, which permanently disables the
logging and any buffering of messages.
Co-authored-by: stringintech <stringintech@gmail.com>
As a first step, implement the equivalent of what was implemented in the
now deprecated libbitcoinconsensus header. Also add a test binary to
exercise the header and library.
Unlike the deprecated libbitcoinconsensus the kernel library can now use
the hardware-accelerated sha256 implementations thanks for its
statically-initialzed context. The functions kept around for
backwards-compatibility in the libbitcoinconsensus header are not ported
over. As a new header, it should not be burdened by previous
implementations. Also add a new error code for handling invalid flag
combinations, which would otherwise cause a crash.
The macros used in the new C header were adapted from the libsecp256k1
header.
To make use of the C header from C++ code, a C++ header is also
introduced for wrapping the C header. This makes it safer and easier to
use from C++ code.
Co-authored-by: stickies-v <stickies-v@protonmail.com>
In `QSortFilterProxyModel`, `invalidateFilter()` is scheduled for
deprecation in Qt 6.13.
`beginFilterChange()` was introduced in Qt 6.9.
`endFilterChange()` was introduced in Qt 6.10.
4e352efa2c qt: add createwallet, createwalletdescriptor, and migratewallet to history filter (WakeTrainDev)
Pull request description:
Added `createwallet`, `createwalletdescriptor` and `migratewallet` RPC commands to the Qt console history filter since they may include passphrases or other sensitive data that should not be stored in command history.
ACKs for top commit:
pablomartin4btc:
utACK 4e352efa2c
hebasto:
ACK 4e352efa2c.
Tree-SHA512: dc6a12b95173b1e476d483381df3d74add88a1e225c90b1b60db59eab6d504a2496b66890ccec28c691745e405a3053d72afda9d80ae96a703f12cd256e4ebd6
When there is no rule to build a target in the makefile, make looks
for a builtin rule.
When --no-builtin-rules is specified make no longer performs this lookup.
E.g. the following in an excerpt from make -d output.
Here, make looks for a rule to build 'all'.
Considering target file 'all'.
File 'all' does not exist.
Looking for an implicit rule for 'all'.
Trying pattern rule with stem 'all'.
Trying implicit prerequisite 'all.o'.
Trying pattern rule with stem 'all'.
Trying implicit prerequisite 'all.c'.
Trying pattern rule with stem 'all'.
Trying implicit prerequisite 'all.cc'.
Trying pattern rule with stem 'all'.
Trying implicit prerequisite 'all.C'.
Trying pattern rule with stem 'all'.
Trying implicit prerequisite 'all.cpp'.
Trying pattern rule with stem 'all'.
Trying implicit prerequisite 'all.p'.
Trying pattern rule with stem 'all'.
Trying implicit prerequisite 'all.f'.
...
Many more lines like this are omitted.
Because this build system does not use make builtin rules or suffixes,
there is no benefit in having builtin rules enabled.
There are 2 benefits in having builtin rules disabled.
1. Improves performance by eliminating redundant lookups.
2. Simplifies troubleshooting by reducing the output of make -d or
make -p.
The options used were wrong in two ways: firstly they were not enforced
as a "choice" (i.e. invalid input valudes could be provided without
error) and one of the options was listed as `gh` when we passed it as
`gha` from ci.yml.
"Fix" this by removing the choice altogether but sanity-testing the
input value against an expected list using a GHA "warning" to notify of
unknown inputs.
fa9d0f994b ci: gha: Set debug_pull_request_number_str annotation (MarcoFalke)
Pull request description:
GitHub Actions does not offer any way to determine the pull request number in a machine readable way from the checks API. See https://github.com/bitcoin/bitcoin/issues/27178#issuecomment-1503475232.
However, the pull request number can be useful for external tools to act on CI results.
Fix that by using a check run annotation for a single task named `debug_pull_request_number_str`.
This should re-enable the 'CI Failed' labelling mechanism via 1f24cc1ab9.
ACKs for top commit:
l0rinc:
code review ACK fa9d0f994b
willcl-ark:
ACK fa9d0f994b
Tree-SHA512: d872b81afeaef603006bb65f18acafdff2771acf2b70af4ab6b46167b0826e96b1ac434bba2020833107922eaf1e73f59a50782a535ba04ea16921f1828d42ca
No need to pass consensusParams, as CheckHeadersPoW already has access
to m_chainparams.GetConsensus()
Co-Authored-By: maflcko <6399679+maflcko@users.noreply.github.com>
Avoid the need to construct a CBlockIndex object just to compute work for a header,
when its nBits value suffices for that.
Co-Authored-By: Pieter Wuille <pieter@wuille.net>
Just don't call this function when it won't have any effect.
Note that we can't remove the LookupBlockIndex call, since `last_received_header`
is needed to check if new headers were received (`received_new_header`).
A few temporary `CCoinsViewCache`'s are destructed right after the `Flush()`, therefore it is not necessary to call `ReallocateCache` to recreate them right before they're killed anyway.
* `Flush()` - retains existing functionality;
* `Flush(/*will_reuse_cache=*/false)` - skips destruction and reallocation of the parent cache since it will soon go out of scope anyway;
For the `will_reuse_cache` parameter we want to see exactly which ones will reallocate memory and which won't - since both can be valid usages.
This change was based on a subset of https://github.com/bitcoin/bitcoin/pull/28945.
Co-authored-by: Martin Ankerl <martin.ankerl@gmail.com>
Previously, when the parent coins cache had no entry and the child did, `BatchWrite` performed a find followed by `try_emplace`, which resulted in multiple `SipHash` computations and bucket traversals on the common insert path.
This change uses a single leading `try_emplace` and branches on the returned `inserted` flag.
In the `FRESH && SPENT` case (only exercised by tests), we erase the just-inserted placeholder (which is constant time with no rehash anyway).
Semantics are unchanged for all valid parent/child state combinations.
This change is a minimal version of 723c49b63b and draws simplification ideas ae76ec7bcf.
Added TODO versions for related pre-existing issues that should be fixed in follow-ups.
Co-authored-by: Martin Ankerl <martin.ankerl@gmail.com>
Co-authored-by: Andrew Toth <andrewstoth@gmail.com>
Co-authored-by: optout <13562139+optout21@users.noreply.github.com>
Currently, if `-asmap` is specified without a filename, bitcoind tries to load
`ip_asn.map` data file.
This change now requires `-asmap=ip_asn.map` or another filename to be
specified explicitly.
The change is intended to make behavior of the option explicit avoid confusion
reported https://github.com/bitcoin/bitcoin/issues/33386 where documentation
specifies a default file which is not actually loaded by default. It was
originally implemented in
https://github.com/bitcoin/bitcoin/pull/33631#issuecomment-3410302383 and
various alternatives are discussed there.
Co-authored-by: Fabian Jahr <fjahr@protonmail.com>
07a926474b node: change a tx-relay on/off flag to enum (Vasil Dimov)
Pull request description:
Previously the `bool relay` argument to `BroadcastTransaction()` designated:
```
relay=true: add to the mempool and broadcast to all peers
relay=false: add to the mempool
```
Change this to an `enum`, so it is more readable and easier to extend with a 3rd option. Consider these example call sites:
```cpp
Paint(true);
// Or
Paint(/*is_red=*/true);
```
vs
```cpp
Paint(RED);
```
The idea for putting `TxBroadcastMethod` into `node/types.h` by Ryan.
---
This is part of [#29415 Broadcast own transactions only via short-lived Tor or I2P connections](https://github.com/bitcoin/bitcoin/pull/29415). Putting it in its own PR to reduce the size of #29415 and because it does not logically depend on the other commits from there.
ACKs for top commit:
optout21:
ACK 07a926474b
kevkevinpal:
ACK [07a9264](07a926474b)
laanwj:
Concept and code review ACK 07a926474b. Agree with the general reasoning and the change in #29415 is a valid motivation to change this interface.
glozow:
utACK 07a926474b
Tree-SHA512: ec8f6fa56a6d2422a0fbd5941ff2792685e8d8e7b9dd50bba9f3e21ed9b4a4a26c89b0d7e4895d48f30b7a635f2eddd894af26b5266410952cbdaf5c40b42966
1a1f46c228 refactor/doc: Add blockman param to `GetTransaction` doc comment and reorder out param (Musa Haruna)
Pull request description:
Follow-up to [#27125](https://github.com/bitcoin/bitcoin/pull/27125#discussion_r1190350876)
This PR addresses a minor documentation and style nit mentioned during review:
- Adds the missing `@param[in] blockman` line to the `GetTransaction()` doc comment.
- Moves the output parameter `hashBlock` to the end of both the function
declaration and definition, as suggested in the comment.
ACKs for top commit:
l0rinc:
ACK 1a1f46c228
maflcko:
re-lgtm-ut-cr-rfm-ACK 1a1f46c228
kevkevinpal:
reACK [1a1f46c](1a1f46c228)
Tree-SHA512: 5807a1ae6c383e691e948648dcb1e029620eaff3dcdff73d88c6dc268a7af5559a30c491b72f038b3f7e812e1845f4f063b49bd3671edfac1bb3a170c84be4f5
b6f8c48946 coins: increase default `dbbatchsize` to 32 MiB (Lőrinc)
8bbb7b8bf8 refactor: Extract default batch size into kernel (Lőrinc)
Pull request description:
This change is part of [[IBD] - Tracking PR for speeding up Initial Block Download](https://github.com/bitcoin/bitcoin/pull/32043)
### Summary
When the in-memory UTXO set is flushed to LevelDB (after IBD or AssumeUTXO load), it does so in batches to manage memory usage during the flush.
A hidden `-dbbatchsize` config option exists to modify this value. This PR only changes the default from `16` MiB to `32` MiB.
Using a larger default reduces the overhead of many small writes and improves I/O efficiency (especially on HDDs). It may also help LevelDB optimize writes more effectively (e.g., via internal ordering).
The change is meant to speed up a critical part of IBD: dumping the accumulated work to disk.
### Context
The UTXO set has grown significantly since [2017](https://github.com/bitcoin/bitcoin/pull/10148/files#diff-d102b6032635ce90158c1e6e614f03b50e4449aa46ce23370da5387a658342fdR26-R27), when the original fixed 16 MiB batch size was chosen.
With the current multi-gigabyte UTXO set and the common practice of using larger `-dbcache` values, the fixed 16 MiB batch size leads to several inefficiencies:
* Flushing the entire UTXO set often requires thousands of separate 16 MiB write operations.
* Particularly on HDDs, the cumulative disk seek time and per-operation overhead from numerous small writes significantly slow down the flushing process.
* Each `WriteBatch` call incurs internal LevelDB overhead (e.g., MemTable handling, compaction triggering logic). More frequent, smaller batches amplify this cumulative overhead.
Flush times of 20-30 minutes are not uncommon, even on capable hardware.
### Considerations
As [noted by sipa](https://github.com/bitcoin/bitcoin/pull/31645#issuecomment-2587500105), flushing involves a temporary memory usage increase as the batch is prepared. A larger batch size naturally leads to a larger peak during this phase. Crashing due to OOM during a flush is highly undesirable, but now that [#30611](https://github.com/bitcoin/bitcoin/pull/30611) is merged, the most we'd lose is the first hour of IBD.
Increasing the LevelDB write batch size from 16 to 32 MiB raised the measured peaks by ~70 MiB in my tests during UTXO dump. The option remains hidden, and users can always override it.
The increased peak memory usage (detailed below) is primarily attributed to LevelDB's `leveldb::Arena` (backing MemTables) and the temporary storage of serialized batch data (e.g., `std::string` in `CDBBatch::WriteImpl`).
Performance gains are most pronounced on systems with slower I/O (HDDs), but some SSDs also show measurable improvements.
### Measurements:
AssumeUTXO proxy, multiple runs with error bars (flushing time is faster that the measured loading + flushing):
* Raspberry Pi, dbcache=500: ~30% faster with 32 MiB vs 16 MiB, peak +~75 MiB and still < 1 GiB.
* i7 + HDD: results vary by dbcache, but 32 MiB usually beats 16 MiB and tracks close to 64 MiB without the larger peak.
* i9 + fast NVMe: roughly flat across 16/32/64 MiB. The goal here is to avoid regressions, which holds.
### Reproducer:
```bash
# Set up a clean demo environment
rm -rfd demo && mkdir -p demo
# Build Bitcoin Core
cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j$(nproc)
# Start bitcoind with minimal settings without mempool and internet connection
build/bin/bitcoind -datadir=demo -stopatheight=1
build/bin/bitcoind -datadir=demo -blocksonly=1 -connect=0 -dbcache=3000 -daemon
# Load the AssumeUTXO snapshot, making sure the path is correct
# Expected output includes `"coins_loaded": 184821030`
build/bin/bitcoin-cli -datadir=demo -rpcclienttimeout=0 loadtxoutset ~/utxo-880000.dat
# Stop the daemon and verify snapshot flushes in the logs
build/bin/bitcoin-cli -datadir=demo stop
grep "FlushSnapshotToDisk: completed" demo/debug.log
```
---
This PR originally proposed 64 MiB, then a dynamic size, but both were dropped: 64 MiB increased peaks more than desired on low-RAM systems, and the dynamic variant underperformed across mixed hardware. 32 MiB is a simpler default that captures most of the gains with a modest peak increase.
For more details see: https://github.com/bitcoin/bitcoin/pull/31645#issuecomment-3234329502
---
While the PR isn't about IBD in general, rather about a critical section of it, I have measured a reindex-chainstate until 900k blocks, showing a 1% overall speedup:
<details>
<summary>Details</summary>
```python
COMMITS="e6bfd95d5012fa1d91f83bf4122cb292afd6277f af653f321b135a59e38794b537737ed2f4a0040b"; \
STOP=900000; DBCACHE=10000; \
CC=gcc; CXX=g++; \
BASE_DIR="/mnt/my_storage"; DATA_DIR="$BASE_DIR/BitcoinData"; LOG_DIR="$BASE_DIR/logs"; \
(echo ""; for c in $COMMITS; do git fetch -q origin $c && git log -1 --pretty='%h %s' $c || exit 1; done; echo "") && \
hyperfine \
--sort command \
--runs 1 \
--export-json "$BASE_DIR/rdx-$(sed -E 's/(\w{8})\w+ ?/\1-/g;s/-$//'<<<"$COMMITS")-$STOP-$DBCACHE-$CC.json" \
--parameter-list COMMIT ${COMMITS// /,} \
--prepare "killall bitcoind 2>/dev/null; rm -f $DATA_DIR/debug.log; git checkout {COMMIT}; git clean -fxd; git reset --hard && \
cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release && ninja -C build bitcoind && \
./build/bin/bitcoind -datadir=$DATA_DIR -stopatheight=$STOP -dbcache=1000 -printtoconsole=0; sleep 10" \
--cleanup "cp $DATA_DIR/debug.log $LOG_DIR/debug-{COMMIT}-$(date +%s).log" \
"COMPILER=$CC ./build/bin/bitcoind -datadir=$DATA_DIR -stopatheight=$STOP -dbcache=$DBCACHE -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0"
e6bfd95d50 Merge bitcoin-core/gui#881: Move `FreespaceChecker` class into its own module
af653f321b coins: derive `batch_write_bytes` from `-dbcache` when unspecified
Benchmark 1: COMPILER=gcc ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=900000 -dbcache=10000 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = e6bfd95d50)
Time (abs ≡): 25016.346 s [User: 30333.911 s, System: 826.463 s]
Benchmark 2: COMPILER=gcc ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=900000 -dbcache=10000 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = af653f321b135a59e38794b537737ed2f4a0040b)
Time (abs ≡): 24801.283 s [User: 30328.665 s, System: 834.110 s]
Relative speed comparison
1.01 COMPILER=gcc ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=900000 -dbcache=10000 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = e6bfd95d50)
1.00 COMPILER=gcc ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=900000 -dbcache=10000 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = af653f321b135a59e38794b537737ed2f4a0040b)
```
</details>
ACKs for top commit:
laanwj:
Concept and code review ACK b6f8c48946
TheCharlatan:
ACK b6f8c48946
andrewtoth:
ACK b6f8c48946
Tree-SHA512: a72008feca866e658f0cb4ebabbeee740f9fb13680e517b9d95eaa136e627a9dd5ee328456a2bf040401f4a1977ffa7446ad13f66b286b3419ff0c35095a3521
51093d6ae1 test: resolve symlinks in which result for capnp (David Gumberg)
Pull request description:
On Fedora, `/bin/` and `/usr/bin` are symlinked, and on one of my boxes (although I could not reproduce this behavior in a docker container), `/bin` comes before `/usr/bin` in `$PATH`, so `which capnp` reports `/bin/capnp`, and `capnp_dir` is set to `/include`, and the test fails:
```console
$ ./build/test/functional/interface_ipc.py
2025-10-30T20:43:43.753812Z TestFramework (INFO): PRNG seed is: 8370468257027235753
2025-10-30T20:43:43.754163Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_b9kjzj2a
terminate called after throwing an instance of 'kj::ExceptionImpl'
what(): mp/proxy.capnp:6: failed: Import failed: /capnp/c++.capnp
Aborted (core dumped)
```
This changes the functional test to resolve any symlinks in the `capnp` binary path reported by `which`.
ACKs for top commit:
TheCharlatan:
utACK 51093d6ae1
ryanofsky:
Code review ACK 51093d6ae1
Tree-SHA512: 17a3e16c3ef50d19e65c18bd12636f287b41e54fc14629e2eb6efb8f9532af7e0e0d404e4e234eeba92473b7ae18d97144a953d28523670308e78e4c4fbb7137
78d4d36730 test: Format strings in `*.rs` (rustaceanrob)
Pull request description:
`format!` strings may contain variables within the string representation. This is a lint as of a recent `rustc` nightly version.
ACKs for top commit:
maflcko:
lgtm ACK 78d4d36730
TheCharlatan:
ACK 78d4d36730
rkrux:
crACK 78d4d36730
Tree-SHA512: d6da94682dfa35964be4d7bba323847bae040dcec921e3d4ee2f25400751fa3af40fafe27805c2d6587d00a8ff54cc6af22ca46bf8911f13a200e73e77daa019
PR #33374 proposed a new Mining IPC method applySolution() which
could be used by clients to obtain the reconstructed block for
inspection, especially in the case of a rejected block.
However it was pointed out during review that submitBlock() modified
the template CBlock in place, so the client can just call getBlock()
and no new method is needed.
This commit adds a test to document that (now intentional) behavior.
When an IPC client requests a new block template via the Mining interface,
we hold on to its CBlock. That way when they call submitSolution() we can
modify it in place, rather than having to reconstruct the full block like
the submitblock RPC does.
Before this commit however we forgot to invalidate
m_checked_witness_commitment, which we should since the client brings a
new coinbase.
This would cause us to accept an invalid chaintip.
Fix this and add a test to confirm that we now reject such a block.
As a sanity check, we add a second node to the test and confirm that will
accept our mined block.
Note that the IPC code takes the coinbase as provided, unlike the
submitblock RPC which calls UpdateUncommittedBlockStructures() and adds
witness commitment to the coinbase if it was missing.
Although that could have been an alternative fix, we instead document that
IPC clients are expected to provide the full coinbase including witness
commitment.
facf8b771a ci: Add missing python3-dev package for riscv64 (MarcoFalke)
Pull request description:
This is required to compile the pip wheels on native riscv64.
ACKs for top commit:
fanquake:
ACK facf8b771a
Tree-SHA512: 7305deda4f2a7c2be5a82f4fcbc110f20a154374d98442e56d50175edda7f37a68b8e4cc1d84fc1fbc69ec1cc28559bbe795cc553fae8bd2e5effc36b0e534a2
fa4b52bd16 fuzz: refactor memcpy to std::ranges::copy to work around ubsan warn (MarcoFalke)
Pull request description:
Using std::ranges::copy from the C++ standard library has a few benefits here:
* It has the additional benefit of being a bit more type safe and document the byte cast explicitly.
* The compiler will likely optimize it to the same asm, but performance doesn't really matter here anyway.
* It has defined semantics for empty source ranges.
Fixes https://github.com/bitcoin/bitcoin/issues/33643
ACKs for top commit:
marcofleon:
tACK fa4b52bd16
dergoegge:
utACK fa4b52bd16
Tree-SHA512: 04fcf096e3cfc526e996c9313ec6e0a4d12c382fa19cb846b51564d33de2f0ef78a588fc6a936da0c76ca8bc9d9db4a824c36d99413db4f538a98239864d48f0
Normally, the values are small enough to fit in size_t, but the risk
that it may not fit sometimes is a reason to use uint64_t consistently
for all architectures.
On 64-bit systems, this refactor is a no-op. On 32-bit systems, it could
avoid bugs in the theoretical and unexpected case where a 32-bit size_t
is too small and overflows.
For example, 32-bit Bitcoin Core versions with CVE-2025-46597 unfixed
may overflow while checking for the bad-blk-length violation when
receiving a malformed and bloated compact block.
The values are small enough to fit in size_t, but to avoid having to
think about it, just use uint64_t consistently for all architectures.
On 64-bit systems, this refactor is a no-op. On 32-bit systems, it could
avoid bugs in the theoretical and unexpected case where a 32-bit size_t
is too small and overflows.
Bitcoin Core already assumes that 'unsigned int' means uint32_t and
'signed int' means int32_t. See src/compat/assumptions.h. Also, any
serialized integral value must be of a fixed size.
So make the fixed size explicit in this documenting refactor, which does
not change the behavior on any platform.
66667d6512 test: Use same rpc timeout for authproxy and cli (MarcoFalke)
Pull request description:
It seems odd to use different timeouts (and timeout factors) depending on whether the Python RPC proxy is used, or the bitcoin rpc command line interface.
Fix it by using the same timeout.
This can be tested by introducing a timeout error and checking it happens with and without `--usecli` after the exact same time.
Example timeout error:
```diff
diff --git a/test/functional/mining_template_verification.py b/test/functional/mining_template_verification.py
index de0833c596..e0f93a2b1e 100755
--- a/test/functional/mining_template_verification.py
+++ b/test/functional/mining_template_verification.py
@@ -173,7 +173,7 @@ class MiningTemplateVerificationTest(BitcoinTestFramework):
self.log.info("Submitting this block should succeed")
assert_equal(node.submitblock(block.serialize().hex()), None)
- node.waitforblockheight(2)
+ node.waitforblockheight(200000)
def transaction_test(self, node, block_0_height, tx):
self.log.info("make block template with a transaction")
```
Example cmd: `./bld-cmake/test/functional/mining_template_verification.py --timeout-factor=0.1 --usecli`.
ACKs for top commit:
brunoerg:
ACK 66667d6512
stickies-v:
tACK 66667d6512
Tree-SHA512: c8c21d8b9fb60ab192e3bbd45b317b96a40e10bf03704148613ac3cbdaae4abc2c03c4afbd504309ea0958201267c0d2a4bc5b40aa020917175c47e080ffe292
5fa81e239a test: add valid tx test with minimum-sized ECDSA signature (8 bytes DER-encoded) (Sebastian Falbesoner)
Pull request description:
Currently in our tests, all ECDSA signatures passing verification have sizes of 69 bytes and above (that's the DER-encoded size, i.e. counted without the sighash flag byte) [1]. This PR adds test coverage for the minimum-sized valid case of 8 bytes, by taking an interesting testnet transaction that I stumbled upon:
https://mempool.space/testnet/tx/c6c232a36395fa338da458b86ff1327395a9afc28c5d2daa4273e410089fd433
Note that this is a very obscure construction that only works because the public key used isn't contained in the locking script, but calculated and provided later at spending time (see https://bitcointalk.org/index.php?topic=1729534.msg17309060#msg17309060 for an explainer), to match the message (sighash) and picked signature. So this doesn't represent a use-case that really makes sense in practice, but it can still appear in a block (not in mempool though, due to `SCRIPT_VERIFY_CONST_SCRIPTCODE`), and having test-coverage seems useful.
Can be tested with same patch below (tests crash with the condition `>= 9`, but pass with `>= 8`).
[1] this can be verified by applying the following patch and running the tests:
```diff
diff --git a/src/pubkey.cpp b/src/pubkey.cpp
index a4ca9a170a..bee0caa603 100644
--- a/src/pubkey.cpp
+++ b/src/pubkey.cpp
@@ -288,7 +288,9 @@ bool CPubKey::Verify(const uint256 &hash, const std::vector<unsigned char>& vchS
/* libsecp256k1's ECDSA verification requires lower-S signatures, which have
* not historically been enforced in Bitcoin, so normalize them first. */
secp256k1_ecdsa_signature_normalize(secp256k1_context_static, &sig, &sig);
- return secp256k1_ecdsa_verify(secp256k1_context_static, &sig, hash.begin(), &pubkey);
+ bool ret = secp256k1_ecdsa_verify(secp256k1_context_static, &sig, hash.begin(), &pubkey);
+ if (ret) assert(vchSig.size() >= 69);
+ return ret;
}
```
ACKs for top commit:
ajtowns:
ACK 5fa81e239a lgtm
fjahr:
tACK 5fa81e239a
real-or-random:
utACK 5fa81e239a interesting case
Tree-SHA512: d1f0612fdb71c9238ca0420f574f6f246e60dbd11970b23f21d082c759a89ff98a13b12a1f6266f14f20539ec437b7ab79322082278da32984ddfee2d8893356
Fixes: 33735
Correct runner type selection for the lint job.
This was erroneously left-out during refactor of the runner selection
mechanism in #33302 causing the lint job to run on GH hosts (and
therefore not be able to acces local cirrus caches).
Using std::ranges::copy from the C++ standard library has a few benefits
here:
* It has the additional benefit of being a bit more type safe and
document the byte cast explicitly.
* The compiler will likely optimize it to the same asm, but performance
doesn't really matter here anyway.
* It works around an UB-Sanitizer bug, when the source range is empty.
Fixes https://github.com/bitcoin/bitcoin/issues/33643
fa0fa0f700 refactor: Revert "disable self-assign warning for tests" (MarcoFalke)
faed118fb3 build: Bump clang minimum supported version to 17 (MarcoFalke)
Pull request description:
Most supported operating systems ship with clang-17 (or later), so bump the minimum to that and allow new code to drop workarounds for previous clang bugs.
(Apart from dropping the small workaround, this bump allows the `ci_native_nowallet_libbitcoinkernel` CI to run on riscv64 without running into an ICE with clang-16.)
This patch will only be released in version 31.x, next year (2026).
For reference:
* https://packages.debian.org/bookworm/clang-19
* https://packages.ubuntu.com/noble/clang (clang-18)
* CentOS-like 8/9/10 ship clang-17 (and later) via Stream
* FreeBSD 12/13 ship clang-17 (and later) via packages
* OpenSuse Tumbleweed ships with https://software.opensuse.org/package/clang (clang21); No idea about OpenSuse Leap
On operating systems where the clang version is not shipped by default, the user would have to use GCC, or install clang in a different way. For example:
* https://packages.debian.org/bookworm/g++ (g++-12)
* https://packages.ubuntu.com/jammy/g++ (g++-11)
* https://apt.llvm.org/, or nix, or guix, or compile clang from source, ...
*Ubuntu 22.04 LTS does not ship with clang-16 (the previous minimum required), nor with clang-17, so one of the above workarounds is needed there.*
macOS 14 is unaffected, and the previous minimum requirement of Xcode15.0 remains, see also 919e6d01e9/depends/hosts/darwin.mk (L3-L4). (Modulo compiling the fuzz tests, which requires 919e6d01e9/.github/workflows/ci.yml (L149))
ACKs for top commit:
janb84:
Concept ACK fa0fa0f700
l0rinc:
Code review ACK fa0fa0f700
hebasto:
ACK fa0fa0f700.
Tree-SHA512: 5973cec39982f80b8b43e493cde012d9d1ab75a0362300b007d155db9f871c6341e7e209e5e63f0c3ca490136b684683de270136d62cb56f6b00b0ac0331dc36
5555bce994 ci: Document why IN_GETOPT_BIN env var is needed on macOS (MarcoFalke)
fabe516440 ci: Export the container id in python script (MarcoFalke)
fa6aa9f42f ci: Retry image building once on failure (MarcoFalke)
fa4dbe04d7 ci: Allow overwriting check option in run() helper (MarcoFalke)
fa8e4de5c3 ci: Use os.environ[key] access when value must be set (MarcoFalke)
Pull request description:
This should fix https://github.com/bitcoin/bitcoin/issues/33640.
It also contains a few refactor cleanups, which are explained in the corresponding commits.
ACKs for top commit:
l0rinc:
Code review reACK 5555bce994
kevkevinpal:
ACK [5555bce](5555bce994)
davidgumberg:
crACK 5555bce994
Tree-SHA512: f1ea95b0650e57d6a9f97c575a11ee461832c0715c3d1a24dbfe12ccc5366f295639d4c4827f1d01da460ddf00917ecaa627e7dbd12e405770db6c53c3778a9c
53b34c80c6 ci: use pycapnp 2.2.1 in mac native job (fanquake)
865432869c ci: remove Python version comment from mac config (fanquake)
Pull request description:
Switch to using v2.2.1 in the mac native job. Remove the git clone & install step.
ACKs for top commit:
maflcko:
lgtm ACK 53b34c80c6
l0rinc:
crACK 53b34c80c6
hebasto:
ACK 53b34c80c6.
Tree-SHA512: e756694c14431aacb3e48104331da88285c7500b4c4599c698f50d721d428ffe61258be075ef526b93c15aa3331f38535ca95249a2ef3ebfc804f61479095d9b
53e4951a5b Switch to ANSI Windows API in `fsbridge::fopen()` function (Hennadii Stepanov)
dbe770d921 Switch to ANSI Windows API in `Win32ErrorString()` function (Hennadii Stepanov)
06d0be4e22 Remove no longer necessary `WinCmdLineArgs` class (Hennadii Stepanov)
f366408492 cmake: Set process code page to UTF-8 on Windows (Hennadii Stepanov)
dccbb17806 Set minimum supported Windows version to 1903 (May 2019 Update) (Hennadii Stepanov)
Pull request description:
The main goal is to remove [deprecated](https://github.com/bitcoin/bitcoin/issues/32361) code (removed in C++26).
This PR employs Microsoft's modern [approach](https://learn.microsoft.com/en-us/windows/apps/design/globalizing/use-utf8-code-page) to handling UTF-8:
> Until recently, Windows has emphasized "Unicode" -W variants over -A APIs. However, recent releases have used the ANSI code page and -A APIs as a means to introduce UTF-8 support to apps. If the ANSI code page is configured for UTF-8, then -A APIs typically operate in UTF-8. This model has the benefit of supporting existing code built with -A APIs without any code changes.
TODO:
- [x] Handle application manifests properly when building with MSVC.
- [x] Bump the minimum supported Windows version to 1903 (May 2019 Update).
- [x] Remove all remaining use cases of the deprecated `std:wstring_convert`.
- The instance in `subprocess.h` will be addressed in a follow-up PR, as additional tests are likely needed.
- The usage in `common/system.cpp` is handled in https://github.com/bitcoin/bitcoin/pull/32566.
Resolves partially https://github.com/bitcoin/bitcoin/issues/32361.
ACKs for top commit:
laanwj:
re-ACK 53e4951a5b
hodlinator:
re-ACK 53e4951a5b
davidgumberg:
untested crACK 53e4951a5b
Tree-SHA512: 0dbe9badca8b979ac2b4814fea6e4a7e53c423a1c96cb76ce894253137d3640a87631a5b22b9645e8f0c2a36a107122eb19ed8e92978c17384ffa8b9ab9993b5
57f7c68821 test: add functional test for `TestShell` (matching doc example) (Sebastian Falbesoner)
53874f7934 doc: test: update TestShell example instructions/options (Sebastian Falbesoner)
Pull request description:
This PR adds a functional framework test for the `TestShell` class. The primary motivation for this is to avoid that the example instructions for the interactive Python shell in `test-shell.md` get outdated or broken without noticing, a problem we had already several times in the past (see #26520, #27906, #31415). Having a copy is still not perfect, as docs and functional test have to be kept in sync, but I don't expect this to be a problem in practice, assuming the hint in the functional test comment is hopefully noticed if changes are made.
Alternatively, the example instructions in `test-shell.md` could be removed with a hint to the functional test (I tend to prefer to keep the docs as-is though, showing the full REPL interaction).
The first commit contain two small removal fix-ups in `test-shell.md` regarding the result of the `createwallet` RPC call and the mentioning of the `noshutdown` option that was removed earlier (see #31620). Could be backported to v30.
ACKs for top commit:
brunoerg:
ACK 57f7c68821
stratospher:
ACK 57f7c68.
pinheadmz:
ACK 57f7c68821
Tree-SHA512: 25c35af46b742bbefce7838708437529bbf613fa3d1f0fba590cacef0acdde82b3a78c7a01459c73eaac26ce3f1041e54092d098f0fc94a8c76ac0b4f4970338
1a7fb5eeee fees: return current block height in estimateSmartFee (ismaelsadeeq)
ab49480d9b fees: rename fees_args to block_policy_estimator_args (ismaelsadeeq)
06db08a435 fees: refactor: rename fees to block_policy_estimator (ismaelsadeeq)
6dfdd7e034 fees: refactor: rename policy_fee_tests.cpp to feerounder_tests.cpp (ismaelsadeeq)
Pull request description:
This PR is a simple refactoring that does four things:
1. Renames `test/policy_fee_tests.cpp` to `test/feerounder_tests.cpp`.
2. Renames `policy/fees.{h,cpp}` to `policy/fees/block_policy_estimator.{h,cpp}`.
3. Renames `policy/fees_args.cpp` to `policy/fees/block_policy_estimator_args.cpp`.
4. Modifies `estimateSmartFee` to return the block height at which the estimate was made by adding a `best_height` unsigned int value to the `FeeCalculation` struct.
**Motivation**
In preparation for adding a new fee estimator, the `fees` directory is created so we can organize code into `block_policy_estimator` and `mempool` because
a) It would be clunky to add more code directly under `fees`.
b) Having `policy/fees.{h,cpp}` and `policy/mempool.{h,cpp}` would also be undesirable.
Therefore, it makes sense to structure the it as `policy/fees/block_policy_estimator`, `policy/fees/mempool`, etc.
Hence test file were also updated accordingly.
The current block height is also returned because later in #30157 we log the height at which each estimate is made (at the debug log category of fee estimation :) ). This feature is particularly useful for empirical data analysis.
ACKs for top commit:
maflcko:
re-ACK 1a7fb5eeee🐤
polespinasa:
re ACK 1a7fb5eeee
willcl-ark:
ACK 1a7fb5eeee
janb84:
re ACK 1a7fb5eeee
Tree-SHA512: fef7ace2a9f262ec0361fb7a46df5108afc46b5c4b059caadf2fd114740aefbb2592389d11646c13d0e28bf0ef2cfcfbab3e659c4d4288b8ebe64725fd1963c0
944e5ff848 doc: mention key removal in rpc interface modification (rkrux)
Pull request description:
A discussion in a previous PR 32618 prompted me to add this note: https://github.com/bitcoin/bitcoin/pull/32618#discussion_r2181951390
<!--
*** Please remove the following help text before submitting: ***
Pull requests without a rationale and clear improvement may be closed
immediately.
GUI-related pull requests should be opened against
https://github.com/bitcoin-core/gui
first. See CONTRIBUTING.md
-->
<!--
Please provide clear motivation for your patch and explain how it improves
Bitcoin Core user experience or Bitcoin Core developer experience
significantly:
* Any test improvements or new tests that improve coverage are always welcome.
* All other changes should have accompanying unit tests (see `src/test/`) or
functional tests (see `test/`). Contributors should note which tests cover
modified code. If no tests exist for a region of modified code, new tests
should accompany the change.
* Bug fixes are most welcome when they come with steps to reproduce or an
explanation of the potential issue as well as reasoning for the way the bug
was fixed.
* Features are welcome, but might be rejected due to design or scope issues.
If a feature is based on a lot of dependencies, contributors should first
consider building the system outside of Bitcoin Core, if possible.
* Refactoring changes are only accepted if they are required for a feature or
bug fix or otherwise improve developer experience significantly. For example,
most "code style" refactoring changes require a thorough explanation why they
are useful, what downsides they have and why they *significantly* improve
developer experience or avoid serious programming bugs. Note that code style
is often a subjective matter. Unless they are explicitly mentioned to be
preferred in the [developer notes](/doc/developer-notes.md), stylistic code
changes are usually rejected.
-->
<!--
Bitcoin Core has a thorough review process and even the most trivial change
needs to pass a lot of eyes and requires non-zero or even substantial time
effort to review. There is a huge lack of active reviewers on the project, so
patches often sit for a long time.
-->
ACKs for top commit:
maflcko:
lgtm ACK 944e5ff848
stickies-v:
ACK 944e5ff848
glozow:
lgtm ACK 944e5ff848
Tree-SHA512: f64c086c99e7c73a3ae7d60b2e8e06c8e7a3a49305a66d5c5a96db9b4ebbd01928ab5ccbcbdac26f400d16662f84469c448625e1f55ec2a9a920eff8a05fc379
This change updates to IWYU 0.25, which is compatible with Clang 21.
Fixes new "modernize-use-default-member-init" warnings.
The warning in `interpreter.cpp` is a false positive, so it has been
suppressed.
The values are small enough to fit in an int, so the cast is at best
redundant. However, UniValue can handle any integer type, so having to
think about the cast here is also confusing.
The serialize related methods were removed in commit
30007fda76.
If someone wants to see the tested methods, they can just read the test
itself, instead of relying on the wrong comment.
02d2b5a11c ci, iwyu: Treat warnings as errors for specific directories (Hennadii Stepanov)
57a3eac387 refactor: Fix includes in `index` directory (Hennadii Stepanov)
bdb8eadcdc refactor: Fix includes in `crypto` directory (Hennadii Stepanov)
56f2a689a2 ci: Do not patch `leveldb` to workaround UB in "tidy" CI job (Hennadii Stepanov)
Pull request description:
This PR is the first step towards treating IWYU warnings as errors. At this stage, it applies only to the `crypto` and `index` directories.
ACKs for top commit:
maflcko:
re-ACK 02d2b5a11c💮
ryanofsky:
Code review ACK 02d2b5a11c. Just rebased and update tidy patch comment again since last review
willcl-ark:
ACK 02d2b5a11c
Tree-SHA512: 1c966e01c47bf3e7d225faa3b819367f757430e2d71e9582fa82d67307aabe3f0d76f69346ee180192e7f5ab194ecc58d2b8ecf178eab26ba3309a6b55bff4b6
59c4898994 guix: remove python-pydantic-core input from LIEF (fanquake)
9f2a6927d3 guix: use Clang & LLVM 19 for macOS build (fanquake)
9570ddbec9 guix: update time-machine to 5cb84f2013c5b1e48a7d0e617032266f1e6059e2 (fanquake)
7b5cc276aa guix: patch around riscv issue with newer (2.40+) binutils (fanquake)
91b5cbaabb ci: use Debian Trixie for macOS cross job (fanquake)
Pull request description:
5cb84f2013 isn't super recent, but it's enough to get access to some newer packages, such as LLVM 19, and avoids having to add any further work arounds for things that we know are fixed later (i.e nsis). Once things upstream have stabilized a bit more (the `core-updates` branch was fairly recently merged), we could look at bumping to something newer.
Package updates:
(base) glibc 2.35 -> 2.39
binutils 2.38 -> 2.41
diffutils 3.8 -> 3.10
gawk 5.2.1 -> 5.3.0
git-minimal 2.45.2 -> 2.46.0
grep 3.8 -> 3.11
gzip 1.12 -> 1.13
linux-headers 6.1.106 -> 6.1.119
make 4.3 -> 4.4.1
xz 5.2.8 -> 5.4.5
CMake 3.30 becomes available.
Clang/LLVM 19 becomes available.
Could be used for #32764.
ACKs for top commit:
hebasto:
re-ACK 59c4898994.
willcl-ark:
ACK 59c4898994
Tree-SHA512: c44965d5a315e4c862f5e40d8e98c645713405fec72a61055f95b6c68b7d2dcc69a61a084e397a4556d4c1df18f1cfa7a905234643fe4a7df9c58d486e26c097
664657ed13 bugfix: disallow label for ranged descriptors & allow external non-ranged descriptors to have label (scgbckbone)
Pull request description:
Motivation:
* ranged descriptors MUST not be able to have label (current impl allows it)
* external non-ranged descriptor MUST be able to have label (current impl disallows it, **if** `internal=false` is provided via importdescriptor user data)
Repro steps:
* create blank wallet and import descriptors
* external has `label=test` (not internal)
```
conn = bitcoind.create_wallet(wallet_name=w_name, disable_private_keys=True, blank=True,
passphrase=None, avoid_reuse=False, descriptors=True)
descriptors = [
{
"timestamp": "now",
"label": "test",
"active": True,
"desc": "wpkh([0f056943/84h/1h/0h]tpubDC7jGaaSE66Pn4dgtbAAstde4bCyhSUs4r3P8WhMVvPByvcRrzrwqSvpF9Ghx83Z1LfVugGRrSBko5UEKELCz9HoMv5qKmGq3fqnnbS5E9r/0/*)#erexmnep",
"internal": False
},
{
"desc": "wpkh([0f056943/84h/1h/0h]tpubDC7jGaaSE66Pn4dgtbAAstde4bCyhSUs4r3P8WhMVvPByvcRrzrwqSvpF9Ghx83Z1LfVugGRrSBko5UEKELCz9HoMv5qKmGq3fqnnbS5E9r/1/*)#ghu8xxfe",
"active": True,
"internal": True,
"timestamp": "now"
},
]
r = conn.importdescriptors(descriptors)
print(r)
```
response:
```
[{'error': {'code': -8,
'message': 'Internal addresses should not have a label'},
'success': False,
'warnings': ['Range not given, using default keypool range']},
{'success': True,
'warnings': ['Range not given, using default keypool range']}]
```
But in above, ONLY external has a label.
If you remove `internal: False` from external descriptor import object - it will import no problem:
```
[{'success': True,
'warnings': ['Range not given, using default keypool range']},
{'success': True,
'warnings': ['Range not given, using default keypool range']}]
```
Even tho it should NOT, as the descriptor is ranged. Current implementation relies on checking user provided data to decide whether desc is ranged.
ACKs for top commit:
achow101:
ACK 664657ed13
rkrux:
lgtm crACK 664657ed13
Tree-SHA512: 9e70aea620019c29950ba417d4ae38d65cd94a4f6fcabbc021d67b031de1c44c27d6f6f5cb7e6950a099eb6e58bed9be764d4c6347195daeccb14a5d95c123b2
0465574c12 test: Fixes send_blocks_and_test docs (Sergi Delgado Segura)
09c95f21e7 test: Adds block tiebreak over restarts tests (Sergi Delgado Segura)
18524b072e Make nSequenceId init value constants (Sergi Delgado Segura)
8b91883a23 Set the same best tip on restart if two candidates have the same work (Sergi Delgado Segura)
5370bed21e test: add functional test for complex reorgs (Pieter Wuille)
ab145cb3b4 Updates CBlockIndexWorkComparator outdated comment (Sergi Delgado Segura)
Pull request description:
This PR grabs some interesting bits from https://github.com/bitcoin/bitcoin/pull/29284 and fixes some edge cases in how block tiebreaks are dealt with.
## Regarding #29284
The main functionality from the PR was dropped given it was not an issue anymore, however, reviewers pointed out some comments were outdated https://github.com/bitcoin/bitcoin/pull/29284#discussion_r1522023578 (which to my understanding may have led to thinking that there was still an issue) it also added test coverage for the aforementioned case which was already passing on master and is useful to keep.
## New functionality
While reviewing the superseded PR, it was noticed that blocks that are loaded from disk may face a similar issue (check https://github.com/bitcoin/bitcoin/pull/29284#issuecomment-1994317785 for more context).
The issue comes from how tiebreaks for equal work blocks are handled: if two blocks have the same amount of work, the one that is activatable first wins, that is, the one for which we have all its data (and all of its ancestors'). The variable that keeps track of this, within `CBlockIndex` is `nSequenceId`, which is not persisted over restarts. This means that when a node is restarted, all blocks loaded from disk are defaulted the same `nSequenceId`: 0.
Now, when trying to decide what chain is best on loading blocks from disk, the previous tiebreaker rule is not decisive anymore, so the `CBlockIndexWorkComparator` has to default to its last rule: whatever block is loaded first (has a smaller memory address).
This means that if multiple same work tip candidates were available before restarting the node, it could be the case that the selected chain tip after restarting does not match the one before.
Therefore, the way `nSequenceId` is initialized is changed to:
- 0 for blocks that belong to the previously known best chain
- 1 to all other blocks loaded from disk
ACKs for top commit:
sipa:
utACK 0465574c12
TheCharlatan:
ACK 0465574c12
furszy:
Tested ACK 0465574c12.
Tree-SHA512: 161da814da03ce10c34d27d79a315460a9c98d019b85ee35bc5daa991ed3b6a2e69a829e421fc70d093a83cf7a2e403763041e594df39ed1991445e54c16532a
51877f2fc5 test: Update BIP324 test vectors (Tim Ruffing)
Pull request description:
This updates the hardcoded test vectors from BIP324. The test vectors had to be regenerated (in the aux files of the BIP) because there was a bug in the script used for generating them (https://github.com/bitcoin/bips/pull/2016).
ACKs for top commit:
jonatack:
ACK 51877f2fc5
theStack:
ACK 51877f2fc5
Tree-SHA512: 59f4075e286067b11fce98667c860f3083b6cca8a2e49da8783ccdce8e32c34fd3e1943191d24dcf5bb68d8a2540726d99f7c29e8b0f104032ccb82423ca8d82
- Also move them to policy/fees/ and update includes
- Note: the block_policy_estimator_args.h include in block_policy_estimator_args.cpp was done manually.
5ded99a7f0 fuzz: MockMempoolMinFee in wallet_fees (brunoerg)
c9a7a198d9 test: move MockMempoolMinFee to util/txmempool (brunoerg)
adf67eb21b fuzz: create FeeEstimatorTestingSetup to set fee_estimator (brunoerg)
ff10a37e99 fuzz: mock CBlockPolicyEstimator in wallet_fuzz (brunoerg)
f591c3beca fees: make estimateSmartFee/HighestTargetTracked virtual for mocking (brunoerg)
19273d0705 fuzz: set mempool options in wallet_fees (brunoerg)
Pull request description:
Some functions in `wallet/fees.cpp` (fuzzed by the wallet_fees target) depends on some mempool stuff - e.g. relay current min fee, smart fee and max blocks estimation, relay dust fee and other ones. For better fuzzing of it, it would be great to have these values/interactions. That said, this PR enhances the `wallet_fees` target by:
- Setting mempool options - `min_relay_feerate`, `dust_relay_feerate` and `incremental_relay_feerate` - when creating the `CTxMemPool`.
- Creates a `ConsumeMempoolMinFee` function which is used to have a mempool min fee (similar approach from `MockMempoolMinFee` from unit test).
- Mock `CBlockPolicyEstimator` - estimateSmartFee/HighestTagretTracket functions, especifically. It's better to mock it then trying to interact to CBlockPolicyEstimator in order to have some effective values due to performance.
Note that I created `FeeEstimatorTestingSetup` because we cannot set `m_node.fee_estimator` in `ChainTestingSetup` since fae8c73d9e.
ACKs for top commit:
maflcko:
re-ACK 5ded99a7f0🎯
ismaelsadeeq:
Code review ACK 5ded99a7f0
Tree-SHA512: 13d2af042098afd237ef349437021ea841069d93d4c3e3a32e1b562c027d00c727f375426709d34421092993398caf7ba8ff19077982cb6f470f8938a44e7754
45bd891465 log: split assumevalid ancestry-failure-reason message (Lőrinc)
6c13a38ab5 log: separate script verification reasons (Lőrinc)
f2ea6f04e7 refactor: untangle assumevalid decision branches (Lőrinc)
9bc298556c validation: log initial script verification state (Lőrinc)
4fad4e992c test: add assumevalid scenarios scaffold (Lőrinc)
91ac64b0a6 log: reword `signature validations` to `script verification` in `assumevalid` log (Lőrinc)
Pull request description:
### Summary
Users can encounter cases where script checks are unexpectedly enabled (e.g. after reindex, or when `assumevalid`/`minimumchainwork` gates fail). Without an explicit line, they must infer state from the absence of a message, which is incomplete and error-prone.
The existing "Assuming ancestors of block …" line does not reliably indicate whether script checks are actually enabled, which makes debugging/benchmarking confusing.
### What this changes
We make the initial **script-verification** state explicit and log **why** checks are enabled to avoid confusion.
* Always log the first script-verification state on startup, **before** the first `UpdateTip`.
* Flatten the nested `assumevalid` conditionals into a linear gating sequence for readability.
* Extend the functional test to assert the old behavior with the new reason strings.
This is a **logging-only** test change it shouldn't change any other behavior.
### Example output
The state (with reason) is logged at startup and whenever the reason changes, e.g.:
* `Disabling script verification at block #904336 (000000000000000000014106b2082b1a18aaf3091e8b337c6fed110db8c56620).`
* `Enabling script verification at block #912527 (000000000000000000010bb6aa3ecabd7d41738463b6c6621776c2e40dbe738a): block too recent relative to best header.`
* `Enabling script verification at block #912684 (00000000000000000001375cf7b90b2b86e559d05ed92ca764d376702ead3858): block height above assumevalid height.`
------
Follow-up to https://github.com/bitcoin/bitcoin/pull/32975#discussion_r2329269037
ACKs for top commit:
Eunovo:
re-ACK 45bd891465
achow101:
ACK 45bd891465
hodlinator:
re-ACK 45bd891465
yuvicc:
ACK 45bd891465
andrewtoth:
ACK 45bd891465
ajtowns:
ACK 45bd891465
Tree-SHA512: 58328d7c418a6fe18f1c7fe1dd31955bb6fce8b928b0df693f6200807932eb5933146300af886a80a1d922228d93faf531145186dae55ad4ad1f691970732eca
b63428ac9c rpc: refactor: use more (Maybe)Arg<std::string_view> (stickies-v)
037830ca0d refactor: increase string_view usage (stickies-v)
b3bf18f0ba rpc: refactor: use string_view in Arg/MaybeArg (stickies-v)
Pull request description:
The `RPCHelpMan::{Arg,MaybeArg}` helpers avoid copying (potentially) large strings by returning them as `const std::string*` (`MaybeArg`) or `const std::string&` (`Arg`). For `MaybeArg`, this has the not-so-nice effect that users need to deal with raw pointers, potentially also requiring new functions (e.g. [`EnsureUniqueWalletName` ](d127b25199 (diff-d8bfcfbdd5fa7d5c52d38c1fe5eeac9ce5c5a794cdfaf683585140fa70a32374R32))) with raw pointers being implemented.
This PR aims to improve on this by returning a trivially copyable `std::string_view` (`Arg`) or `std::optional<std::string_view>` (`MaybeArg`), modernizing the interface without introducing any additional copying overhead. In doing so, it also generalizes whether we return by value or by pointer/reference using `std::is_trivially_copyable_v` instead of defining the types manually.
In cases where functions currently take a `const std::string&` and it would be too much work / touching consensus logic to update them (`signmessage.cpp`), a `std::string` copy is made (which was already happening anyway).
The last 2 commits increase usage of the `{Arg,MaybeArg}<std::string_view>` helpers, and could be dropped/pruned if anything turns out to be controversial - I just think it's a nice little cleanup.
ACKs for top commit:
maflcko:
re-ACK b63428ac9c🎉
achow101:
ACK b63428ac9c
pablomartin4btc:
re-ACK [b63428a](b63428ac9c)
w0xlt:
reACK b63428ac9c
Tree-SHA512: b4942c353a1658c22a88d8c9b402c288ad35265a3b88aa2072b1f9b6d921cd073194ed4b00b807cb48ca440f47c87ef3d8e0dd1a5d814be58fc7743f26288277
65a10fc3c5 p2p: add assertion for BlockTransactionsRequest indexes (frankomosh)
58be359f6b fuzz: add a target for DifferenceFormatter Class (frankomosh)
Pull request description:
Adds a fuzz test for the [`DifferenceFormatter`](e3f416dbf7/src/blockencodings.h (L22-L42)) (used in [`BlockTransactionsRequest`](https://github.com/bitcoin/bitcoin/blob/master/src/blockencodings.h#L44-L54), [BIP 152](https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki)). The DifferenceFormatter class implements differential encoding for compact block transactions (BIP 152). This PR ensures that its strictly-monotonic property is maintained. It complements the tests in [`blocktransactionsrequest_deserialize`](9703b7e6d5/src/test/fuzz/deserialize.cpp (L314)).
Additionally, there's an added invariant check after GETBLOCKTXN deserialization in `net_processing.cpp`.
ACKs for top commit:
Crypt-iQ:
tACK 65a10fc3c5
achow101:
ACK 65a10fc3c5
dergoegge:
Code review ACK 65a10fc3c5
Tree-SHA512: 70659cf045e99bb5f753763c7ddac094cb2883c202c899276cbe616889afa053b2d5e831f99d6386d4d1e4118cd35fa0b14b54667853fe067f6efe2eb77b4097
This refactor does not change behavior, but it has a few benefits:
* The shellcheck SC2086 warning is disabled for the whole command, but
is only needed for the CI_CONTAINER_CAP env var. So in Python, only
pass this one env var to shlex.split() for proper word splitting.
* Future logic improvements can be implemented in Python.
The comments are moved, which can be checked via the git options:
--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
fa70e23de7 ci: Drop libFuzzer from msan fuzz task (MarcoFalke)
Pull request description:
libFuzzer is mostly unmaintained (https://llvm.org/docs/LibFuzzer.html#status), and it isn't really needed by the CI tasks. While it provides some additional stats like rss or the max input byte size, they are not essential. Dropping libFuzzer here would also drop the "60 seconds sanity check" for empty folders, but I think this is an acceptable price to pay to silence false-positives that were hit for years.
Also, there seems to be a history of intermittent false-positive msan warnings (https://github.com/bitcoin/bitcoin/pull/33600#issuecomment-3391921802).
It is unclear what exactly is causing the false-positives, so just disable libFuzzer in this task for now, to work around them.
ACKs for top commit:
kevkevinpal:
ACK [fa70e23](fa70e23de7)
dergoegge:
ACK fa70e23de7
Tree-SHA512: c3e5958b8378ba30f51d923f97a84dec2ee60af8b9c2a4f13bc8de486a490031468371120e421384aa198ffec591db554e636935ab3c6d4de5e870238f5079f2
fa37153288 util: Abort on failing CHECK_NONFATAL in debug builds (MarcoFalke)
fa0dc4bdff test: Allow testing of check failures (MarcoFalke)
faeb58fe66 refactor: Set G_ABORT_ON_FAILED_ASSUME when G_FUZZING_BUILD (MarcoFalke)
Pull request description:
A failing `CHECK_NONFATAL` will throw an exception. This is fine and even desired in production builds, because the program may catch the exception and give the user a way to easily report the bug upstream.
However, in debug development builds, exceptions for internal bugs are problematic:
* The exception could accidentally be caught and silently ignored
* The exception does not include a full stacktrace, possibly making debugging harder
Fix all issues by turning the exception into an abort in debug builds.
This can be tested by reverting the hunks to `src/rpc/node.cpp` and `test/functional/rpc_misc.py` and then running the functional or fuzz tests.
ACKs for top commit:
achow101:
ACK fa37153288
ryanofsky:
Code review ACK fa37153288, just catching subprocess.CalledProcessError in test fixing up a comment since last review
stickies-v:
ACK fa37153288
Tree-SHA512: 2d892b838ccef6f9b25a066e7c2f6cd6f5acc94aad1d91fce62308983bd3f5c5d724897a76de4e3cc5c3678ddadc87e2ee8c87362965373526038e598dfb0101
cc5dda1de3 headerssync: Make HeadersSyncState more flexible and move constants (Hodlinator)
8fd1c2893e test(headerssync): Test returning of pow_validated_headers behavior (Hodlinator)
7b00643ef5 test(headerssync): headers_sync_chainwork test improvements (Hodlinator)
04eeb9578c doc(test): Improve comments (Hodlinator)
fe896f8faa refactor(test): Store HeadersSyncState on the stack (Hodlinator)
f03686892a refactor(test): Break up headers_sync_state (Hodlinator)
e984618d0b refactor(headerssync): Process spans of headers (Hodlinator)
a4ac9915a9 refactor(headerssync): Extract test constants ahead of breakup into functions (Hodlinator)
Pull request description:
### Background
As part of the release process we often run *contrib/devtools/headerssync-params.py* and increase the values of the constants `HEADER_COMMITMENT_PERIOD` and `REDOWNLOAD_BUFFER_SIZE` in *src/headerssync.cpp* as per *doc/release-process.md* (example: 11a2d3a63e). This helps fine tune the memory consumption per `HeadersSyncState`-instance in the face of malicious peers.
(The `REDOWNLOAD_BUFFER_SIZE`/`HEADER_COMMITMENT_PERIOD` ratio determines how many Headers Sync commitment bits must match between PRESYNC & REDOWNLOAD phases before we start permanently storing headers from a peer. For more details see comments in *src/headerssync.h* and *contrib/devtools/headerssync-params.py*).
### Problem: Not feeding back headers until completing sync
During v30 release process #33274 made `REDOWNLOAD_BUFFER_SIZE` exceed the `target_blocks` constant used to control the length of chains generated for testing Headers Sync (`15000`, *headers_sync_chainwork_tests.cpp*).
The `HeadersSyncState::m_redownloaded_headers`-buffer now does not reach the `REDOWNLOAD_BUFFER_SIZE`-threshold during those unit tests. As a consequence `HeadersSyncState::PopHeadersReadyForAcceptance()` will not start feeding back headers until the PoW threshold has been met. While this will not cause the unit test to start failing on master, it means we have gone from testing behavior that resembles mainnet (way more than `REDOWNLOAD_BUFFER_SIZE` headers to reach the PoW limit), to behavior that is not possible/expected there.
### Solution
Avoid testing this unrealistic condition of completing Headers Sync before reaching `REDOWNLOAD_BUFFER_SIZE` by making tests able to define their own values through the new `HeadersSyncParams` instead of having them hard-coded for all chains & tests.
### Commits
* First 6 commits refactor and improve the unit tests in order to clarify latter changes.
* We then add checks for the behavior around the `REDOWNLOAD_BUFFER_SIZE` threshold.
* The main change: we extract the section from *headerssync.cpp* containing the constants to *kernel/chainparams.cpp*, making `HeadersSyncState` no longer hard-coded to mainnet.
### Notes
This PR used to be called "headerssync: Preempt unrealistic unit test behavior".
ACKs for top commit:
l0rinc:
reACK cc5dda1de3
marcofleon:
code review ACK cc5dda1de3
danielabrozzoni:
reACK cc5dda1de3
Tree-SHA512: ccc824dcbbb8ad5ae98c3bf5808b38467aac0230739898a758c9b939eecd74f982df088fa0ba81cc1c1732f19a607b135a6e9577bb9fcf7f8570567ce92f66e6
d0e1bbad01 test: repeat block malleability test with relayable block over P2P (Musa Haruna)
Pull request description:
This PR adds a functional test to repeat the existing malleability check for oversized coinbase witness nonce size using a block that is small enough to be relayed over the P2P network.
This addresses the TODO in test_block_malleability by ensuring behavior is consistent between submitblock RPC and P2P relay.
ACKs for top commit:
maflcko:
lgtm ACK d0e1bbad01
janb84:
re ACK d0e1bbad01
glozow:
utACK d0e1bbad01
Tree-SHA512: 05aec4fade5af8043f40274a8d2f3cf3f540acd038138975bdefbbbc81e105792d6d2588256a2ee5ddb1e05b37fe2d0b3d287160d2dbe86e1aac7cfa9cc02116
faa9d10c84 refactor: Construct g_verify_flag_names on first use (MarcoFalke)
Pull request description:
The current usage of the `g_verify_flag_names` map seems fine and I can not see a static initialization order fiasco here.
However, it seems brittle to hope this remains the case in the future. Also, it triggers a msan false-positive in the fuzz CI task. (C.f https://github.com/bitcoin-core/qa-assets/actions/runs/18352815555/job/52413137315?pr=241#step:7:5245)
So just apply the "Construct on first use" idiom.
ACKs for top commit:
kevkevinpal:
ACK [faa9d10](faa9d10c84)
ajtowns:
ACK faa9d10c84
janb84:
lgtm ACK faa9d10c84
stickies-v:
ACK faa9d10c84
Tree-SHA512: 6685dfc91c99a8245722e07fac99a7a6d58586c30964be7ccd74a176dfbf00c6255c8594621e2909640763924f51d3efd4ce65ed65eaeeb1d05c2fd01fe63604
8f7673257a miner: fix empty mempool case for waitNext() (Sjors Provoost)
Pull request description:
Block template fees are calculated by looping over `new_tmpl->vTxFees` and return (early) once the `fee_threshold` is exceeded.
This left an edge case when the mempool is empty, which this commit fixes and adds a test for.
Also update `test/functional/interface_ipc.py` to reflect the new behavior,
Fixes https://github.com/Sjors/sv2-tp/issues/9
ACKs for top commit:
optout21:
ACK 8f7673257a
cedwies:
tACK 8f76732
sipa:
utACK 8f7673257a
zaidmstrr:
Concept ACK [8f76732](8f7673257a)
Tree-SHA512: ef200fe95e96f810e425283bc37f945c4bf5efa16f4b74820b8a07968f30c5146bca213a372124be84b48beead5dfd35f2b5d10d188d0a465f847ebab61de10a
e9cd45e3d3 test: set number of RPC server threads to 2 (furszy)
Pull request description:
The default `-rpcthreads` value spawns 16 HTTP server threads for each node.
Running the functional test suite with default `rpcthreads` can exhaust file
descriptors or hit other resource limits very easily (more when tests are run
in parallel).
Furthermore, having 16 threads is unnecessary since they are mostly idle. We
run RPC calls on a single RPC connection and wait for it result synchronously.
There is (almost) never two RPC calls occurring concurrently.
Because of this, the threads are mostly idle, so we can safely limit the number
of them to two.
Note for reviewers:
I checked this does not introduce any timing regression but would be good
to double-check it on your end too. We could add another thread if needed.
Just the 16 threads default value is too high and unnecessary.
ACKs for top commit:
maflcko:
lgtm ACK e9cd45e3d3
l0rinc:
ACK e9cd45e3d3
kevkevinpal:
ACK [e9cd45e](e9cd45e3d3)
andrewtoth:
ACK e9cd45e3d3
Tree-SHA512: a777286f4a890fb87f5df72cd2ccfdc628657206a4b3e995044e5a0d12987b8c78a7cf7d684cc4e92605aa782aaeebc44e9f754752c3a524152fac94fa30f4b5
The default `-rpcthreads` value spawns 16 HTTP server threads for each node.
Running the functional test suite with default `rpcthreads` can exhaust file
descriptors or hit other resource limits very easily.
Moreover, having 16 threads is unnecessary since they are mostly idle. We
run RPC calls on a single RPC connection and wait for it result synchronously.
There is (almost) never two RPC calls occurring concurrently.
Because of this, the threads are mostly idle, so we can safely limit the number
of them to two.
The build scripts inside the image retry after a failure. However, there
may be some rare network failures inside the container engine. For
example, when pulling the underlying base image, or when pulling the
docker cache.
Thus, retry after a failure once, which should hopefully fix
https://github.com/bitcoin/bitcoin/issues/33640.
The other code in this file is using this pattern to throw when a key is
unset, instead of silently returning a None when using os.getenv(key)
with no default value specified.
So use the pattern here as well. As the env vars are always set, this
should be a refactor that does not change the behavior.
fabe0e07de ci: Only write docker build images to Cirrus cache (MarcoFalke)
fab64a5d6f ci: Move buildx command to python script (MarcoFalke)
fa72a2bd5c ci: Remove unused MAYBE_CPUSET (MarcoFalke)
Pull request description:
The `DOCKER_BUILD_CACHE_ARG` env var holds the options on how to use cache providers. Storing the image layers is useful for the Cirrus cache provider, because it offers 10GB per runner (https://cirrus-runners.app/setup/#speeding-up-the-cache). The cached image layers can help to avoid issues when the upstream package manager infra (apt native, apt llvm, pip, apk, git clone, ...) has outages or network issues.
However, on the GitHub Actions cache provider, a *total* cache of 10GB is offered for the whole repo. This cache must be shared with the depends cache, and the ccache, as well as the previous releases cache. So it is already full and trying to put the docker build layers into it will lead to an overflow.
Fix it by only writing to the docker cache on Cirrus.
Also, `DOCKER_BUILD_CACHE_ARG` requires a `shellcheck disable=SC2086` on the full build command. Fix that as well by using `shlex.split` from Python on just this variable.
ACKs for top commit:
m3dwards:
ACK fabe0e07de
cedwies:
reACK fabe0e0
l0rinc:
Code review ACK fabe0e07de
willcl-ark:
ACK fabe0e07de
Tree-SHA512: 4f471f080007fdd0c3bc97b0cfe0e9c0457e5029a7ccde1d784d30eb4752e5eb309cd4b122b182bce31f1b986c8a9f3e9a49da1768bedbb2b1f64f70183680ba
9610b0d1e2 randomenv: Fix MinGW dllimport warning for `environ` (Lőrinc)
Pull request description:
Related to https://github.com/bitcoin/bitcoin/pull/33550#issuecomment-3378978210
Extends 7703884 to guard environ declaration on all Windows builds, not just MSVC.
In the `mingw-w64` headers (used by `llvm-mingw`), `environ` is defined as a macro which expands through [`_environ`](cdb052f1d4/mingw-w64-headers/crt/stdlib.h (L262-L264)) to `(* __p__environ())`, a call to a `dllimport` function, causing the same inconsistent linkage warning as MSVC.
Use `WIN32` instead of `_MSC_VER` to match the platform-specific guards already used throughout the file.
The warning occurs with `llvm-mingw` (both `UCRT` and `MSVCRT` variants as tested by Hebasto), but not with the `mingw-w64` toolchain currently used in CI (as mentioned by fanquake).
----
The error was reproduced by adding a temporary [nightly build](https://github.com/l0rinc/bitcoin-core-nightly/pull/4) pointing to https://github.com/l0rinc/bitcoin/pull/45. On `master` the failure can be seen in https://github.com/l0rinc/bitcoin-core-nightly/pull/2
before:
https://github.com/l0rinc/bitcoin-core-nightly/actions/runs/18327936488/job/52196728885?pr=2
<details>
<summary>Details</summary>
```
/home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/src/randomenv.cpp:61:15: warning: '__p__environ' redeclared without 'dllimport' attribute: previous 'dllimport' ignored [-Winconsistent-dllimport]
61 | extern char** environ; // NOLINT(readability-redundant-declaration): Necessary on some platforms
| ^
/home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/llvm_mingw_toolchain/aarch64-w64-mingw32/include/stdlib.h:656:17: note: expanded from macro 'environ'
656 | #define environ _environ
| ^
/home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/llvm_mingw_toolchain/aarch64-w64-mingw32/include/stdlib.h:225:21: note: expanded from macro '_environ'
225 | #define _environ (* __p__environ())
| ^
/home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/llvm_mingw_toolchain/aarch64-w64-mingw32/include/stdlib.h:221:27: note: previous declaration is here
221 | _CRTIMP char ***__cdecl __p__environ(void);
| ^
/home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/llvm_mingw_toolchain/aarch64-w64-mingw32/include/stdlib.h:221:3: note: previous attribute is here
221 | _CRTIMP char ***__cdecl __p__environ(void);
| ^
/home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/llvm_mingw_toolchain/aarch64-w64-mingw32/include/_mingw.h:52:40: note: expanded from macro '_CRTIMP'
52 | # define _CRTIMP __attribute__ ((__dllimport__))
| ^
1 warning generated.
```
</details>
after:
https://github.com/l0rinc/bitcoin-core-nightly/actions/runs/18329616268/job/52201940831?pr=4
<details>
<summary>Details</summary>
```
[ 28%] Building CXX object src/util/CMakeFiles/bitcoin_util.dir/__/randomenv.cpp.obj
```
</details>
Note that there are some other remaining warnings in the logs that will be fixed in separate PRs
ACKs for top commit:
sipa:
utACK 9610b0d1e2 if this makes the compilers happy
laanwj:
Code review ACK 9610b0d1e2
hebasto:
re-ACK 9610b0d1e2.
Tree-SHA512: a9e39d288b663ed24cbbbae228850e6f02d417d8781a3ac3d0b3db0b7ff734bbd62fddb9f57b8f77daab4e9c016ff66906ebc5fb2de7635ef539ef7f4dc2eaba
fa20275db3 test: Use unassigned p2p_port instead of hardcoded 60000 in p2p_i2p_ports.py (MarcoFalke)
Pull request description:
The goal is to fix https://github.com/bitcoin/bitcoin/issues/30030.
The root cause it unclear. However, hard-coding the port to 60000 does not seem ideal anyway. This could break in an unlikely setting where so many functional tests are run, such that the port is occupied. Also, it could fail when `TEST_RUNNER_PORT_MIN` is set sufficiently high. (This is purely theoretical, as I don't think anyone would run a command like this, but on current master it fails, and on this pull it passes: `TEST_RUNNER_PORT_MIN=60000 ./bld-cmake/test/functional/p2p_i2p_ports.py --portseed=0`)
So fix those issues (and hopefully also 30030) by using an unoccupied p2p_port.
The logic is similar to the `extra_port()` logic in the `feature_bind_extra.py` test.
ACKs for top commit:
laanwj:
Code review ACK fa20275db3
mzumsande:
ACK fa20275db3
Tree-SHA512: ac5487ca195db9ca746b78b8add91d0b9ef59cc3be0cdb7fbd9f76d42549eea68a61c32b4f5a162e01f3777959110f9f8d56ff05af6a13a9f61ea5be5b7d8631
c864a4c194 Simplify fs::path by dropping filename() and make_preferred() overloads (Ryan Ofsky)
b0113afd44 Fix windows libc++ fs::path fstream compile errors (Ryan Ofsky)
Pull request description:
Drop support for passing `fs::path` directly to `std::ifstream` and `std::ofstream` constructors and `open()` functions, because as reported by hebasto in https://github.com/bitcoin/bitcoin/issues/33545, after https://wg21.link/lwg3430 there is no way this can continue to work in windows builds, and there are already compile errors compiling for windows with newer versions of libc++.
Instead, add an `fs::path::std_path()` method that returns `std::filesystem::path` references and use it where needed.
ACKs for top commit:
hebasto:
ACK c864a4c194.
l0rinc:
Code review ACK c864a4c194
maflcko:
re-ACK c864a4c194 🌥
Tree-SHA512: d22372692ab86244e2b2caf4c5e9c9acbd9ba38df5411606b75e428474eabead152fc7ca1afe0bb0df6b818351211a70487e94b40a17b68db5aa757604a0ddf6
This has a few benefits:
* The shellcheck SC2086 warning is disabled for the whole command, but
is only needed for the DOCKER_BUILD_CACHE_ARG env var. So in Python,
only pass this one env var to shlex.split() for proper word splitting.
* Future logic improvements can be implemented in Python.
The comments are moved, which can be checked via the git options:
--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
The option is currently unused. If it is used again in the future, it
could trivially be added back.
Also, the logic is just a single undocumented python command one-liner.
So remove it for now.
4b41f99d57 build: Move CMAKE_SKIP_INSTALL_RPATH from CMake to Guix script (Henry Romp)
Pull request description:
Remove `CMAKE_SKIP_INSTALL_RPATH` from CMakeLists.txt and add `CMAKE_SKIP_RPATH` to the Guix build script. This keeps build-environment-specific settings in the build scripts rather than hardcoded in the CMake configuration.
ACKs for top commit:
purpleKarrot:
ACK 4b41f99d57
janb84:
re ACK 4b41f99d57
Tree-SHA512: 74d6af382476d731f10f9833978d670e9981c160ba306d0e9d4b1ad1e9b9960b8d03a3b9b608e234edb1c0c2c7a2b4f9f606a2a7887b7a153792159e71ae9b21
fa75ef4328 test: Move export_env_build_path to util.py (MarcoFalke)
fa9f495308 test: Move get_binary_paths and Binaries to util.py (MarcoFalke)
Pull request description:
Having the binary related utils sit in the test_framework.py is fine. However, they are mostly stand-alone utils, which may be used externally.
So move them to utils.py, to allow easier external use. The diff is trivial and can be reviewed via the git options `--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space`.
ACKs for top commit:
kevkevinpal:
ACK [fa75ef4](fa75ef4328)
Sjors:
lgtm ACK fa75ef4328
yuvicc:
Code review ACK fa75ef4328
janb84:
ACK fa75ef4328
musaHaruna:
Code Review ACK [fa75ef4](fa75ef4328)
enirox001:
ACK [fa75ef4](fa75ef4328)
Tree-SHA512: f382118484cb5495e8888214437e72c81727d54f97b3c09dfd996faab6cb6643c4c2d816b89ab82de73fc091c36ed7b8744c7d34a443b6adc415eb06697ef6ea
3cbf7cb3e6 Squashed 'src/secp256k1/' changes from b9313c6e1a..d543c0d917 (fanquake)
Pull request description:
Updates the subtree to d543c0d917
Related to #33284.
ACKs for top commit:
hebasto:
ACK 879c21045e.
janb84:
ACK 879c21045e
Tree-SHA512: 1802cd84959b5c935170792f458651f30431fe8340ead7966ff381c1c0c3a9f6c21bbb8dd96a07482ffed49642ded49e80b61802e688b8351956b111dffd5a78
Remove CMAKE_SKIP_INSTALL_RPATH from CMakeLists.txt and add CMAKE_SKIP_RPATH to the Guix build script. This keeps build-environment-specific settings in the build scripts rather than hardcoded in the CMake configuration.
3d22282564 [doc] correct topology requirements in submitpackage helptext (glozow)
Pull request description:
This doc is outdated since #31385. Also made it explicit that a singleton is ok.
Can be backported to 30.x, but doesn't need to be backported earlier ("if any" covers #31096).
ACKs for top commit:
janb84:
ACK 3d22282564
instagibbs:
ACK 3d22282564
Tree-SHA512: 95e40630a5b2a571029c0657c20a5e2a1cf1789913b868cee314c1a9fcb9a09fccdd3c87f3f15a8eb95c5ff9b83f8adee0661f86619bf21965866b6f6a76dfd0
f21162d819 Squashed 'src/leveldb/' changes from aba469ad6a..cad64b151d (fanquake)
Pull request description:
Rather than continue to close PRs/"Send these upstream" i.e: #33638, #33148, #22664, #13781; just fix the typos.
Includes https://github.com/bitcoin-core/leveldb-subtree/pull/57.
ACKs for top commit:
l0rinc:
ACK 54ffe3de5b
cedwies:
ACK 54ffe3d
stickies-v:
ACK 54ffe3de5b
Tree-SHA512: cc4d758ee95a1943f14e800472dfef24d5598a1dfafede32300821bc27e02a80ae97ea12ee87643b395b204262c7bc28e64d421a3d375d46bef7782381fd4362
9b43428c96 TxGraph: change m_excluded_clusters (Greg Sanders)
Pull request description:
Change BlockBuilderImpl's m_excluded_clusters to unordered set since ordering is not used.
Change the set to a set of sequence numbers for a modest stability increase under fuzz testing.
ACKs for top commit:
sipa:
ACK 9b43428c96
marcofleon:
tACK 9b43428c96
glozow:
ACK 9b43428c96
Tree-SHA512: 140a492af93f3eff756847a8168aab2624bb7df407f177dde6f3b07e9db2d0ced6b125e2b126f4957ccd054272056bedf74f9f0e64a80d90c16fd94e0fa86a44
24d861da78 coins: only adjust `cachedCoinsUsage` on `EmplaceCoinInternalDANGER` insert (Lőrinc)
d7c9d6c291 coins: fix `cachedCoinsUsage` accounting to prevent underflow (Lőrinc)
39cf8bb3d0 refactor: remove redundant usage tracking from `CoinsViewCacheCursor` (Lőrinc)
67cff8bec9 refactor: assert newly-created parent cache entry has zero memory usage (Lőrinc)
Pull request description:
### Summary
This PR fixes `cachedCoinsUsage` accounting bugs in `CCoinsViewCache` that caused UBSan `unsigned-integer-overflow` violations during testing. The issues stemmed from incorrect decrement timing in `AddCoin()`, unconditional reset in `Flush()` on failure, and incorrect increment in `EmplaceCoinInternalDANGER()` when insertion fails.
### Problems Fixed
**1. `AddCoin()` underflow on exception**
- Previously decremented `cachedCoinsUsage` *before* the `possible_overwrite` validation
- If validation threw, the map entry remained unchanged but counter was decremented
- This corrupted accounting and later caused underflow
- **Impact**: Test-only in current codebase, but unsound accounting that could affect future changes
**2. `Flush()` accounting drift on failure**
- Unconditionally reset `cachedCoinsUsage` to 0, even when `BatchWrite()` failed
- Left the map populated while the counter read zero
- **Impact**: Test-only (production `BatchWrite()` returns `true`), but broke accounting consistency
**3. Cursor redundant usage tracking**
- `CoinsViewCacheCursor::NextAndMaybeErase()` subtracted usage when erasing spent entries
- However, `SpendCoin()` already decremented and cleared the `scriptPubKey`, leaving `DynamicMemoryUsage()` at 0
- **Impact**: Redundant code that obscured actual accounting behavior
**4. `EmplaceCoinInternalDANGER()` double-counting**
- Incremented `cachedCoinsUsage` even when `try_emplace` did not insert (duplicate key)
- Inflated the counter on duplicate attempts
- **Impact**: Mostly test-reachable (AssumeUTXO doesn't overwrite in production), but incorrect accounting
### Testing
To reproduce the historical UBSan failures on the referenced baseline and to verify the fix, run:
```
MAKEJOBS="-j$(nproc)" FILE_ENV="./ci/test/00_setup_env_native_fuzz.sh" ./ci/test_run_all.sh
```
The change was tested with the related unit and fuzz test, and asserted before/after each `cachedCoinsUsage` change (in production code and fuzz) that the calculations are still correct by recalculating them from scratch.
<details>
<summary>Details</summary>
```C++
bool CCoinsViewCache::CacheUsageValid() const
{
size_t actual{0};
for (auto& entry : cacheCoins | std::views::values) actual += entry.coin.DynamicMemoryUsage();
return actual == cachedCoinsUsage;
}
```
or
```patch
diff --git a/src/coins.cpp b/src/coins.cpp
--- a/src/coins.cpp(revision fd3b1a7f4bb2ac527f23d4eb4cfa40a3215906e5)
+++ b/src/coins.cpp(revision 872a05633bfdbd06ad82190d7fe34b42d13ebfe9)
@@ -96,6 +96,7 @@
fresh = !it->second.IsDirty();
}
if (!inserted) {
+ Assert(cachedCoinsUsage >= it->second.coin.DynamicMemoryUsage());
cachedCoinsUsage -= it->second.coin.DynamicMemoryUsage();
}
it->second.coin = std::move(coin);
@@ -133,6 +134,7 @@
bool CCoinsViewCache::SpendCoin(const COutPoint &outpoint, Coin* moveout) {
CCoinsMap::iterator it = FetchCoin(outpoint);
if (it == cacheCoins.end()) return false;
+ Assert(cachedCoinsUsage >= it->second.coin.DynamicMemoryUsage());
cachedCoinsUsage -= it->second.coin.DynamicMemoryUsage();
TRACEPOINT(utxocache, spent,
outpoint.hash.data(),
@@ -226,10 +228,12 @@
if (itUs->second.IsFresh() && it->second.coin.IsSpent()) {
// The grandparent cache does not have an entry, and the coin
// has been spent. We can just delete it from the parent cache.
+ Assert(cachedCoinsUsage >= itUs->second.coin.DynamicMemoryUsage());
cachedCoinsUsage -= itUs->second.coin.DynamicMemoryUsage();
cacheCoins.erase(itUs);
} else {
// A normal modification.
+ Assert(cachedCoinsUsage >= itUs->second.coin.DynamicMemoryUsage());
cachedCoinsUsage -= itUs->second.coin.DynamicMemoryUsage();
if (cursor.WillErase(*it)) {
// Since this entry will be erased,
@@ -279,6 +283,7 @@
{
CCoinsMap::iterator it = cacheCoins.find(hash);
if (it != cacheCoins.end() && !it->second.IsDirty() && !it->second.IsFresh()) {
+ Assert(cachedCoinsUsage >= it->second.coin.DynamicMemoryUsage());
cachedCoinsUsage -= it->second.coin.DynamicMemoryUsage();
TRACEPOINT(utxocache, uncache,
hash.hash.data(),
```
</details>
ACKs for top commit:
optout21:
reACK 24d861da78
andrewtoth:
ACK 24d861da78
sipa:
ACK 24d861da78
w0xlt:
ACK 24d861da78
Tree-SHA512: ff1b756b46220f278ab6c850626a0f376bed64389ef7f66a95c994e1c7cceec1d1843d2b24e8deabe10e2bdade2a274d9654ac60eb2b9bf471a71db8a2ff496c
444409ff2b ci: Reduce Alpine musl task to md runner size (MarcoFalke)
fa6b2e9efe ci: Turn centos config into alpine musl config (MarcoFalke)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/33437
Historically, the centos task was added to add CI coverage for old packages and 32-bit depends builds, but both are now covered by different tasks.
The CentOS task aligns with Ubuntu/Debian CI tasks in terms of libc usage, but (slightly) differs in package naming and update philosophy. I am not aware of the task ever discovering a centos-related issue, so it seems fine to recycle it into an Alpine Linux task.
The main difference would be that musl libc is now used. Also, busybox is used in Alpine, so in theory the busybox install could be removed from the arm CI task in the future.
Packaging considerations: All packages should roughly be the same (gcc remains at version 14, python remains at version 3.12, etc). Also, all packages are from the Alpine main track, coming with 2 years of support. The only exception is the py3-pip package (https://pkgs.alpinelinux.org/packages?name=py3-pip&branch=v3.22&repo=&arch=riscv64) from the community track, however, I don't expect any issues arising from that.
ACKs for top commit:
janb84:
reACK 444409ff2b
willcl-ark:
ACK 444409ff2b
Tree-SHA512: fd1a1da0fd766591e44a57dbdb84f9b3b47ca92113a429bba139ee5fef54714b8fe509c321e7b3a470c29b4af7d9eab9786e1660b9effb862ecea52824f458aa
3a10d700bc test: P2SH sig ops are only counted with `SCRIPT_VERIFY_P2SH` flag (brunoerg)
Pull request description:
This PR adds a test case for `GetTransactionSigOpCost` to check that P2SH sig ops are only counted when `SCRIPT_VERIFY_P2SH` flag is set.
Kills the following [mutant](https://corecheck.dev/mutation/src/consensus/tx_verify.cpp#L150):
```diff
diff --git a/src/consensus/tx_verify.cpp b/src/consensus/tx_verify.cpp
index 9d09872597..cc7cdaaf8f 100644
--- a/src/consensus/tx_verify.cpp
+++ b/src/consensus/tx_verify.cpp
@@ -147,7 +147,7 @@ int64_t GetTransactionSigOpCost(const CTransaction& tx, const CCoinsViewCache& i
if (tx.IsCoinBase())
return nSigOps;
- if (flags & SCRIPT_VERIFY_P2SH) {
+ if (1==1) {
nSigOps += GetP2SHSigOpCount(tx, inputs) * WITNESS_SCALE_FACTOR;
}
```
ACKs for top commit:
l0rinc:
Tested ACK 3a10d700bc
maflcko:
re-lgtm ACK 3a10d700bc
instagibbs:
ACK 3a10d700bc
janb84:
tested ACK 3a10d700bc
Tree-SHA512: f560b4f9f2ce5c5fdd0a86e7e1f8ea27a8c6fda0327a6186a0c21e2c06ef13beeb017686db1688cace68812a01701abe46e8e1a095afefc6f2aed6ed96ba8288
d543c0d917 Merge bitcoin-core/secp256k1#1734: Introduce (mini) unit test framework
f44c1ebd96 Merge bitcoin-core/secp256k1#1719: ci: DRY workflow using anchors
a44a339384 Merge bitcoin-core/secp256k1#1750: ci: Use clang-snapshot in "MSan" job
15d014804e ci: Drop default for `inputs.command` in `run-in-docker-action`
1decc49a1f ci: Use YAML anchor and aliases for repeated "CI script" steps
dff1bc107d ci, refactor: Generalize use of `matrix.configuration.env_vars`
4b644da199 ci: Use YAML anchor and aliases for repeated "Print logs" steps
a889cd93df ci: Bump `actions/checkout` version
574c2f3080 ci: Use YAML anchor and aliases for repeated "Checkout" steps
53585f93b7 ci: Use clang-snapshot in "MSan" job
6894c964f3 Fix Clang 21+ `-Wuninitialized-const-pointer` warning when using MSan
2b7337f63a Merge bitcoin-core/secp256k1#1756: ci: Fix image caching and apply other improvements
f163c35897 ci: Set `DEBIAN_FRONTEND=noninteractive`
70ae177ca0 ci: Bump `docker/build-push-action` version
b2a95a420f ci: Drop `tags` input for `docker/build-push-action`
122014edb3 ci: Add `scope` parameter to `cache-{to,from}` options
2f4546ce56 test: add --log option to display tests execution
95b9953ea4 test: Add option to display all available tests
953f7b0088 test: support running specific tests/modules targets
0302c1a3d7 test: add --help for command-line options
9ec3bfe22d test: adapt modules to the new test infrastructure
48789dafc2 test: introduce (mini) unit test framework
baa265429f Merge bitcoin-core/secp256k1#1727: docs: Clarify that callback can be called more than once
4d90585fea docs: Improve API docs of _context_set_illegal_callback
895f53d1cf docs: Clarify that callback can be called more than once
de6af6ae35 Merge bitcoin-core/secp256k1#1748: bench: improve context creation in ECDH benchmark
5817885153 Merge bitcoin-core/secp256k1#1749: build: Fix warnings in x86_64 assembly check
ab560078aa build: Fix warnings in x86_64 assembly check
10dab907e7 Merge bitcoin-core/secp256k1#1741: doc: clarify API doc of `secp256k1_ecdsa_recover` return value
dfe284ed2d bench: improve context creation in ECDH benchmark
7321bdf27b doc: clarify API doc of `secp256k1_ecdsa_recover` return value
b475654302 Merge bitcoin-core/secp256k1#1745: test: introduce group order byte-array constant for deduplication
9cce703863 refactor: move 'gettime_i64()' to tests_common.h
0c91c56041 test: introduce group order byte-array constant for deduplication
88be4e8d86 Merge bitcoin-core/secp256k1#1735: musig: Invalidate secnonce in secp256k1_musig_partial_sign
36e76952cb Merge bitcoin-core/secp256k1#1738: check-abi: remove support for obsolete CMake library output location (src/libsecp256k1.so)
399b582a5f Split memclear into two versions
4985ac0f89 Merge bitcoin-core/secp256k1#1737: doc: mention ctx requirement for `_ellswift_create` (not secp256k1_context_static)
7ebaa134a7 check-abi: remove support for obsolete CMake library output location (src/libsecp256k1.so)
806de38bfc doc: mention ctx requirement for `_ellswift_create` (not secp256k1_context_static)
03fb60ad2e Merge bitcoin-core/secp256k1#1681: doc: Recommend clang-cl when building on Windows
d93380fb35 Merge bitcoin-core/secp256k1#1731: schnorrsig: Securely clear buf containing k or its negation
8113671f80 Merge bitcoin-core/secp256k1#1729: hash: Use size_t instead of int for RFC6979 outlen copy
325d65a8cf Rename and clear var containing k or -k
960ba5f9c6 Use size_t instead of int for RFC6979 outlen copy
737912430d ci: Add more tests for clang-cl
7379a5bed3 doc: Recommend clang-cl when building on Windows
f36afb8b3d Merge bitcoin-core/secp256k1#1725: tests: refactor tagged hash verification
5153cf1c91 tests: refactor tagged hash tests
d2dcf52091 Merge bitcoin-core/secp256k1#1726: docs: fix broken link to Tromer's cache.pdf paper
489a43d1bf docs: fix broken link to eprint cache.pdf paper
d599714147 Merge bitcoin-core/secp256k1#1722: docs: Exclude modules' `bench_impl.h` headers from coverage report
0458def51e doc: Add `--gcov-ignore-parse-errors=all` option to `gcovr` invocations
1aecce5936 doc: Add `--merge-mode-functions=separate` option to `gcovr` invocations
106a7cbf41 doc: Exclude modules' `bench_impl.h` headers from coverage report
a9e955d3ea autotools, docs: Adjust help string for `--enable-coverage` option
e523e4f90e Merge bitcoin-core/secp256k1#1720: chore(ci): Fix typo in Dockerfile comment
24ba8ff168 chore(ci): Fix typo in Dockerfile comment
74b8068c5d Merge bitcoin-core/secp256k1#1717: test: update wycheproof test vectors
c25c3c8a88 test: update wycheproof test vectors
20e3b44746 Merge bitcoin-core/secp256k1#1688: cmake: Avoid contaminating parent project's cache with `BUILD_SHARED_LIBS`
2c076d907a Merge bitcoin-core/secp256k1#1711: tests: update Wycheproof
7b07b22957 cmake: Avoid contaminating parent project's cache with BUILD_SHARED_LIBS
5433648ca0 Fix typos and spellings
9ea54c69b7 tests: update Wycheproof files
git-subtree-dir: src/secp256k1
git-subtree-split: d543c0d917a76a201578948701cc30ef336e0fe6
e4c04f7759 ci: add libcpp hardening flags to macOS fuzz job (fanquake)
Pull request description:
Follows up to https://github.com/bitcoin/bitcoin/pull/33425#issuecomment-3323149107.
ACKs for top commit:
maflcko:
lgtm ACK e4c04f7759. The qa-assets repo has a libc++ debug run, so this isn't required, but it seems fast enough to not hurt.
Tree-SHA512: 6c0dc90528ca867df49027eebf2d1c417a7395f9f94779076ace48e3e4b21771e7d99e8b3ed75ca56da87153418a446075429aa6b9ec5cd4b6b8cb5b0c25c1d7
Previously the `bool relay` argument to `BroadcastTransaction()`
designated:
```
relay=true: add to the mempool and broadcast to all peers
relay=false: add to the mempool
```
Change this to an `enum`, so it is more readable and easier to extend
with a 3rd option. Consider these example call sites:
```cpp
Paint(true);
// Or
Paint(/*is_red=*/true);
```
vs
```cpp
Paint(RED);
```
The idea for putting `TxBroadcastMethod` into `node/types.h` by Ryan.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
ac599c4a9c test: Test MuSig2 in the wallet (Ava Chow)
68ef954c4c wallet: Keep secnonces in DescriptorScriptPubKeyMan (Ava Chow)
4a273edda0 sign: Create MuSig2 signatures for known MuSig2 aggregate keys (Ava Chow)
258db93889 sign: Add CreateMuSig2AggregateSig (Ava Chow)
bf69442b3f sign: Add CreateMuSig2PartialSig (Ava Chow)
512b17fc56 sign: Add CreateMuSig2Nonce (Ava Chow)
82ea67c607 musig: Add MuSig2AggregatePubkeys variant that validates the aggregate (Ava Chow)
d99a081679 psbt: MuSig2 data in Fill/FromSignatureData (Ava Chow)
4d8b4f5336 signingprovider: Add musig2 secnonces (Ava Chow)
c06a1dc86f Add MuSig2SecNonce class for secure allocation of musig nonces (Ava Chow)
9baff05e49 sign: Include taproot output key's KeyOriginInfo in sigdata (Ava Chow)
4b24bfeab9 pubkey: Return tweaks from BIP32 derivation (Ava Chow)
f14876213a musig: Move synthetic xpub construction to its own function (Ava Chow)
fb8720f1e0 sign: Refactor Schnorr sighash computation out of CreateSchnorrSig (Ava Chow)
a4cfddda64 tests: Clarify why musig derivation adds a pubkey and xpub (Ava Chow)
39a63bf2e7 descriptors: Add a doxygen comment for has_hardened output_parameter (Ava Chow)
2320184d0e descriptors: Fix meaning of any_key_parsed (Ava Chow)
Pull request description:
This PR implements MuSig2 signing so that the wallet can receive and spend from imported `musig(0` descriptors.
The libsecp musig module is enabled so that it can be used for all of the MuSig2 cryptography.
Secnonces are handled in a separate class which holds the libsecp secnonce object in a `secure_unique_ptr`. Since secnonces must not be used, this class has no serialization and will only live in memory. A restart of the software will require a restart of the MuSig2 signing process.
ACKs for top commit:
fjahr:
tACK ac599c4a9c
rkrux:
lgtm tACK ac599c4a9c
theStack:
Code-review ACK ac599c4a9c🗝️
Tree-SHA512: 626b9adc42ed2403e2f4405321eb9ce009a829c07d968e95ab288fe4940b195b0af35ca279a4a7fa51af76e55382bad6f63a23bca14a84140559b3c667e7041e
0626b90f50 multiprocess: align our logging with libmultiprocess's (Cory Fields)
9d068225ee multiprocess: update multiprocess EventLoop construction to use options (Cory Fields)
Pull request description:
This fixes https://github.com/bitcoin-core/libmultiprocess/issues/215 on Core's side. ~It depends on https://github.com/bitcoin-core/libmultiprocess/pull/220 being merged upstream, and a PR to update our subtree. I've included a subtree merge from my repo here for now, but will rebase on top of the merge from upstream once it's in.~ Edit: Rebased on top of #33518.
For context: before https://github.com/bitcoin-core/libmultiprocess/pull/220, libmultiprocess serializesd every log message parameter, even if that message was ultimately going to be discarded. The upstream PR accomplished 2 main things:
- Creates logging categories, similar to Core's
- Using macros, avoids serializing parameters for disabled log levels.
That allowed the expensive serialization to be skipped, but the default log level is `Trace`. This PR updates our usage of libmultiprocess options to honor our log categories and levels.
Because of the substantial unnecessary overhead (see the [flamegraphs](https://github.com/bitcoin-core/libmultiprocess/issues/215). Logging accounts for 50% of my application's cpu time, and nearly 10% of bitcoin-node's, both of which go to ~0% once fixed), it'd be a shame to ship the first multiprocess binaries without this fixed. So I propose that we also backport this (and the required libmultiprocess subtree merge) ~to v30. Sorry about the timing~ :(
Edit: Didn't make it for v30, but it would still make sense to backport for a v30.1.
ACKs for top commit:
Sjors:
ACK 0626b90f50
TheCharlatan:
ACK 0626b90f50
sipa:
utACK 0626b90f50
Tree-SHA512: 70b63b62d1f6de547f4d4775538d7bcaf32f57d8a72c5b26762b57755810c8be6942d9dfebab43cf1c1d8d025a555f72a48e9ebf3d84f8d40d6592ca801cda5d
Change BlockBuilderImpl's m_excluded_clusters to unordered
set since ordering is not used.
Change the set to a set of sequence numbers for a modest
stability increase under fuzz testing.
7b544341c0 test: change log rate limit version gate from 299900 to 290100 (Eugene Siegel)
Pull request description:
Change the version gate from 299900 to 290100 for bypassing the log rate limit in case an explicit version is set in the functional test framework.
See discussion here: https://github.com/bitcoin/bitcoin/pull/33225#discussion_r2287838255
ACKs for top commit:
maflcko:
lgtm ACK 7b544341c0
janb84:
ACK 7b544341c0
stickies-v:
ACK 7b544341c0
Tree-SHA512: c07c8741dfdeca87c49748b7082c2ecb829da391908316f35daef7292bc017814a89f04e16e738f3a105541bbc38e4feb5bca3fb6ab718a1dc1de7c70a9c8a58
023cd5a546 txgraph: add SingletonClusterImpl (mem optimization) (Pieter Wuille)
e346250732 txgraph: give Clusters a range of intended tx counts (preparation) (Pieter Wuille)
e93b0f09cc txgraph: abstract out creation of empty Clusters (refactor) (Pieter Wuille)
6baf12621f txgraph: comment fixes (doc fix) (Pieter Wuille)
726b995739 txgraph: make Cluster an abstract class (refactor) (Pieter Wuille)
2602d89edd txgraph: avoid accessing other Cluster internals (refactor) (Pieter Wuille)
04c808ac4c txgraph: expose memory usage estimate function (feature) (Pieter Wuille)
7680bb8fd4 txgraph: keep track of Cluster memory usage (preparation) (Pieter Wuille)
4ba562e5f4 txgraph: keep data structures compact (mem optimization) (Pieter Wuille)
bb5cb222ae depgraph: add memory usage control (feature) (Pieter Wuille)
b1637a90de txgraph: avoid holes in DepGraph positions (mem optimization) (Pieter Wuille)
2b1d302508 txgraph: move some sanity checks from Cluster to TxGraphImpl (refactor) (Pieter Wuille)
d40302fbaf txgraph: Make level of Cluster implicit (optimization) (Pieter Wuille)
Pull request description:
Part of #30289.
This adds a few optimizations to reduce `TxGraph`'s memory usage, and makes sure that dynamic memory it uses doesn't linger after shrinking clusters. Finally, it exposes a function `GetMainMemoryUsage()` to compute `TxGraph`'s approximate memory usage.
It makes the `Cluster` type abstract, with two instances (`SingletonClusterImpl` for 1-transaction clusters, and `GenericClusterImpl` for others).
On my 64-bit system, I obtain the following numbers:
* `SingletonClusterImpl`: 48 bytes, plus 16 bytes malloc overhead in its `unique_ptr`, plus 8-byte pointer in `m_clusters`
* `GenericClusterImpl`: 104 bytes, plus 16 bytes malloc overhead in its `unique_ptr`, plus 8-byte pointer in `m_clusters`, plus 72 bytes malloc overhead inside its vectors and `DepGraph`, plus 40 bytes per transaction in those.
* `TxGraphImpl::Entry`: 72 bytes per transaction
* `TxGraphImpl::ChunkData`: 8 bytes, plus 56 bytes in `std::set` overhead + malloc overhead, all per chunk.
* `TxGraph::Ref`: 16 bytes per transaction
This overall amounts to 200 bytes per cluster, plus 64 bytes per chunk, plus 128 bytes per transaction, but only 224 bytes overall per singleton cluster.
ACKs for top commit:
l0rinc:
code review reACK 023cd5a546
instagibbs:
reACK 023cd5a546
ismaelsadeeq:
reACK 023cd5a546🚢
glozow:
reACK 023cd5a546
Tree-SHA512: c957b27f47318be7c25d71453df2ae9d4e7bf21dab13b6e5e975cca122a221a99b15c584872491225785d276a9165f090675ee0f4460a2775bd3271933e3b246
Extends 7703884 to guard environ declaration on all Windows builds, not just MSVC.
In the mingw-w64 headers (used by llvm-mingw), environ is defined as a macro which expands through _environ to (* __p__environ()), a call to a dllimport function, causing the same inconsistent linkage warning as MSVC.
Use WIN32 instead of _MSC_VER to match the platform-specific guards already used throughout the file.
The warning occurs with llvm-mingw (both UCRT and MSVCRT variants as tested by Hebasto), but not with the mingw-w64 toolchain currently used in CI (as mentioned by fanquake).
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
bc706955d7 ci: expose all ACTIONS_* vars (willcl-ark)
Pull request description:
When using `docker buildx build` in conjunction with the `gha` backend cache type (as we do in our CI) it's important to specify the URL and TOKEN needed to authenticate.
On Cirrus runners this is working with only `ACTIONS_CACHE_URL` and `ACTIONS_RUNTIME_TOKEN`, but this is not enough for the GitHub backend.
Fix this by exporting all `ACTIONS_*` variables.
This fixes docker build layer cache restore/save on forks or where GH-hosted runners are being used, and addresses https://github.com/bitcoin/bitcoin/issues/31965#issuecomment-3324707093
ACKs for top commit:
m3dwards:
ACK bc706955d7
maflcko:
lgtm ACK bc706955d7
Tree-SHA512: 13e973bb1c1ca5448dd6c3c176fb5ce39c725886ba2012d3253158205309a7038a1430135b37400e1f2f69408a9d0f4e2b3c5f0515154a593ec382ab7db10266
Block template fees are calculated by looping over new_tmpl->vTxFees
and return (early) once the fee_threshold is exceeded.
This left an edge case when the mempool is empty, which this commit
fixes and adds a test for. It does so by using std::accumulate instead
of manual loops.
Also update interface_ipc.py to account for the new behavior.
Co-authored-by: Raimo33 <claudio.raimondi@protonmail.com>
fa6fd16f36 ci: Use native platform for win-cross task (MarcoFalke)
Pull request description:
Forcing the architecture to amd64 is no longer required. Dropping it should have some benefits:
* Faster CI speed on other arches (riscv64, arm, ...)
* Unlock the CI task to run on riscv64 at all
ACKs for top commit:
hebasto:
ACK fa6fd16f36, tested on Ubuntu 24.04, RISC-V.
Tree-SHA512: 68a3fc90cc22ab085d6946deb106e50b22e06eebc61523a9dcb53b38a50021a19da26cc29e2cd20f4673ffc5cc10f441dacca7cc799782258351609d9fa04969
671b774d1b depends: Use $(package)_file_name when downloading from the fallback (Ava Chow)
Pull request description:
The server hosting the fallbacks uses `make download` so the files are only available with their overridden names rather than the original name on the upstream source. We should therefore also use the overridden name when downloading from the fallback.
Fixes https://github.com/bitcoin-core/bitcoincore.org/issues/1168
ACKs for top commit:
theuni:
utACK 671b774d1b. I was going to PR the same change.
janb84:
ut ACK 671b774d1b
hebasto:
ACK 671b774d1b, tested with the following patch:
Tree-SHA512: ba010adb64900d8d748487cc1a658e2b163872354f4e7b38c4dfc37a14fcb22fec4379a635d2c6788c64dd46bef0d94aa3eb6f522ec700680e886d5468678031
`EmplaceCoinInternalDANGER()` incremented `cachedCoinsUsage` even when `try_emplace` did not insert (duplicate key), inflating the counter.
This is mostly reachable in tests today since `AssumeUTXO` does not overwrite.
Increment only on successful insert, and capture `coin.DynamicMemoryUsage()` before the move so accounting uses the correct value.
Fuzz: add an `EmplaceCoinInternalDANGER` path to exercise insert-only accounting.
Unit test: emplace two different coins at the same outpoint (with different `DynamicMemoryUsage()`), verify `SelfTest()` passes and `AccessCoin(outpoint)` returns the first coin.
Co-authored-by: Andrew Toth <andrewstoth@gmail.com>
Co-authored-by: w0xlt <woltx@protonmail.com>
Move the `cachedCoinsUsage` subtract in `AddCoin()` to after the `possible_overwrite` check.
Previously a throw before assignment decremented the counter without changing the entry, which corrupted accounting and later underflowed.
In `Flush()`, reset `cachedCoinsUsage` to `0` only when `BatchWrite()` succeeds and `cacheCoins` is actually cleared. In production `BatchWrite()` returns `true`, so this mostly affects tests. On failure, leave the counter unchanged to keep it in sync with the cache.
The existing `Flush()` workaround in fuzzing was also removed now that the source of the problem was fixed, so the fuzzer no longer needs `coins_view_cache.Flush()` to realign `cachedCoinsUsage` after an exception.
Replace the prior `expected_code_path` tracking with direct assertions. The role of the variable was to verify that code execution follows only expected paths, either successful addition, or if it's an exception, the message is verified and checked that overwrite was disallowed.
With these changes the counter stays consistent across success and exception paths, so we can finally remove the `UBSan` suppressions for `CCoinsViewCache` that were masking the issue.
Included a unit test as well, attempting to add a different coin to the same outpoint without allowing overwrites and make sure it throws.
We use `SelfTest()` to validates accounting, and check that the cache remains usable.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Co-authored-by: w0xlt <woltx@protonmail.com>
When a coin is spent via `SpendCoin()`, `cachedCoinsUsage` is already decremented and the coin's `scriptPubKey` is cleared, so `DynamicMemoryUsage()` is `0`.
`CoinsViewCacheCursor::NextAndMaybeErase()` was subtracting usage again when erasing spent entries.
Replace it with an assert that documents spent coins have zero dynamic memory usage by the time the cursor encounters them.
Remove the now-unnecessary `usage` reference from the cursor's constructor and member variables.
During `BatchWrite`, the parent entry is created under a guard that guarantees insertion, so the new `Coin` is default-constructed and empty.
Assert this invariant to document why there is no `cachedCoinsUsage` decrement before the assignment at this site.
Co-authored-by: Andrew Toth <andrewstoth@gmail.com>
This adds a specialized Cluster implementation for singleton clusters, saving
a significant amount of memory by avoiding the need for m_depgraph, m_mapping,
and m_linearization, and their overheads.
This adds 4 functions to Cluster to help implement Merge() and Split() without
needing access to the internals of the other Cluster. This is a preparation for
a follow-up that will make Clusters a virtual class whose internals are abstracted
away.
This reduces per-Cluster memory usage by making Clusters not aware of their
own level. Instead, track it either in calling code, or infer it based on
the transactions in them.
Without this change, logging (even if unused) may account for a
substantial portion of bitcoin-node's and/or client's runtime cpu usage, due
to libmultiprocess's expensive message serialization.
This (along with some recent upstream changes) avoids the overhead by opting
out of log handling for messages that we're not interested in.
Info, Warning, and Error are logged unconditionally to match our behavior
elsewhere. See BCLog::Logger::GetCategoryLogLevel .
f6567527d8 doc: bump the template macOS version (kevkevinpal)
Pull request description:
Motivated by https://github.com/bitcoin/bitcoin/pull/33489#issuecomment-3361601497
The minimum version of MacOS for this repo is now 14 and above so it makes sense to update the issue template to reflect that.
We are now using a higher version but since it is just a bug template, there is no need to put the lowest version we support.
ACKs for top commit:
maflcko:
lgtm ACK f6567527d8
l0rinc:
ACK f6567527d8
janb84:
ACK f6567527d8
Tree-SHA512: 701b161bda25245996c94b6d2119b5cc85a34917551dcf8c92ffacf3aa80fa7fe84bb3497edc7e600c5b2443de13a6f6107fc7289721e585b16c4972d07a796c
This test (which would fail without the previous commit) checks
that after the stalling block was received, we don't incorrectly
mark another peer as a staller immediately.
Simplify and improve the logic for calculating pindexLastCommonBlock, in order to calculate
nWindowEnd better.
The previous logic would not take into account when the chain tip had moved forward, so that
FindNextBlocks could iterate over many blocks already downloaded and
connected, which could result in blocks not being requested for download that should have been
requested, and peers being wrongly marked as staller.
It also removes extra logic from commit 49d569cb1f
for the situation right after a snapshot was loaded:
After snapshot loading, our tip becomes the snapshot block.
For peers that have the most-work chain, which inlcludes the snapshot,
our tip is an ancestor of the peer's best block, hence the general
advancement logic will move pindexLastCommonBlock
from any pre-snapshot position to the snapshot height automatically.
Co-authored-by: stringintech <stringintech@gmail.com>
Co-authored-by: Pieter Wuille <pieter@wuille.net>
e4335a3192 Revert "depends: Update URL for `qrencode` package source tarball" (Ava Chow)
a89a822e6e Revert "depends: Use hash instead of file name for package download stamp" (Ava Chow)
Pull request description:
The new URL breaks CI on the current release branches, see https://github.com/bitcoin/bitcoin/pull/33494#issuecomment-3380802351.
The old URL also no longer exists so the tarball is fetched from the depends sources cache that we host, but the original tarball has already been overwritten on there. We will need to manually reinstate the original tarball.
ACKs for top commit:
m3dwards:
utACK e4335a3192
maflcko:
review ACK e4335a3192💤
glozow:
ACK e4335a3192
Tree-SHA512: a5028342d77b4768daaec8688acd364795d683aed2bea0407c7827d44f814a97d50cc3b30c2de2a8296a2b212115fe1e76c57685a74e93387fc57afdabb93bd2
a1226bc760 doc: how to update a subtree (Sjors Provoost)
Pull request description:
We have instructions on how to verify a subtree update, but not on how to perform one.
ACKs for top commit:
yuvicc:
ACK a1226bc760
achow101:
ACK a1226bc760
janb84:
ACK a1226bc760
furszy:
ACK a1226bc760
Tree-SHA512: ba3ccc56a9f1c7f461e0db9699612e1fd64b7c72bfd1dae63d4cb830db416871a493820d3a7924c19b6ce353fc20c5fe07578b053dec6ea68273a007cbebc512
ceeb53adcd ci: Properly include $FILE_ENV in DEPENDS_HASH (Ava Chow)
Pull request description:
$FILE_ENV has a full relative path already, prepending with ci/test/ results in a non-existent path which means that DEPENDS_HASH was not actually committing to the test's environment file.
ACKs for top commit:
maflcko:
lgtm ACK ceeb53adcd
Tree-SHA512: 80a7a23676ff8bf2f48a7d3c5897217f11d7d4d4f8a54897d2b7c42689585d2d63e45fad2b8f4c442111f128a87eeb6edeac2b25c79862e6bc035eeb1ebc7f4e
b35341b9ba Update ci.yml (Coder)
Pull request description:
Release notes:https://github.com/actions/download-artifact/releases/tag/v5.0.0
Change:
uses: actions/download-artifact@v4 -> uses: actions/download-artifact@v5
ACKs for top commit:
maflcko:
lgtm ACK b35341b9ba
willcl-ark:
ACK b35341b9ba
hebasto:
ACK b35341b9ba, I have reviewed the code and it looks OK.
Tree-SHA512: f82dd0fe3ca8d431b9ff6ef9f23a4f2e92a1463c6f55fbe9b46b9e13750d311bd2aa915a8570f76600363b3a1ccbf394c95216cfac0f6db30846d9be7ec7c4cf
The `createwallet` RPC doesn't return the empty passphrase
warning anymore if no passphrase was passed explicitly.
The `noshutdown` parameter key was removed in commit
fa0dc09b90, so remove it from
the table.
$FILE_ENV has a full relative path already, prepending with ci/test/
results in a non-existent path which means that DEPENDS_HASH was not
actually committing to the test's environment file.
156927903d ci: Check windows manifests for all executables (Max Edwards)
e1a1b14c93 ci: use a more generic way of finding mt.exe (Max Edwards)
7ae0497eef ci: remove 3rd party js from windows dll gha job (Max Edwards)
Pull request description:
The windows job uses the external dependency `ilammy/msvc-dev-cmd` which runs javascript. We use this to put various tools on the path such as `MSBuild.exe` and `mt.exe`. We can remove this dependency and use `vswhere.exe` directly to find these tools and create a "[Developer command prompt](https://github.com/microsoft/vswhere/wiki/Start-Developer-Command-Prompt#using-powershell)" as someone would on their dev machine.
While in this area of the code, this PR also runs some additional manifest checks on the windows binaries.
Fixes: #32508
ACKs for top commit:
davidgumberg:
crACK 156927903d
hebasto:
ACK 156927903d.
Tree-SHA512: df640dff27579a1c95daddc5a5ba8fd655bbd0a6f2aff74d0f63439c7185c0b18a90abfee3f1f032fe833cd19b822ef71812f44b24c4c044222e46d01c271864
652424ad16 test: additional test coverage for script_verify_flags (Anthony Towns)
417437eb01 script/verify_flags: extend script_verify_flags to 64 bits (Anthony Towns)
3cbbcb66ef script/interpreter: make script_verify_flag_name an ordinary enum (Anthony Towns)
bddcadee82 script/verify_flags: make script_verify_flags type safe (Anthony Towns)
a5ead122fe script/interpreter: introduce script_verify_flags typename (Anthony Towns)
4577fb2b1e rpc: have getdeploymentinfo report script verify flags (Anthony Towns)
a3986935f0 validation: export GetBlockScriptFlags() (Anthony Towns)
5db8cd2d37 Move mapFlagNames and FormatScriptFlags logic to script/interpreter.h (Anthony Towns)
Pull request description:
We currently use 21 of 32 possible bits for `SCRIPT_VERIFY_*` flags, with open PRs that may use 8 more (#29247, #31989, #32247, #32453). The mutinynet fork that has included many experimental soft fork features is [already reusing bits here](d4a86277ed/src/script/interpreter.h (L175-L195)). Therefore, bump this to 64 bits.
In order to make it easier to update this logic in future, this PR also introduces a dedicated type for the script flags, and disables implicit conversion between that type and the underlying integer type. To make verifying that this change doesn't cause flags to disappear, this PR also resurrects the changes from #28806 so that the script flags that are consensus enforced on each block can be queried via getdeploymentinfo.
ACKs for top commit:
instagibbs:
reACK 652424ad16
achow101:
ACK 652424ad16
darosior:
ACK 652424ad16
theStack:
Code-review ACK 652424ad16🎏
Tree-SHA512: 7b30152196cdfdef8b9700b571b7d7d4e94d28fbc5c26ea7532788037efc02e4b1d8de392b0b20507badfdc26f5c125f8356a479604a9149b8aae23a7cf5549f
3635d62f5a chain: make use of pskip in LastCommonAncestor (optimization) (Pieter Wuille)
2e09d66fbb tests: add unit tests for CBlockIndex::GetAncestor and LastCommonAncestor (Pieter Wuille)
Pull request description:
In theory, the `LastCommonAncestor` function in chain.cpp can take $\mathcal{O}(n)$ time, walking over the entire chain, if the forking point is very early, which could take ~milliseconds. I expect this to be very rare in normal occurrences, but it seems nontrivial to reason about worst cases as it's accessible from several places in net_processing.
This PR modifies the algorithm to make use of the `CBlockIndex::pskip` skip pointers to find the forking point in sublinear time (a simulation shows that for heights up to $34 \cdot 4^k - 2$ and $k \geq 8$, no more than $k^2 + 10k + 13$ steps are ever needed), in a way that should be nearly free - at worst the same number of memory accesses should be made, with a tiny increase in computation.
As it appears we didn't really have tests for this function, unit tests are added for that function as well as `CBlockIndex::GetAncestor()`.
This is inspired by https://github.com/bitcoin/bitcoin/pull/32180#discussion_r2394877881
ACKs for top commit:
optout21:
ACK 3635d62f5a
achow101:
ACK 3635d62f5a
vasild:
ACK 3635d62f5a
mzumsande:
Code Review ACK 3635d62f5a
furszy:
ACK 3635d62f5a
stratospher:
ACK 3635d62f5a.
Tree-SHA512: f9b7dea1e34c1cc1ec1da3fb9e90c4acbf4aaf0f04768844f538201efa6b11eeeefc97b720509e78c21878977192e2c4031fd8974151667e2e756247002b8164
a4f929696490 Merge bitcoin-core/libmultiprocess#224: doc: fix typos
f4344ae87da0 Merge bitcoin-core/libmultiprocess#222: test, ci: Fix threadsanitizer errors in mptest
1434642b3804 doc: fix typos
73d22ba2e930 test: Fix tsan race in thread busy test
b74e1bba014d ci: Use tsan-instrumented cap'n proto in sanitizers job
c332774409ad test: Fix failing exception check in new thread busy test
ca3c05d56709 test: Use KJ_LOG instead of std::cout for logging
7eb1da120ab6 ci: Use tsan-instrumented libcxx in sanitizers job
ec86e4336e98 Merge bitcoin-core/libmultiprocess#220: Add log levels and advertise them to users via logging callback
515ce93ad349 Logging: Pass LogData struct to logging callback
213574ccc43d Logging: reclassify remaining log messages
e4de0412b430 Logging: Break out expensive log messages and classify them as Trace
408874a78fdc Logging: Use new logging macros
67b092d835cd Logging: Disable logging if messsage level is less than the requested level
d0a1ba7ebf21 Logging: add log levels to mirror Core's
463a8296d188 Logging: Disable moving or copying Logger
83a2e10c0b03 Logging: Add an EventLoop constructor to allow for user-specified log options
58cf47a7fc8c Merge bitcoin-core/libmultiprocess#221: test default PassField impl handles output parameters
db03a663f514 Merge bitcoin-core/libmultiprocess#214: Fix crash on simultaneous IPC calls using the same thread
afcc40b0f1e8 Merge bitcoin-core/libmultiprocess#213: util+doc: Clearer errors when attempting to run examples + polished docs
6db669628387 test In|Out parameter
29cf2ada75ea test default PassField impl handles output parameters
1238170f68e8 test: simultaneous IPC calls using same thread
eb069ab75d83 Fix crash on simultaneous IPC calls using the same thread
ec03a9639ab5 doc: Precision and typos
2b4348193551 doc: Where possible, remove links to ryanofsky/bitcoin/
286fe469c9c9 util: Add helpful error message when failing to execute file
git-subtree-dir: src/ipc/libmultiprocess
git-subtree-split: a4f92969649018ca70f949a09148bccfeaecd99a
1aaaaa078b fuzz: Drop unused workaround after Apple-Clang bump (MarcoFalke)
fadad7a494 Drop support for EOL macOS 13 (MarcoFalke)
Pull request description:
Now that macOS 13 is EOL (https://en.wikipedia.org/wiki/MacOS_Ventura), it seems odd to still support it.
(macOS Ventura 13.7.8 received its final security update on 20 Aug 2025: https://support.apple.com/en-us/100100)
This patch will only be released in version 31.x, another 6 months out from now.
So:
* Update the depends build and release note template to drop EOL macOS 13.
* As a result, update the earliest Xcode to version 16 in CI.
* Also, bump the macOS CI runner to version 15, to avoid issues when version 14 will be at its EOL in about 1 year.
This also allows to drop a small workaround in the fuzz tests and unlocks libcpp hardening (https://github.com/bitcoin/bitcoin/pull/33462)
ACKs for top commit:
stickies-v:
re-ACK 1aaaaa078b
l0rinc:
code review ACK 1aaaaa078b
hodlinator:
re-ACK 1aaaaa078b
hebasto:
ACK 1aaaaa078b.
Tree-SHA512: 6d247a8432ef8ea8c6ff2a221472b278f8344346b172980299507f9898bb9e8e16480c128b1f4ca692bcbcc393da2b2fd6895ac5f118bc09e0f30f910529d20c
c76de2eea1 net: support overriding the proxy selection in ConnectNode() (Vasil Dimov)
Pull request description:
Normally `ConnectNode()` would choose whether to use a proxy and which one. Make it possible to override this from the callers and same for `OpenNetworkConnection()` - pass down the proxy to `ConnectNode()`.
Document both functions.
This is useful if we want to open connections to IPv4 or IPv6 peers through the Tor SOCKS5 proxy.
Also have `OpenNetworkConnection()` return whether the connection succeeded or not. This can be used when the caller needs to keep track of how many (successful) connections were opened.
---
This is part of [#29415 Broadcast own transactions only via short-lived Tor or I2P connections](https://github.com/bitcoin/bitcoin/pull/29415). Putting it in its own PR to reduce the size of #29415 and because it does not depend on the other commits from there.
ACKs for top commit:
stratospher:
ACK c76de2e.
optout21:
ACK c76de2eea1
mzumsande:
Code Review ACK c76de2eea1
andrewtoth:
ACK c76de2eea1
Tree-SHA512: 1d266e4280cdb1d0599971fa8b5da58b1b7451635be46abb15c0b823a1e18cf6e7bcba4a365ad198e6fd1afee4097d81a54253fa680c8b386ca6b9d68d795ff0
These overloads were needed to allow passing `fs::path` objects directly to
libstdc++'s `fstream` constructors, but after the previous commit, there is no
longer any remaining code that does pass `fs::path` objects to `fstream`
constructors. Writing new code which does this is also discouraged because the
standard has been updated in https://wg21.link/lwg3430 to disallow it.
Dropping these also means its no longer possible to pass `fs::path` arguments
directly to `fstream::open` in libstdc++, which is somewhat unfortunate but not
a big loss because it is already not possible to pass them to the constructor.
So this commit updates `fstream::open` calls.
Additionally, this change required updates to src/bitcoin.cpp since it was
relying on the overloaded filename() method.
As reported by hebasto in https://github.com/bitcoin/bitcoin/issues/33545,
newer libc++ versions implementing https://wg21.link/lwg3430 will no longer
implicitly convert `fs::path` objects to `std::filesystem::path` objects when
constructing `std::ifstream` and `std::ofstream` types.
This is not a problem in Unix systems since `fs::path` objects use
`std::string` as their native string type, but it causes compile errors on
Windows which use `std::wstring` as their string type, since `fstream`s can't
be constructed from `wstring`s.
Fix the windows libc++ compile errors by adding a new `fs::path::std_path()`
method and using it construct `fstream`s more portably.
Additionally, delete `fs::path`'s implicit `native_string` conversion so these
errors will not go undetected in the future, even though there is not currently
a CI job testing Windows libc++ builds.
Update doc/multisig-tutorial.md to use a single multipath descriptor
instead of separate external/internal descriptors, per PR #22838.
Extract one xpub per participant, build a multipath descriptor with
<0;1> change index, and use getdescriptorinfo to append the checksum.
Clarify importdescriptors expands multipath descriptors into internal
and external forms. Tested shell snippets to confirm equivalent
listdescriptors output as the two-descriptor method.
Added missing loadwallet command for multisig_wallet_01
test: Use multipath descriptors in the functional wallet test
wallet_multisig_descriptor_psbt as this is intended as documentation
doc: replace `bitcoin-cli` with `bitcoin rpc` in multisig-tutorial.md
removed -named parameter where possible.
fixed a couple bugs where -signet was not passed
the call to getcoins.py requires the bitcoin-cli command still
0f7d4ee4e8 p2p: Use different inbound inv timer per network (Martin Zumsande)
94db966a3b net: use generic network key for addrcache (Martin Zumsande)
Pull request description:
Currently, `NextInvToInbounds` schedules each round of `inv` at the same time for all inbound peers. It's being done this way because with a separate timer per peer (like it's done for outbounds), an attacker could do multiple connections to learn about the time a transaction arrived. (#13298).
However, having a single timer for inbounds of all networks is also an obvious fingerprinting vector: Connecting to a suspected pair of privacy-network and clearnet addresses and observing the `inv` pattern makes it trivial to confirm or refute that they are the same node.
This PR changes it such that a separate timer is used for each network.
It uses the existing method from `getaddr` caching and generalizes it to be saved in a new field `m_network_key` in `CNode` which will be used for both `getaddr` caching and `inv` scheduling, and can also be used for any future anti-fingerprinting measures.
ACKs for top commit:
sipa:
utACK 0f7d4ee4e8
stratospher:
reACK 0f7d4ee.
naiyoma:
Tested ACK 0f7d4ee4e8
danielabrozzoni:
reACK 0f7d4ee4e8
Tree-SHA512: e197c3005b2522051db432948874320b74c23e01e66988ee1ee11917dac0923f58c1252fa47da24e68b08d7a355d8e5e0a3ccdfa6e4324cb901f21dfa880cd9c
dda5228e02 test: set par=2 in default config for functional test framework (Andrew Toth)
Pull request description:
Depending on the host machine, a default `par` value can spawn up to 15 script verification threads for each node. Running the functional test suite with default `par` can exhaust file descriptors or hit other resource limits when many threads are spawned. These threads are mostly idle and the same code paths are executed with a value of `par=2`. Limit this to 2 for functional tests that do not override the default option.
ACKs for top commit:
maflcko:
lgtm ACK dda5228e02
pablomartin4btc:
ACK dda5228e02
l0rinc:
Code review ACK dda5228e02
theStack:
ACK dda5228e02
Tree-SHA512: 4459972330ff50ac7391141db6382579de09d84e68959eaeb5f20972bb9daf9aac1bd68355028ded9ee65e838c12dbd53e6f3bb6cdc375d269f666c19a19eaec
8e47ed6906 test: addrman: check isTerrible when time is more than 10min in the future (brunoerg)
Pull request description:
This PR adds test coverage to kill the following mutant (https://corecheck.dev/mutation/src/addrman.cpp#L76):
```diff
diff --git a/src/addrman.cpp b/src/addrman.cpp
index 9c3a24db90..0ffd349315 100644
--- a/src/addrman.cpp
+++ b/src/addrman.cpp
@@ -73,7 +73,7 @@ bool AddrInfo::IsTerrible(NodeSeconds now) const
}
if (nTime > now + 10min) { // came in a flying DeLorean
- return true;
+ return false;
}
```
When the `nTime` is set 10 minutes in the future the addr should be marked as terrible.
ACKs for top commit:
Crypt-iQ:
crACK 8e47ed6906
danielabrozzoni:
tACK 8e47ed6906
marcofleon:
Nice, code review ACK 8e47ed6906
Tree-SHA512: b53b3aa234a73ec7808cb1555916ac64dd707f230ec290a1712493ece8e274a060e16d862b31df0f744804ebd3c0c2825c49becb7d3040cc358e48c4002524cb
14ae71f323 test: make notfound_on_unannounced more reliable (David Gumberg)
99bc552980 test: fix (w)txid confusion in p2p_leak_tx.py (Martin Zumsande)
576dd97cb9 test: increase timeout in p2p_leak_tx.py (Martin Zumsande)
Pull request description:
This fixes two issues with `p2p_leak_tx.py`:
1.) #33090: As far as I can see, this is just the randomness of `NextInvToInbounds`/ `rand_exp_duration`, which has a probability of `e^-(60s/5s) = 6.14×10^−6` to result in a period > 60s (our waiting time), so that the test would fail every 160k runs... Doubling the timeout should be sufficient to lower the probability drastically.
2.) The subtest `test_notfound_on_unannounced_tx` has some (w)txid confusion: we send a `MSG_TX`-type getdata with a `wtxid` in it, which necessarily always results in a NOTFOUND. Fixed this, and change the subtest to be more deterministic based on `mocktime`.
ACKs for top commit:
stratospher:
ACK 14ae71f. nice restructuring using mocktime!
davidgumberg:
reACK 14ae71f323
vasild:
ACK 14ae71f323
Tree-SHA512: be5a4ca7bf56f82b6fa04d90ef9312dc2e6f8ff7ddf70b39d979dc42fbdd823157109b8b5dc46eb7f81ac1e816f40e6966b3c8a7d384aadee01e2189c20d3e3a
7b5261f7ef contrib: fix using macdploy script without translations. (amisha)
Pull request description:
**Description**
From what I deciphered reading the line https://github.com/bitcoin/bitcoin/blob/master/contrib/macdeploy/macdeployqtplus#L390 is that qt translations are optional to have hence we should be able to build without it but the case where the flag translations_dir falls back to its default Null value it raises this error.
The config comments also mentioned that adding translation file is optional.
```
./macdeployqtplus --help
usage: macdeployqtplus [-h] [-verbose [VERBOSE]] [-no-plugins] [-no-strip] [-translations-dir path] [-zip zip] app-bundle
Improved version of macdeployqt. Outputs a ready-to-deploy app in a folder "dist" and optionally wraps it in a .zip file. Note, that the "dist" folder will be deleted before deploying on each run. Optionally, Qt translation files
(.qm) can be added to the bundle.
```
**Steps to reproduce**
So I was following the general steps to set up app on macos however I didn't download any qt translations presuming it was optional from the comment linkedin in PR, so to reproduce if you have translation directories in place ull need to delete them and then try to build the file, otherwise don't download it at all and try to build it. It should fail on that flag as translations dir was never downloaded.
**Approach taken**
I have moved the code which adds language files under the if statement that first checks if the value of the flag is not Null before referencing it.
ACKs for top commit:
ismaelsadeeq:
ACK 7b5261f7ef
Tree-SHA512: 8d51b17569e42c9feb95e1be17b1551c708a05eb44b82c74db0b25e07006b4ee223d64484f8bdb2ee1420f6e571686561ae1c09bd3362f77dcbb507bc5085f86
eca50854e1 depends: static libxcb_cursor (fanquake)
Pull request description:
Remove the runtime requirement of `libxcb-cursor`. This library is no-longer present on modern Ubuntu.
Fixes#33432.
Also related to #32097.
ACKs for top commit:
davidgumberg:
Addendum ACK eca50854e1
willcl-ark:
Code review ACK eca50854e1
Tree-SHA512: d545a03baf5030de64874b79add87b6ef5f95eb5ca31aa66007ee03554103d2eda5e56dfd4395d0a12e24b2e489457e4f19ed9e6d390351c72a0da630f03cc42
By using the pskip pointer, which regularly allows jumping back much faster
than pprev, the forking point between two CBlockIndex entries can be found
much faster.
A simulation shows that no more than 136 steps are needed to jump anywhere
within the first 2^20 block heights, and on average 65 jumps for uniform
forking points around that height.
The other executables have manifests and these should be checked in
addition to bitcoind. Skipping fuzz.exe, bench_bitcoin.exe and
test_bitcoin-qt.exe as they do not have manifests.
This sets up a vs developer command prompt and should hopefully should
be more resilient to upstream changes
Co-authored-by: David Gumberg <davidzgumberg@gmail.com>
06df14ba75 test: add more TRUC reorg coverge (Greg Sanders)
26e71c237d Mempool: Do not enforce TRUC checks on reorg (Greg Sanders)
bbe8e9063c fuzz: don't bypass_limits for most mempool harnesses (Greg Sanders)
Pull request description:
This was the intended behavior but our tests didn't cover the scenario where in-block transactions themselves violate TRUC topological constraints.
The behavior in master will potentially lead to many erroneous evictions during a reorg, where evicted TRUC packages may be very high feerate and make sense to mine all together in the next block and are well within the normal anti-DoS chain limits.
This issue exists since the merge of https://github.com/bitcoin/bitcoin/pull/28948/files#diff-97c3a52bc5fad452d82670a7fd291800bae20c7bc35bb82686c2c0a4ea7b5b98R956
ACKs for top commit:
sdaftuar:
ACK 06df14ba75
glozow:
ACK 06df14ba75
ismaelsadeeq:
Code review ACK 06df14ba75
Tree-SHA512: bdb6e4dd622ed8b0b11866263fff559fcca6e0ca1c56a884cca9ac4572f0026528a63a9f4c8a0660df2f5efe0766310a30e5df1d6c560f31e4324ea5d4b3c1a8
Use the {Arg,MaybeArg}<std::string_view> helper in all places where
it is a trivial change. In many places, this simplifies the logic
and reduces duplication of default values.
Update select functions that take a const std::string& to take a
std::string_view instead. In a next commit, this allows us to use
the {Arg,MaybeArg}<std::string_view> helper.
Modernizes interface by not forcing users to deal with raw pointers,
without adding copying overhead. Generalizes the logic of whether
we return by value or by optional/pointer.
In cases where functions take a `const std::string&` and it would
be too much work to update them, a string copy is made (which was
already happening anyway).
Normally `ConnectNode()` would choose whether to use a proxy and which
one. Make it possible to override this from the callers and same for
`OpenNetworkConnection()` - pass down the proxy to `ConnectNode()`.
Document both functions.
This is useful if we want to open connections to IPv4 or IPv6 peers
through the Tor SOCKS5 proxy.
Also have `OpenNetworkConnection()` return whether the connection
succeeded or not. This can be used when the caller needs to keep track
of how many (successful) connections were opened.
When the assumevalid ancestry check fails, log a precise reason:
- "block height above assumevalid height" if the block is above the assumevalid block (the default reason)
- "block not in of assumevalid chain" otherwise
The new split was added under the existing condition to simplify conceptually that the two cases are related.
It could still be useful to know when the block is just above the assumevalid block or when it's not even on the same chain.
Update the functional test to assert the new reason strings. No behavior change.
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
Flatten nested conditionals into a linear gating sequence for readability and precise logging. No functional change, TODOs are addressed in next commit
Replaced `atomic<bool>` with `std::optional<bool>` (logged once on first observation). Safe because `ConnectBlock` holds `cs_main`.\
After this change, the state is logged before the very first `UpdateTip` line.
Co-authored-by: Eunovo <eunovo9@gmail.com>
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
Co-authored-by: w0xlt <woltx@protonmail.com>
Increase the test to 6 nodes and add flows for baseline, deep anchor, and too-recent cases, plus scaffolding for off-best-header, not-in-assumevalid,
and reindex gates.
Assertions are minimal here; follow-ups add reason checks.
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
87e7f37918 doc: clarify peer address in getpeerinfo and addnode RPC help (Vasil Dimov)
2a4450ccbb net: change FindNode() to not return a node and rename it (Vasil Dimov)
4268abae1a net: avoid recursive m_nodes_mutex lock in DisconnectNode() (Vasil Dimov)
3a4d1a25cf net: merge AlreadyConnectedToAddress() and FindNode(CNetAddr) (Vasil Dimov)
Pull request description:
`CConnman::FindNode()` would lock `m_nodes_mutex`, find the node in `m_nodes`, release the mutex and return the node. The current code is safe but it is a dangerous interface where a caller may end up using the node returned from `FindNode()` without owning `m_nodes_mutex` and without having that node's reference count incremented.
Change `FindNode()` to return a boolean since all but one of its callers used its return value to check whether a node exists and did not do anything else with the return value.
Remove a recursive lock on `m_nodes_mutex`.
Rename `FindNode()` to better describe what it does.
ACKs for top commit:
achow101:
ACK 87e7f37918
furszy:
Code review ACK 87e7f37918
hodlinator:
re-ACK 87e7f37918
Tree-SHA512: 44fb64cd1226eca124ed1f447b4a1ebc42cc5c9e8561fc91949bbeaeaa7fa16fcfd664e85ce142e5abe62cb64197c178ca4ca93b3b3217b913e3c498d0b7d1c9
Signal m_tip_block_cv when Ctrl-C is pressed or SIGTERM is received, the same
way it is currently signalled when the `stop` RPC is called. This lets RPC
calls like `waitforblockheight` and IPC calls like `waitTipChanged` be
interrupted, instead of waiting for their original timeouts and delaying
shutdown.
Historical notes:
- The behavior where `stop` RPC signals `m_tip_block_cv`, but CTRL-C does not,
has been around since the condition variable was introduced in #30409
(7eccdaf160).
- The signaling was later moved without changing behavior in #30967
(5ca28ef28b). This commit moves it again to
the Interrupt() function, which is probably the place it should have been
added initially, so it works for Ctrl-C shutdowns as well as `stop`
shutdowns.
- A Qt shutdown bug calling wait methods was fixed previously in #18452
(da73f1513a), and this change updates that
fix to avoid the hang happening again in Qt.
The returned value in `getpeerinfo/addr` could be a hostname as well as
an IP address and the `:port` part could be missing. It is displayed
from `CNode::m_addr_name` which could have been set from RPC `addnode`
where the argument is allowed to be a hostname and an optional port.
All callers of `CConnman::FindNode()` use its return value `CNode*` only
as a boolean null/notnull. So change that method to return `bool`.
This removes the dangerous pattern of handling a `CNode` object (the
return value of `FindNode()`) without holding `CConnman::m_nodes_mutex`
and without having that object's reference count incremented for the
duration of the usage.
Also rename the method to better describe what it does.
Have `CConnman::DisconnectNode()` iterate `m_nodes` itself instead of
using `FindNode()`. This avoids recursive mutex lock and drops the only
caller of `FindNode()` which used the return value for something else
than a boolean found/notfound.
50194029e7 ci: Remove bash -c from cmake invocation using eval (Brandon Odiwuor)
Pull request description:
Follow up to https://github.com/bitcoin/bitcoin/pull/32970https://github.com/bitcoin/bitcoin/pull/32970#r2213730157
> Does `cmake -S ...` still need to be wrapped in `bash -c "..."`?
https://github.com/bitcoin/bitcoin/pull/32970#r2213741192
> It is not trivial to replace. Maybe the `eval` hack from below can be used:
>
> ```shell
> # parses TEST_RUNNER_EXTRA as an array which allows for multiple arguments such as TEST_RUNNER_EXTRA='--exclude "rpc_bind.py --ipv6"'
>
> eval "TEST_RUNNER_EXTRA=($TEST_RUNNER_EXTRA)"
> ```
>however, I haven't tried this yet.
https://github.com/bitcoin/bitcoin/pull/32970#r2213801696
> Yeah, the eval hack should work:
>
> ```
> $ export T="-DREDUCE_EXPORTS=ON -DCMAKE_CXX_FLAGS='-Wno-psabi -Wno-error=maybe-uninitialized'"; eval "T=($T)"; for i in "${T[@]}"; do echo "_${i}_" ; done
> _-DREDUCE_EXPORTS=ON_
> _-DCMAKE_CXX_FLAGS=-Wno-psabi -Wno-error=maybe-uninitialized_
> ```
>
> (can be done in a follow-up)
This replaces the `bash -c` wrapper with an eval-based array parsing to preserve spaces in flag values (e.g., in CMAKE_CXX_FLAGS), allowing ShellCheck to lint the cmake command
ACKs for top commit:
maflcko:
lgtm ACK 50194029e7
hebasto:
ACK 50194029e7.
Tree-SHA512: 6fd22569e2c719a8d13805f18e1e7e3b8eb57d0a6307f2e7175988b25750eafb7c8260796c8e7350db67d622dbe97e6af7bab8ee52187bb8e8eeae3740a47c01
0802398e74 fuzz: make it possible to mock (fuzz) CThreadInterrupt (Vasil Dimov)
6d9e5d130d fuzz: add CConnman::SocketHandler() to the tests (Vasil Dimov)
3265df63a4 fuzz: add CConnman::InitBinds() to the tests (Vasil Dimov)
91cbf4dbd8 fuzz: add CConnman::CreateNodeFromAcceptedSocket() to the tests (Vasil Dimov)
50da7432ec fuzz: add CConnman::OpenNetworkConnection() to the tests (Vasil Dimov)
e6a917c8f8 fuzz: add Fuzzed NetEventsInterface and use it in connman tests (Vasil Dimov)
e883b37768 fuzz: set the output argument of FuzzedSock::Accept() (Vasil Dimov)
Pull request description:
Extend `CConnman` fuzz tests to also exercise the methods `OpenNetworkConnection()`, `CreateNodeFromAcceptedSocket()`, `InitBinds()` and `SocketHandler()`.
Previously fuzzing those methods would have resulted in real socket functions being called in the operating system which is undesirable during fuzzing. Now that https://github.com/bitcoin/bitcoin/pull/21878 is complete all those are mocked to a fuzzed socket and a fuzzed DNS resolver (see how `CreateSock` and `g_dns_lookup` are replaced in the first commit).
ACKs for top commit:
achow101:
ACK 0802398e74
jonatack:
Review re-ACK 0802398e74
dergoegge:
Code review ACK 0802398e74
Tree-SHA512: a717d4e79f42bacf2b029c821fdc265e10e4e5c41af77cd4cb452cc5720ec83c62789d5b3dfafd39a22cc8c0500b18169aa7864d497dded729a32ab863dd6c4d
451ba9ada4 datacarrier: Undeprecate configuration option (Anthony Towns)
Pull request description:
Removes the deprecation for the `datacarrier` and `datacarriersize` options by reverting commit 0b4048c733 from https://github.com/bitcoin/bitcoin/pull/32406
**Many current Bitcoin Core users want to continue using this option**
This statement is based on public postings from many Bitcoin Core users and not a formal survey. AJ Towns’ observation from [#32406](0b4048c733 (r2084024874)) that “_for now there seem to be a bunch of users who like the option_” has only become more apparent in the months since.
**The deprecation intent is unclear to users**
This echo’s Ava Chow’s comment from #32714 that “_IMO we should not have removal warnings if there is no current plan to actually remove them._” In months since that comment, partially due to increased feedback from Bitcoin Core users wanting to keep this option, there is even less likelihood of a near term plan to remove these options. That leaves Bitcoin Core users in an unclear situation: the option could be removed in the next version or perhaps never. Removing the deprecation gives clarity for their planning purposes. Deprecating the option in the future, preferably with a removal schedule to better inform users, would still be possible.
**Minimal downsides to removing deprecation**
As a best practice, Bitcoin Core has avoided an option when the developers cannot articulate when they should be used. There is non-zero maintenance cost to keeping this code around (although leaving the options deprecated for a long time has the same effect). “Don’t offer users footguns” is also a good principle, but with this option, there seems to be only small impacts that can quickly be remedied by changing the option value by Bitcoin Core users. There already exist in Bitcoin Core more potentially-user-harmful options/values than what datacarrier might cause.
ACKs for top commit:
ajtowns:
ACK 451ba9ada4
darosior:
That said, certain users care strongly about using those options. In these conditions, i do not see the project removing the option anytime soon. Therefore i think it's technically incorrect (and confusing) to mark it as deprecated. utACK 451ba9ada4 on removing the deprecation.
instagibbs:
crACK 451ba9ada4
Raimo33:
ACK 451ba9ada4
Ademan:
utACK 451ba9a
ryanofsky:
Code review ACK 451ba9ada4
marcofleon:
ACK 451ba9ada4
achow101:
ACK 451ba9ada4
moonsettler:
ACK 451ba9ada4
ismaelsadeeq:
utACK 451ba9ada4🛰️
jonatack:
ACK 451ba9ada4
Zero-1729:
crACK 451ba9ada4
vasild:
ACK 451ba9ada4
Tree-SHA512: b83fc509f5dd820976596e1ae9fb69a22ada567e0e0ac88da5fc5e940a46d8894b40cc70c3eff2cbdabd4da5ec913f0d18c1632fc906f210b308855868410699
fabc2615af test: Use extra_port() helper in feature_bind_extra.py (MarcoFalke)
Pull request description:
This is a refactor for self-validating and self-documenting code.
Currently, the test assumes that extra ports are available and just increments them without checking. However, this may not be the case when the test is modified to use more ports. In this case, the tests may fail intermittently and the failure is hard to debug.
Fix this confusion, by calling `p2p_port` each time. This ensures the required `assert n <= MAX_NODES` is checked each time.
Closes https://github.com/bitcoin/bitcoin/issues/33250
ACKs for top commit:
achow101:
ACK fabc2615af
janb84:
crACK fabc2615af
w0xlt:
ACK fabc2615af
Tree-SHA512: 1eff00be7f43104ae8a66e79fbf64075ec22bb20f392ac1e4c8a7dd694d4f1760aa44ea54ab7b1f2b947ab018851ab3c10d3c717714c0bee4d8d24617594c2bb
By using mocktime, we will always hit both the notfound
branch and the tx sent branch.
The previous version didn't achieve that due to timing
issues.
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
With a low but not negligible probability in the order
of 10^-6 the exponential timer NextInvToInBounds can lead
to an interval >60s, making the test fail.
Also uses mocktime to speed up the test and fixes a
non-matching on_inv override.
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Currently when CTRL-C is pressed and there is an active `waitforblockheight`,
or `waitforblock`, or `waitfornewblock` RPC call, or a mining interface
`waitTipChanged` IPC call with a long timeout, the node will not shut down
right away, and will wait for the timeout to be reached before exiting.
This behavior is not ideal and only happens when the node is stopped with
CTRL-C or SIGTERM. When the node is stopped with `bitcoin-cli stop`, the wait
calls are interrupted and the node does shut down right away.
The next commit improves node behavior. This commit just adds test coverage to
simplify the next commit and clarify the change in behavior there.
When creating Taproot signatures, if the key being signed for is known
to be a MuSig2 aggregate key, do the MuSig2 signing algorithms.
First try to create the aggregate signature. This will fail if there are
not enough partial signatures or public nonces. If it does fail, try to
create a partial signature with all participant keys. This will fail for
those keys that we do not have the private keys for, and if there are
not enough public nonces. Lastly, if the partial signatures could not be
created, add our own public nonces for the private keys that we know, if
they do not yet exist.
A common pattern that MuSig2 functions will use is to aggregate the
pubkeys to get the keyagg_cache and then validate the aggregated pubkey
against a provided aggregate pubkey. A variant of MuSig2AggregatePubkeys
is added which does that.
The functionality of GetMuSig2KeyAggCache and GetCPubKeyFromMuSig2KeyAggCache
are included in MuSig2AggregatePubkeys (and used internally) so there is
no expectation that callers will need these so they are made static.
Adds GetMuSig2SecNonces which returns secp256k1_musig_secnonce*, and
DeleteMuSig2Session which removes the MuSig2 secnonce from wherever it
was retrieved. FlatSigningProvider stores it as a pointer to a map of
session id to secnonce so that deletion will actually delete from the
object that actually owns the secnonces.
The session id is just a unique identifier for the caller to determine
what secnonces have been created.
fc861332b3 wallet, log: reduce unconditional logging during load (furszy)
Pull request description:
Currently the unconditional log during init with a default wallet happens three times:
```
2025-09-03T19:57:16Z init message: Verifying wallet(s)…
2025-09-03T19:57:16Z Using SQLite Version 3.45.1
2025-09-03T19:57:16Z Using wallet XXX/.bitcoin/regtest
2025-09-03T19:57:16Z Using SQLite Version 3.45.1
2025-09-03T19:57:16Z Using wallet XXX/.bitcoin/regtest
(...)
2025-09-03T19:57:16Z Using SQLite Version 3.45.1
2025-09-03T19:57:16Z Using wallet XXX/.bitcoin/regtest
2025-09-03T19:57:16Z init message: Loading wallet…
```
For non-default wallets it's logged two times.
That seems a bit too much, so just log the SQLite version just one, and remove the log for the full path of the wallet, since it's already clear from other logs which wallet is being loaded.
ACKs for top commit:
achow101:
ACK fc861332b3
furszy:
utACK fc861332b3
stickies-v:
ACK fc861332b3
Tree-SHA512: ca45c8ede985e6feab0cb93d718a6d633691276ca6e5f13f6471759f11dee98b312e1c802a7fb42c7fa859b6edc44a8c54b9e2ca389655cf028aebf2dabe51f6
Currently nodes schedule their invs to all inbound peers at the same time.
It is trivial to make use this timing pattern for fingerprinting
identities on different networks. Using a separate timers for each network will
make the fingerprinting harder.
The https://fukuchi.org/ homepage no longer links to the source tarball,
and previously available files appear to have been removed. The homepage
now instructs users to download source tarballs from the GitHub releases
page instead.
The diff between the source trees is immaterial.
When using `docker buildx build` in conjunction with the `gha` backend
cache type, it's important to specify the URL and TOKEN needed to
authenticate.
On Cirrus runners this is working with only `ACTIONS_CACHE_URL` and
`ACTIONS_RUNTIME_TOKEN`, but this is not enough for the GitHub backend.
Fix this by exporting all `ACTIONS_*` variables.
This fixes cache restore/save on forks or where GH-hosted runners are
being used.
d4f47f9771 ci: use latest versions of lint deps (fanquake)
Pull request description:
Some of the versions used here are > 2 years old. i.e `mypy`. Use the latest avilable versions, except for LIEF, which is generally changed with Guix.
Side note. I can't remember the last time one of these tools (mypy, ruff, vulture) actually caught an issue in the lint job.
ACKs for top commit:
maflcko:
lgtm ACK d4f47f9771
janb84:
lgtm ACK d4f47f9771
hebasto:
ACK d4f47f9771, I have reviewed the code and it looks OK.
Tree-SHA512: 8b312535c9fea8e76d58f517ada6d6ea7a119c5e03c8cb84a41b5b6ca80dfaaff65a81478bdc1a5acf734cfb0bc66a8b3ba5400db8973c43ca913b07568abfe4
Not enforcing TRUC topology on reorg was the intended
behavior, but the appropriate bypass argument was not
checked.
This mistake means we could potentially invalidate a long
chain of perfectly incentive-compatible transactions that
were made historically, including subsequent non-TRUC
transactions, all of which may have been very high feerate.
Lastly, it wastes CPU cycles doing topology checks since
this behavior cannot actually enforce the topology in
general for the reorg setting.
Using bypass_limits=true is essentially fuzzing part of a
reorg only, and results in TRUC invariants unable to be
checked. Remove most instances of bypassing limits, leaving
one harness able to do so.
The removed statements were logged up to two or three times for each loaded
wallet. The SQLite version only needs to be logged once.
The full wallet path is dropped, since the existing unconditional
logging while loading wallets is sufficient (also reduces anonymization
efforts in case of sharing logs).
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
`CConnman::AlreadyConnectedToAddress()` is the only caller of
`CConnman::FindNode(CNetAddr)`, so merge the two in one function.
The unit test that checked whether `AlreadyConnectedToAddress()` ignores
the port is now unnecessary because now the function takes a `CNetAddr`
argument. It has no access to the port.
ff05bebcc4 doc: rpc: fix case typo in `finalizepsbt` help (final_scriptwitness) (Sebastian Falbesoner)
Pull request description:
The lower-case spelling matches the `decodepsbt` result field:
200150beba/src/rpc/rawtransaction.cpp (L871)200150beba/src/rpc/rawtransaction.cpp (L1253)
ACKs for top commit:
l0rinc:
ACK ff05bebcc4
rkrux:
Ah crACK ff05bebcc4
Tree-SHA512: c0a0e29e95fed3fcee4df4f3fc87b32774d76bebadcda5aa010bc45142727536d6a71e4c0e70564db8bdb734e8647c80953793ac9ecd6c434345e972f8d9b7b0
Depending on the host machine, a default `par` value can spawn up to 15 script verification threads for each node.
Running the functional test suite with default `par` can exhaust file descriptors or hit other resource limits when many threads are spawned.
These threads are mostly idle and the same code paths are executed with a value of `par=2`.
Limit this to 2 for functional tests that do not override the default option.
Co-authored-by: maflcko <6399679+maflcko@users.noreply.github.com>
Only leaves messages we never found in the final assert message of the functions, which is more helpful (remaining_expected).
Avoids repeatedly searching for messages we have already found (pop()).
Stops searching for other expected messages if we already failed finding one. Still need to clean remaining_expected at the end, but *only if we fail*.
Co-authored-by: Lőrinc <pap.lorinc@gmail.com>
print_log was recalculated every 0.05s in assert_debug_log(), even during successful circumstances - changed to only be computed upon failure.
Simplified terminology from "(does not) partially match(es)" to "(not) found in" since it seems to reference the first function having used regular expression matching, while it always escaped the search strings (see parent commit). (Simplified grammar also avoids issues with singular/plural "was/were not found").
75e6984ec8 test/refactor: use test deque to avoid quadratic iteration (Lőrinc)
Pull request description:
Extracted from https://github.com/bitcoin/bitcoin/pull/33141#discussion_r2323012972.
-----
In Python, [list `pop(0)` is linear](https://docs.python.org/3/tutorial/datastructures.html#using-lists-as-queues), so consuming all items in the test results in quadratic iteration.
Switching to `collections.deque` with `popleft()` expresses FIFO intent and avoids the O(n^2) path.
Behavior is unchanged - for a few hundred items the perf impact is likely negligible.
ACKs for top commit:
maflcko:
lgtm ACK 75e6984ec8
theStack:
re-ACK 75e6984ec8
enirox001:
reACK 75e6984
w0xlt:
reACK 75e6984ec8
Tree-SHA512: 290f6aeeb33d8b12b7acbbfede7ce0bef1c831a7ab9efc9c3a08c049986572e289cdece0844db908cf198395f574575ce4073c268033bf6dbaadc3828c96c1d8
1ff9e92948 key: use static context for libsecp256k1 calls where applicable (Sebastian Falbesoner)
Pull request description:
The dynamically created [signing context](2d6a0c4649/src/key.cpp (L19)) for libsecp256k1 calls is only needed for functions that involve generator point multiplication with a secret key, i.e. different variants of public key creation and signing. The API docs hint to those by stating "[(not secp256k1_context_static)](b475654302/include/secp256k1.h (L645))" for the context parameter. In our case that applies to the following calls:
- `secp256k1_ec_pubkey_create`
- `secp256k1_keypair_create`
- `secp256k1_ellswift_create`
- `secp256k1_ecdsa_sign`
- `secp256k1_ecdsa_sign_recoverable`
- `secp256k1_schnorrsig_sign32`
- `ec_seckey_export_der` (not a direct secp256k1 function, but calls `secp256k1_ec_pubkey_create` inside)
For all the other secp256k1 calls we can simply use the static context. This is done for consistency to other calls that already use `secp256k1_context_static`, and also to reduce dependencies on the global signing context variable. Looked closer at this in the course of reviewing #29675, where some functions used the signing context that didn't need to, avoiding a move to another module (see https://github.com/bitcoin/bitcoin/pull/29675#discussion_r2333831377).
ACKs for top commit:
Eunovo:
ACK 1ff9e92948
furszy:
ACK 1ff9e92948
rkrux:
crACK 1ff9e92948
Tree-SHA512: f091efa56c358057828f3455d4ca9ce40ec0d35f3e38ab147fe3928bb5dbf7ffbc27dbf97b71937828ab95ea4e9be5f96d89a2d29e2aa18df4542aae1b33e258
QT translations are optional, but the script would error when
'translations_dir' falls back to its default value NULL.
This PR fixes it by moving the set-up of QT translations under
the check for 'translations_dir' presence.
316a0c5132 rpc: addpeeraddress: throw on invalid IP (John Moffett)
Pull request description:
Right now we return an opaque `{"success" : false}` in `addpeeraddress` for an empty or invalid IP. This changes it to throw `RPC_CLIENT_INVALID_IP_OR_SUBNET` with the error message `Invalid IP address`. Tests updated to match.
ACKs for top commit:
sipa:
utACK 316a0c5132
achow101:
ACK 316a0c5132
vasild:
ACK 316a0c5132
pablomartin4btc:
tACK 316a0c5132
Tree-SHA512: 79a8ce127d0a24b2eb1f31bc3294b895d0c6424032a6b49168259e0e94aff69723d067adf1b4dc3c9b79e597531e5b65e4b8fc5a8e21fba0b81f99168de12b96
453b0fa286 bitcoin: Make wrapper not require -m (Ryan Ofsky)
29e836fae6 test: add tool_bitcoin to test bitcoin wrapper behavior (Ryan Ofsky)
0972f55040 init: add exe name to bitcoind, bitcoin-node -version output to be able to distinguish these in tests (Ryan Ofsky)
Pull request description:
This change makes the `bitcoin` command respect IPC command line options and _bitcoin.conf_ settings, so IPC listening can be enabled by just running `bitcoin node -ipcbind=unix` or `bitcoin node` with `ipcbind=unix` in the configuration file, and there is no longer a need to specify a multiprocess `-m` option like `bitcoin -m node [...]`
sipa and theuni in #31802 pointed out that users shouldn't be exposed to multiprocess implementation details just to use IPC features, so current need to specify the `bitcoin -m` option in conjunction with `-ipcbind` could be seen as a design mistake and not just a usage inconvenience.
This PR also adds a dedicated functional test for the `bitcoin` wrapper command and to make sure it calls the right binaries and test the new functionality.
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).
ACKs for top commit:
Sjors:
re-ACK 453b0fa286
achow101:
ACK 453b0fa286
TheCharlatan:
Re-ACK 453b0fa286
Tree-SHA512: 9e49cb7e183fd220fa7a4e8ac68cef55f3cb2ccec40ad2a9d3e3f31db64c4953db8337f8caf7fce877bc97002ae97568dcf47ee269a06ca1f503f119bfe392c1
df67bb6fd8 test: Remove convert_to_json_for_cli (Ava Chow)
44a493e150 cli: Allow arguments to be both strings and json (Ava Chow)
Pull request description:
There are some RPCs where the argument can be either JSON that needs to be parsed, or a string that we can pass straight through. However, `bitcoin-cli` would always parse those arguments as JSON which makes for some cumbersome argument passing when using those RPCs. Notably, `hash_or_height` in `getblockstats` and `gettxoutsetinfo` do this, and results in a more cumbersome command of `bitcoin-cli getblockstats '"<hash>"'`. Otherwise, using a normal invocation of `bitcoin-cli getblockstats <hash>` results in `error: Error parsing JSON`. This PR marks those particular options as also being a string so that when `bitcoin-cli` fails to parse the argument as JSON, it will assume that the argument is a string and pass it straight through.
ACKs for top commit:
ryanofsky:
Code review ACK df67bb6fd8, just rebased since last review. I do still think it would be good to improve the test (https://github.com/bitcoin/bitcoin/pull/33230#discussion_r2369570345)
rkrux:
Light code review, lgtm ACK df67bb6fd8
mzumsande:
Code Review ACK df67bb6fd8
Tree-SHA512: 6c488570fbb24d0cf10508416c56accfc7af5163b7a7187d22d78c812424a9e3ecc95906d3e295fbf6af54bf80903aa448fd879dd6a9944ba8b4d1a33eb29ef2
We can use vswhere.exe directly to create a vs developer
prompt and so can remove this third party dependency.
Co-authored-by: David Gumberg <davidzgumberg@gmail.com>
b807dfcdc5 miner: fix `addPackageTxs` unsigned integer overflow (ismaelsadeeq)
Pull request description:
This PR fixes an unsigned integer overflow in the `addPackageTxs` method of the `BlockAssembler`.
The overflow is a rare edge case that might occur on master when a miner reserves 2000 WU and wants to create an block to be empty.
i.e, by starting with `-blockmaxweight=2000`, `-blockreservedweight=2000`, or just `blockmaxweight=2000`, and then calling the mining interface `createNewBlock` with `blockReservedWeight` set to `2000`.
Instead of bailing out after going through transactions equivalent to `MAX_CONSECUTIVE_FAILURES`, the loop never breaks until all mempool transactions are visited.
See https://github.com/bitcoin/bitcoin/pull/33421#issuecomment-3324859282
The fix avoids the overflow by using addition instead adding `BLOCK_FULL_ENOUGH_WEIGHT_DELTA` to the block weight and comparing it with `m_options.nBlockMaxWeight`.
Another alternative that preserves the same structure is to use `static_cast`. See c9530cf35d.
This fix can be tested by cherry-picking the commits from #33421 without the static cast fix and running:
```bash
echo "AQAAAAAAA
AAnJycnAAAAAAAAAAAAAAAAAA" | base64 --decode > miner.crash
FUZZ=block_template_cache ./build_fuzz/bin/fuzz miner.crash
```
---
This is part of a larger inconsistency in how size/weight is represented in the codebase. It may be worth defining a dedicated type for size/weight.
ACKs for top commit:
glozow:
nice, utACK b807dfcdc5
furszy:
Code ACK b807dfcdc5
Tree-SHA512: c1d2f7e500f9b0624a4c22a146921a1644017065e6c94d0c5027486392321f5de26c61751a24765e025e45b34c535adfd6d0e2ac809dea6846b99f37d13043c9
After an incomplete reindex the blocks will need to be replayed.
This results in excessive `Rolling back` and `Rolling forward` messages which quickly triggers the recently introduced log rate limiter.
Change the logging strategy to:
- Add single `LogInfo` messages showing the full range being replayed for both rollback and roll forward;
- Log progress at `LogInfo` level only every 10,000 blocks to track the long operations.
Reproducer:
* Start a normal IBD, stop after some progress
* Do a reindex, stop before it finishes
* Restart the node normally without specifying the reindex parameter
It should start rolling the blocks forward.
Before this change the excessive logging would show:
```
[*] Rolling forward 000000002f4f55aecfccc911076dc3f73ac0288c83dc1d79db0a026441031d40 (46245)
[*] Rolling forward 0000000017ffcf34c8eac010c529670ba6745ea59cf1edf7b820928e3b40acf6 (46246)
```
After the change it shows:
```
Replaying blocks
Rolling forward to 00000000000000001034012d7e4facaf16ca747ea94b8ea66743086cfe298ef8 (326223 to 340991)
Rolling forward 00000000000000000faabab19f17c0178c754dbed023e6c871dcaf74159c5f02 (330000)
Rolling forward 00000000000000000d9b2508615d569e18f00c034d71474fc44a43af8d4a5003 (340000)
...
Rolled forward to 00000000000000001034012d7e4facaf16ca747ea94b8ea66743086cfe298ef8
```
(similarly to rolling back)
Co-authored-by: Anthony Towns <aj@erisian.com.au>
Co-authored-by: Vasil Dimov <vd@freebsd.org>
bf7996cbc3 rpc: fix getblock(header) returns target for tip (Sjors Provoost)
4c3c1f42cf test: add block 2016 to mock mainnet (Sjors Provoost)
Pull request description:
A `target` field was added to the `getblock` and `getblockheader` RPC calls in #31583, but it mistakingly always used the tip value.
This PR fixes it to return the target for the given block. Because regtest does not have difficulty adjustment, the mainnet test is expanded to cover the fix.
A preliminary commit deals with mining block 2016 that's needed for the test. It also:
- renames the `create_coinbase` `retarget_period` argument to `halving_period`. Before #31583 this was hardcoded for regtest where these values are the same.
- drops unused `fees` argument from `mine` helper
- expands the CPU miner instructions for generating the alternative mainnet chain
Fixes#33440
ACKs for top commit:
sipa:
utACK bf7996cbc3
luke-jr:
crACK bf7996cbc3
TheCharlatan:
ACK bf7996cbc3
ismaelsadeeq:
Code review ACK bf7996cbc3
Tree-SHA512: 2a2e11efd91f4aaccf9d2ec4dff9fd82c366b8a7e797ce5981dca2e6f08028f69154f4e6a27aef20d78b0e6c3304416789267c2fad42d7aa5072f8537d0c8b0d
8e434a8499 macdeploy: rename macOS output to bitcoin-macos-app.zip (fanquake)
05353d9cf0 macdeploy: combine appname & -zip arguments (fanquake)
Pull request description:
Output `bitcoin-macos-app.zip`, similar to what we do for Windows: `bitcoin-win64-setup.exe`.
ACKs for top commit:
hodlinator:
re-ACK 8e434a8499
willcl-ark:
ACK 8e434a8499
Tree-SHA512: e762c9866630c4f8c577027ee9492d74a5c7f4b194df73876d702703b9100c356a30986c2f209ba3f3e2d483017f5e61596a2a7cdfae0a684f8dc244420cd108
ef20c2d11d build, msvc: Update vcpkg manifest baseline (Hennadii Stepanov)
Pull request description:
This PR updates the vcpkg manifest baseline from the ["2025.03.19 Release"](https://github.com/microsoft/vcpkg/releases/tag/2025.03.19) to the ["2025.08.27 Release"](https://github.com/microsoft/vcpkg/releases/tag/2025.08.27), with the following package
changes:
- `boost`: 1.87.0 --> 1.88.0
- `qtbase`: 6.8.2#1 -> 6.9.1
- `qttools`: 6.8.2 -> 6.9.1
- `sqlite3`: 3.49.1 --> 3.50.4
The previous update was made in https://github.com/bitcoin/bitcoin/pull/32213.
ACKs for top commit:
hodlinator:
ACK ef20c2d11d
Tree-SHA512: 3c95fea911e1481b3536958d83dcaa52012abdff350cd08c21b30b3df61a501b2f3272e879882820bb59456066e9270de820bcb47810d3d1b8e8a1267d987d90
88b0647f02 wallet: Always write last hardened cache flag in migrated wallets (Ava Chow)
8a08eef645 tests: Check that the last hardened cache upgrade occurs (Ava Chow)
Pull request description:
#32597 set the descriptor cache upgraded flag for newly created wallets, but migrated wallets still did not have the flag set when they are migrated. For consistency, and to avoid an unnecessary upgrade, we should be setting this flag for migrated wallets.
The flag would end up being set anyways at the end of migration when the wallet is reloaded as it would perform the automatic upgrade at that time. However, this is unnecessary and we should just set it from the get go.
This PR also adds a couple tests to verify that the flag is being set, and that the upgrade is being performed.
ACKs for top commit:
cedwies:
re-ACK 88b0647
rkrux:
lgtm ACK 88b0647f02
pablomartin4btc:
ACK 88b0647f02
Tree-SHA512: 7d0850db0ae38eedd1e6a3bfaa548c6c612182291059fb1a47279a4c4984ee7914ecd02d8c7e427ef67bf9f5e67cbc57a7ae4412fad539e1bf3e05c512a60d69
2427939935 test: forbid copying of DebugLogHelper (Daniel Pfeifer)
d6aa266d43 test: don't throw from the destructor of DebugLogHelper (Vasil Dimov)
Pull request description:
Throwing an exception from the destructor of a class is a bad practice because the destructor will be called when an object of that type is alive on the stack and another exception is thrown, which will result in "exception during the exception". This would terminate the program without any messages.
Instead print the message to the standard error output and call `std::abort()`.
---
This change is part of https://github.com/bitcoin/bitcoin/pull/26812. It is an improvement on its own, so creating a separate PR for it following the discussion at https://github.com/bitcoin/bitcoin/pull/32604#discussion_r2345091587. Getting it in will reduce the size of #26812.
ACKs for top commit:
Crypt-iQ:
crACK 2427939
l0rinc:
Code review reACK 2427939935
optout21:
crACK 2427939935
furszy:
utACK 2427939935
Tree-SHA512: 918c1e40d2db4ded6213cd78a18490ad10a9f43c0533df64bdf09f0b216715415030e444712981e4407c32ebf552fbb0e3cce718e048df10c2b8937caf015564
The generic key can also be used in other places
where behavior between different network identities should
be uncorrelated to avoid fingerprinting.
This also changes RANDOMIZER_ID - since it is not
being persisted to disk, there are no compatibility issues.
2738b63e02 test: validate behaviour of getpeerinfo last_inv_sequence and inv_to_send (Anthony Towns)
77b2ebb811 rpc/net: report per-peer last_inv_sequence (Anthony Towns)
adefb51c54 rpc/net: add per-peer inv_to_send sizes (Anthony Towns)
Pull request description:
Adds per-peer entries to `getpeerinfo` for the size of the inv_to_send queue and the mempool sequence number as at the last INV. Can be helpful for debugging tx relay performance and privacy/fingerprinting issues.
ACKs for top commit:
sipa:
utACK 2738b63e02
instagibbs:
ACK 2738b63e02
Tree-SHA512: e3c9c52e8e38b099d405a177ffba6783c5821cc5ce1432b98218843e00906986ce2141dcd5b04a67006c328211a672e519fa3390e012688499bfc9ac99767599
b77137a564 ci: link against -lstdc++ in native fuzz with msan job (fanquake)
Pull request description:
Remove the Clang build from msan fuzz by using the apt install LLVM / Clang, and just linking against `-lstdc++`.
ACKs for top commit:
maflcko:
lgtm ACK b77137a564
Tree-SHA512: dc32b22a93196120a343d91265db3f42f6dc00afc887929986987ea62f2513580c855e98d088f037adb4c2e62358f98e47b914a412ef9c1069037917a36c0b03
cad9a7fd73 rpc: Always return per-wtxid entries in submitpackage tx-results (John Moffett)
Pull request description:
Follow-up to #28848
When `submitpackage` produced no per-transaction result for a member, the RPC set `"error": "unevaluated"` but then continued without inserting the entry into `tx-results`, making it impossible for callers to know which `wtxids` were unevaluated.
This inserts the error result before continuing, updates help text, and adjusts functional tests to expect entries for all submitted `wtxids`.
ACKs for top commit:
instagibbs:
ACK cad9a7fd73
glozow:
ACK cad9a7fd73
Tree-SHA512: 8df5c9b3d1f17aaf0311c38f028ae4b55d4c52a660f85171f105c4f65d130b14ab00698ac5d7c27403a0c37fff391c154c3ad44cc99ba4d549d9c30751b8360f
fbde8d9a81 doc: remove unrelated `bitcoin-wallet` binary from `libbitcoin_ipc` description (Sebastian Falbesoner)
Pull request description:
`bitcoin-wallet` as-is is merely an offline wallet inspection tool (introduced more than 9 years ago in PR #13926) that doesn't have any relation with IPC/multiprocess, so remove it from the list of binaries that use `libbitcoin_ipc`.
ACKs for top commit:
pablomartin4btc:
ACK fbde8d9a81
Tree-SHA512: e11720d35596575cd9785b9b00e6b11e46ba4c8aad6fe98e952d4aa4310f9e5c719dd2f177da8b5c3abefc831cbace0e1a0620f428d847f9bdcf7252a8889641
00c253d494 ci: disable cirrus cache in 32bit arm job (will)
ff18b6bbaf ci: refactor docker action to return provider str (will)
Pull request description:
Add an optional matrix field allowing opt-out of configuring cirrus GHA cache when not using cirrus runners.
This is not needed for the cirruslabs/[save|restore]-cache actions, as they automatically fallback based on runner type.
Addresses https://github.com/bitcoin/bitcoin/issues/31965#issuecomment-3252638785
ACKs for top commit:
m3dwards:
ACK 00c253d494
Tree-SHA512: 4c79deec2b0018f62a982b2d1051c78e94e242a1b8faf5db037353b05b707827dafded56c9b5ffbc861fcadac5a90571077e6ab69410975f7a2f40c755630a8e
`bitcoin-wallet` as-is is merely an offline wallet inspection tool
(introduced more than 9 years ago in PR #13926) that doesn't have any
relation with IPC/multiprocess, so remove it from the list of binaries
that use `libbitcoin_ipc`.
56791b5829 test: split out `system_ram_tests` to signal when total ram cannot be determined (Lőrinc)
337a6e7386 system: improve handling around GetTotalRAM() (Vasil Dimov)
Pull request description:
1. Fix unused variable warning (https://github.com/bitcoin/bitcoin/pull/33333#discussion_r2362493046)
2. Enable `GetTotalRAM()` on other platforms where it was tested to work.
3. Skip the `GetTotalRAM()` unit test on unsupported platforms.
Prior discussion: https://github.com/bitcoin/bitcoin/pull/33333#discussion_r2362493046
ACKs for top commit:
l0rinc:
ACK 56791b5829
hebasto:
ACK 56791b5829.
Tree-SHA512: bc419aa55edad77473dbcf810f02d02fa0c45a6355a93d17f7881051117b753c584296ab3840893270ecdc9bb2bee0fe4e070607c6560b794e97a25da733c47d
A target field was added to the getblock and getblockheader RPC calls in bitcoin#31583, but it mistakingly always used the tip value.
Because regtest does not have difficulty adjustment, a test is added for mainnet instead.
The next commit requires an additional mainnet block which changes the difficulty.
Also fix a few minor mistakes in the test (suite):
- rename the create_coinbase retarger_period argument to halving_period. Before bitcoin#31583 this was hardcoded for regtest where these values are the same.
- drop unused fees argument from mine helper
Finally the CPU miner instructions for generating the alternative mainnet chain are expanded.
6a33970fef fuzz: Reduce iterations in slow targets (marcofleon)
Pull request description:
The `mini_miner`, `txdownloadman`, `txdownloadman_impl`, and `tx_pool_standard` fuzz targets are all slow-running targets. Fix this by reducing the iteration count in the `LIMITED_WHILE` loops.
This should help decrease the run time of the fuzz CI jobs. See https://github.com/bitcoin/bitcoin/pull/33425.
Addresses https://github.com/bitcoin/bitcoin/issues/32870 as well.
ACKs for top commit:
Crypt-iQ:
crACK 6a33970fef
dergoegge:
utACK 6a33970fef
enirox001:
Concept ACK 6a33970
brunoerg:
ACK 6a33970fef
Tree-SHA512: d03d687507f497e587f7199866266298ca67d9843985dc96d1c957a6fbffb3c6cd5144a4876c471b84c84318295b0438908c745f3a4ac0254dca3e72655ecc14
79752b9c0b build(windows): Remove lingering registry entries and shortcuts upon install (Hodlinator)
Pull request description:
### Problem
Prior to fb2b05b125 / #32132 we installed using paths with an extra " (64-bit)"-suffix. Installing a version including that commit on top of a version that does not results in 2 entries in the "Installed apps" list. Both of them end up running the same `C:\Program Files\Bitcoin\uninstall.exe`. However, only one of the entries is removed by the uninstaller. The left over registry entry will now point to an executable that no longer exists and fail to work.
Removing the left over "Installed apps" entry on master currently requires the user to manually remove the Windows Registry entries (or run the correct old/new installer to ensure the uninstaller exists again).
### Solution
This PR automates removal of old entries (& shortcuts) when installing the new version.
### Disclaimer
Not an NSIS expert - confirmed that added deletion commands work without causing any visible errors both when prior items exist and when they don't.
ACKs for top commit:
achow101:
ACK 79752b9c0b
hebasto:
ACK 79752b9c0b.
Tree-SHA512: d23bd2e8f035ca93c3bd6187b3e5545c89c541b51d7b2b91b79bae1ebe328cd08c38b57e75a39bb376771fc85a537fe1d628903b9eadd32d04c3eb976c2e6d87
b81f37031c p2p: Increase tx relay rate (Anthony Towns)
Pull request description:
In the presence of smaller transactions on the network, blocks can sustain a higher relay rate than 7tx/second. In this event, the per-peer inventory queues can grow too large.
This commit bumps the rate up to 14 tx/s (for inbound peers), increasing the safety margin by a factor of 2.
Outbound peers continue to receive relayed transactions at 2.5x the rate of inbound peers, for a rate of 35tx/second.
ACKs for top commit:
sipa:
ACK b81f37031c
achow101:
ACK b81f37031c
darosior:
utACK b81f37031c.
glozow:
utACK b81f37031c
Tree-SHA512: 854ea0824d5f4c629f1dceb9ee61cc9226c8f0d4d26664737e68db917f65341d4800362ab55ed32673db920b2b59aa116b4cb9ee063367b2e43c94a904b41c08
Prior releases installed using these paths. Especially annoying was that the lingering registry entry for the uninstaller would show up as "Bitcoin Core (64-bit)" besides the current "Bitcoin Core" entry in the list of installed programs, and whichever was uninstalled last would fail to work as they would default to the same install directory.
When submitpackage produced no per-transaction result for a member,
the RPC previously set "error": "unevaluated" but then continued
without inserting the entry into tx-results, making it impossible for
callers to know which wtxids were unevaluated.
Insert the placeholder result before continuing, update help text, and
adjust functional tests to expect entries for all submitted wtxids.
Throwing an exception from the destructor of a class is a bad practice,
avoid that and instead print the message to the standard error output
and call `std::abort()`.
Even though not all script verification is turned off currently (e.g. we're still doing the cheaper sigop counts), this naming is more consistent with other usages.
67f632b6de net: remove unnecessary casts in socket operations (Matthew Zipkin)
Pull request description:
During review of https://github.com/bitcoin/bitcoin/pull/32747 several casting operations were questioned in existing code that had been copied or moved. That lead me to find a few other similar casts in the codebase.
It turns out that since the `Sock` class wraps syscalls with its own internal casting (see https://github.com/bitcoin/bitcoin/pull/24357 and https://github.com/bitcoin/bitcoin/pull/20788 written in 2020-2022) we no longer need to cast the arguments when calling these functions. The original argument-casts are old and were cleaned up a bit in https://github.com/bitcoin/bitcoin/pull/12855 written in 2018.
The casting is only needed for windows compatibility, where those syscalls require a data argument to be of type `char*` specifically:
https://learn.microsoft.com/en-us/windows/win32/api/winsock/nf-winsock-getsockopt
```
int getsockopt(
[in] SOCKET s,
[in] int level,
[in] int optname,
[out] char *optval,
[in, out] int *optlen
);
```
but on POSIX the argument is `void*`:
https://www.man7.org/linux/man-pages/man2/getsockopt.2.html
```
int getsockopt(socklen *restrict optlen;
int sockfd, int level, int optname,
void optval[_Nullable restrict *optlen],
socklen_t *restrict optlen);
```
ACKs for top commit:
Raimo33:
ACK 67f632b6de
achow101:
ACK 67f632b6de
hodlinator:
ACK 67f632b6de
vasild:
ACK 67f632b6de
davidgumberg:
ACK 67f632b6de
Tree-SHA512: c326d7242698b8d4d019f630fb6281398da2773c4e5aad1e3bba093a012c2119ad8815f42bd009e61a9a90db9b8e6ed5c75174aac059c9df83dd3aa5618a9ba6
168360f4ae coins: warn on oversized -dbcache (Lőrinc)
6c720459be system: add helper for fetching total system memory (Lőrinc)
Pull request description:
### Summary
Oversized allocations can cause out-of-memory errors or [heavy swapping](https://github.com/getumbrel/umbrel-os/issues/64#issuecomment-663637321), [grinding the system to a halt](https://x.com/murchandamus/status/1964432335849607224).
### Fix
Added a minimal system helper to query total physical RAM on [Linux/macOS/Windows](https://stackoverflow.com/a/2513561) (on unsupported platforms we just disable this warning completely).
The added test checks if the value is roughly correct by checking if the CI platforms are returning any value and if the value is at least 1 GB (as a simple property test checking if the unit size is correct, e.g. doesn't return megabytes or bits).
### Details
`LogOversizedDbCache()` now emits a startup warning if the configured `-dbcache` exceeds a cap derived from system RAM, using the same parsing/clamping as cache sizing via `CalculateDbCacheBytes()`. This isn't meant as a recommended setting, rather a likely upper limit.
Note that we're not modifying the set value, just issuing a warning.
Also note that the 75% calculation is rounded for the last two numbers since we have to divide first before multiplying, otherwise we wouldn't stay inside `size_t` on 32-bit systems - and this was simpler than casting back and forth.
We could have chosen the remaining free memory for the warning (e.g. warn if free memory is less than 1 GiB), but this is just a heuristic, we assumed that on systems with a lot of memory, other processes are also running, while memory constrained ones run only Core.
### Cap
If total RAM < 2 GiB, cap is `DEFAULT_DB_CACHE` (`450 MiB`), otherwise it's 75% of total RAM.
The threshold is chosen to be close to values commonly used in [raspiblitz](https://github.com/raspiblitz/raspiblitz/blob/dev/home.admin/_provision.setup.sh#L98-L115) for common setups:
| Total RAM | `dbcache` (MiB) | raspiblitz % | proposed cap (MiB) |
|----------:|----------------:|-------------:|-------------------:|
| 1 GiB | 512 | 50.0% | 450* |
| 2 GiB | 1536 | 75.0% | 1536 |
| 4 GiB | 2560 | 62.5% | 3072 |
| 8 GiB | 4096 | 50.0% | 6144 |
| 16 GiB | 4096 | 25.0% | 12288 |
| 32 GiB | 4096 | 12.5% | 24576 |
[Umbrel issues](https://github.com/getumbrel/umbrel-os/issues/64#issuecomment-663816367) also mention 75% being the upper limit.
### Reproducer
Starting `bitcoind` on an 8 GiB rpi4b with a dbcache of 7 GiB:
> ./build/bin/bitcoind -dbcache=7000
warns now as follows:
```
2025-09-07T17:24:29Z [warning] A 7000 MiB dbcache may be too large for a system memory of only 7800 MiB.
Warning: A 7000 MiB dbcache may be too large for a system memory of only 7800 MiB.
2025-09-07T17:24:29Z Cache configuration:
2025-09-07T17:24:29Z * Using 2.0 MiB for block index database
2025-09-07T17:24:29Z * Using 8.0 MiB for chain state database
2025-09-07T17:24:29Z * Using 6990.0 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
```
### Manual testing
Besides the [godbolt](https://godbolt.org/z/ec81Tjvrj) reproducers for the new total memory method, we also tested the warnings manually on:
- [x] Apple M4 Max, macOS 15.6.1
- [x] Intel Core i9-9900K, Ubuntu 24.04.2 LTS
- [x] Raspberry Pi 4 Model B, Armbian Linux 6.12.22-current-bcm2711
- [x] Intel Xeon x64, Windows 11 Home Version 24H2, OS Build 26100.4351
ACKs for top commit:
achow101:
ACK 168360f4ae
w0xlt:
reACK 168360f4ae
hodlinator:
re-ACK 168360f4ae
danielabrozzoni:
reACK 168360f4ae
Tree-SHA512: aa0c9b1034d55a6a4212685a19715d8cd89668ab7c33c688711a15559e6ad81aa65f3cd8b488c91385306e1e16cd9eeefa8f659ba90ef19ce9c7a2e64f8b561a
Throw RPC_CLIENT_INVALID_IP_OR_SUBNET when LookupHost(addr, false) fails
in addpeeraddress. This aligns with setban/addconnection and avoids the
opaque {"success": false} result for input errors. The JSON {success,
error?} object remains for addrman outcomes only. Update test to match.
e9c52272eb test: Avoid interface_ipc.py Duplicate ID errors (Ryan Ofsky)
Pull request description:
This change should fix issue https://github.com/bitcoin/bitcoin/issues/33417 reported by zaidmstrr. It's possible to reproduce the `mp/proxy.capnp:0: failed: Duplicate ID @0xcc316e3f71a040fb` error by installing libmultiprocess system-wide, or to one of the locations listed in the python test's `imports` list before the local libmultiprocess subtree, and then running the test.
ACKs for top commit:
zaidmstrr:
Tested ACK [e9c5227](e9c52272eb)
Tree-SHA512: 5df7fe767989b91245ce96f7c43b6767b7af49ec6c7007175e462341ffd69e161f21632697804060ce286b3e102a8d141a57a53f7e0e32299ef9a3a69ca8794a
Oversized allocations can cause out-of-memory errors or [heavy swapping](https://github.com/getumbrel/umbrel-os/issues/64#issuecomment-663637321), [grinding the system to a halt](https://x.com/murchandamus/status/1964432335849607224).
`LogOversizedDbCache()` now emits a startup warning if the configured `-dbcache` exceeds a cap derived from system RAM, using the same parsing/clamping as cache sizing via CalculateDbCacheBytes(). This isn't meant as a recommended setting, rather a likely upper limit.
Note that we're not modifying the set value, just issuing a warning.
Also note that the 75% calculation is rounded for the last two numbers since we have to divide first before multiplying, otherwise we wouldn't stay inside size_t on 32-bit systems - and this was simpler than casting back and forth.
We could have chosen the remaining free memory for the warning (e.g. warn if free memory is less than 1 GiB), but this is just a heuristic, we assumed that on systems with a lot of memory, other processes are also running, while memory constrained ones run only Core.
If total RAM < 2 GiB, cap is `DEFAULT_DB_CACHE` (`450 MiB`), otherwise it's 75% of total RAM.
The threshold is chosen to be close to values commonly used in [raspiblitz](https://github.com/raspiblitz/raspiblitz/blob/dev/home.admin/_provision.setup.sh#L98-L115) for common setups:
| Total RAM | `dbcache` (MiB) | raspiblitz % | proposed cap (MiB) |
|----------:|----------------:|-------------:|-------------------:|
| 1 GiB | 512 | 50.0% | 450* |
| 2 GiB | 1536 | 75.0% | 1536 |
| 4 GiB | 2560 | 62.5% | 3072 |
| 8 GiB | 4096 | 50.0% | 6144 |
| 16 GiB | 4096 | 25.0% | 12288 |
| 32 GiB | 4096 | 12.5% | 24576 |
[Umbrel issues](https://github.com/getumbrel/umbrel-os/issues/64#issuecomment-663816367) also mention 75% being the upper limit.
Starting `bitcoind` on an 8 GiB rpi4b with a dbcache of 7 GiB:
> ./build/bin/bitcoind -dbcache=7000
warns now as follows:
```
2025-09-07T17:24:29Z [warning] A 7000 MiB dbcache may be too large for a system memory of only 7800 MiB.
2025-09-07T17:24:29Z Cache configuration:
2025-09-07T17:24:29Z * Using 2.0 MiB for block index database
2025-09-07T17:24:29Z * Using 8.0 MiB for chain state database
2025-09-07T17:24:29Z * Using 6990.0 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
```
Besides the [godbolt](https://godbolt.org/z/EPsaE3xTj) reproducers for the new total memory method, we also tested the warnings manually on:
- [x] Apple M4 Max, macOS 15.6.1
- [x] Intel Core i9-9900K, Ubuntu 24.04.2 LTS
- [x] Raspberry Pi 4 Model B, Armbian Linux 6.12.22-current-bcm2711
- [x] Intel Xeon x64, Windows 11 Home Version 24H2, OS Build 26100.4351
Co-authored-by: stickies-v <stickies-v@protonmail.com>
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
Co-authored-by: w0xlt <woltx@protonmail.com>
Added a minimal system helper to query total physical RAM on [Linux/macOS/Windows](https://stackoverflow.com/a/2513561) (on other platforms we just return an empty optional).
The added test checks if the value is roughly correct by checking if the CI platforms are returning any value and if the value is at least 1 GiB and not more than 10 TiB.
The max value is only validated on 64 bits, since it's not unreasonable for 32 bits to have max memory, but on 64 bits it's likely an error.
https://learn.microsoft.com/en-us/windows/win32/api/sysinfoapi/ns-sysinfoapi-memorystatusex
> ullTotalPhys The amount of actual physical memory, in bytes.
https://man7.org/linux/man-pages/man3/sysconf.3.html:
> _SC_PHYS_PAGES The number of pages of physical memory. Note that it is possible for the product of this value and the value of _SC_PAGESIZE to overflow.
> _SC_PAGESIZE Size of a page in bytes. Must not be less than 1.
See https://godbolt.org/z/ec81Tjvrj for further details
This change should fix issue https://github.com/bitcoin/bitcoin/issues/33417
reported by zaidmstrr. It's possible to reproduce the `mp/proxy.capnp:0:
failed: Duplicate ID @0xcc316e3f71a040fb` error by installing libmultiprocess
system-wide, or to one of the locations listed in the python test's `imports`
list before the local libmultiprocess subtree, and then running the test.
47d79db8a552 Merge bitcoin-core/libmultiprocess#201: bug: fix mptest hang, ProxyClient<Thread> deadlock in disconnect handler
f15ae9c9b9fb Merge bitcoin-core/libmultiprocess#211: Add .gitignore
4a269b21b8c8 bug: fix ProxyClient<Thread> deadlock if disconnected as IPC call is returning
85df96482c49 Use try_emplace in SetThread instead of threads.find
ca9b380ea91a Use std::optional in ConnThreads to allow shortening locks
9b0799113557 doc: describe ThreadContext struct and synchronization requirements
d60db601ed9b proxy-io.h: add Waiter::m_mutex thread safety annotations
4e365b019a9f ci: Use -Wthread-safety not -Wthread-safety-analysis
15d7bafbb001 Add .gitignore
fe1cd8c76131 Merge bitcoin-core/libmultiprocess#208: ci: Test minimum cmake version in olddeps job
b713a0b7bfbc Merge bitcoin-core/libmultiprocess#207: ci: output CMake version in CI script
0f580397c913 ci: Test minimum cmake version in olddeps job
d603dcc0eef0 ci: output CMake version in CI script
git-subtree-dir: src/ipc/libmultiprocess
git-subtree-split: 47d79db8a5528097b408e18f7b0bae11a6702d26
7584a4fda9 cmake: Install `bitcoin` manpage (Hennadii Stepanov)
Pull request description:
This PR is an amendment to https://github.com/bitcoin/bitcoin/pull/31375.
ACKs for top commit:
ryanofsky:
Code review ACK 7584a4fda9.
Tree-SHA512: 66810c1d65fa8ae469b8161a5f807aa7b43a7b18e88d40b05617c7110b2e03e07bcb8f310c1736fb2c3738e274fc524032ff5d34d5c644824a4edd64372f1e9f
f563ce9081 net: Do not apply whitelist permission to onion inbounds (Martin Zumsande)
Pull request description:
Tor inbound connections do not reveal the peer's actual network address. Do not apply whitelist permissions to them since address-based matching is ineffective.
ACKs for top commit:
darosior:
ACK f563ce9081
furszy:
ACK f563ce9081
vasild:
ACK f563ce9081
Tree-SHA512: 49ae70e382fc2f78b7073553fe649a6843a41214b2986ea7f77e285d02b7bd00fe0320a1b71d1aaca08713808fb14af058f0b1f19f19adb3a77b97cb9d3449ce
This change updates the vcpkg manifest baseline from the "2025.03.19
Release" to the "2025.08.27 Release", with the following package
changes:
- boost: 1.87.0 --> 1.88.0
- qtbase: 6.8.2#1 -> 6.9.1
- qttools: 6.8.2 -> 6.9.1
- sqlite3: 3.49.1 --> 3.50.4
The dynamically created signing context for libsecp256k1 calls is only
needed for functions that involve generator point multiplication with a
secret key, i.e. different variants of public key creation and signing.
The API docs hint to this by stating "not secp256k1_context_static" for
the context parameter. In our case that applies to the following calls:
- `secp256k1_ec_pubkey_create`
- `secp256k1_keypair_create`
- `secp256k1_ellswift_create`
- `secp256k1_ecdsa_sign`
- `secp256k1_ecdsa_sign_recoverable`
- `secp256k1_schnorrsig_sign32`
- `ec_seckey_export_der` (not a direct secp256k1 function, but calls
`secp256k1_ec_pubkey_create` inside)
For all the other secp256k1 calls we can simply use the static context.
Tor inbound connections do not reveal the peer's actual network address.
Therefore do not apply whitelist permissions to them.
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
0a26731c4c test: Add submitblock test in interface_ipc (TheCharlatan)
Pull request description:
Expands the ipc mining test a bit with submitting a solved block and checking its validity.
ACKs for top commit:
Sjors:
ACK 0a26731c4c
marcofleon:
code review ACK 0a26731c4c
zaidmstrr:
Tested ACK [0a26731](0a26731c4c)
Tree-SHA512: 35c87d88496eec469bddedf2ae82c494626abb47ae15d5a45d6ab0400199c86501199c3e569e83836549830042be76b197b470e1100a317bdfef2578a9d5a92f
These methods in the Sock class wrap corresponding syscalls,
accepting void* arguments and casting to char* internally, which is
needed for Windows support and ignored on other platforms because
the syscall itself accepts void*:
Send()
Recv()
GetSockOpt()
SetSockOpt()
bdf01c6f61 test: Prevent disk space warning during node_init_tests (Ryan Ofsky)
Pull request description:
mzumsande pointed out https://github.com/bitcoin/bitcoin/pull/32345#issuecomment-3286964369 that this test was print a warning:
```
Warning: Disk space for "/tmp/test_common bitcoin/node_init_tests/init_test/bf78678cb7723a3e84b5/blocks" may not accommodate the block files. Approximately 810 GB of data will be stored in this directory.
```
Fix by setting regtest instead of mainnet network before running the test.
ACKs for top commit:
achow101:
ACK bdf01c6f61
Eunovo:
Tested ACK bdf01c6f61:
janb84:
ACK bdf01c6f61
l0rinc:
tested ACK bdf01c6f61
mzumsande:
utACK bdf01c6f61
enirox001:
utACK bdf01c6
Tree-SHA512: ac4e1e48246c84a4c4b80ccb25e962b0090359ab0e541ee4f1a9e18ac9da8ec35a78c9a55501d231423053e945ff785862f0db141d4b620d622327670c764f8c
mzumsande pointed out https://github.com/bitcoin/bitcoin/pull/32345#issuecomment-3286964369 that this test was causing a warning:
Warning: Disk space for "/tmp/test_common bitcoin/node_init_tests/init_test/bf78678cb7723a3e84b5/blocks" may not accommodate the block files. Approximately 810 GB of data will be stored in this directory.
Fix by setting regtest instead of mainnet network before running the test.
Move calculated constants from the top of src/headerssync.cpp into src/kernel/chainparams.cpp.
Instead of being hardcoded to mainnet parameters, HeadersSyncState can now vary depending on chain or test. (This means we can reset TARGET_BLOCKS back to the nice round number of 15'000).
Signet and testnets got new HeadersSyncParams constants through temporarily altering headerssync-params.py with corresponding GENESIS_TIME and MINCHAINWORK_HEADERS (based off defaultAssumeValid block height comments, corresponding to nMinimumChainWork). Regtest doesn't have a default assume valid block height, so the values are copied from Testnet 4. Since the constants only affect memory usage, and have very low impact unless dealing with a largely malicious chain, it's not that critical to keep updating them for non-mainnet chains.
GENESIS_TIMEs (UTC):
Testnet3: 1296688602 = datetime(2011, 2, 2)
Testnet4: 1714777860 = datetime(2024, 5, 3)
Signet: 1598918400 = datetime(2020, 9, 1)
Adding these checks necessitates increasing the length of the generated test chains so that we can properly exceed the REDOWNLOAD_BUFFER_SIZE during the test.
One can check out this commit and locally revert the TARGET_BLOCKS value change to prove the need for tests being able to control the buffer size, as is done by the next commit. Beyond the current REDOWNLOAD_BUFFER_SIZE of 15'009 we need 3 extra - 15'012 TARGET_BLOCKS:
* 1 for the genesis block.
* 1 for the test wanting to check that we start receiving headers for permanent storage *before* the final header (first_chain.back()).
* 1 to exceed REDOWNLOAD_BUFFER_SIZE in HeadersSyncState::PopHeadersReadyForAcceptance().
(The release process includes an occasional increase of the REDOWNLOAD_BUFFER_SIZE value, see release-process.md and history of headerssync.cpp).
Introduces CHECK_RESULT for consistently validating ProcessingResult.
* Verifies HeadersSyncState::State directly after ProcessNextHeaders().
* Uses BOOST_REQUIRE_EQUAL for HeadersSyncState::State - Nicer failure output and prevents continuing test in nonsensical state.
* Encourages checking Locator and result.pow_validated_headers.
Changes happy_path to test both full & non-full headers messages.
Helps logically separate the scenarios being tested.
Also adds missing comment for part 4.
(unique_ptrs and ProcessingResults will be cleaned up in next commit).
113a422822 wallet: Add m_cached_from_me to cache "from me" status (Ava Chow)
609d265ebc test: Add a test for anchor outputs in the wallet (Ava Chow)
c40dc822d7 wallet: Throw an error in sendall if the tx size cannot be calculated (Ava Chow)
39a7dbdd27 wallet: Determine IsFromMe by checking for TXOs of inputs (Ava Chow)
e76c2f7a41 test: Test wallet 'from me' status change (Ava Chow)
Pull request description:
One of the ways that the wallet would determine if a transaction was sent from the wallet was by checking if the total amount being spent by a transaction from outputs known to the wallet was greater than 0. This has worked fine until recently since there was no reason for 0-value outputs to be created. However, with ephemeral dust and P2A, it is possible to create standard 0-value outputs, and the wallet was not correctly identifying the spends of such outputs. This PR updates `IsFromMe` to only check whether the wallet knows any of the inputs, rather than checking the debit amount of a transaction.
Additionally, a new functional test is added to test for this case, as well as a few other anchor output related scenarios. This also revealed a bug in `sendall` which would cause an assertion error when trying to spend all of the outputs in a wallet that has anchor outputs.
Fixes#33265
ACKs for top commit:
rkrux:
lgtm ACK 113a422822
enirox001:
Tested ACK 113a422. Ran the full functional test suite including `wallet_anchor.py`; all tests passed. Fix for 0 value anchor detection and sendall size errors looks good. LGTM.
furszy:
ACK 113a422822
Tree-SHA512: df2ce4b258d1875ad0b4f27a5b9b4437137a5889a7d5ed7fbca65f904615e9572d232a8b8d070760f75ac168c1a49b7981f6b5052308575866dc610d191ca964
93a29ff283 trace: Workaround GCC bug compiling with old systemtap (Luke Dashjr)
Pull request description:
ACKs for top commit:
0xB10C:
lgtm ACK 93a29ff283 - I did not test this.
Tree-SHA512: 9ce9ed8b7733af721134462073a3417e52d67e9e9853eebbddfa795842b381de98e28756ebfa6652536cbfdd08181142eccd198f4dc00a57d8748801b362b4b7
b736052e39 ci: always use tag for LLVM checkout (fanquake)
Pull request description:
Rather than trying to match the apt installed clang version, which is prone to intermittent issues. i.e #33345.
ACKs for top commit:
davidgumberg:
ACK b736052e39
willcl-ark:
ACK b736052e39
Tree-SHA512: 8e3fcc8219f573cec65941576c7995f21cae3330bcdbf615f799e8c5facd1146d3239a7284e9af7b013c37170ddf7435d7df6d2966f63fe7b4a8e4937311ff36
fa96a4afea ci: Enable CI_LIMIT_STACK_SIZE=1 in i686_no_ipc task (MarcoFalke)
facfde2cdc test: Fix CLI_MAX_ARG_SIZE issues (MarcoFalke)
Pull request description:
`CLI_MAX_ARG_SIZE` has many edge case issues:
* It seems to be lower on some systems, but it is unknown how to reproduce locally: https://github.com/bitcoin/bitcoin/pull/33079#issuecomment-3139957274
* `MAX_ARG_STRLEN` is a limit per arg, but we probably want "The maximum length of [all of] the arguments": See https://www.man7.org/linux/man-pages/man3/sysconf.3.html, section `ARG_MAX - _SC_ARG_MAX`.
* It doesn't account for the additional args added by the `bitcoin` command later on: 73220fc0f9/src/bitcoin.cpp (L85-L92)
* It doesn't account for unicode encoding a string to bytes before taking its length.
The issues are mostly harmless edge cases, but it would be good to fix them. So do that here, by:
* Replacing `max()` by `sum()`, to correctly take into account all args, not just the largest one.
* Reduce `CLI_MAX_ARG_SIZE`, to account for the `bitcoin` command additional args.
Also, there is a test. The test can be called with `ulimit` to hopefully limit the max args size to the hard-coded value in the test framework. For reference:
```
$ ( ulimit -s 512 && python3 -c 'import os; print(os.sysconf("SC_ARG_MAX") )' )
131072
```
On top of this pull it should pass, ...
```
bash -c 'ulimit -s 512 && BITCOIN_CMD="bitcoin -M" ./bld-cmake/test/functional/rpc_misc.py --usecli -l DEBUG'
```
... and with the test_framework changes reverted, it should fail:
```
OSError: [Errno 7] Argument list too long: 'bitcoin'
```
Also, there is a commit to enable `CI_LIMIT_STACK_SIZE=1` in the i686 task, because it should now be possible and no longer hit the hard-to-reproduce issue mentioned above.
ACKs for top commit:
cedwies:
ACK fa96a4a
achow101:
ACK fa96a4afea
enirox001:
ACK fa96a4a — thanks for addressing the nits and clarifying the test; LGTM.
mzumsande:
Code Review ACK fa96a4afea
Tree-SHA512: d12211bd097d692d560c3615970ec0e911707d8c6cbbb145591abc548beed55f487a80b08f0a8c89d4eef4d76a9fbd6a33edc0b42b5860a93dd7b954355bc887
653a9849d5 common: Make arith_uint256 trivially copyable (Fabian Jahr)
Pull request description:
Makes `arith_uint256`/`base_uint` trivially copyable by removing the custom copy constructor and copy assignment operators. Removing of the custom code should not result in a change of behavior since `base_uint` contains a simple array of `uint32_t` and compiler generated versions of the code could be better optimized.
This was suggested by maflcko here: https://github.com/bitcoin/bitcoin/pull/30469#pullrequestreview-3186533494
ACKs for top commit:
Raimo33:
ACK 653a9849d5
l0rinc:
ACK 653a9849d5
achow101:
ACK 653a9849d5
hodlinator:
re-ACK 653a9849d5
Tree-SHA512: 38db5220a2cf773c0c5fb5591671e329b6b87458d972db4f5f3f98c025ec329a8c39b32b5bc24ef8b50b1002b43bb248d8b35aa1c9a56c68c6bbd1d470485bd7
75d9b72475 kernel: make blockTip index const (stickies-v)
Pull request description:
Notification interface subscribers need to view, but not mutate, the index.
This change allows improving the #30595 kernel interface, see e.g. `BlockTreeEntry` where [currently](https://github.com/bitcoin/bitcoin/pull/30595/files#diff-4d05cd02fdce641be603f0f9abcecfeaf76944285d4539ba4bbc40337fa9bbc2R617) a `View` is constructed from a non-const pointer, whereas really this should be a `const btck_BlockTreeEntry* entry`.
ACKs for top commit:
achow101:
ACK 75d9b72475
TheCharlatan:
ACK 75d9b72475
l0rinc:
Code review ACK 75d9b72475
yuvicc:
Code review ACK 75d9b72475
Tree-SHA512: 6151374a040cead36490c5fa5ce9dc4d93499a02110f444c50bd90f9095912747bc5b2fd7294815e6794c96a6843f43eb0507706d41d7296af96071b5f704ff4
fa4885ef2f test: Remove polling loop from test_runner (MarcoFalke)
Pull request description:
(This picks up my prior attempt from https://github.com/bitcoin/bitcoin/pull/13384)
Currently, the test_runner is using a `time.sleep` before polling to check if any tests have completed. This is largely fine when running a few tests, or when the tests take a long time.
However, when running many fast tests, this can accumulate and leave the CPU idle for no reason.
A trivial improvement would be to only sleep when really needed:
```diff
diff --git a/test/functional/test_runner.py b/test/functional/test_runner.py
index 7c8c15f391..1d9f28cee4 100755
--- a/test/functional/test_runner.py
+++ b/test/functional/test_runner.py
@@ -747,7 +747,6 @@ class TestHandler:
dot_count = 0
while True:
# Return all procs that have finished, if any. Otherwise sleep until there is one.
- time.sleep(.5)
ret = []
for job in self.jobs:
(name, start_time, proc, testdir, log_out, log_err) = job
@@ -771,6 +770,7 @@ class TestHandler:
ret.append((TestResult(name, status, int(time.time() - start_time)), testdir, stdout, stderr, skip_reason))
if ret:
return ret
+ time.sleep(.5)
if self.use_term_control:
print('.', end='', flush=True)
dot_count += 1
```
However, ideally there is no sleep at all. So do that by using a `ThreadPoolExecutor`.
This can be tested via something like:
```
time ./bld-cmake/test/functional/test_runner.py $(for i in {1..200}; do echo -n "tool_rpcauth "; done) -j 200
```
The result should show:
* Current `master` is the slowest
* The "sleep patch" from above is a bit faster (1.5x improvement)
* This pull request is the fastest (2x improvement)
ACKs for top commit:
achow101:
ACK fa4885ef2f
l0rinc:
tested ACK fa4885ef2f
Eunovo:
ReACK fa4885ef2f
Tree-SHA512: f097636c5d9e005781012d8e20c2886cd9968544d4d555b1d2e28982d420ff63fec15cfabb6bd30e4d3c389b8b8350a1ddad721cceaf4b7760cad38b95160175
d45f3717d2 txgraph: use enum Level instead of bool main_only (Pieter Wuille)
Pull request description:
Part of #30289. Inspired by https://github.com/bitcoin/bitcoin/pull/28676#discussion_r2331387778.
Since there has been more than one case in the development of #28676 of calling a `TxGraph` function without correctly setting the `bool main_only` argument that many of its interface functions have, make these mandatory and explicit, using an `enum class Level`:
```c++
enum class Level {
TOP, //!< Refers to staging if it exists, main otherwise.
MAIN //!< Always refers to the main graph, whether staging is present or not.
};
```
ACKs for top commit:
instagibbs:
ACK d45f3717d2
vasild:
ACK d45f3717d2
glozow:
code review ACK d45f3717d2
Tree-SHA512: d1c4b37e8ab3ec91b414df8970cb47aa080803f68da5881c8e1cbdc6939dea7851e0f715192cf3edd44b7f328cd6b678474d41f9cd9da8cb68f6c5fd78cb71b1
Replacing the custom code with default behavior should not result in a change of behavior since base_uint contains a simple array of uint32_t and compiler generated versions of the code could be better optimized.
Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
Co-authored-by: Max Edwards <youwontforgetthis@gmail.com>
Add an optional matrix field allowing opt-out of configuring cirrus
GHA cache when not using cirrus runners.
This is not needed for the cirruslabs/[save|restore]-cache actions, as
they automatically fallback based on runner type.
6a371b70c8 gui: Avoid pathological QT text/markdown behavior... (David Gumberg)
Pull request description:
...during text selection by only setting plaintext mime data.
Fixes the OOM described in #887.
The issue is related to the construction of the [`text/markdown`](b617d11765/src/widgets/widgets/qwidgettextcontrol.cpp (L3539)) MIME data for the selection. Using the `heaptrack` utility, I observed that nearly all of the allocations when reproducing happen in [`QTextMarkdownWriter::writeFrame`](b617d11765/src/gui/text/qtextmarkdownwriter.cpp (L95)). I am not 100% sure what is causing this issue in QT's conversion of our HTML to markdown; I have tried changing the [HTML tags](689a321976/src/qt/rpcconsole.cpp (L916-L924)) (e.g. using `<p></p`> and `<ul><li></li></ul>` in place of tables) used in our `rpcconsole` messages, but the issue recurs.
The solution applied here is to override `createMimeDataFromSelection()` to avoid construction of the (likely never-used anyways) `text/markdown` mime data, and only set plaintext mime data in the clipboard.
ACKs for top commit:
hebasto:
ACK 6a371b70c8.
Tree-SHA512: 3edc4da47e6dbe939f27664d2265376938eed4f83ded3706e4b73677eac5c9a4ba8819f241428b45a08e8834982ee7759ee096afd090586db3b523d0ccbbbf73
### Summary
When triggering a reindex, users had no indication of how many files remained or how far along the process was.
### Fix
This patch prefetches the target file block file count to be able to show progress information.
Instead of just displaying which block file is being processed, it now indicates how many files remain.
### Reproducer + expected results
Running
```bash
cmake -B build && make -C build -DCMAKE_BUILD_TYPE=Release && ./build/bin/bitcoind -datadir=demo -reindex
```
Shows the block files one-by-one currently, there's no way to see how much work is left:
```
Reindexing block file blk00000.dat...
Loaded 119920 blocks from external file in 1228ms
Reindexing block file blk00001.dat...
Loaded 10671 blocks from external file in 284ms
Reindexing block file blk00002.dat...
Loaded 5459 blocks from external file in 263ms
Reindexing block file blk00003.dat...
Loaded 5595 blocks from external file in 267ms
```
After the change:
```
Reindexing block file blk00000.dat (0% complete)...
Loaded 119920 blocks from external file in 1255ms
Reindexing block file blk00001.dat (1% complete)...
Loaded 10671 blocks from external file in 303ms
Reindexing block file blk00002.dat (2% complete)...
Loaded 5459 blocks from external file in 278ms
Reindexing block file blk00003.dat (3% complete)...
Loaded 5595 blocks from external file in 285ms
```
4d4789dffa net: Prevent node from binding to the same CService (woltx)
Pull request description:
Currently, if the node inadvertently starts with repeated `-bind` options (e.g. `./build/bin/bitcoind -listen -bind=0.0.0.0 -bind=0.0.0.0`), the user will receive a misleading message followed by the node shutdown:
```
[net:error] Unable to bind to 0.0.0.0:8333 on this computer. Bitcoin Core is probably already running.
[error] Unable to bind to 0.0.0.0:8333 on this computer. Bitcoin Core is probably already running.
```
And the user might spend some time looking for a `bitcoind` process or what application is using port 8333, when what happens is that Bitcoin Core successfully connected to port 8333 and then tries again, generating this fatal error.
This PR proposes that repeated `-bind` options have no effect.
ACKs for top commit:
l0rinc:
ACK 4d4789dffa
yuvicc:
re-ACK 4d4789dffa
sipa:
utACK 4d4789dffa
achow101:
ACK 4d4789dffa
vasild:
ACK 4d4789dffa
naiyoma:
Tested ACK 4d4789dffa
Tree-SHA512: f1042c00417da16550403cfcb75cb8b12740e67cf92a1d8e3c007ae81fcf741907088a633129ce12a6a48ad07fc9f320602792cafed73ec33f6306cd854514b4
d3c5e47391 wallet, refactor: Remove Legacy check and error (pablomartin4btc)
30c6f64eed test: Remove unnecessary LoadWallet() calls (pablomartin4btc)
Pull request description:
Remove dead code due to legacy wallet removal.
Leftovers from previous #32481.
---
**Note**:
While attempting to remove the legacy check in `CWallet::UpgradeDescriptorCache()` (which is called from `DBErrors WalletBatch::LoadWallet(CWallet* pwallet))`, I once again ran into the fact that `LoadWallet()` is used in two distinct scenarios — something I was already aware of:
- Wallet creation – the upgrade is ignored here because no wallet flags are yet set; attempting to set a flag (ie `WALLET_FLAG_LAST_HARDENED_XPUB_CACHED` at the end of the upgrade function, if the legacy check is removed) would produce a failure (`DBErrors CWallet::LoadWallet()` -> `Assert(m_wallet_flags == 0)`).
- Wallet loading – the upgrade proceeds correctly and the flag `WALLET_FLAG_LAST_HARDENED_XPUB_CACHED` is set.
While revisiting this, I also noticed that some `LoadWallet()` calls in the wallet tests are unnecessary and I've removed them in the first commit.
The following change in `UpgradeDescriptorCache()` could be done in PR #32636 as part of the separation between wallet loading and creation responsibilities.
```diff --git a/src/wallet/wallet.cpp b/src/wallet/wallet.cpp
void CWallet::UpgradeDescriptorCache()
{
+ // Only descriptor wallets can upgrade descriptor cache
+ Assert(IsWalletFlagSet(WALLET_FLAG_DESCRIPTORS));
+
- if (!IsWalletFlagSet(WALLET_FLAG_DESCRIPTORS) || IsLocked() || IsWalletFlagSet(WALLET_FLAG_LAST_HARDENED_XPUB_CACHED)) {
+ if (IsLocked() || IsWalletFlagSet(WALLET_FLAG_LAST_HARDENED_XPUB_CACHED)) {
return;
}
```
ACKs for top commit:
davidgumberg:
crACK d3c5e47391
achow101:
ACK d3c5e47391
l0rinc:
code review ACK d3c5e47391
Tree-SHA512: ead37cf4061dfce59feb41ac50e807e6790e1a5e6b358e3b9c13e63d61a9cb82317a2e596cecb543f62f88a4338171788b651452425c1f40b5c1bec7fe78339e
53e6db91ef contrib: add placeholder manpage for bitcoin binary (fanquake)
f5887a8de4 contrib: add bitcoin binary to gen-manpages (fanquake)
Pull request description:
This was missed in #31375.
ACKs for top commit:
dergoegge:
ACK 53e6db91ef
Tree-SHA512: ff283ee02fadb57dbb335425d0348533b1322c6de323020f3ce5b6f01ff958cc731cb2191b8a774cd6a53b462f831e0ee86bbd522283357a6f6121962ef0abf1
Currently, if the user inadvertently starts the node with duplicate bind options,
such as `-bind=0.0.0.0 -bind=0.0.0.0`, it will cause a fatal error with the
misleading message "Bitcoin Core is probably already running".
This commit adds early validation to detect duplicate bindings across all binding
configurations (-bind, -whitebind, and onion bindings) before attempting to bind.
When duplicates are detected, the node terminates with a clear, specific error
message: "Duplicate binding configuration for address <addr>. Please check your
-bind, -bind=...=onion and -whitebind settings."
The validation catches duplicates both within the same option type (e.g.,
`-bind=X -bind=X`) and across different types (e.g., `-bind=X -whitebind=Y@X`),
helping users identify and fix configuration mistakes.
m_cached_from_me is used to track whether a transaction is "from me", i.e. has
any inputs which belong to the wallet. This is held in memory only in
the same way that a transaction's balances are.
Instead of checking whether the total amount of inputs known by the
wallet is greater than 0, we should be checking for whether the input is
known by the wallet. This enables us to determine whether a transaction
spends an of output with an amount of 0, which is necessary for marking
0-value dust outputs as spent.
If something is imported into the wallet, it can change the 'from me'
status of a transaction. This status is only visible through
gettransaction's "fee" field which is only shown for transactions that
are 'from me'.
Regenerated `.clang-format` from current configs to replace deprecated keys with up-to-date equivalents.
Also added all current formatter default values to guard against version differences.
The configs were updated with the following command (using v16 for maximal compatibility):
$(brew --prefix llvm@16)/bin/clang-format -dump-config -style=file:src/.clang-format
The new config was tested with:
$(brew --prefix llvm@16)/bin/clang-format -i src/deploymentinfo.h
Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
The default database write batch size is increased from 16 MiB to 32 MiB to improve I/O efficiency and performance during UTXO flushes, particularly during Initial Block Download and `assumeutxo` loads.
On systems with slower I/O, a larger batch size reduces overhead from numerous small writes. Measurements show this change provides a modest performance improvement on most hardware during a critical section, with a minimal peak memory increase (approx. 75 MiB on default settings).
Adds a functional test that repeats the existing witness nonce size
malleability check using a block under MAX_BLOCK_WEIGHT so it can be
relayed over the P2P network, addressing the TODO in test_block_malleability.
Includes rejection check for 'bad-witness-nonce-size' and confirmation
that a corrected block is accepted.
`using script_verify_flags = uint32_t` allows implicit conversion to
and from int, so replace it with a class to have the compiler ensure we
use the correct type. Provide from_int and as_int to allow for explicit
conversions when desired.
Introduces the type `script_verify_flag_name` for the individual flag
name enumeration.
Previously the SCRIPT_VERIFY_* flags were specified as either uint32_t,
unsigned int, or unsigned. This converts them to a common type alias in
preparation for changing the underlying type.
Moves FormatScriptFlags logic into GetScriptFlagNames which returns a
vector of strings. For completeness, also has GetScriptFlagNames report
on any bits that do not match a known script flag.
Did both in this commit, since the return value of `WriteReindexing` was ignored anyway - which existed only because of the constant `Erase` being called
`WriteBatch` can only ever return `true` - its errors are handled by throwing a `throw dbwrapper_error` instead.
The boolean return value is quite confusing, especially since it's symmetric with `CDBWrapper::Read`, which catches the exceptions and returns a boolean instead.
We're removing the constant return value and inlining `true` for its usages.
When loading an older wallet without the last hardened cache, an
automatic upgrade should be performed. Check this in
wallet_backwards_compatibility.py
When migrating a wallet, the migrated wallet should always have the last
hardened cache, so verify in wallet_migration.py
Adds tests to make sure we are consistent on activating the same chain over
a node restart if two or more candidates have the same work when the node is shutdown
Before this, if we had two (or more) same work tip candidates and restarted our node,
it could be the case that the block set as tip after bootstrap didn't match the one
before stopping. That's because the work and `nSequenceId` of both block will be the same
(the latter is only kept in memory), so the active chain after restart would have depended
on what tip candidate was loaded first.
This makes sure that we are consistent over reboots.
This requires adjusting some tests to force exceptions over aborts, or
accept either exceptions or aborts.
Also, remove a fuzz test in integer.cpp that is mostly redundant with
the unit test added in the prior commit.
This allows specific tests to mock the check behavior to consistently
use exceptions instead of aborts for intentionally failing checks in all
build configurations.
* Make the methods of `CThreadInterrupt` virtual and store a pointer to
it in `CConnman`, thus making it possible to override with a mocked
instance.
* Initialize `CConnman::m_interrupt_net` from the constructor, making it
possible for callers to supply mocked version.
* Introduce `FuzzedThreadInterrupt` and `ConsumeThreadInterrupt()` and
use them in `src/test/fuzz/connman.cpp` and `src/test/fuzz/i2p.cpp`.
This improves the CPU utilization of the `connman` fuzz test.
As a nice side effect, the `std::shared_ptr` used for
`CConnman::m_interrupt_net` resolves the possible lifetime issues with
it (see the removed comment for that variable).
Now that all network calls done by `CConnman::OpenNetworkConnection()`
are done via `Sock` they can be redirected (mocked) to `FuzzedSocket`
for testing.
`FuzzedSock::Accept()` properly returns a new socket, but it forgot to
set the output argument `addr`, like `accept(2)` is expected to.
This could lead to reading uninitialized data during testing when we
read it, e.g. from `CService::SetSockAddr()` which reads the `sa_family`
member.
Set `addr` to a fuzzed IPv4 or IPv6 address.
The unserialization flows of the PSBT types work based on few underlying
assumptions of functions from `serialize.h` & `stream.h` that takes some
to understand when read the first time.
Add few comments that highlight these assumptions hopefully making it easier
to grasp. Also, mention key/value format types as per BIP 174.
Move the periodic flush inside the outer loop of ActivateBestChain.
For very long activations, such as with reindex-chainstate, this
calls periodic flushes so progress can be saved to disk.
Co-Authored-By: l0rinc <pap.lorinc@gmail.com>
* do not only check user provided range data to decide whether descriptor is ranged
* properly handle std::optional<bool> when checking if descriptor is internal
On Alpine Linux 3.12.3, compiling the `libevent` package produces
multiple warnings:
```
$ gmake -C depends -j $(nproc) libevent
<snip>
<command-line>: warning: "_FORTIFY_SOURCE" redefined
<built-in>: note: this is the location of the previous definition
<snip>
```
This was introduced by commit ab9edbd6b6.
It appears the original author may have gotten tired and pasted the wrong
error code into this 1 place. Every other situation where the value stack
lacks the required number of arguments for the op-code,
SCRIPT_ERR_INVALID_STACK_OPERATION is reported. Not so here.
This commit fixes the situation.
Also in this commit:
- Fix script_tests to adjust to the corrected error message
- Fix p2p_invalid_tx functional test to produce the desired error message
In the presence of smaller transactions on the network, blocks can sustain a
higher relay rate than 7tx/second. In this event, the per-peer inventory queues
can grow too large.
This commit bumps the rate up to 14 tx/s (for inbound peers), increasing the
safety margin by a factor of 2.
Outbound peers continue to receive relayed transactions at 2.5x the rate of
inbound peers, for a rate of 35tx/second.
Co-Authored-By: Suhas Daftuar <sdaftuar@gmail.com>
2024-12-03 17:24:45 +10:00
1587 changed files with 32292 additions and 16448 deletions
timeout-minutes:360# Use maximum time, see https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#jobsjob_idtimeout-minutes. Assuming a worst case time of 1 hour per commit, this leads to a --max-count=6 below.
# Can't use ctest here like other jobs as we don't have a CMake build tree.
run:|
./bin/test_bitcoin-qt.exe
./bin/test_bitcoin.exe -l test_suite # Intentionally run sequentially here, to catch test case failures caused by dirty global state from prior test cases.
exportCI_IMAGE_NAME_TAG="mirror.gcr.io/ubuntu:24.04"# Check that https://packages.ubuntu.com/noble/g++-arm-linux-gnueabihf (version 13.x, similar to guix) can cross-compile
exportCI_IMAGE_NAME_TAG="mirror.gcr.io/debian:trixie"# Check that https://packages.debian.org/trixie/g++-arm-linux-gnueabihf (version 14.x, similar to guix) can cross-compile
exportCI_IMAGE_PLATFORM="linux/arm64"
exportUSE_BUSY_BOX=true
exportRUN_UNIT_TESTS=true
exportRUN_FUNCTIONAL_TESTS=false
exportGOAL="install"
exportCI_LIMIT_STACK_SIZE=1
# -Wno-psabi is to disable ABI warnings: "note: parameter passing for argument of type ... changed in GCC 7.1"
# This could be removed once the ABI change warning does not show up by default
exportCI_IMAGE_NAME_TAG="mirror.gcr.io/debian:trixie"# Check that https://packages.debian.org/trixie/clang (version 19, similar to guix) can cross-compile
exportCI_IMAGE_NAME_TAG="mirror.gcr.io/debian:trixie"# Check that https://packages.debian.org/trixie/clang (version 19, similar to guix) can cross-compile
# Use minimum supported python3.10 and gcc-11, see doc/dependencies.md
exportPACKAGES="gcc-11 g++-11 python3-zmq"
exportDEP_OPTS="CC=gcc-11 CXX=g++-11"
# Use minimum supported python3.10 and gcc-12, see doc/dependencies.md
exportPACKAGES="gcc-12 g++-12 python3-zmq"
exportDEP_OPTS="CC=gcc-12 CXX=g++-12"
exportTEST_RUNNER_EXTRA="--previous-releases --coverage --extended --exclude feature_dbcrash"# Run extended tests so that coverage does not fail, but exclude the very slow dbcrash
exportTEST_RUNNER_EXTRA="--exclude rpc_bind,feature_bind_extra"# Excluded for now, see https://github.com/bitcoin/bitcoin/issues/17765#issuecomment-602068547
# bind tests excluded for now, see https://github.com/bitcoin/bitcoin/issues/17765#issuecomment-602068547
# Copyright (c) 2019-present The Bitcoin Core developers
# Copyright (c) 2025-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
exportLC_ALL=C.UTF-8
exportCONTAINER_NAME=ci_win64
exportCI_IMAGE_NAME_TAG="mirror.gcr.io/ubuntu:24.04"# Check that https://packages.ubuntu.com/noble/g++-mingw-w64-x86-64-posix (version 13.x, similar to guix) can cross-compile
exportCI_IMAGE_PLATFORM="linux/amd64"
exportHOST=x86_64-w64-mingw32
exportPACKAGES="g++-mingw-w64-x86-64-posix nsis"
exportCI_IMAGE_NAME_TAG="mirror.gcr.io/debian:trixie"# Check that https://packages.debian.org/trixie/g++-mingw-w64-ucrt64 can cross-compile
# Copyright (c) 2019-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
exportLC_ALL=C.UTF-8
exportCONTAINER_NAME=ci_win64_msvcrt
exportCI_IMAGE_NAME_TAG="mirror.gcr.io/debian:trixie"# Check that https://packages.debian.org/trixie/g++-mingw-w64-x86-64-posix (version 14.x, similar to guix) can cross-compile
# Using buildx is required to properly load the correct driver, for use with registry caching. Neither build, nor BUILDKIT=1 currently do this properly
echo"Creating $CI_IMAGE_NAME_TAG container to run in"
# Use buildx unconditionally
# Using buildx is required to properly load the correct driver, for use with registry caching. Neither build, nor BUILDKIT=1 currently do this properly
ap.add_argument("-no-strip", dest="strip", action="store_false", default=True, help="don't run 'strip' on the binaries")
ap.add_argument("-translations-dir", nargs=1, metavar="path", default=None, help="Path to Qt's translations. Base translations will automatically be added to the bundle's resources.")
ap.add_argument("-zip", nargs="?", const="", metavar="zip", help="create a .zip containing the app bundle")
ap.add_argument("-zip", nargs=1, metavar="zip", help="create a .zip containing the app bundle")
# Copyright (c) 2014-2021 The Bitcoin Core developers
# Copyright (c) 2014-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.