Previously, the signal was using btcsignals::optional_last_value<bool>.
However, this only worked by accident:
The return value was influenced by the order in which the connections
were done. The noui callbacks would always overwrite the return value
with false. This makes the code overall brittle, and confusing.
For example, the following patch that changes the order of connections
would break the only and single place where the return value actually
matters:
```diff
diff --git a/src/qt/bitcoin.cpp b/src/qt/bitcoin.cpp
index 0b89c605b9..976549470e 100644
--- a/src/qt/bitcoin.cpp
+++ b/src/qt/bitcoin.cpp
@@ -488,3 +488,2 @@ int GuiMain(int argc, char* argv[])
btcsignals::scoped_connection handler_message_box = ::uiInterface.ThreadSafeMessageBox_connect(noui_ThreadSafeMessageBox);
- btcsignals::scoped_connection handler_question = ::uiInterface.ThreadSafeQuestion_connect(noui_ThreadSafeQuestion);
btcsignals::scoped_connection handler_init_message = ::uiInterface.InitMessage_connect(noui_InitMessage);
@@ -663,2 +662,3 @@ int GuiMain(int argc, char* argv[])
app.createWindow(networkStyle.data());
+ btcsignals::scoped_connection handler_question = ::uiInterface.ThreadSafeQuestion_connect(noui_ThreadSafeQuestion);
// Perform base initialization before spinning up initialization/shutdown thread
```
This can be tested by applying the patch and then calling:
(May have to be started twice to trigger the question)
```
bitcoin-qt -regtest -datadir=/tmp -mocktime=123456789
```
Before the changes in this commit (on current master), pressing `OK`
would not have any effect and would abort the program.
After the changes in this commit, pressing `OK` will correctly trigger a
-reindex and leave the program running.
The message will always return false (a constant) and the return value
is never used.
Also, annotate ThreadSafeMessageBox in the GUI code as [[nodiscard]],
because it may actually return a value, which is handled for questions
(but not for messages).
5a2e359213 clarify blockfilterindex cache allocation rationale (Sebastian van Staa)
d06dabf26b node: allocate index caches proportional to usage patterns (Sebastian van Staa)
Pull request description:
The current cache allocation for optional indexes (txindex, txospenderindex, blockfilterindex) uses a sequential total_cache / 8 approach where each index gets 1/8 of the remaining budget after the previous index has been allocated. This means the order in which indexes appear in the code silently determines how much cache each one gets.
|Index|Current share of total|
|---|---|
|txindex|~12%|
|txospenderindex|~11%|
|blockfilterindex|~10%|
This is unintuitive, undocumented, and probably doesn't reflect actual usage patterns. This PR replaces the sequential 1/8 allocation with explicit percentages based on how the indexes are typically used. The current values are an educated guess, and subject to further benchmark and research of typical client usage patterns.
|Index|Allocation|Rationale|
|---|---|---|
|txindex |10% |Serves getrawtransaction RPCs with mostly unique lookups across the entire blockchain: low cache reuse|
|txospenderindex|5%|Serves gettxspendingprevout RPCs with very specific outpoint queries: likely the least repetitive access pattern|
|blockfilterindex|5%|Serves BIP 157 light clients that repeatedly query the same recent blocks: highest cache benefit|
UPDATE: blockfilterindex allocation changed from 15% to 5% in the course of the discussion
This is a continuation of the related discussion: https://github.com/bitcoin/bitcoin/pull/24539#discussion_r2809088034 and https://github.com/bitcoin/bitcoin/pull/31483.
Further feedback and input is very much appreciated.
ACKs for top commit:
fjahr:
ACK 5a2e359213
rustaceanrob:
ACK 5a2e359213
achow101:
ACK 5a2e359213
sedited:
ACK 5a2e359213
Tree-SHA512: 69be2b0c274b975da58aef2513c3042be8a4c8acf0a86af86b962d4ebfd8cf90bcb1d9251d53995652b4825d0d1da24aabe92cdada9148c627690f8ad2ad8a29
3f44f9aef7 test: Add coverage for m_blocks_unlinked invariant in LoadBlockIndex (marcofleon)
0e4b0bacec validation: Don't add pruned blocks to m_blocks_unlinked on startup (marcofleon)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/35050
The `m_blocks_unlinked` map keeps track of blocks that have transactions but whose parent (or any ancestor) does not. This happens when a block is received before its parent, or during a reorg, when `FindMostWorkChain()` encounters a block whose ancestors were pruned.
The bug this PR addresses is a rare interaction of these two cases, which happens on startup when `BlockManager::LoadBlockIndex()` rebuilds `m_blocks_unlinked`. The check there only considers whether a block has transactions, and pruned blocks keep `nTx > 0` but clear `BLOCK_HAVE_DATA`. So if there's a pruned block on a stale fork whose parent has no transactions, that block is added to `m_blocks_unlinked` without having data on disk. This violates an [assertion](ad3f73862b/src/validation.cpp (L5352)) in `CheckBlockIndex()`.
Get rid of this unintended case by gating on `BLOCK_HAVE_DATA` before adding to `m_blocks_unlinked`.
ACKs for top commit:
achow101:
ACK 3f44f9aef7
sedited:
Re-ACK 3f44f9aef7
stratospher:
ACK 3f44f9a. nice!
Tree-SHA512: 275d0f8588524c01c4e701c8635973cd4a086d31c10d252a498c1ef668bdb3895ba1cae265dbe88f8983ca7ddbe32247824753c7c1f49e59c8bce0df377b784c
ec6cf49b91 blockstorage: Remove cs_LastBlockFile recursive mutex (sedited)
Pull request description:
The `cs_LastBlockFile` mutex is redundant: all critical sections are already covered by cs_main. This is demonstrated in this patch by replacing all instances of locking `cs_LastBlockFile` with pairs of `AssertLockHeld(::cs_main)` and `EXCLUSIVE_LOCKS_REQUIRED(::cs_main)` annotations. No additional `::cs_main` LOCK(...)s are introduced (besides for test-only code).
It is also not clear for which sections `cs_LastBlockFile` is responsible for. It is annotated for `m_blockfile_cursors`, but sporadically and inconsistently also covers `m_blockfile_info` (e.g. in `LoadBlockIndexDB`).
Since it has no semantic meaning, and seems confusing to developers, remove it.
An alternative to this patch would be expanding the scope of what `cs_LastBlockFile` covers and turning it into a non-recursive mutex. I prepared such a patch some time ago, but found it unsatisfactory. It was not clear to me if the lock was now covering too much or too little, and its purpose remained unclear. If this patch is accepted, I would expect the project to eventually implement a separate, narrowly-scoped block storage lock to allow for a more parallelizable block processing routine.
ACKs for top commit:
stickies-v:
re-ACK ec6cf49b91
janb84:
re- ACK ec6cf49b91
pablomartin4btc:
ACK ec6cf49b91
Tree-SHA512: e5942bc87300b0db9a0b91d5fe26dab455049e6cef7c96bb12b28141fa04711d46c6af105c0e1a83a9f261edde2c8b8b43ecf577a27d54b4610d784676a85627
The cs_LastBlockFile mutex is redundant: all critical sections are
already covered by cs_main. This is demonstrated in this patch by
replacing all instances of locking cs_LastBlockFile with pairs of
`AssertLockHeld(::cs_main)` and `EXCLUSIVE_LOCKS_REQUIRED(::cs_main)`
annotations. No additional `::cs_main` LOCK(...)s are introduced.
It is also not clear for which sections `cs_LastBlockFile` is
responsible for. It is annotated for `m_blockfile_cursors`, but
sporadically and inconsistently also covers `m_blockfile_info`.
Since it has no semantic meaning, and seems confusing to developers,
remove it.
BIP434 defines FEATURE messages which are sent between VERSION and VERACK
to indicate support for new P2P protocol features. This commit provides
the infrastructure for easily using BIP434 negotiation when implementing
such new P2P protocol features. Note that advertised protocol version
is bumped to 70017, as per BIP434's specification.
Add a submitBlock method to the Mining IPC interface, similar to the
submitblock RPC. This accepts a fully assembled block, validates it, and
if accepted as new, processes it into chainstate.
This is needed for Stratum v2 Job Declarator Server (JDS), where accepted
solutions may correspond to jobs not tied to a Bitcoin Core BlockTemplate.
JDS receives PushSolution fields and reconstructs full blocks; without an
IPC submitBlock method, final submission requires the submitblock RPC.
The method returns detailed status (reason/debug strings) matching the
checkBlock pattern, giving callers enough information to handle
validation failures.
Introduce a SubmitBlock() helper in node/miner.cpp that wraps
ProcessNewBlock submission and captures validation state through
the BlockChecked callback.
Route submitSolution through the helper before adding any new IPC method.
No behavior change.
Read configured mining options once during startup instead of parsing
options like -blockmaxweight each time a block template is generated.
Store the parsed startup options as BlockCreateOptions. Members left unset
keep representing unset options; hardcoded defaults are applied by
FlattenMiningOptions() before the options are used. IPC overrides and node
defaults can then be merged with the same option type used by
BlockAssembler.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Check BlockCreateOptions before block template creation instead of
clamping runtime values. This makes invalid runtime block creation
options, including those passed by IPC mining clients, fail explicitly
instead of silently mining with different values than the caller
requested.
Runtime option validation now uses the same error wording as startup
option validation. Startup validation also rejects -blockmaxweight
values lower than -blockreservedweight instead of allowing them to be
clamped later.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Move block template defaulting into helper functions for
BlockCreateOptions. FlattenMiningOptions() fills hardcoded defaults and
MergeMiningOptions() overlays defaults without replacing caller-provided
values.
Use the shared option type in BlockAssembler so IPC callers and internal
callers can pass through the same options path. This commit does not
change behavior, except for dropping the "Specified " prefix from startup
option error messages.
Keep the -blockmintxfee ParseMoney check in ReadMiningArgs() instead of
CheckMiningOptions(), because CheckMiningOptions() only sees the parsed
CFeeRate value and not the original string.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
This new optional replaces nBlockMaxWeight.
Use uint64_t for the block weight options to match BlockAssembler's
nBlockWeight accounting and the IPC schema's blockReservedWeight type.
A negative -blockmaxweight value is now parsed as 0 instead of an
overflowed signed value before validation rejects it.
The new block_max_weight option is not exposed to IPC clients.
Move the argument parsing for -blockmaxweight, -blockreservedweight,
-blockmintxfee out of init.cpp to a dedicated mining_args.cpp.
This is mostly a refactor and keeps the existing validation checks. It
does switch the weight arguments to GetArg<uint64_t>, introduced in
bitcoin/bitcoin#34582, so very large or negative weight values can be
reported differently in error messages.
Have most tests, benchmarks and fuzzers go through the mining interface.
This is a refactor: it does not change what blocks are created, just how
the creation calls are made. This avoids most direct test, benchmark and
fuzzer use of node::BlockAssembler::Options, making it easier to drop in a
later commit.
Two exceptions which use BlockAssembler directly:
- one check in test/miner_tests.cpp needs m_package_feerates
- fuzz/tx_pool.cpp Finish() doesn't have access to a NodeContext
Move test_block_validity from BlockAssembler::Options to
BlockCreateOptions so bench/block_assemble.cpp can continue to set it.
Just like coinbase_output_script, this is not exposed to IPC clients.
Inline options variable in places where it's only needed once.
We also drop one unused PrepareBlock declaration and one unused
implementation.
TestChain100Setup::CreateBlock no longer needs a chainstate argument,
which in turn means it can be dropped from CreateAndProcessBlock. Using
the Mining interface here also requires marking the test
KernelNotifications chainstate as loaded after LoadVerifyActivateChainstate().
The next commit switches test helpers that take a const NodeContext& to
create blocks through the Mining interface, so MakeMining needs to
accept a const NodeContext too.
02b2c41103 logging: use util/log.h where possible (Anthony Towns)
57d7495fe5 IWYU fixes (Anthony Towns)
611878b46f scripted-diff: logging: Drop LogAcceptCategory (Anthony Towns)
34332dba2f util/log, logging: Provide ShouldDebugLog and ShouldTraceLog instead of a generic ShouldLog (Anthony Towns)
abea304dd6 logging: Move GetLogCategory into Logger class (Anthony Towns)
58113e5833 util/log: Rename LogPrintLevel_ into detail_ namespace (Anthony Towns)
f69d1ae56d util/log: Provide util::log::NO_RATE_LIMIT to avoid rate limits (Anthony Towns)
72e92d67df logging: Protect ShrinkDebugFile by m_cs (Anthony Towns)
904c0d07bb util/stdmutex: Drop StdLockGuard (Anthony Towns)
Pull request description:
`ShrinkDebugFile` now takes the logging mutex for its entire run; though it's only called in init so shouldn't have any races in the first place.
Adds a `NO_RATE_LIMIT` tag that can be used with info/warning/error logs to avoid rate-limiting. This allows `LogPrintLevel_` to be restricted to being an internal API.
The `GetLogCategory` function is moved out of the global namespace.
`ShouldLog` is split into separate `ShouldDebugLog` and `ShouldTraceLog` so that filtering checks are somewhat more enforced via function signature checks.
Redundant `LogAcceptCategory` function is removed.
More files are pointed at util/log.h instead of logging.h.
ACKs for top commit:
maflcko:
review ACK 02b2c41103📅
sedited:
Re-ACK 02b2c41103
l0rinc:
untested ACK 02b2c41103
ryanofsky:
Code review ACK 02b2c41103435d8dbaa77a526e484066471b2b8c! Overall a lot of nice improvements here.
Tree-SHA512: 3bffdca91afbe5c45a522815fe82e6f4cfa96529a4a243b29aad21234650502d6cac780126b584ee3e7ec129d8fdd50670d8a05036cc5c36e586b8c4c3563970
The previous commit made CreateNewBlock() ignore the
include_dummy_extranonce flag (OP_0 is now appended only at heights
<= 16 to satisfy bad-cb-length, regardless of caller-supplied options).
Remove the now-unused field from BlockCreateOptions and clean up all
the call sites that still set it (RPC, bench, fuzz, test setups).
Drop the include_dummy_extranonce branch from the OP_0 padding
condition in CreateNewBlock(), so that the dummy extraNonce is
only appended when consensus actually requires it (heights <= 16,
where the BIP34 height push alone would yield a 1-byte scriptSig
and trigger bad-cb-length).
The include_dummy_extranonce option struct field is now unused by
the miner and is removed in the next commit. Callers still set it,
so that this commit compiles.
Regenerate the hardcoded coinbase / block hashes throughout the
unit and functional test suites and update the regtest assumeutxo
snapshot in chainparams.
Additional side-effects:
- Without the dummy extranonce, coinbase scriptSigs are 1 byte
shorter at heights > 16, making every block 1 byte smaller.
This shifts where block files wrap and therefore where pruning
boundaries land.
- feature_assumeutxo malleation cases:
- case 1: error message changes due to UTXO reordering, similar
to 8f2078af6a
- case 4: the corruption byte is swapped from \x82 to \x83
because \x82 happened to be the actual value at that
offset in the new snapshot.
Since #32420, createNewBlock has thrown `bad-cb-length` errors when called at
low block heights because `OP_0` padding stopped being added to coinbase
transactions. (#32420 did add an `include_dummy_extranonce` option which could
bypass this, but it was not exposed to IPC clients.) Fix the problem by padding
coinbase transactions with `OP_0` when necessary to produce valid blocks.
Additionally this commit stops adding `OP_0` padding to the template
`script_sig_prefix` field when `include_dummy_extranonce` is true. This is safe
because non-IPC clients don't use this field, and IPC clients could never set
the option to true, and are expected to add their own nonces in any case.
This also improves documentation about the `script_sig_prefix` field and
`getCoinbaseTx` method.
d084bc88be doc: clarify IWYU workflow (Lőrinc)
7c7cec4567 ci: update IWYU patch reference (Lőrinc)
Pull request description:
### Problem
This was prompted by https://github.com/bitcoin/bitcoin/pull/34435#discussion_r3123255248, where it was not clear to me how (and where) exceptional IWYU cases should be documented.
### Fix
This PR documents the IWYU CI wrapper as the reproducible local entrypoint.
The developer notes now recommend reducing suspected IWYU false positives to a minimal upstream reproducer, treat `IWYU pragma` as a narrow workarounds, and ask for nearby rationale comments on non-obvious IWYU pragma use. An example comment was also added.
The IWYU patch comment is also updated to point at the current `clang_22` include picker reference.
### Reproducer
Create a dummy commit on top that adds an unused include, then run the command from the developer notes.
Without the dummy commit, the command should pass.
<details><summary>IWYU demo commit</summary>
```diff
diff --git a/src/kernel/bitcoinkernel.cpp b/src/kernel/bitcoinkernel.cpp
--- a/src/kernel/bitcoinkernel.cpp(revision c92b329e7b7d49476b5977d26c24d7c4982c6024)
+++ b/src/kernel/bitcoinkernel.cpp(revision ad2c5ba2ba69156e77061c1e6c098b725c28f322)
@@ -43,6 +43,7 @@
#include <functional>
#include <list>
#include <memory>
+#include <vector>
#include <span>
#include <stdexcept>
#include <string>
```
</details>
> [!NOTE]
> After repeated failing runs, `docker container rm -f ci_native_iwyu` may be needed because the local CI wrapper can leave the detached container running when the inner test command fails.
ACKs for top commit:
hebasto:
ACK d084bc88be.
sedited:
ACK d084bc88be
Tree-SHA512: 0aac42d468a1fdfa9f4a3856372e05fb43ec9f0973aeb3a4194fff948fc61e8e72e3b280cde10e74b8da88b6cff93962b3a7f7390eb042113ef92aa6b51d6d8f
9fa4076b20 test: Test merging implicit PSBTv0 with explicit PSBTv0 (w0xlt)
1660c18232 doc: Release notes for psbtv2 (Ava Chow)
470e52a5f8 fuzz: Enforce additional version invariants in PSBT fuzzer (Antoine Poinsot)
5bd0579c09 test: Tests for PSBT AddInput and AddOutput (Ava Chow)
b8b6e7f0c2 tests: Add PSBT unit test for ComputeTimeLock (Ava Chow)
0bc1c2e508 tests: Add test vectors from BIP 370 (Ava Chow)
e0e4dbdeb5 psbt: Change default psbt version to 2 (Ava Chow)
bcc1dca77b Add psbt_version to PSBT RPCs and default to v2 (Ava Chow)
ab38c30195 Implement PSBTv2 field merging (Ava Chow)
93e339e29f Implement PSBTv2 AddInput and AddOutput (Ava Chow)
b39c86ae60 Allow specifying PSBT version in constructor (Ava Chow)
dcc9a3c8df Implement PSBTv2 in decodepsbt (Ava Chow)
5770dbd39f Add PSBT::ComputeLockTime() (Ava Chow)
863cf47b33 Update test_framework/psbt.py for PSBTv2 (Ava Chow)
925161eaf0 Implement PSBTv2 fields de/ser (Ava Chow)
d9cf658ee0 Restrict joinpsbts to PSBTv0 only (Ava Chow)
3da0e16012 Replace PSBT.tx with PSBT::GetUnsignedTx and PSBT::GetUniqueID (Ava Chow)
c568624ff2 psbt: Return std::optional from PrecomputePSBTData (Ava Chow)
092de4f1f6 Replace PSBT::GetInputUTXO with PSBTInput::GetUTXO (Ava Chow)
82c9fe3179 psbt: Use PSBTInput and PSBTOutput fields instead of accessing global tx (Ava Chow)
95897507e9 psbt: AddInput and AddOutput should take only PSBTInput and PSBTOutput (Ava Chow)
1b7d323a72 Add PSBTInput::GetOutPoint (Ava Chow)
543d3e1cdc psbt: add PSBTv2 global tx fields (Ava Chow)
c01c7f068c psbt: Remove default constructor (Ava Chow)
9671aa08c2 psbt: add tx input and output fields in PSBTInput and PSBTOutput (Ava Chow)
990b084f11 Have PSBTInput and PSBTOutput know the PSBT's version (Ava Chow)
7eacc21ff6 psbt: make PSBT structs into classes (Ava Chow)
f926c326bb gui: Store PSBT in std::optional in PSBTOperationsDialog (Ava Chow)
1e2d146b47 psbt: Refactor duplicate key lookup and size checks (Ava Chow)
88384180d3 test: PSBTs should roundtrip through RPCs that do nothing (Ava Chow)
001877500d test: construct psbt with unknown field programmatically (David Gumberg)
0cb884e6df psbt: Fill hash preimages and taproot builder from SignatureData (Ava Chow)
Pull request description:
BIP 370 PSBTv2 introduces several new fields and different invariants for PSBT. This PR implements those new fields and restructures the PSBT implementation to match PSBTv2 but still remain compatible with PSBTv0.
ACKs for top commit:
nervana21:
re-ACK 9fa4076b20
theStack:
re-ACK 9fa4076b20
w0xlt:
ACK 9fa4076b20
Tree-SHA512: ab0a5ada4fa5fca27ba9ec9c291a44b30e69d6db11971957572d86c58c71c4caa4557dc25f403e1170ba4fac751306d074cc582defefc6e2fdd37be51c3d9dd0
1aa78cdab6 clusterlin: adopt STL ranges algorithms (refactor) (Pieter Wuille)
747da25360 feefrac: drop comparison and operator{<<,>>} for sorted wrappers (Pieter Wuille)
Pull request description:
Instead of having an unintuitive but strong implicit sort order on `FeeFrac` (first increasing feerate, then decreasing size), and separate overloaded `operator<<` and `operator>>` that implement a weak ordering that only looks at feerate, replace these with explicit wrapper classes which make the behavior more explicit (`ByRatio` and `ByRatioNegSize`).
This allows for things like `ByRatio{a} <= ByRatio{b}`, instead of the earlier `!(a >> b)`. It also supports usage inside `std::min`/`std::max`/`std::less`, and `std::greater`, so one can use:
* `std::max<ByRatioNegSize<FeeFrac>>(a, b)` to get the highest-feerate `FeeFrac`, tie-breaking by smallest size.
* `std::ranges::sort(v, std::greater<ByRatioNegSize<FeeFrac>>{});` to sort a vector that way.
Suggested in https://github.com/bitcoin/bitcoin/pull/34257#discussion_r2780475893.
ACKs for top commit:
achow101:
ACK 1aa78cdab6
sedited:
ACK 1aa78cdab6
ajtowns:
ACK 1aa78cdab6
Tree-SHA512: d76657b15f6d745e5ca01c67fd5b101fdc418e6301646d14e575b6564bfa2fe0eb40a95a7ff95a4420624ef6b67224d35e4713aa5bbc0d293e08fe44c0cc6db0
The global unsigned tx is decomposed into separate fields inside of
PSBT, which mirrors what PSBTv2 will do. However, we still need to get
the global unsigned tx so PSBT::GetUnsignedTx is introduced to do that.
In order to also have a stable unique ID, we also introduce
PSBT::GetUniqueID to replace uses of PSBT.tx.GetHash().
PSBTInput now has the previous txid and output index, and PSBTOutput has
the amount and script. We no longer need to access the global unsigned
tx for these fields.
Additionally, we can change iterating tx.vin and tx.vout to psbtx.inputs
and psbtx.outputs.
This is in prepration for use with PSBTv2 where the global unsigned tx
will not exist.
0bc9d354df multi_index: fix compilation failure with boost >= 1.91 (Cory Fields)
Pull request description:
This effectively reverts a3cb309e7c from PR #30194.
That PR reduced the `multi_index` type signatures as recommended upstream, but this is no longer supported as of boost 1.91 because it is no longer necessary. 1.91 drops support for the pre-c++11 work-arounds that bloated the type signatures to begin with.
The upstream `BOOST_MULTI_INDEX_ENABLE_MPL_SUPPORT` define is meant to provide compatibility with removed features, but it does not work for this case. Using `indexed_by` directly when defining the `multi_index` (as opposed to inheriting from it) works with all versions, and avoids the use of the back-compat define.
This is a slight regression when building against boost < 1.91 because the bloated type signatures are reintroduced in that case, but it's not significant enough to go to the trouble of introducing version detection and ifdefs.
ACKs for top commit:
maflcko:
review ACK 0bc9d354df🎶
hebasto:
ACK 0bc9d354df.
w0xlt:
ACK 0bc9d354df
Tree-SHA512: 883ee998efd16d944628653ca204e3d2acaf2554b2eced40556143a66d6072a3625861d961d1a4a194a7b8d4d448562581e5d11a09380754a5635a871d2a0aa1
Document the CI wrapper as the reproducible IWYU entrypoint instead of suggesting ad hoc native runs.
Also describe how to handle suspected false positives, explain when local `IWYU pragma` workarounds are appropriate, and add an example rationale to an existing pragma.
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
Co-authored-by: Daniel Pfeifer <daniel@pfeifer-mail.de>
This effectively reverts a3cb309e7c from PR #30194.
That PR reduced the multi_index type signatures as recommended upstream, but
this is no longer supported as of boost 1.91 because it is no longer necessary.
1.91 drops support for the pre-c++11 work-arounds that bloated the type
signatures to begin with.
The upstream `BOOST_MULTI_INDEX_ENABLE_MPL_SUPPORT` define is meant to provide
compatibility with removed features, but it does not work for this case. Using
`indexed_by` directly when defining the `multi_index` (as opposed to inheriting
from it) works with all versions, and avoids the use of the back-compat define.
This is a slight regression when building against boost < 1.91 because the
bloated type signatures are reintroduced in that case, but it's not significant
enough to go to the trouble of introducing version detection and ifdefs.
Replace the sign, finalize , bip32derivs and sighash_type arguments which
are passed to FillPSBT() and SignPSBTInput() with a PSBTFillOptions struct.
This makes it easier to add additional options later without large code
churn, such as avoid_script_path proposed in #32857. It also makes the
use of default boolean options safer compared to positional arguments
that can easily get mixed up.
LoadBlockIndex() adds to m_blocks_unlinked based only on nTx > 0, without
checking BLOCK_HAVE_DATA. Pruning preserves nTx but clears BLOCK_HAVE_DATA,
so a pruned block whose parent was header-only gets re-added on every
restart, causing the CheckBlockIndex() assertion that entries must have
data on disk to fail.
Check that BLOCK_HAVE_DATA is set before inserting into m_blocks_unlinked.
Fixes#35050.
af0ee28eb6 refactor: use _MiB consistently for Mebibyte conversions (Lőrinc)
b3edd30aa2 util: add _GiB for Gibibyte conversions (Lőrinc)
Pull request description:
### Problem
Byte-size conversions in the codebase currently show up in many equivalent formats (multiplication/division chains, shifts, hex/binary literals), which creates a maintenance burden and makes review error-prone - especially considering the architectural differences of `size_t`.
Inspired by https://github.com/bitcoin/bitcoin/pull/34305#discussion_r2734720002, it seemed appropriate to unify `Mebibyte` usage across the codebase and add `Gibibyte` support with 32/64 bit `size_t` validation.
### Fix
This PR refactors those call sites to use `""_MiB` (existing) and `""_GiB` (new), and adds the encountered value/pattern replacements to unit tests to make review straightforward, and to ensure the conversions remain valid.
The literals are overflow-checked when converting to `size_t`, and unit tests cover the 32-bit boundary cases.
Concretely, it replaces patterns such as:
* `1024*1024`, `1<<20`, `0x100000`, `1048576`, `/ 1024 / 1024`, `* (1.0 / 1024 / 1024)` → `1_MiB` or `double(1_MiB)`
* `1024*1024*1024`, `1<<30`, `0x40000000`, `1024_MiB`, `>> 30` → `1_GiB`
(added unit tests for each replacement category to ease review)
Additionally, declarations whose initializer reads a `_MiB`/`_GiB` literal are switched to braced initialization so a future oversized value is rejected at compile time through the narrowing check instead of silently truncating.
### Note
In the few places where arithmetic involves signed values, the result is identical to the previous code assuming those quantities never become negative.
ACKs for top commit:
achow101:
ACK af0ee28eb6
janb84:
ACK af0ee28eb6
maflcko:
review ACK af0ee28eb6 🖍
hodlinator:
re-ACK af0ee28eb6
Tree-SHA512: 55286ce3f833f88335394a74e9e0b95c7d023e5cdc9ded40accbbbcd870101e4dcc05926865d6bef4c1be1ebd648aa3fdf947ef9575633ccfe56691f145d7a2d
fa1015bbcb refactor: Use NodeClock::time_point for m_connected (MarcoFalke)
fa244b984c refactor: Use NodeClock::time_point for m_last_send/recv and m_ping_start (MarcoFalke)
fa2605b204 refactor: Use NodeClock::time_point for CNetMessage::m_time (MarcoFalke)
fa644e625b refactor: Use NodeClock::duration for m_last_ping_time/m_min_ping_time/m_ping_wait (MarcoFalke)
333316f6be doc: Fix typo "eviction criterium" -> "eviction criterion" (MarcoFalke)
fa54fb0129 refactor: gui: Accept up to nanoseconds in formatDurationStr, but clarify they are ignored (MarcoFalke)
fab88884b7 refactor: Avoid manual chrono casts with * or / (MarcoFalke)
facfce37f6 util: Add NodeClock::epoch alias (MarcoFalke)
fa41e072b3 refactor: Use NodeClock alias over deprecated GetTime (MarcoFalke)
Pull request description:
It is a bit confusing to have some code use the deprecated `GetTime`, which returns a duration and not a time point, and other code to use `NodeClock` time points.
Fix a few more places to properly use time_point types.
ACKs for top commit:
stickies-v:
re-ACK fa1015bbcb
seduless:
re-ACK fa1015bbcb
naiyoma:
ACK fa1015bbcb
sedited:
ACK fa1015bbcb
Tree-SHA512: 7c8df1a9025271b08a40fd0d176bcbbf90920bc4d83a6e1c8cfaad2a894632af2b9a1aca5c3c9ddc3803e559dd168244121fd188ef22f399d60075ff194a9140
Replace hard-coded MiB byte conversions (e.g. `1024*1024`, `1<<20`, `1048576`) with the existing `_MiB` literal to improve readability and avoid repeating constants.
In the few spots where arithmetic involves signed values, the result is identical to the previous code assuming those quantities never turn negative.
Also switch to brace init on every declaration assigned from `_MiB`/`_GiB` literals so a future oversized value (e.g. `unsigned int x{4096_MiB}`) becomes a compile error through the C++11 narrowing check instead of silently truncating.
Extend unit tests to cover the 32-bit `size_t` overflow boundary and to assert equivalence for integer and floating-point conversions.
Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
Co-authored-by: w0xlt <94266259+w0xlt@users.noreply.github.com>
Introduce `operator""_GiB`, sharing the overflow-checked conversion logic with the existing `operator""_MiB`.
Use `1_GiB` in a few existing places where it is a drop-in replacement (e.g. `1024_MiB`, `1<<30`) and extend unit tests to cover boundary behavior.
fc736013a5 rpc: Add in_memory option to dumptxoutset with rollback (Fabian Jahr)
d0fd718948 test: Extend named pipe sqlite tool test to use rollback (Fabian Jahr)
ab9463efac test: Add dumptxoutset fork test (Fabian Jahr)
49d5e835a8 rpc: Don't invalidate blocks in dumptxoutset (Fabian Jahr)
fe58eb9850 blockstorage: Add DeletePruneLock (Fabian Jahr)
Pull request description:
This is an alternative approach to implement `dumptxoutset` with rollback that was discussed a few times. It does not rely on `invalidateblock` and `reconsiderblock` and instead creates a temporary copy of the coins DB, modifies this copy by rolling back as many blocks as necessary and then creating the dump from this temp copy DB. See also https://github.com/bitcoin/bitcoin/pull/29553#issuecomment-1978480989, https://github.com/bitcoin/bitcoin/issues/32817#issuecomment-3012406102 and #29565 discussions.
The nice side-effects of this are that forks can not interfere with the rollback and network activity does not have to be suspended. But there are also some downsides when comparing to the current approach: this does require some additional disk space for the copied coins DB and performance is slower (master took 3m 17s vs 9m 16s in my last test with the code here, rolling back ~1500 blocks). However, there is also not much code being added here, network can stay active throughout and performance would stay constant with this approach while it would impact master if there were forks that needed to be invalidated as well (see #33444 for the alternative approach), so this could still be considered a good trade-off.
ACKs for top commit:
stratospher:
tested ACK fc73601. very nice!
sedited:
Re-ACK fc736013a5
theStack:
re-ACK fc736013a5
Tree-SHA512: d3d674f68184ac3ada87d969d0fca7bc38203ee939853864adcd235ee3a954914c7e351b817800b885a495606e323392c27d88ba8d8e018eaf8567c098eb0e9c
8783cc8056 refactor: inline `CCoinsViewBacked` implementation (Lőrinc)
86296f276d coins: make `CCoinsView` methods pure virtual (Lőrinc)
b637566c8d coins: add explicit `CoinsViewEmpty` noop backend (Lőrinc)
90c635c01c fuzz: keep backend assertions aligned to active backend (Lőrinc)
a9f92e3497 refactor: normalize CCoinsView whitespace and signatures (Lőrinc)
38a99f3344 scripted-diff: normalize `CCoinsView` naming (Lőrinc)
06172ef0d5 refactor: rename `hashBlock` to `m_block_hash` to avoid shadowing (Lőrinc)
Pull request description:
### Problem
`CCoinsView` is the coins view interface, but historically it also provided built-in no-op behavior:
* It provided default no-op implementations (returning `std::nullopt`, `uint256()`, `false`, or `nullptr`) instead of being pure virtual.
* Callers could instantiate a bare `CCoinsView` and get silent no-op behavior.
* Mixing the interface definition with a built-in dummy implementation blurred the abstraction boundary.
### Context
This is part of the ongoing coins caching cleanup in #34280.
### Fix
This PR separates the interface from no-op behavior and makes `CCoinsView` pure virtual in incremental steps:
* Add `CoinsViewEmpty` as an explicit no-op coins view for tests, benchmarks, and temporary backends.
* Replace direct bare-`CCoinsView` test and dummy instantiations with `CoinsViewEmpty`.
* Make all `CCoinsView` methods pure virtual (`PeekCoin`, `GetCoin`, `HaveCoin`, `GetBestBlock`, `GetHeadBlocks`, `BatchWrite`, `Cursor`, `EstimateSize`).
* Remove the legacy default implementations from `coins.cpp`.
* Update fuzz and dummy backends to use `CoinsViewEmpty` explicitly.
ACKs for top commit:
w0xlt:
reACK 8783cc8056
ryanofsky:
Code review ACK 8783cc8056. Just updated comments and variable name since last review. The fuzz test code is much clearer now IMO
andrewtoth-exo:
ACK 8783cc8056
ajtowns:
ACK 8783cc8056
Tree-SHA512: cfc831578aa309788c1b5dafbfecca3de388cc4215534c3f3df24f90d7770ed37b1fd7aa134df91d611d0a1ca75929accb98d5ed7df7b52851c259e04f08e4a3
242b0ebb5c btcsignals: use a single shared_ptr for liveness and callback (Cory Fields)
b12f43a0a8 signals: remove boost::signals2 from depends and vcpkg (Cory Fields)
a4b1607983 signals: remove boost::signals2 mentions in linters and docs (Cory Fields)
375397ebd9 signals: remove boost includes where possible (Cory Fields)
091736a153 signals: re-add forward-declares to interface headers (Cory Fields)
9958f4fe49 Revert "signals: Temporarily add boost headers to bitcoind and bitcoin-node builds" (Cory Fields)
34eabd77a2 signals: remove boost compatibility guards (Cory Fields)
e60a0b9a22 signals: Add a simplified boost-compatible implementation (Cory Fields)
63c68e2a3f signals: add signals tests (Cory Fields)
edc2978058 signals: use an alias for the boost::signals2 namespace (Cory Fields)
9ade3929aa signals: remove forward-declare for signals (Cory Fields)
037e58b57b signals: use forwarding header for boost signals (Cory Fields)
2150153f37 signals: Temporarily add boost headers to bitcoind and bitcoin-node builds (Cory Fields)
fd5e9d9904 signals: Use a lambda to avoid connecting a signal to another signal (Cory Fields)
Pull request description:
This drops our dependency on `boost::signals2`, leaving `boost::multi_index` as the only remaining boost dependency for bitcoind.
`boost::signals2` is a complex beast, but we only use a small portion of it. Namely: it's a way for multiple subscribers to connect to the same event, and the ability to later disconnect individual subscribers from that event.
`btcsignals` adheres to the subset of the `boost::signals2` API that we currently use, and thus is a drop-in replacement. Rather than implementing a complex `slot` tracking class that we never used anyway (and which was much more useful in the days before std::function existed), callbacks are simply wrapped directly in `std::function`s.
The new tests work with either `boost::signals2` or the new `btcsignals` implementation. Reviewers can verify
functional equivalency by running the tests in the commit that introduces them against `boost::signals2`, then again with `btcsignals`.
The majority of the commits in this PR are preparation and cleanup. Once `boost::signals2` is no longer needed, it is removed from depends. Additionally, a few CMake targets no longer need boost includes as they were previously only required for signals.
I think this is actually pretty straightforward to review. I kept things simple, including keeping types unmovable/uncopyable where possible rather than trying to define those semantics. In doing so, the new implementation has even fewer type requirements than boost, which I believe is due to a boost bug. I've opened a PR upstream for that to attempt to maintain parity between the implementations.
See individual commits for more details.
Closes#26442.
ACKs for top commit:
fjahr:
Code review ACK 242b0ebb5c
maflcko:
re-review ACK 242b0ebb5c🎯
w0xlt:
reACK 242b0ebb5c
Tree-SHA512: 9a472afa4f655624fa44493774a63b57509ad30fb61bf1d89b6d0b52000cb9a1409a5b8d515a99c76e0b26b2437c30508206c29a7dd44ea96eb1979d572cd4d4
