8891949bdcb25093d3a6703ae8228c3c3687d3a4 index: Improve BaseIndex::BlockUntilSyncedToCurrentChain reliability (Ryan Ofsky)
Pull request description:
Since commit f08c9fb0c6a799e3cb75ca5f763a746471625beb from PR https://github.com/bitcoin/bitcoin/pull/21726, index `BlockUntilSyncedToCurrentChain` behavior has been less reliable, and there has also been a race condition in the `coinstatsindex_initial_sync` unit test.
It seems better for `BlockUntilSyncedToCurrentChain` to actually wait for the last connected block to be fully processed, than to be able to return before prune locks are set, so this switches the order of `m_best_block_index = block;` and `UpdatePruneLock` statements in `SetBestBlockIndex` to make it more reliable.
Also since commit f08c9fb0c6a799e3cb75ca5f763a746471625beb, there has been a race condition in the `coinstatsindex_initial_sync` test. Before that commit, the atomic index best block pointer `m_best_block_index` was updated as the last step of `BaseIndex::BlockConnected`, so `BlockUntilSyncedToCurrentChain` could safely be used in tests to wait for the last `BlockConnected` notification to be finished before stopping and destroying the index. But after that commit, calling `BlockUntilSyncedToCurrentChain` is no longer sufficient, and there is a race between the test shutdown code which destroys the index object and the new code introduced in that commit calling `AllowPrune()` and `GetName()` on the index object. Reproducibility instructions for this are in https://github.com/bitcoin/bitcoin/issues/25365#issuecomment-1259744133
This commit fixes the `coinstatsindex_initial_sync` race condition, even though it will require an additional change to silence TSAN false positives, https://github.com/bitcoin/bitcoin/pull/26188, after it is fixed. So this partially addresses but does not resolve the bug reporting TSAN errors https://github.com/bitcoin/bitcoin/issues/25365.
There is no known race condition outside of test code currently, because the bitcoind `Shutdown` function calls `FlushBackgroundCallbacks` not `BlockUntilSyncedToCurrentChain` to safely shut down.
Co-authored-by: vasild
Co-authored-by: MarcoFalke
ACKs for top commit:
mzumsande:
re-ACK 8891949bdcb25093d3a6703ae8228c3c3687d3a4
Tree-SHA512: 52e29e3772a0c92873c54e5ffb31dd66a909b68a2031b7585713cd1d976811289c98bd9bb41679a8689062f03be4f97bb8368696e789caa4607c2fd8b1fe289b
fabbbe32ee09430d356fd1843f7d5c716b5f46cc Remove unused CDataStream::rdbuf method (MacroFake)
Pull request description:
It is unused and seems unlikely to be ever used.
ACKs for top commit:
theStack:
Code-review ACK fabbbe32ee09430d356fd1843f7d5c716b5f46cc
aureleoules:
ACK fabbbe32ee09430d356fd1843f7d5c716b5f46cc
Tree-SHA512: 5804642658f96a0fb51482ebf3a062bb0f997c1e0527455afa4aceeeb6c1ad139a98b14a7c8a0909daba733a83bdc24fcadad45060ead4be6eb3dc3e66c129e2
33b12e5df6aa14344dd084e0c6f2d34158fca383 docs: improve docs where MemPoolLimits is used (stickies-v)
6945853c0bf3b1941bfd18b5ffbd5a81be0e56da test: use NoLimits() in MempoolIndexingTest (stickies-v)
3a86f24a4c1e4e985b1d90eddc135b8dd17049a4 refactor: mempool: use CTxMempool::Limits (stickies-v)
b85af25f8770974bae4ef3fae64e75ef6dd2d3c2 refactor: mempool: add MemPoolLimits::NoLimits() (stickies-v)
Pull request description:
Mempool currently considers 4 limits regarding ancestor and descendant count and size, which get passed around between functions quite a bit. This PR uses `CTxMemPool::Limits` introduced in https://github.com/bitcoin/bitcoin/pull/25290 to simplify those signatures and callsites.
The purpose of this PR is to improve readability and maintenance, without behaviour change.
As noted in the first commit "refactor: mempool: change MemPoolLimits members to uint", we currently have an underflow issue where a user could pass a negative `-limitancestorsize`, which is eventually cast to an unsigned integer. This behaviour already exists. Because it's orthogonal and to minimize scope, I think this should be fixed in a separate PR.
ACKs for top commit:
hebasto:
ACK 33b12e5df6aa14344dd084e0c6f2d34158fca383, I have reviewed the code and it looks OK, I agree it can be merged.
glozow:
reACK 33b12e5df6aa14344dd084e0c6f2d34158fca383
Tree-SHA512: 591c6dcee1894f1c3ca28b34a680eeadcf0d40cda92451b4a422c03087b27d682b5e30ba4367abd75a99b5ccb115b7884b0026958d3c7dddab030549db5a4056
01bf4af4f2e67420b0180ffd39098758b1fc5e1b docs: fix m_children to be a member of CTxMemPoolEntry (stickies-v)
Pull request description:
Small documentation fix to reflect that `m_children` [is a member](73b61717a9/src/txmempool.h (L99)) of `CTxMemPoolEntry`, not `CTxMemPool`
ACKs for top commit:
hebasto:
ACK 01bf4af4f2e67420b0180ffd39098758b1fc5e1b, wrong wording was introduced in bitcoin/bitcoin#19478.
glozow:
ACK 01bf4af4f2
Tree-SHA512: b66c43b92fda44682b1f67c43073ca9e133a6dc03cd28253e571e67170531138c20b22ffdb08f312fb2d47a1f869b876611646b54325c8b614d12049befad578
97007e2b9b7a578315df2b1549cd6075130e8f05 test: Prevent UB in `minisketch_tests.cpp` (Hennadii Stepanov)
Pull request description:
[`std::optional::operator*`](https://en.cppreference.com/w/cpp/utility/optional/operator*), which follows after the changed line, can cause UB.
This PR addresses https://github.com/bitcoin/bitcoin/issues/26262#issuecomment-1268855418
ACKs for top commit:
stickies-v:
ACK 97007e2b9b7a578315df2b1549cd6075130e8f05
Tree-SHA512: a7dde8dac0cbdfa362fa1158b4564eccff9405852612227d581690c9a34084b3467ae6d4c0269262688d75339dcea90aaa38fccbba9be92d2643c2113860f3d6
0f40d653218789aa176ca2f844e3222d2ad890a3 refactor: remove duplicate code from BlockAssembler (James O'Beirne)
Pull request description:
Found while reminding myself how transactions are chosen for blocks. Take it or leave it!
ACKs for top commit:
glozow:
ACK 0f40d653218789aa176ca2f844e3222d2ad890a3
theStack:
Concept and code-review ACK 0f40d653218789aa176ca2f844e3222d2ad890a3
Tree-SHA512: 8a2694e670ce3fe897ab8f64f64c8df5f8487fc1264527a3abbcba0e5b921fb693416497ccd62508295bc33f202c65556b91b6af463acb91aab43138d2492c14
Since commit f08c9fb0c6a799e3cb75ca5f763a746471625beb from PR
https://github.com/bitcoin/bitcoin/pull/21726, index
`BlockUntilSyncedToCurrentChain` behavior has been less reliable, and there has
also been a race condition in the `coinstatsindex_initial_sync` unit test.
It seems better for `BlockUntilSyncedToCurrentChain` to actually wait for the
last connected block to be fully processed, than to be able to return before
prune locks are set, so this switches the order of `m_best_block_index =
block;` and `UpdatePruneLock` statements in `SetBestBlockIndex` to make it more
reliable.
Also since commit f08c9fb0c6a799e3cb75ca5f763a746471625beb, there has been a
race condition in the `coinstatsindex_initial_sync` test. Before that commit,
the atomic index best block pointer `m_best_block_index` was updated as the
last step of `BaseIndex::BlockConnected`, so `BlockUntilSyncedToCurrentChain`
could safely be used in tests to wait for the last `BlockConnected`
notification to be finished before stopping and destroying the index. But
after that commit, calling `BlockUntilSyncedToCurrentChain` is no longer
sufficient, and there is a race between the test shutdown code which destroys
the index object and the new code introduced in that commit calling
`AllowPrune()` and `GetName()` on the index object. Reproducibility
instructions for this are in
https://github.com/bitcoin/bitcoin/issues/25365#issuecomment-1259744133
This commit fixes the `coinstatsindex_initial_sync` race condition, even though
it will require an additional change to silence TSAN false positives,
https://github.com/bitcoin/bitcoin/pull/26188, after it is fixed. So this
partially addresses but does not resolve the bug reporting TSAN errors
https://github.com/bitcoin/bitcoin/issues/25365.
There is no known race condition outside of test code currently, because the
bitcoind `Shutdown` function calls `FlushBackgroundCallbacks` not
`BlockUntilSyncedToCurrentChain` to safely shut down.
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Co-authored-by: MacroFake <falke.marco@gmail.com>
5c5b85d0e7e7bb6eea47be60e20140b9fa9fa745 refactor: Make 64-bit shift explicit (Hennadii Stepanov)
Pull request description:
[`std::array::at()`](https://en.cppreference.com/w/cpp/container/array/at) expects an argument of the `size_t` type. This PR avoids implicit type conversion (for both 64-bit and 32-bit systems).
Also it enables MSVC warning [C4334](https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-3-c4334) for all codebase.
ACKs for top commit:
MarcoFalke:
ACK 5c5b85d0e7e7bb6eea47be60e20140b9fa9fa745 🚎
jonatack:
Code review ACK 5c5b85d0e7e7bb6eea47be60e20140b9fa9fa745
Tree-SHA512: fda850a42068f2ada9f877fac9ff8af1e22b5dcb3e708f5b95c316e77c52c72d33cd9ec6507a7f5d1731d1afdf5af6dc65025d388cc480f82c46f4d88ef2d306
The (100, 1000000, 1000, 1000000) limits are arbitrarily high and
don't restrict anything, they are just meant to calculate ancestors
properly. Using NoLimits() makes this intent more clear and simplifies
the code.
Simplifies function signatures by removing repetition of all the
ancestor/descendant limits, and increases readability by being
more verbose by naming the limits, while still reducing the LoC.
There are quite a few places in the codebase that require us to
construct a CTxMemPool without limits on ancestors and descendants.
This helper function allows us to get rid of all that duplication.
fa0437655417a9179a25e7717fdc1506f915bb08 Remove clang-format from lint task (MacroFake)
Pull request description:
clang-format could be used in scripted diffs, but remained largely unused.
So remove the install bloat, as it is unlikely to be used in the future.
ACKs for top commit:
fanquake:
ACK fa0437655417a9179a25e7717fdc1506f915bb08
hebasto:
ACK fa0437655417a9179a25e7717fdc1506f915bb08
Tree-SHA512: e0a3ee47d2aa2565dd34676914c558c985eaeb522a05f10bcaac115871edcf0d7f101b517e4d452ca5223c40b18ad02883c31e2da3d1f4ff86464a9af0097b11
8a6b6dfcd8d26b62c3a13beba72440635fcc5e19 fuzz: pass max fee into ConsumeTxMemPoolEntry (fanquake)
eb155692804b4278f6638c74273c1d9d35cd6ab7 fuzz: add util/mempool/h.cpp (fanquake)
Pull request description:
Moving the heavy (Boost) mempool code out of fuzz/util.h. Means that (for ex) a crypto_common fuzz unit doesn't need to care about seeing endless Boost headers. This results in a ~10% speedup (for me) when compiling the fuzz tests. Your results may vary.
ACKs for top commit:
MarcoFalke:
review ACK 8a6b6dfcd8d26b62c3a13beba72440635fcc5e19 🍮
Tree-SHA512: 27dc9d9581ac0b1b319cc0dc08fe5f8fbf9269386a5cb23f6fd5d8231bf015ed942ab4414d8001220541be0013756354578ddab1fec607c6fba04daf421bc870
fa6054e952f4522b98dc89609033950a3cbfd06c ci: Allow PIP_PACKAGES on centos (MacroFake)
fac085a05cc518b14271353128bb1fa830b3c612 ci: Remove unused package (MacroFake)
Pull request description:
This was added in 7fc5e865b93af59364e9c8bf75ec68b4decc7e5d but I can't see a reason why this should be forbidden.
This is also needed for other changes (bumping the minimum python version).
ACKs for top commit:
hebasto:
re-ACK fa6054e952f4522b98dc89609033950a3cbfd06c
Tree-SHA512: e8ead9ee00079024eb1e8c6e7b31c78cf2a3392159b444765c2ea9a58bed2a7550bf71083210692a45bb8ed7896cb882b72bf70baa13a2384864b2b510b73005
fa9436e90859c31d9aa7bc0b7f223ab93ec14bdc test: Remove unused fCheckpointsEnabled from miner_tests (MacroFake)
Pull request description:
The earliest checkpoint is at height 11111, so this can't possibly have any impact on this test.
ACKs for top commit:
fanquake:
ACK fa9436e90859c31d9aa7bc0b7f223ab93ec14bdc - given the low number of blocks, having the additional check in `ContextualCheckBlockHeader()` enabled should be a no-op, so disabling and re-enabling is dead code.
Tree-SHA512: 7d1b952c297c915e9588761f82f5006cf5186b7ff30e8a1c702302e0b44afe536bde9eda8acf2995825ae01d2ad9d2393ae2feefb29f15676aaf71881941579b
b8d361ab6fa2c672a08114e20a29a12537148e2d ci: Workaround Windows filesystem executable bit loss (Hennadii Stepanov)
Pull request description:
Fixes https://cirrus-ci.com/task/5946581265416192:
```
From https://github.com/bitcoin/bitcoin
* branch refs/pull/26103/merge -> FETCH_HEAD
error: Your local changes to the following files would be overwritten by checkout:
ci/lint/04_install.sh
ci/lint/06_script.sh
contrib/devtools/gen-manpages.py
contrib/devtools/symbol-check.py
contrib/signet/getcoins.py
contrib/signet/miner
test/functional/feature_proxy.py
test/functional/feature_taproot.py
test/functional/interface_rest.py
test/functional/mining_prioritisetransaction.py
test/functional/rpc_blockchain.py
test/functional/rpc_fundrawtransaction.py
test/functional/rpc_help.py
test/functional/rpc_rawtransaction.py
test/functional/test_framework/test_framework.py
test/functional/test_runner.py
test/functional/wallet_basic.py
test/functional/wallet_bumpfee.py
test/functional/wallet_hd.py
test/functional/wallet_importmulti.py
test/functional/wallet_listdescriptors.py
test/functional/wallet_multiwallet.py
test/functional/wallet_resendwallettransactions.py
test/functional/wallet_sendall.py
test/lint/lint-locale-dependence.py
Please commit your changes or stash them before you switch branches.
Aborting
```
Top commit has no ACKs.
Tree-SHA512: 2b33d882a515bb17c7c2ae8cfe73541483cdc15e736909afaf42befc8f648dba5dc83ff58ebd6d38a5650a8eca01907ae6c61537927ac9718bd9582d8501647d
37cf4720635b63cbe36a900a2411718704b63899 ci: Use same `merge_script` implementation for Windows as for all (Hennadii Stepanov)
ac1d99240af6c5d3ed5db2beea1479903d949a37 ci: Move `git config` commands into script where they are used (Hennadii Stepanov)
Pull request description:
This PR is a continuation of bitcoin/bitcoin#26202 and it suggests the same approach for the "Win64 native" CI task.
Top commit has no ACKs.
Tree-SHA512: 3154c9f30bc62549d738dc337e24f66614420417c349770c8381cc29f825f75695c9abbbb8dc57abbfda1375bf353f7c68e1a3766fd6d2440792e2d7fb68e15e
7d14577d0f9316feef3bcb5220aa3037748615d3 refactor: move DEFAULT_BLOCKFILTERINDEX from val to blockfilterindex (fanquake)
c87d569189992c08d932fd3bf2f9aa8ef689a398 refactor: move DEFAULT_COINSTATSINDEX from validation to coinstatsindex (fanquake)
2bfc1e6aaaf8aa392f1f85050686ea6293aa817c refactor: move DEFAULT_TXINDEX from validation to txindex (fanquake)
Pull request description:
Move `*index` default constants out of `validation.h`.
ACKs for top commit:
stickies-v:
re-ACK 7d14577d0f9316feef3bcb5220aa3037748615d3
aureleoules:
ACK 7d14577d0f9316feef3bcb5220aa3037748615d3
Tree-SHA512: 3021db1a63ceb714dee4b91f755d1fb9a6633adb6f1081e34e4179900e7543e3a7b06fe47507d580a3a2caf52f7ede784cb36716d521c76b0404bdc798f0186a
4bee62e9b8fb06749b5521d717a475192886a45d kernel: remove util/bytevectorhash.cpp (fanquake)
Pull request description:
This is no-longer used.
ACKs for top commit:
hebasto:
ACK 4bee62e9b8fb06749b5521d717a475192886a45d, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: 4d61f87b640ef3c759008631433b3e6d2bd2ac54bbe0b287f32ea1569760048f17a66cfe846b94ec458a7db5d064be6da59299b9280572a3dc649df60760c63f
d0d9cf7aea2ff36a14a19e6999400e4070b7b0c9 test: Check external coin effective value is used in CoinSelection (Aurèle Oulès)
76b79c1a177afa006184d716bd3d5b22ebadb168 wallet: Use correct effective value when checking target (Aurèle Oulès)
Pull request description:
Fixes#26185. The following assert failed because it was not checked in the parent function.
2bd9aa5a44/src/wallet/coinselection.cpp (L391)
ACKs for top commit:
glozow:
reACK d0d9cf7aea2ff36a14a19e6999400e4070b7b0c9
furszy:
ACK d0d9cf7a
Tree-SHA512: e126daba1115e9d143f2a582c6953e7ea55e96853b6e819c7744fd7a23668f7d9854681d43ef55d8774655bc54e7e87c1c9fccd746d9e30fbf3caa82ef808ae9
30cc1c6609ad7868f73e88afe0b0233d395ec08c refactor: Drop `owns_lock()` call (Hennadii Stepanov)
bff4e068b69edd40a00466156f860bde2df29268 refactor: Do not discard `try_lock()` return value (Hennadii Stepanov)
Pull request description:
Microsoft's C++ Standard Library uses the `[[nodiscard]]` attribute for `try_lock()`.
See: https://github.com/microsoft/STL/blob/main/stl/inc/mutex
This change allows to drop the current suppression for the warning C4838 and helps to prevent the upcoming warning C4858.
See: 539c26c923Fixesbitcoin/bitcoin#26017.
Split from bitcoin/bitcoin#25819.
ACKs for top commit:
vasild:
ACK 30cc1c6609ad7868f73e88afe0b0233d395ec08c
Tree-SHA512: ce17404e1c78af4f763129753caf8e5a0e1c91ba398778fe912f9fcc56a847e8112460d1a1a35bf905a593b7d8e0b16c6b099ad74976b67dca5f4f3eda6ff621
9cbfe40d8af8567682284890c080b0c3cf434490 net: remove useless call to IsReachable() from CConnman::Bind() (Vasil Dimov)
Pull request description:
`CConnman::Bind()` is called without `BF_EXPLICIT` only when passed
either `0.0.0.0` or `::`. For those addresses `IsReachable()` is always
true (regardless of the `-onlynet=` setting!), meaning that the `if`
condition never evaluates to true.
`IsReachable()` is always true for the "any" IPv4 and IPv6 addresses
because `CNetAddr::GetNetwork()` returns `NET_UNROUTABLE` instead of
`NET_IPV4` or `NET_IPV6` and the network `NET_UNROUTABLE` is always
considered reachable.
It follows that `BF_EXPLICIT` is unnecessary, remove it too.
ACKs for top commit:
naumenkogs:
ACK 9cbfe40d8af8567682284890c080b0c3cf434490
aureleoules:
ACK 9cbfe40d8af8567682284890c080b0c3cf434490
mzumsande:
ACK 9cbfe40d8af8567682284890c080b0c3cf434490
Tree-SHA512: 4e53ee8a73ddd133fd4ff25635135b65e5c19d1fc56fe5c30337406560664616c0adff414dca47602948919f34c81073aae6bfc2871509f3912663d86750928e
91bee4d898d0be4257484ec7ae519d7e3aca5c84 ci: Run `bench_bitcoin.exe --sanity-check` in "Win64 native" task (Hennadii Stepanov)
Pull request description:
This PR adds [`--sanity-check`](https://github.com/bitcoin/bitcoin/pull/25107) flag to `src\bench_bitcoin.exe` invocation as its results are been discarded.
Also a better name used for the script as it follows GNU's `make check`.
ACKs for top commit:
fanquake:
ACK 91bee4d898d0be4257484ec7ae519d7e3aca5c84
Tree-SHA512: fd5feeda72d1ef46c5fbfc2aa5c042ab2e3de7772546379da4596306b5658ab95f62939fba237c0bd7a1b09c85de20fc1cd9e5df1efe11bdae50d4a7b8081f74
079cf88c0df6de038b8f8933d55c0d17af007b43 refactor: move Boost datetime usage to wallet (fanquake)
Pull request description:
This means we don't need Boost Datetime in a `--disable-wallet` build, and it isn't included in the kernel (via time.h/cpp). Split from a larger boost removal branch/effort.
ACKs for top commit:
hebasto:
re-ACK 079cf88c0df6de038b8f8933d55c0d17af007b43
aureleoules:
re-ACK 079cf88c0df6de038b8f8933d55c0d17af007b43 - rebased and two additional unit tests since my last review.
jarolrod:
crACK 079cf88c0df6de038b8f8933d55c0d17af007b43
Tree-SHA512: c84f47158a4f21902f211c059d8c4bd55ffe95a256835deee723653be08cca49eeddfc33a2316b0cd31805e81cf77eaa39c6c9dcff4cda11a26ba4c1c143974e
5c9a27a46f4b07f872f850f7e918c10a33595cfd test: Use proper Boost macros instead of assertions (Hennadii Stepanov)
Pull request description:
On the master branch:
```
$ src/test/test_bitcoin -l test_suite -t banman_tests
Running 1 test case...
...
Test case banman_tests/file did not check any assertions
...
```
This PR suggests to use proper Boost [macros](https://www.boost.org/doc/libs/1_80_0/libs/test/doc/html/boost_test/utf_reference/testing_tool_ref.html).
Top commit has no ACKs.
Tree-SHA512: e0c8e5e6371acd0e0a80070fffdf1445f264c62499f8d9811822994c89735a913c18c8ed730495578400abdd93d2d500345504f2a9246401d53fb2f9f71be8c5
51a08f41ffab914d6dfa009b2ba31065b86444a4 signet/miner: reduce default interblock interval limit to 30min (Anthony Towns)
Pull request description:
Reduces the cap on the time between blocks from 60 minutes to 30 minutes, and makes it configurable.
Top commit has no ACKs.
Tree-SHA512: 7b880c50e47d055a2737c057fab190017748849d264c6c39dde465959a544d502221d12c6307d4de693f51badb4779b9c147e48775ede6ec6613e808067ab279
a9d20eeceb6d96eca579e35fd7bf90a275300f92 doc: bump bips.md up-to-date version to v24.0 (Sebastian Falbesoner)
Pull request description:
This is a trivial follow-up to #26124.
ACKs for top commit:
jarolrod:
ACK a9d20eeceb6d96eca579e35fd7bf90a275300f92
Tree-SHA512: 24c17c72498f96f9122d8fb041f1f6f63bd186e25ac3cb5a661bb1993106c6632f5efd95a15d19681004d30d38eca2d2a16b383a7a1f1c3db17f887ae1fcd02a
