9aa50152c1cfa1c41215b2b51ed7a516aa67137a Add destroy to BlockTemplate schema (Sjors Provoost)
Pull request description:
This ensures that if a client no longer needs a block template, the node can clear its memory as soon as possible.
A block template may hold on to transactions that are no longer in the mempool, so this can be significant.
This has a trivial silent merge conflict with #31283 because it also used the number `@9` (gaps are not allowed). I'll rebase whichever is merged last.
ACKs for top commit:
TheCharlatan:
Re-ACK 9aa50152c1cfa1c41215b2b51ed7a516aa67137a
ryanofsky:
Code review ACK 9aa50152c1cfa1c41215b2b51ed7a516aa67137a
Tree-SHA512: 393571b4455969cba71c7572feaeff4503738205331ab198b9181c156c75eb65933a8e5ceff66fc06d1efb8f2528bdb254e5eee7df75735b912526de1e7b166d
When BasicTestingSetup is used in fuzz-tests it will now create test directories containing the fuzz target names. Example:
/tmp/test_common bitcoin/tx_package_eval/153d7906294f7d0606a7/
This is already implemented for bench and unit tests.
5736d1ddacc4019101e7a5170dd25efbc63b622a tracing: pass if replaced by tx/pkg to tracepoint (0xb10c)
a4ec07f1944999c2eead41d08d7dd4fc3aa71243 doc: add comments for CTxMemPool::ChangeSet (Suhas Daftuar)
83f814b1d1100baac9dca9c176f89b0ec2555dbc Remove m_all_conflicts from SubPackageState (Suhas Daftuar)
d3c8e7dfb63f7986a1f9654ea2393aabe3cd78da Ensure that we don't add duplicate transactions in rbf fuzz tests (Suhas Daftuar)
d7dc9fd2f7bc675256687b9c55fdbec9cc8ac781 Move CalculateChunksForRBF() to the mempool changeset (Suhas Daftuar)
284a1d33f1dcbc3b3404ea40a948ff6600239613 Move prioritisation into changeset (Suhas Daftuar)
446b08b599bc492bbec10ccc2292aee6f90c58e7 Don't distinguish between direct conflicts and all conflicts when doing cluster-size-2-rbf checks (Suhas Daftuar)
b53041021abc4f9ee7203341413e8676e2d5a7ca Duplicate transactions are not permitted within a changeset (Suhas Daftuar)
b447416fddcb8c8647391502cca3dbfd1552e02e Public mempool removal methods Assume() no changeset is outstanding (Suhas Daftuar)
2b30f4d36c86f775ac637b171d27d42a02309c5b Make RemoveStaged() private (Suhas Daftuar)
18829194ca68152ac0b38d34e94b9265ee74c410 Enforce that there is only one changeset at a time (Suhas Daftuar)
7fb62f7db60c7d793828ae45f87bc3f5c63cc989 Apply mempool changeset transactions directly into the mempool (Suhas Daftuar)
34b6c5833d11ea84fbd4b891e06408f6f4ca6fac Clean up FinalizeSubpackage to avoid workspace-specific information (Suhas Daftuar)
57983b8add72a04721d3f2050c063a3c4d8683ed Move LimitMempoolSize to take place outside FinalizeSubpackage (Suhas Daftuar)
01e145b9758f1df14a7ea18058ba9577bf88e459 Move changeset from workspace to subpackage (Suhas Daftuar)
802214c0832de00f24268183f7763fa984ba7903 Introduce mempool changesets (Suhas Daftuar)
87d92fa340195d9c87be3d023ca133b90b3b7d4e test: Add unit test coverage of package rbf + prioritisetransaction (Suhas Daftuar)
15d982f91e6b0f145c9dd4edf29827cfabb37a3f Add package hash to package-rbf log message (Suhas Daftuar)
Pull request description:
part of cluster mempool: #30289
It became clear while working on cluster mempool that it would be helpful for transaction validation if we could consider a full set of proposed changes to the mempool -- consisting of a set of transactions to add, and a set of transactions (ie conflicts) to simultaneously remove -- and perform calculations on what the mempool would look like if the proposed changes were to be applied. Two specific examples of where we'd like to do this:
- Determining if ancestor/descendant/TRUC limits would be violated (in the future, cluster limits) if either a single transaction or a package of transactions were to be accepted
- Determining if an RBF would make the mempool "better", however that idea is defined, both in the single transaction and package of transaction cases
In preparation for cluster mempool, I have pulled this reworking of the mempool interface out of #28676 so it can be reviewed on its own. I have not re-implemented ancestor/descendant limits to be run through the changeset, since with cluster mempool those limits will be going away, so this seems like wasted effort. However, I have rebased #28676 on top of this branch so reviewers can see what the new mempool interface could look like in the cluster mempool setting.
There are some minor behavior changes here, which I believe are inconsequential:
- In the package validation setting, transactions would be added to the mempool before the `ConsensusScriptChecks()` are run. In theory, `ConsensusScriptChecks()` should always pass if the `PolicyScriptChecks()` have passed and it's just a belt-and-suspenders for us, but if somehow they were to diverge then there could be some small behavior change from adding transactions and then removing them, versus never adding them at all.
- The error reporting on `CheckConflictTopology()` has slightly changed due to no longer distinguishing between direct conflicts and indirect conflicts. I believe this should be entirely inconsequential because there shouldn't be a logical difference between those two ideas from the perspective of this function, but I did have to update some error strings in some tests.
- Because, in a package setting, RBFs now happen as part of the entire package being accepted, the logging has changed slightly because we do not know which transaction specifically evicted a given removed transaction.
- Specifically, the "package hash" is now used to reference the set of transactions that are being accepted, rather than any single txid. The log message relating to package RBF that happen in the `TXPACKAGES` category has been updated as well to include the package hash, so that it's possible to see which specific set of transactions are being referenced by that package hash.
- Relatedly, the tracepoint logging in the package rbf case has been updated as well to reference the package hash, rather than a transaction hash.
ACKs for top commit:
naumenkogs:
ACK 5736d1ddac
instagibbs:
ACK 5736d1ddacc4019101e7a5170dd25efbc63b622a
ismaelsadeeq:
reACK 5736d1ddacc4019101e7a5170dd25efbc63b622a
glozow:
ACK 5736d1ddacc
Tree-SHA512: 21810872e082920d337c89ac406085aa71c5f8e5151ab07aedf41e6601f60a909b22fbf462ef3b735d5d5881e9b76142c53957158e674dd5dfe6f6aabbdf630b
a6ca8f324396522e9748c9a7bbefb3bf1c74a436 fuzz: Fix difficulty target generation in p2p_headers_presync (marcofleon)
fa327c77e34e0cfb7994842c23f539ab11bf5d3b util: Add ConsumeArithUInt256InRange fuzzing helper (marcofleon)
Pull request description:
In the `p2p_headers_presync` fuzz target, this assertion failed:
```
assert(total_work < chainman.MinimumChainWork());
```
Input that triggered the failure: [p2ppresync_crash.txt](https://github.com/user-attachments/files/17620203/p2ppresync_crash.txt)
The test previously used `ConsumeIntegralInRange` to generate header difficulty targets within a hardcoded range. The fuzzer found specific values in that range that correspond to very low thresholds due to how [`SetCompact`][setcompact-link] works. The total work of a long enough test chain ended up exceeding `MinimumChainWork`.
Fix this by adding a new `ConsumeArithUInt256InRange` helper function and use it in the fuzz test to generate target values within the originally intended range. The target is then converted to an `nBits` value using `GetCompact()`.
For some more context, see https://github.com/bitcoin/bitcoin/pull/30918.
[setcompact-link]: 6463117a29/src/arith_uint256.h (L251-L271)
ACKs for top commit:
instagibbs:
ACK a6ca8f3243
dergoegge:
Code review ACK a6ca8f324396522e9748c9a7bbefb3bf1c74a436
brunoerg:
code review ACK a6ca8f324396522e9748c9a7bbefb3bf1c74a436
Tree-SHA512: 92013d9d37bd3f11992ee678ba9745196efbdc4d773fd14994116629260bea46ffc9fa3923d443af7b623d39c6211900ce98a349c62ad1976e12312c37ef9df0
637f437a1642ce845d7f750b45b71320a82d83b5 doc: remove PR Review Club frequency (Gabriele Bocchi)
Pull request description:
The PR Review Club is mentioned as weekly in the CONTRIBUTING.md file, but it is held monthly as per the official [Bitcoin Core PR Review Club website](https://bitcoincore.reviews/). This PR updates the documentation to just remove the frequency.
ACKs for top commit:
fanquake:
ACK 637f437a1642ce845d7f750b45b71320a82d83b5
Tree-SHA512: 27bf8a0e32edd8bedb5301ceb3c744ff4629403292a7ad00b633921f36278443ae297cd53708a533b1d6e6eab863b831e11247b95277b94cce28e3d5ddb7d9b9
fa7857ccda5d82739bba16eeb7244433978d28f3 build: Enable -Wbidi-chars=any (MarcoFalke)
Pull request description:
I don't see a use-case for UTF-8 bidirectional control characters in this codebase. So disable them for now.
Ref: https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wbidi-chars_003d
ACKs for top commit:
fanquake:
ACK fa7857ccda5d82739bba16eeb7244433978d28f3
Tree-SHA512: 29cf78a2bd0fd94f919f4cd1d1038009a574b4d011146c69bf94d3c06951606200b7d1f827ac6f2fb4540e8f45118ba72b3ccf6c20ef8048e819974371d8f67a
bcd82b13f4649e57d7d106856aab7b2a6296d728 Remove pkgconfig from toolchain file (TheCharlatan)
319a4e82614283afb3dbc5d38ff3b9d17fb911b3 depends: drop sqlite pkgconfig file (fanquake)
a8fe1fd38bf496356dc4f28963d4edfa75fe04a5 depends: better cleanup after fontconfig (fanquake)
17e79c92607e2e32b48ffd388828184c5d1a65df depends: fully remove libtool archives from Qt build (fanquake)
8ca85651c8350a6edb069eae0f88cf03c6eae0d5 guix: move pkg-config to Linux builds (fanquake)
e3e648cf410d30185927d031c81a85d5fa890b8c depends: drop pkg-config option from Qt build (fanquake)
0d185bd99f9e40913f678af0fc224add2e1d2f14 doc: update depends doc to prefer .cmake outputs (fanquake)
Pull request description:
After #31181, `pkg-config` is no-longer needed for macOS or Windows Guix builds. It's still needed for Linux, as it's used by a Qt subdependency (fontconfig to find freetype). However we should also no-longer need it for Qt itself, when building using depends.
ACKs for top commit:
TheCharlatan:
ACK bcd82b13f4649e57d7d106856aab7b2a6296d728
Tree-SHA512: 89ae68281030d43fcb6c5c96429cd038a21f13a8ca19ea828ada47e8f9f0aa7407854a67c9003652817e47ab9565573b7028342e3e11bb1cca1d823c483081cd
faaaf59f71ede057b2c1d369ef8db973c2f2dbc2 test: Make g_rng_temp_path rand, not dependent on SeedRandomForTest (MarcoFalke)
fa80b08fef0eaa600339caa678fdf80a8aec3ce3 test: Revert to random path element (MarcoFalke)
Pull request description:
The randomness in the path element is required to allow a single fuzz test to run in parallel. Previous releases used a uint256 random value, but 10 random bytes should be sufficient as well, while avoiding a `MAX_PATH` violation on Windows.
The issue was introduced by myself, by suggesting to use the current time in https://github.com/bitcoin/bitcoin/pull/31000#discussion_r1835351305.
ACKs for top commit:
kevkevinpal:
reACK faaaf59f71
hodlinator:
ACK faaaf59f71ede057b2c1d369ef8db973c2f2dbc2
tdb3:
re ACK faaaf59f71ede057b2c1d369ef8db973c2f2dbc2
dergoegge:
ACK faaaf59f71ede057b2c1d369ef8db973c2f2dbc2
Tree-SHA512: f12256c8b353618291030f71bf36eab97a25ffeaa28e36a5f2c6718dfc1fbbc8548c71475edec53d59026f2a779a05778db83f0530dd3e1d1faf6e4fc0ee7d70
111465d72dd35e42361fc2a089036f652417ed37 test: Remove unused attempts parameter from wait_until (Fabian Jahr)
5468a23eb9a3fd2b0c08dbca69fe3df58af42530 test: Add check_interval parameter to wait_until (Fabian Jahr)
16c87d91fd4d7709fa9d8824d5b641ef71821931 test: Introduce ensure_for helper (Fabian Jahr)
Pull request description:
A repeating pattern in the functional tests is that the test sleeps for a while to ensure that a certain condition is still true after some amount of time has elapsed. Most recently a new case of this was added in #30807. This PR here introduces an `ensure` helper to streamline this functionality.
Some approach considerations:
- It is possible to construct this by reusing `wait_until` and wrapping it in `try` internally. However, the logger output of the failing wait would still be printed which seems irritating. So I opted for simplified but similar internals to `wait_until`.
- This implementation starts for a failure in the condition right away which has the nice side-effect that it might give feedback on a failure earlier than is currently the case. However, in some cases, it may be expected that the condition may still be false at the beginning and then turns true until time has run out, something that would work when the test sleeps without checking in a loop. I decided against this design (and even against adding it as an option) because such a test design seems like it would be racy either way.
- I have also been going back and forth on naming. To me `ensure` works well but I am also not a native speaker, happy consider a different name if others don't think it's clear enough.
ACKs for top commit:
maflcko:
re-ACK 111465d72dd35e42361fc2a089036f652417ed37 🍋
achow101:
ACK 111465d72dd35e42361fc2a089036f652417ed37
tdb3:
code review re ACK 111465d72dd35e42361fc2a089036f652417ed37
furszy:
utACK 111465d72dd35e42361fc2a089036f652417ed37
Tree-SHA512: ce01a4f3531995375a6fbf01b27d51daa9d4c3d7cd10381be6e86ec5925d2965861000f7cb4796b8d40aabe3b64c4c27e2811270e4e3c9916689575b8ba4a2aa
b2d536100282bd901d3e0be7f7f4a6966e0ef817 build: Temporarily disable compiling `fuzz/utxo_snapshot.cpp` with MSVC (Hennadii Stepanov)
Pull request description:
This PR suggests a temporary workaround for a compiler bug [introduced](https://github.com/bitcoin/bitcoin/issues/31303) in Visual Studio 2022 version 17.12.
This workaround is required to fix the CI until the upstream compiler bug is resolved.
ACKs for top commit:
maflcko:
lgtm ACK b2d536100282bd901d3e0be7f7f4a6966e0ef817
TheCharlatan:
ACK b2d536100282bd901d3e0be7f7f4a6966e0ef817
brunoerg:
ACK b2d536100282bd901d3e0be7f7f4a6966e0ef817
Tree-SHA512: 8f8b772253f6f61d9e24c6ae8692511c7c1229c7d20a45fe680ad6a3909c59b58b504589f06d3135d9708a61684be4aba6a426f7f5f60020551f52b9090d3030
This ensures the options are applied consistently from contexts where
they might not pass through the args manager, such as in some tests, or
when used through the kernel library.
This is similar to the patch applied in 09ef322acc0a88a9e119f74923399598984c68f6.
This ensures that if a client no longer needs a block template,
the node can clear its memory as soon as possible.
A block template may hold on to transactions that are no longer
in the mempool, so this can be significant.
Requested by clang-tidy:
src/wallet/salvage.cpp:119:18: error: use emplace_back instead of push_back [modernize-use-emplace,-warnings-as-errors]
119 | warnings.push_back(Untranslated("Salvage: Database salvage found errors, all data may not be recoverable."));
| ^~~~~~~~~~
| emplace_back(
This is required for a future commit. Can be reviewed via the git
options --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
Also move util::detail::Hex to a proper namespace instead of an inline
namespace so it doesn't conflict with the new util::detail namespace, and
won't create other problems for callers trying to use the inline namespaces.
Also fix a misleading comment in util_string_tests.cpp.
Co-Authored-By: Ryan Ofsky <ryan@ofsky.org>
fa1177e3d7ca9ef003ded4d0c915fa6dc22bd37d refactor: Avoid std::string format strings (MarcoFalke)
Pull request description:
This changes some unchecked `std::string` format strings to use string literals, which are `consteval` checked at compile-time.
Split out, because it is used in several pull requests.
ACKs for top commit:
l0rinc:
ACK fa1177e3d7ca9ef003ded4d0c915fa6dc22bd37d
tdb3:
code review and light test ACK fa1177e3d7ca9ef003ded4d0c915fa6dc22bd37d
rkrux:
tACK fa1177e3d7ca9ef003ded4d0c915fa6dc22bd37d
ryanofsky:
Code review ACK fa1177e3d7ca9ef003ded4d0c915fa6dc22bd37d
Tree-SHA512: 7cc70a49b07dadc386336687b463043e79e94a46d18db0184c9813218536e87e954a1afeb8739d5d8706e7b2f355d3f7984048c7de2725851b463985f1c5369f
