This adds a large simulation fuzz test for all TxOrphanage public interface
functions, using a mix of comparison with expected behavior (in case it is
fully specified), and testing of properties exhibited otherwise.
This is preparation for the simulation fuzz test added in a later commit. Since
AddChildrenToWorkSet consumes randomness, there is no way for the simulator to
exactly predict its behavior. By returning the set of made-reconsiderable announcements
instead, the simulator can instead test that it is *a* valid choice, and then
apply it to its own data structures.
For the default number of peers (125), allows each to relay a default
descendant package (up to 25-1=24 can be missing inputs) of small (9
inputs or fewer) transactions out of order.
This limit also gives acceptable bounds for worst case LimitOrphans iterations.
Functional tests aren't changed to check for larger cap because it would
make the runtime too long.
Also deletes the now-unused DEFAULT_MAX_ORPHAN_TRANSACTIONS.
This is largely a reimplementation using boost::multi_index_container.
All the same public methods are available. It has an index by outpoint,
per-peer tracking, peer worksets, etc.
A few differences:
- Limits have changed: instead of a global limit of 100 unique orphans,
we have a maximum number of announcements (which can include duplicate
orphans) and a global memory limit which scales with the number of
peers.
- The maximum announcements limit is 100 to match the original limit,
but this is actually a stricter limit because the announcement count
is not de-duplicated.
- Eviction strategy: when global limits are reached, a per-peer limit
comes into play. While limits are exceeded, we choose the peer whose
“DoS score” (max usage / limit ratio for announcements and memory
limits) is highest and evict announcements by entry time, sorting
non-reconsiderable ones before reconsiderable ones. Since announcements
are unique by (wtxid, peer), as long as 1 announcement remains for a
transaction, it remains in the orphanage.
- This eviction strategy means no peer can influence the eviction of
another peer’s orphans.
- Also, since global limits are a multiple of per-peer limits, as long
as a peer does not exceed its limits, its orphans are protected from
eviction.
- Orphans no longer expire, since older announcements are generally
removed before newer ones.
- GetChildrenFromSamePeer returns the transactions from newest to
oldest.
Co-authored-by: Pieter Wuille <pieter@wuille.net>
Move towards a model where TxOrphanage is initialized with limits that
it remembers throughout its lifetime.
Remove the param. Limiting by number of unique orphans will be removed
in a later commit.
Now that -maxorphantx is gone, this does not change the node behavior.
The parameter is only used in tests.
c18bf0bd9b refactor: cleanup index logging (Sjors Provoost)
Pull request description:
This PR removes the use of `__func__` from index logging, since we have `-logsourcelocations`.
It also improves readability by putting `GetName()` in a more logical place.
Before
> coinstatsindex: best block of the index not found. Please rebuild the index.
After:
> best block of coinstatsindex not found. Please rebuild the index.
I found myself maintaining this commit as part of https://github.com/Sjors/bitcoin/pull/86, but since that might never land here, it seemed better to split it into its own PR (or get rid of it).
ACKs for top commit:
l0rinc:
Lightweight code review ACK c18bf0bd9b
maflcko:
review ACK c18bf0bd9b🚣
Tree-SHA512: 755948371e3ff7a5515b63ce48075631ec7868d69c3c1469176d5be0e8b28e1c071e206ae3f7320f87d8c441f815894acfef61621f05795b5ff6b8a5a3031e3b
node1 (with 24 blocks) causes node0 (with 6 blocks) to silently
reorg. so move the subtest to a point before the 20 blocks are
generated so that node1's state doesn't cause node0 to silently
reorg.
44f3bae300 depends: Force `CMAKE_EXPORT_NO_PACKAGE_REGISTRY=TRUE` (Hennadii Stepanov)
Pull request description:
When using CMake policies 3.14 and below, the `export(PACKAGE)` command by default populates the user package registry, which is stored outside the build tree. Setting the `CMAKE_EXPORT_NO_PACKAGE_REGISTRY` variable disables this side effect.
In CMake 3.15 and later, this behavior is disabled by default, and the variable has no effect.
This PR forces `CMAKE_EXPORT_NO_PACKAGE_REGISTRY=TRUE` globally, rather than managing it for each dependency package individually rather. It may be reverted once all CMake-based packages have updated to policies 3.15 or newer.
Fixes https://github.com/bitcoin/bitcoin/issues/32938.
ACKs for top commit:
fanquake:
ACK 44f3bae300
Tree-SHA512: 0aac398b7182e80185b064d59f81aece4d8477a609fad9cc3fee317da2aff43b66ef7db1efec0135b4f0feaad23b1db664e33bd035fe659712c5b2a9bf2d6fb6
fad191ff48 ci: Avoid cd into build dir (MarcoFalke)
Pull request description:
Changing into the build dir is confusing and brittle, because the following commands implicitly assume it. So they could break on unrelated changes.
The changes are required for stuff like:
* cmake presets (see https://github.com/bitcoin/bitcoin/pull/30871#issuecomment-2344031208)
* meta ci tests (like https://github.com/bitcoin/bitcoin/pull/32874)
So remove the `cd` and just make the build dir explicit.
ACKs for top commit:
hebasto:
ACK fad191ff48, I have reviewed the code and it looks OK.
Tree-SHA512: a88a9341445ffe28a0dac3815f235ec8eb0459d10a91a80829fd3184762d3c807d0f68c56243b20c04a6efa5becd8a7fad568f43c2b1e6af1ff8ba07b140ef87
12a6959892 cmake: Drop no longer necessary "cmakeMinimumRequired" object (Hennadii Stepanov)
Pull request description:
The minimum required CMake version is 3.22:6a13a6106e/CMakeLists.txt (L10)
ACKs for top commit:
fanquake:
ACK 12a6959892 - has been unneeded since it was introduced (minimum was already 3.22).
Tree-SHA512: 26f97662bfe52986e19e38dbf4ab8e1e7558bc78c3a65593cbecd1f35887bba7a9f7d8a3d08ccfab8396f41c2334cdad5b0e503999a759cfa158d3bb8d0d14d7
84ef5524d5 fix spelling in tor.md docs (stutxo)
Pull request description:
This PR is to fix some spelling mistakes i found of the word occurrences! there are two occurrences of this mistake.
thanks!
ACKs for top commit:
maflcko:
lgtm ACK 84ef5524d5
willcl-ark:
ACK 84ef5524d5
delta1:
ACK 84ef5524d5
Tree-SHA512: 4ba71b772fdc8cf36ada7493d29fb5b312a7a6085099347162eb3495db4de984b0417b7861f2927c617cbd552741356e26688479601bdf7e835c15e097aa28f3
8f766f39df ci: enable -Werror=dev (fanquake)
7b420ca834 guix: configure with -Werror=dev (fanquake)
44097ddb19 cmake: enable -Werror=dev in dev-mode preset (fanquake)
Pull request description:
Pass `-Werror=dev` in the CI, Guix and the `dev-mode` preset.
https://cmake.org/cmake/help/latest/manual/cmake.1.html#cmdoption-cmake-Werror:
> Make developer warnings errors.
> Make warnings that are meant for the author of the CMakeLists.txt files errors. By default this will also turn on deprecated warnings as errors.
Pulled out of #32865.
ACKs for top commit:
Sjors:
re-ACK 8f766f39df
hebasto:
ACK 8f766f39df, tested on Ubuntu 24.04.
Tree-SHA512: 0fa321b77d2519b5249d90590664c4e5938ac86209b068658647adf97ab55ea4d54c913aae2f622385fe2f41d7c851cd5d7371905fdad38b66cb124371e16ac7
Expiry is going away in a later commit.
This is only an RPC change. Behavior of the orphanage does not change.
Note that getorphantxs is marked experimental.
a60f863d3e scripted-diff: Replace GenTxidVariant with GenTxid (marcofleon)
c8ba199598 Remove old GenTxid class (marcofleon)
072a198ea4 Convert remaining instances of GenTxid to GenTxidVariant (marcofleon)
1b528391c7 Convert `txrequest` to GenTxidVariant (marcofleon)
bde4579b07 Convert `txdownloadman_impl` to GenTxidVariant (marcofleon)
c876a892ec Replace GenTxid with Txid/Wtxid overloads in `txmempool` (marcofleon)
de858ce2be move-only: make GetInfo a private CTxMemPool member (stickies-v)
eee473d9f3 Convert `CompareInvMempoolOrder` to GenTxidVariant (marcofleon)
243553d590 refactor: replace get_iter_from_wtxid with GetIter(const Wtxid&) (stickies-v)
fcf92fd640 refactor: make CTxMemPool::GetIter strongly typed (marcofleon)
11d28f21bb Implement GenTxid as a variant (marcofleon)
Pull request description:
Part of the [type safety refactor](https://github.com/bitcoin/bitcoin/pull/32189).
This PR changes the GenTxid class to a variant, which holds both Txids and Wtxids. This provides compile-time type safety and eliminates the manual type check (bool m_is_wtxid). Variables that can be either a Txid or a Wtxid are now using the new GenTxid variant, instead of uint256.
ACKs for top commit:
w0xlt:
ACK a60f863d3e
dergoegge:
Code review ACK a60f863d3e
maflcko:
review ACK a60f863d3e🎽
theStack:
Code-review ACK a60f863d3e
Tree-SHA512: da9b73b7bdffee2eb9281a409205519ac330d3336094d17681896703fbca8099608782c9c85801e388e4d90af5af8abf1f34931f57bbbe6e9674d802d6066047
fa894b0f3e log: Properly log warnings with warn loglevel in addrdb (MarcoFalke)
Pull request description:
The logging in addrdb is confusing, because it uses `LogPrintf` (info level) to log warnings.
Fix this by properly using the `warn` level, where needed. Also, drop unused trailing `\n` while touching the lines.
ACKs for top commit:
stickies-v:
ACK fa894b0f3e
dergoegge:
utACK fa894b0f3e
Tree-SHA512: 96d3823623ea8e1698e8cb541ca97cbab7b2a9934b2f894884171045abbca7be796f07965082e997001c97d06d1e0c4d13b29354eb4fe71c3a2ee680eada5516
- don't log function name
- take into account that GetName() always ends with " index"
- replace deprecated LogPrintf with LogInfo
- remove trailing \n
- adjusted log level where needed
fa8862723c fuzz: CheckGlobals in init (MarcoFalke)
fa26bfde98 test: Avoid resetting mocktime in testing setup (MarcoFalke)
fa6b45fa8e Add SetMockTime for time_point types (MarcoFalke)
Pull request description:
(Tracking issue https://github.com/bitcoin/bitcoin/issues/29018)
During fuzzing, `AppInitParameterInteraction` may actually disable a previously set mocktime. This is confusing and can also cause non-determinism.
Fix this issue, by
* fixing the erroneous `-mocktime` parsing in `AppInitParameterInteraction`.
* adding the missing `SetMockTime` calls to the affected fuzz init functions.
* adding a `CheckGlobals` to the fuzz init, to prevent this issue in the future.
This can be tested by
* Cherry-picking the `CheckGlobals`-commit onto current master and observing a fuzz failure in the touched fuzz targets.
* Reverting the touched fuzz fixups and observing a fuzz failure for each target.
ACKs for top commit:
w0xlt:
ACK fa8862723c
dergoegge:
utACK fa8862723c
Tree-SHA512: 5a9400f0467c82fa224713af4cc2b525afbefefc7c3f419077110925ad7af6c7fda3dcd2b50f7facf0ee7df2547c6ac20336906d707adcdfd1d652a9d9a735fe
Starting with version 13.x, the mingw headers will define the value of
NTDDI_VERSION, based on the value of _WIN32_WINNT, if that version is <
Windows 10. Given that libevent was undefining our _WIN32_WINNT, and
redefining it to a value < Windows 10 (0x0501), NTDDI_VERSION was also
being defined to that value, leading to functions not being exposed in
the mingw-w64 headers; see here:
9c2668ef77/mingw-w64-headers/include/iphlpapi.h (L36-L41).
Imports a commit from usptream (a14ff91254f40cf36e0fee199e26fb11260fab49).
Fixes#32707.
f43571010e Resolve guix non-determinism with emplace_back instead of push_back (Ava Chow)
Pull request description:
For some reason, building x86_64-w64-mingw32 on x86_64 and aarch64 results in a single instruction difference which can be traced down to prevector.h:174. The ultimate caller of this is the copy constructor for a prevector that ends up being called by std::vector::push_back in walletmodel.cpp:183. By replacing the push_back with an emplace_back, somehow this non-determinism goes away.
Closes#32923
ACKs for top commit:
l0rinc:
code review ACK f43571010e
Sjors:
utACK f43571010e
maflcko:
lgtm ACK f43571010e
Tree-SHA512: 5bf0571f32cb72efc0c533e16d2704cfc3a79bcef2943f0892743572808610fb00ca8ab41223897536f8e5090bf4030735be910942de8116652d02bc3f231e2e
83bb414557 test: less ambiguous error if bitcoind is missing (Sjors Provoost)
Pull request description:
Before this change, when a functional test is run without building the source, the error message suggested that previous release binaries were missing.
When no previous release version is set, make the error message more specifically about bitcoind.
To test, try this before and after:
```sh
git clean -dfx
cmake -B build
build/test/functional/mining_basic.py
cmake --build build
build/test/functional/mining_basic.py
build/test/functional/wallet_backwards_compatibility.py
test/get_previous_releases.py
build/test/functional/wallet_backwards_compatibility.py
```
ACKs for top commit:
achow101:
ACK 83bb414557
janb84:
ACK 83bb414557
w0xlt:
ACK 83bb414557
Tree-SHA512: c6df65019de99d6c214951cf70944c4ddca9b635c5ab60ac2c47e4589478e9c65d5e079c394ace9b470a7eaeea3c9cf68b7246dd413e802c4a1e071913a7fc32
150b5c99ca wallet: replace `reload_wallet` with inline functionality (rkrux)
0f86da382d wallet: remove dead code in legacy wallet migration (rkrux)
Pull request description:
A discussion on a previous [PR 32481](https://github.com/bitcoin/bitcoin/pull/32481#discussion_r2145152084) related to legacy wallet dead
code removal made me realize that checking if the legacy
wallet was loaded prior to the start of the migration is not
required ever since legacy wallets can't be loaded in the first
place. I also verified that the `load_on_start` persistent
setting can also not cause the legacy wallets to be loaded, which
further makes the case for removal of the above mentioned checks
during migration.
The current test coverage also shows these lines uncovered.
ACKs for top commit:
achow101:
ACK 150b5c99ca
furszy:
ACK 150b5c99ca
Tree-SHA512: 9bc7043cac1f4051228557208895e43648de3c7ffae6860c0676d1aa2db3a8ed3a09d1f9defacd96ca50bbb9699ba86652ccb0c5e55cc88be248a1fe727c13d9
d7fca5c171 clusterlin: add big comment explaning the relation between tests (Pieter Wuille)
b64e61d2de clusterlin: abstract try-permutations into ExhaustiveLinearize function (Pieter Wuille)
1fa55a64ed clusterlin tests: verify that chunks are minimal (Pieter Wuille)
da23ecef29 clusterlin tests: support non-empty ReadTopologicalSubset() (Pieter Wuille)
94f3e17c33 clusterlin tests: compare with fuzz-provided linearizations (Pieter Wuille)
5f92ebee0d clusterlin tests: compare with fuzz-provided topological sets (Pieter Wuille)
6e37824ac3 clusterlin tests: optimize clusterlin_simple_linearize (Pieter Wuille)
98c1c88b6f clusterlin tests: separate testing of SimpleLinearize and Linearize (Pieter Wuille)
10e90f7aef clusterlin tests: make SimpleCandidateFinder always find connected (Pieter Wuille)
a38c38951e clusterlin tests: separate testing of Search- and SimpleCandidateFinder (Pieter Wuille)
77a432ee70 clusterlin tests: count SimpleCandidateFinder iterations better (Pieter Wuille)
Pull request description:
Part of the cluster mempool project: #30289
The current cluster linearization fuzz tests contain two tests which combine testing of production code with testing of the test code itself:
* `clusterlin_search_finder`: establishes the correctness of `SearchCandidateFinder` by comparing against both `SimpleCandidateFinder` and `ExhaustiveCandidateFinder` (which is even more simple than `SimpleCandidateFinder`). If `SimpleCandidateFinder` works correctly, then this comparison with `ExhaustiveCandidateFinder` is redundant. If it isn't, we ought to find that in a test specific to `SimpleCandidateFinder` rather than as a side-effect of testing `SearchCandidateFinder`. Split this functionality out into a new `clusterlin_simple_finder`.
* `clusterlin_linearize`: establishes the correctness of `Linearize` by comparing against both `SimpleLinearize` and literally every valid linearization for the cluster. Again, if `SimpleLinearize` works correctly, then this comparison with all valid linearizations is redundant, and if it isn't we should find it in a test for `SimpleLinearize`. Do so by splitting off that functionality into `clusterlin_simple_linearize`.
After that, a few general improvements to the affected tests are made (comparing with linearizations and subsets read from the fuzz input, plus a performance improvement).
ACKs for top commit:
marcofleon:
Re ACK d7fca5c171
ismaelsadeeq:
re-ACK d7fca5c171
monlovesmango:
ACK d7fca5c171
Tree-SHA512: 33cb76bd9b9547a5f3ee231fa452e928f064ad03af98e3d9e64246eb972f2b026c13e7367257ccdac1ae57982ee8ef98c907684588ecbb4bc4c82cbec160b3e8
For some reason, building x86_64-w64-mingw32 on x86_64 and aarch64
results in a single instruction difference which can be traced down to
prevector.h:174. The ultimate caller of this is the copy constructor for
a prevector that ends up being called by std::vector::push_back in
walletmodel.cpp:183. By replacing the push_back with an emplace_back,
somehow this non-determinism goes away.
When using CMake policies 3.14 and below, the `export(PACKAGE)` command
by default populates the user package registry, which is stored outside
the build tree. Setting the `CMAKE_EXPORT_NO_PACKAGE_REGISTRY` variable
disables this side effect.
In CMake 3.15 and later, this behavior is disabled by default, and the
variable has no effect.
