21a9e94dbbc47939f19fbfe39d85cb8ff2ea206a ci: remove hardcoded tag list from ci scripts (josibake)
d530ba390e7703433d3a77673c9a2b52bc446901 doc: update test/README.md (josibake)
614d4682badaadac74b825a45aaee9c2309a3e81 script: default to necessary tags in get_previous_releases.py (josibake)
Pull request description:
Almost every time I need to use this script, I forget the tag list is needed and that a specific set of tags is needed for the backwards compatibility tests to work. I end up wasting time reading through the script and googling to find the tag list before remembering it is in `test/README.md`
I assume (hope) I'm not the only one this happens to, so I figured it would make more sense to have the script default to downloading/building the necessary tags. This has the added benefit of making the script the source of truth: the script already needs to be updated with the SHA256_SUM of the binary for every new tag that is added, so it makes sense to use `SHA256_SUMS` list as the necessary tag list. This means there is less risk of the README and the script drifting (i.e updating the readme with a new tag and forgetting to update the script, or updating the script and forgetting to update the README). Now all that needs to happen is to update the `SHA256_SUMS` list in the script and everything Just Works (TM)
ACKs for top commit:
Sjors:
re-tACK 21a9e94dbbc47939f19fbfe39d85cb8ff2ea206a
Tree-SHA512: 97b488227a89a6827584edd251820a7074fad75dfd7f26f1aa5f858e2521d2e02effd0f11e6dc4676e1155d3d5aba6ff94a4b58ffef80dc201376afd5927deb9
fadd8b2676f6d68ec87189871461c9a6a6aa3cac addrman: Use system time instead of adjusted network time (MarcoFalke)
Pull request description:
This changes addrman to use system time for address relay instead of the network adjusted time.
This is an improvement, because network time has multiple issues:
* It is non-monotonic, even if the system time is monotonic.
* It may be wrong, even if the system time is correct.
* It may be wrong, if the system time is wrong. For example, when the node has limited number of connections (`4`), or the system time is wrong by too much (more than +-70 minutes), or the system time only got wrong after timedata collected more than half of the entries while the time was correct, ...)
This may slightly degrade addr relay for nodes where timedata successfully adjusted the time. Addr relay can already deal with minor offsets of up to 10 minutes. Offsets larger than this should still allow addr relay and not result in a DoS.
ACKs for top commit:
dergoegge:
Code review ACK fadd8b2676f6d68ec87189871461c9a6a6aa3cac
Tree-SHA512: b6c178fa01161544e5bc76c4cb23e11bcc30391f7b7a64accce864923766647bcfce2e8ae21d36fb1ffc1afa07bc46415aca612405bd8d4cc1f319c92a08498f
This command iterates over the inputs and outputs of the given transactions, and tallies up the balance change for the given wallet. This can be useful e.g. when verifying that a coin join like transaction doesn't contain unexpected inputs that the wallet will then sign for unintentionally.
fa2537cf0a7629d81df1bc5b4ae6a22dc572647b test: Target exact weight in MiniWallet _bulk_tx (MacroFake)
Pull request description:
Seems better to target the exact weight than a weight that is up to more than 2000 WU larger.
Also, replace a broad `-acceptnonstdtxn=1` with `-datacarriersize=100000` to document the test assumptions better.
ACKs for top commit:
theStack:
Code-review ACK fa2537cf0a7629d81df1bc5b4ae6a22dc572647b
Tree-SHA512: cf02c3082a13195b8aa730866aeaf2575ce01974ae2b0244739d8cfc12e60c66312729ed703bb3214651744166a3b560bfaa8dc302ef46ed79fc4d1fe7fcc214
b01f336708019f8c8274ea701d3446e4123e7af2 util, refactor: Drop explicit conversion to fs::path (Hennadii Stepanov)
138c668e2b4d64279ddefbe07c1d9b7c3d3c537c util, refactor: Use GetPathArg to read "-rpccookiefile" value (Hennadii Stepanov)
1276090705060fcc97072481c2383bbaaa556194 util, refactor: Use GetPathArg to read "-conf" value (Hennadii Stepanov)
Pull request description:
This PR is a continuation of bitcoin/bitcoin#24265 and bitcoin/bitcoin#24306.
Now the following command-line arguments / configure options been read with the `GetPathArg` method:
- `-conf`, also `includeconf` values been normalized
- `-rpccookiefile`
ACKs for top commit:
jarolrod:
Code Review ACK b01f336708019f8c8274ea701d3446e4123e7af2
ryanofsky:
Code review ACK b01f336708019f8c8274ea701d3446e4123e7af. Changes since last review: just dropping first commit (NormalizedPathFromString) as suggested
Tree-SHA512: 2d26d50b73542acdbcc63a32068977b2a49a017d31ca337471a0446f964eb0a6e3e4e3bb1ebe6771566a260f2cae3bc2ebe93b4b523183cea0d51768daab85c9
faab8dceb37a944d0763fcca390342111e6a9fcc Remove unused SetTip(nullptr) code (MacroFake)
Pull request description:
Now that this path is no longer used after commit b51e60f91472da5216116626afc032acd5616e85, we can remove it.
Future code should reset `CChain` by simply discarding it and constructing a fresh one.
ACKs for top commit:
ryanofsky:
Code review ACK faab8dceb37a944d0763fcca390342111e6a9fcc. Just moved an assert statement since last review
Tree-SHA512: 7dc273b11133d85d32ca2a69c0c7c07b39cdd338141ef5b51496e7de334a809864d5459eb95535497866c8b1e468aae84ed8f91b543041e6ee20130d5622874e
0f3a2532c38074dd9789d1c4c667db6ca46ff0ab validationcaches: Use size_t for sizes (Carl Dong)
41c5201a90bbc2893333e334e8945759ef24e7dd validationcaches: Add and use ValidationCacheSizes (Carl Dong)
82d3058539f54ebad745e2b02b61df01aa832a54 cuckoocache: Check for uint32 overflow in setup_bytes (Carl Dong)
b370164b319df1a500b70694b077f92265a777fb validationcaches: Abolish arbitrary limit (Carl Dong)
08dbc6ef72db48168dc03991f5f838dae42c8dfd cuckoocache: Return approximate memory size (Carl Dong)
0dbce4b1034b53d19b88af332385a006098b6d48 tests: Reduce calls to InitS*Cache() (Carl Dong)
Pull request description:
This is part of the `libbitcoinkernel` project: #24303, https://github.com/bitcoin/bitcoin/projects/18
This PR is **_NOT_** dependent on any other PRs.
-----
a.k.a. "Stop calling `gArgs.GetIntArg("-maxsigcachesize")` from validation code"
This PR introduces the `ValidationCacheSizes` struct and its corresponding `ApplyArgsManOptions` function, removing the need to call `gArgs` from `Init{Signature,ScriptExecution}Cache()`. This serves to further decouple `ArgsManager` from `libbitcoinkernel` code.
More context can be gleaned from the commit messages.
ACKs for top commit:
glozow:
re ACK 0f3a2532c3
theStack:
Code-review ACK 0f3a2532c38074dd9789d1c4c667db6ca46ff0ab
ryanofsky:
Code review ACK 0f3a2532c38074dd9789d1c4c667db6ca46ff0ab. Rebase and comment tweak since last
Tree-SHA512: a492ca608466979807cac25ae3d8ef75d2f1345de52a156aa0d222c5a940f79f1b65db40090de69183cccdb12297ec060f6c64e57a26a155a94fec80e07ea0f7
Similar to 85885919656a3f606f3d7f208378aabe95f3f62d.
```bash
ERROR: test_revocation_mode_soft (tests.test_validate.ValidateTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/guix-build-python-certvalidator-0.1-1.a145bf2.drv-0/source/tests/test_validate.py", line 85, in test_revocation_mode_soft
validate_path(context, path)
File "/tmp/guix-build-python-certvalidator-0.1-1.a145bf2.drv-0/source/tests/../certvalidator/validate.py", line 50, in validate_path
return _validate_path(validation_context, path)
File "/tmp/guix-build-python-certvalidator-0.1-1.a145bf2.drv-0/source/tests/../certvalidator/validate.py", line 358, in _validate_path
raise PathValidationError(pretty_message(
certvalidator.errors.PathValidationError: The path could not be validated because the end-entity certificate expired 2022-07-27 12:00:00Z
```
8df063e53712b69a346a83c599ca9cb83905b70b build: Fix help string for `--enable-external-signer` configure option (Hennadii Stepanov)
Pull request description:
This PR is a follow up of bitcoin/bitcoin#24065 and fixes the help string according to the actual default value 816ca01650/configure.ac (L324-L327)
ACKs for top commit:
kristapsk:
cr utACK 8df063e53712b69a346a83c599ca9cb83905b70b
jarolrod:
ACK 8df063e53712b69a346a83c599ca9cb83905b70b
Tree-SHA512: ad3f457a53c9238ddd8ded9efd1224e564e6cb9da8b7ff7733a11e32a7daad5c0f6c6223509218f44944a874470cb0d2447897662eaf4e78c763b30785717c50
For some reason, the primary consumer of getWalletTxs requires the
transactions to be in hash order when it is processing them. std::map
will iterate in hash order so the transactions end up in that order when
placed into the vector. To ensure this order when mapWallet is no longer
ordered, the vector is replaced with a set which will maintain the hash
order.
In order to avoid constantly re-deriving the same keys in
DescriptorScriptPubKeyMan, cache the SigningProviders generated inside
of GetSigningProvider.
Also:
- Make DEFAULT_MAX_SIG_CACHE_SIZE into constexpr
DEFAULT_MAX_SIG_CACHE_BYTES to utilize the compile-time integer
arithmetic overflow checking available to constexpr.
- Fix comment (MiB instead of MB) for DEFAULT_MAX_SIG_CACHE_BYTES.
- Pass in max_size_bytes parameter to InitS*Cache(), modify log line to
no longer allude to maxsigcachesize being split evenly between the two
validation caches.
- Fix possible integer truncation and add a comment.
[META] I've kept the integer types as int64_t in order to not introduce
unintended behaviour changes, in the next commit we will make
them size_t.
This fixes an potential overflow which existed prior to this patchset.
If CuckooCache::cache<Element, Hash>::setup_bytes is called with a
`size_t bytes` which, when divided by sizeof(Element), does not fit into
an uint32_t, the implicit conversion to uint32_t in the call to setup
will result in an overflow.
At least on x86_64, this overflow is possible:
static_assert(std::numeric_limits<size_t>::max() / 32 <= std::numeric_limits<uint32_t>::max());
static_assert(std::numeric_limits<size_t>::max() / 4 <= std::numeric_limits<uint32_t>::max());
This commit detects such cases and signals to callers that the `size_t
bytes` input is too large.
1. -maxsigcachesize is a DEBUG_ONLY option
2. Almost 7 years has passed since its semantics change in
830e3f3d027ba5c8121eed0f6a9ce99961352572 from "number of entries" to
"number of mebibytes"
3. A std::new_handler was added to the codebase after the original PR
which introduced this limit, which will terminate immediately instead
of causing trouble by being caught somewhere unexpected.
Returning the approximate total size eliminates the need for
InitS*Cache() to do nElems*sizeof(uint256). The cuckoocache has a better
idea of this information.
In src/test/fuzz/script_sigcache.cpp, we should really be setting up a
full working BasicTestingSetup. The initialize_ function is only run
once anyway.
In src/test/txvalidationcache_tests.cpp, the Dersig100Setup inherits
from BasicTestingSetup, which should have already set up a global script
execution cache without the need to explicitly call
InitScriptExecutionCache.
Rename `BResult` class to `util::Result` and update the class interface to be
more compatible with `std::optional` and with a full-featured result class
implemented in https://github.com/bitcoin/bitcoin/pull/25665. Motivation for
this change is to update existing `BResult` usages now so they don't have to
change later when more features are added in #25665.
This change makes the following improvements originally implemented in #25665:
- More explicit API. Drops potentially misleading `BResult` constructor that
treats any bilingual string argument as an error. Adds `util::Error`
constructor so it is never ambiguous when a result is being assigned an error
or non-error value.
- Better type compatibility. Supports `util::Result<bilingual_str>` return
values to hold translated messages which are not errors.
- More standard and consistent API. `util::Result` supports most of the same
operators and methods as `std::optional`. `BResult` had a less familiar
interface with `HasRes`/`GetObj`/`ReleaseObj` methods. The Result/Res/Obj
naming was also not internally consistent.
- Better code organization. Puts `src/util/` code in the `util::` namespace so
naming reflects code organization and it is obvious where the class is coming
from. Drops "B" from name because it is undocumented what it stands for
(bilingual?)
- Has unit tests.
fad5bc432b4e96aaf14bb1e02c1f6f3355a31441 test: Add missing static to IsStandardTx helper (MacroFake)
Pull request description:
Requested in https://github.com/bitcoin/bitcoin/pull/25648#discussion_r935985961
Also remove line break from the other two helpers.
ACKs for top commit:
glozow:
utACK fad5bc432b4e96aaf14bb1e02c1f6f3355a31441
aureleoules:
ACK fad5bc432b4e96aaf14bb1e02c1f6f3355a31441.
theStack:
ACK fad5bc432b4e96aaf14bb1e02c1f6f3355a31441
Tree-SHA512: 771411e1fb5939a58491ecf719e1929ab0150b0faae2078ac72bd13117f1d4dcffdeed5027bfae53e4336af25a4f1db47d564abc06a5a2c9ec006a9f67bae104
f2f6068b69b1b532db92b276f024c89b56f38294 test: MiniWallet: add `send_self_transfer_chain` to create chain of txns (Andreas Kouloumos)
1d6b438ef0ccd05e1522ac38b44f847c1d93e72f test: use MiniWallet to simplify mempool_package_limits.py tests (Andreas Kouloumos)
Pull request description:
While `wallet.py` includes the MiniWallet class and some helper methods, it also includes some methods that have been moved there without having any direct relation with the MiniWallet class. Specifically `make_chain`, `create_child_with_parents` and `create_raw_chain` methods that were extracted from `rpc_packages.py` at f8253d69d6f02850995a11eeb71fedc22e6f6575 in order to be used on both `mempool_package_limits.py` and `rpc_packages.py`.
Since that change, due to the introduction of additional methods in MiniWallet, the functionality of those methods can now be replicated with the existing MiniWallet methods and simultaneously simplify those tests by using the MiniWallet.
This PR's goals are
- to simplify the `mempool_package_limits.py` functional tests with usage of the MiniWallet.
- to make progress towards the removal of the `make_chain`, `create_child_with_parents` and `create_raw_chain` methods of `wallet.py`.
For the purpose of the aforementioned goals, a helper method `MiniWallet.send_self_transfer_chain` is introduced and method `bulk_transaction` has been integrated in `create_self_transfer*` methods using an optional `target_weight` option.
ACKs for top commit:
MarcoFalke:
ACK f2f6068b69b1b532db92b276f024c89b56f38294 👜
Tree-SHA512: 3ddfa0046168cbf7904ec6b1ca233b3fdd4f30db6aefae108b6d7fb69f34ef6fb2cf4fa7cef9473ce1434a0cc8149d236441a685352fef35359a2b7ba0d951eb
ddddd6913b1bdee1cad89a32d363306ea1f7b8d7 sort after scripted-diff (MacroFake)
fac812ca835e0d843aba1d4db0e49d183018a29e scripted-diff: Move mempool_args to src/node (MacroFake)
66664384a6fec39ecb4d8d06db66a4f193a06e33 Remove ::g_max_datacarrier_bytes global (MacroFake)
fad0b4fab849eb5f1f0aa54ebc290f85a473ec91 Pass datacarrier setting into IsStandard (MacroFake)
fa2a6b8516b24d7e9ca11926a49cf2b07f661e81 Combine datacarrier globals into one (MacroFake)
fa477d32eefcc3dd2f06b452066290d9936d8c5d Remove ::GetVirtualTransactionSize() alias (MacroFake)
fa2f6c1a611dffe5a3f63fe1b453f1dd420371b1 Remove ::fIsBareMultisigStd global (MacroFake)
fadc14e4f514e7167723285e0ac3d4a7149bbee6 Remove ::dustRelayFee (MacroFake)
fa8a7f01fe1b6db98097021276ed5d929faadbec Remove ::IsStandardTx(tx, reason) alias (MacroFake)
fa7a9114e59b81b50584311a4ab2b3e9a8d956bd test: Remove unused cs_main (MacroFake)
fa9cba7afb73c01bd2c8fefd662dfc80dd98c5e8 Remove ::incrementalRelayFee and ::minRelayTxFee globals (MacroFake)
fa148602e67fe035b1b21eff6c0b656919ac2d45 Remove ::fRequireStandard global (MacroFake)
fa468bdfb62dec286cb977db78d3e47b64dafeba Return optional error from ApplyArgsManOptions (MacroFake)
Pull request description:
This change is good because:
* It moves module-specific init-logic out of the bloated init.cpp
* It removes a global from validation.cpp and places it into the data structure that needs it (mempool)
ACKs for top commit:
glozow:
re ACK ddddd69
ryanofsky:
Code review ACK ddddd6913b1bdee1cad89a32d363306ea1f7b8d7
ariard:
Light Code Review ACK ddddd69
Tree-SHA512: 9de2ce601cfcaa4dfd7d1c92270568895ce8702ccdffb59829fbe9618eab0fd88d738afef33ed66988c66861115e0340e881056bfb71e2aed4af2440bd37eb1e
9e04cfaa76cf9dda27f10359dd43e78dd3268e09 test: add coverage for wallet inconsistent state during sync (furszy)
77de5c693ffe8dc0afa5e40126e9b0e9cc547e04 wallet: guard and alert about a wallet invalid state during chain sync (furszy)
Pull request description:
Follow-up work to my comment in #25239.
Guarding and alerting the user about a wallet invalid state during chain synchronization.
#### Explanation
if the `AddToWallet` tx write fails, the method returns a wtx `nullptr` without removing the recently added transaction from the wallet's map.
Which makes that `AddToWalletIfInvolvingMe` return false (even when the tx is on the wallet's map already), --> which makes `SyncTransaction` skip the `MarkInputsDirty` call --> which leads to a wallet invalid state where the inputs of this new transaction are not marked dirty, while the transaction that spends them still exist on the in-memory wallet tx map.
Plus, as we only store the arriving transaction inside `AddToWalletIfInvolvingMe` when we synchronize/scan block/s from the chain and nowhere else, it makes sense to treat the transaction db write error as a runtime error to notify the user about the problem. Otherwise, the user will lose all the not stored transactions after a wallet shutdown (without be able to recover them automatically on the next startup because the chain sync would be above the block where the txs arrived).
Note:
On purpose, the first commit adds test coverage for it. Showing how the wallet can end up in an invalid state. The second commit corrects it with the proposed solution.
ACKs for top commit:
achow101:
re-ACK 9e04cfaa76cf9dda27f10359dd43e78dd3268e09
jonatack:
ACK 9e04cfaa76cf9dda27f10359dd43e78dd3268e09
Tree-SHA512: 81f765eca40547d7764833d8ccfae686b67c7728c84271bc00dc51272de643dafc270014079dcc9727b47577ba67b340aeb5f981588b54e69a06abea6958aa96
It is part of the node library. Also, it won't be moved to the kernel
lib, as it will be pruned of ArgsManager.
-BEGIN VERIFY SCRIPT-
# Move module
git mv src/mempool_args.cpp src/node/
git mv src/mempool_args.h src/node/
# Replacements
sed -i 's:mempool_args\.h:node/mempool_args.h:g' $(git grep -l mempool_args)
sed -i 's:mempool_args\.cpp:node/mempool_args.cpp:g' $(git grep -l mempool_args)
sed -i 's:MEMPOOL_ARGS_H:NODE_MEMPOOL_ARGS_H:g' $(git grep -l MEMPOOL_ARGS_H)
-END VERIFY SCRIPT-
fa7bef2e80c9c290b3e97114cfa7afdea5cbd53c univalue: Remove unused and confusing set*() return value (MacroFake)
Pull request description:
The value is:
* currently unused, and useless without `[[nodiscard]]`
* confusing, because it is always `true`, unless a num-string is set
Instead of adding `[[nodiscard]]`, throw when setting is not possible.
ACKs for top commit:
shaavan:
ACK fa7bef2e80c9c290b3e97114cfa7afdea5cbd53c
aureleoules:
ACK fa7bef2e80c9c290b3e97114cfa7afdea5cbd53c.
Tree-SHA512: 0d74f96f34cb93b66019ab75e12334c964630cc83434f22e58cc7a4fff2ee96a5767e42ab37f08acb67aeacba6811b09c75f1edc68d5e903ccfc59b1c82de891
