a2324e4d3f47f084b07a364c9a360a0bf31e86a0 test: Improve naming and logging of avoid_reuse tests (Fabian Jahr)
1abbdac6777bc5396d17a6772c8176a354730997 wallet: Prefer full destination groups in coin selection (Fabian Jahr)
Pull request description:
Fixes#17603 (together with #17843)
In the case of destination groups of >10 outputs existing in a wallet with `avoid_reuse` enabled, the grouping algorithm is adding left-over outputs as an "incomplete" group to the list of groups even when a full group has already been added. This leads to the strange behavior that if there are >10 outputs for a destination the transaction spending from that will effectively use `len(outputs) % 10` as inputs for that transaction.
From the original PR and the code comment I understand the correct behavior should be the usage of 10 outputs. I opted for minimal changes in the current code although there maybe optimizations possible for cases with >20 outputs on a destination this sounds like too much of an edge case right now.
ACKs for top commit:
jonatack:
Re-ACK a2324e4
achow101:
ACK a2324e4d3f47f084b07a364c9a360a0bf31e86a0
kallewoof:
ACK a2324e4d3f47f084b07a364c9a360a0bf31e86a0
meshcollider:
Tested ACK a2324e4d3f47f084b07a364c9a360a0bf31e86a0 (verified the new test fails on master without this change)
Tree-SHA512: 4743779c5d469fcd16df5baf166024b1d3c8eaca151df1e8281b71df62b29541cf7bfee3f8ab48d83e3b34c9256e53fd38a7b146a54c79f9caa44cce3636971a
69ffddc83e0f3e265bf6cf7ae31489ae629fe6be refactor: Remove unused methods CBloomFilter::reset()/clear() (Sebastian Falbesoner)
Pull request description:
The method `CBloomFilter::reset()` was introduced by commit d2d7ee0e863b286e1c9f9c54659d494fb0a7712d in 2015, but was never ever used, as far as I could find. As discovered by MarcoFalke, the method `clear()` is also unused outside of unit tests and is hence also removed.
ACKs for top commit:
MarcoFalke:
re-ACK 69ffddc83e0f3e265bf6cf7ae31489ae629fe6be
jonatack:
ACK 69ffddc83e0f3e2, code review, compiled a fuzz build and started the bloom_filter fuzz test as a sanity check.
promag:
ACK 69ffddc83e0f3e265bf6cf7ae31489ae629fe6be.
Tree-SHA512: 6c53678545ad8e2fa1ffc0a8838e450462f26748a60632f738dc020f0eb494ae2c32841e6256e266ed9140177257a78b707123421942f3819a14ffcb9a99322f
9b5950db8683f9b4be03f79ee0aae8a780b01a4b bnb: exit selection when best_waste is 0 (Andrew Chow)
Pull request description:
If we find a solution which has no waste, just use that. This solution
is what we would consider to be optimal, and other solutions we find
would have to also have 0 waste, so they are equivalent to the first
one with 0 waste. Thus we can optimize by just choosing the first one
with 0 waste.
Closes#18257
ACKs for top commit:
instagibbs:
utACK 9b5950db86
meshcollider:
utACK 9b5950db8683f9b4be03f79ee0aae8a780b01a4b
Tree-SHA512: 59565ff4a3d8281e7bc0ce87065a34c8d8bf8a95f628ba96b4fe89f1274979165aea6312e5f1f21b418c8c484aafc5166d22d9eff9d127a8192498625d58c557
faf989f93695d29099f6e152d5a2e117cd304183 util: Document why ArgsManager (con/de)structor is not inline (MarcoFalke)
fae00a77e2589cc784650e3e60e1b99c22ca8a7b bench: Remove unused argsman.ClearArgs (MarcoFalke)
fa46aebeb1fc419b227524eab8352d9a7fc1f981 scripted-diff: Replace gArgs with local argsman in bench (MarcoFalke)
fa2bc4141df59f2e38fef863723b433250295d20 tools: Add unused argsman to bench_bitcoin (MarcoFalke)
Pull request description:
All utilities use the same gArgs global that the node uses. This is generally fine and does not lead to issues unless a bench test is going to spin up a NodeContext via the TestingSetup. In that case the two uses of gArgs conflict and currently it needs to be cleared:
544709763e/src/bench/bench_bitcoin.cpp (L76)
One solution would be to do nothing, because the current code works with that workaround. Another solution would be to not use the same global in all binaries.
ACKs for top commit:
promag:
ACK faf989f93695d29099f6e152d5a2e117cd304183.
ryanofsky:
Code review ACK faf989f93695d29099f6e152d5a2e117cd304183. Just new commit added restoring forward declaration
Tree-SHA512: 8ee4b28eee294d41c002f801fa844b0c23c919a3061f5109638701db0947b3b0ea28caa7311ae5f126fc660648bbaa0890853e6b06bdc5868692f52ba8c05f66
e44aeefaaed8d698d1b9004b66f85384397b1a75 gitian: Add missing automake package to gitian-win-signer.yml (Andrew Chow)
Pull request description:
automake is needed to build osslsigncode otherwise autogen.sh fails with the docker virtualization method.
ACKs for top commit:
hebasto:
ACK e44aeefaaed8d698d1b9004b66f85384397b1a75, for `osslsigncode-1.7.1` we did not run `autogen.sh` in the past.
fanquake:
ACK e44aeefaaed8d698d1b9004b66f85384397b1a75
jonatack:
ACK e44aeef
Tree-SHA512: a0e615c1b099ee1c469ce41f886f2ece6746234a5a800743a4e8be671e4114fd30e1c35bc0ddcb75778409564129d0fde7ac4e3d70b0f7691f97f729f34c8e0c
0c6318788beaf1a31aeba5a21f3f8bb5c07cea6c ci: Limit cache size regardless of NO_DEPENDS (Hennadii Stepanov)
Pull request description:
Close#18666.
ACKs for top commit:
MarcoFalke:
ACK 0c6318788beaf1a31aeba5a21f3f8bb5c07cea6c . Depends has ccache disabled anyway and is cached regardless of whether ccache is there or not, see #17248
Tree-SHA512: b1bf98be0f844b4704abd177841b014f3900be8160496f0d12596310db607b4f544547e8c3cbfcf17c086a78afd251653363f3dd467b769ac0062bc19adc8144
bee88b8c5887e6beb75f26f0db97888a48fa7e7c tests: have coins simulation test also use CCoinsViewDB (James O'Beirne)
Pull request description:
Before this change, the coins simulation test uses a base view of type
CCoinsViewTest, which has no relevance outside of the unittest suite. Might as
well reuse this testcase with a more realistic configuration that has
CCoinsViewDB (i.e. in-memory leveldb) at the bottom of the view structure.
This adds explicit use of CCoinsViewDB in the unittest suite.
#### Before change
```
./src/test/test_bitcoin --run_test=coins_tests --catch_system_errors=no 21.99s user 0.04s system 99% cpu 22.057 total
```
#### After change
```
./src/test/test_bitcoin --run_test=coins_tests --catch_system_errors=no 78.80s user 0.04s system 100% cpu 1:18.82 total
```
ACKs for top commit:
ryanofsky:
Code review ACK bee88b8c5887e6beb75f26f0db97888a48fa7e7c
Tree-SHA512: 75296b2bcbae2f46e780489aafb032592544a15c384d569d016005692fe79fe60d7f05857cf25cc7b0f9ab1c53b47886a6c71cca074a03fb9afec30e1f376858
a95af77eb264646b4160836e4e9a2c4b45f87bb8 qt: Make bitcoin.ico non-executable (practicalswift)
Pull request description:
Make `bitcoin.ico` non-executable.
No need to execute icons and having +x bits laying around breaks `find … -executable` :)
Before this patch:
```sh
$ find $(git ls-files -- ":(exclude)*.sh" ":(exclude)*.py") -executable
ci/retry/retry
contrib/macdeploy/macdeployqtplus
depends/config.guess
depends/config.sub
src/qt/res/icons/bitcoin.ico
src/secp256k1/src/modules/recovery/main_impl.h
```
After this patch:
```sh
$ find $(git ls-files -- ":(exclude)*.sh" ":(exclude)*.py") -executable
ci/retry/retry
contrib/macdeploy/macdeployqtplus
depends/config.guess
depends/config.sub
src/secp256k1/src/modules/recovery/main_impl.h
```
FWIW:
```
$ file $(find $(git ls-files -- ":(exclude)*.sh" ":(exclude)*.py") -executable)
ci/retry/retry: Bourne-Again shell script, UTF-8 Unicode text executable
contrib/macdeploy/macdeployqtplus: Python script, ASCII text executable
depends/config.guess: POSIX shell script, ASCII text executable
depends/config.sub: POSIX shell script, ASCII text executable
src/qt/res/icons/bitcoin.ico: MS Windows icon resource - 10 icons, 48x48, 16 colors, 4 bits/pixel, 32x32, 16 colors, 4 bits/pixel
src/secp256k1/src/modules/recovery/main_impl.h: C source, ASCII text
```
ACKs for top commit:
MarcoFalke:
ACK a95af77eb264646b4160836e4e9a2c4b45f87bb8 gitian build finished, so it doesn't look like the icon used in Windows resource files needs to be executable. Though, I didn't read the documentation.
jonatack:
ACK a95af77eb264646
Tree-SHA512: ecf9154077824ae4c274b4341e985797f3648c0cb0c31cb25ce382163b923a3acbc7048683720be4ae3663501801129cd0f48c441a36f049cc304ebe9f30994e
f63dec189c3c8eee1ab2187681d5d0b2513b1b2e [REFACTOR] Initialize PrecomputedTransactionData in CheckInputScripts (Pieter Wuille)
Pull request description:
This is a single commit taken from the Schnorr/Taproot PR #17977.
Add a default constructor to `PrecomputedTransactionData`, which doesn't initialize the struct's members. Instead they're initialized inside the `CheckInputScripts()` function. This allows a later commit to add the spent UTXOs to that structure. The spent UTXOs are required for the schnorr signature hash, since it commits to the scriptPubKeys. See https://github.com/bitcoin/bips/blob/master/bip-0341.mediawiki#common-signature-message for details.
By itself, this isn't really an improvement to the code, but I think it makes sense to separate out the refactor/moveonly commits from PR #17977 so that PR is only the logical changes needed for Schnorr/Taproot.
ACKs for top commit:
jonatack:
Re-ACK f63dec1 `git diff 851908d f63dec1` shows no change since last ACK.
sipa:
utACK f63dec189c3c8eee1ab2187681d5d0b2513b1b2e
theStack:
re-ACK f63dec189c3c8eee1ab2187681d5d0b2513b1b2e
fjahr:
Re-ACK f63dec189c3c8eee1ab2187681d5d0b2513b1b2e
ariard:
Code Review ACK f63dec1
Tree-SHA512: ecf9154077824ae4c274b4341e985797f3648c0cb0c31cb25ce382163b923a3acbc7048683720be4ae3663501801129cd0f48c441a36f049cc304ebe9f30994e
fa69f88486e900aacf3fc768671f947927173226 fuzz: Disable debug log file (MarcoFalke)
fa0cbd48c418ec14e1d91bffea206bce20bd1e56 test: Add optional extra_args to testing setup (MarcoFalke)
fad4fa7e2fb95b7ced9007060ebfd0e8f181f5d8 node: Add args alias for gArgs global (MarcoFalke)
Pull request description:
There are several issues with writing to a debug log file when fuzzing:
* Disk access is slow, but fuzzing should be fast (Note: I could not verify this claim with data)
* Disks have a limited size and will eventually run out of space, but fuzzing should run continuous
Fix both issues by disabling the debug log file for fuzz tests
ACKs for top commit:
practicalswift:
ACK fa69f88486e900aacf3fc768671f947927173226 -- patch looks correct
Tree-SHA512: f61beb6c94a9ab664deb191685fcad601e228b77bb1c43db6ec40616ae393c9dd35c51474f1b0759ac0bc29b5ca8456a329906a3695bd0f18fa4372210c8b54a
808ef36b89ea9ce72116bbd7ee479b984367dc60 [doc] Update thread information in developer docs (John Newbery)
Pull request description:
- DumpAddresses thread was removed in #5964
- Shutdown thread was removed in #5679
- Add new threads (scheduler, RPC worker, indexer, tor control)
- Small changes to documentation of other threads
ACKs for top commit:
MarcoFalke:
ACK 808ef36b89
hebasto:
ACK 808ef36b89ea9ce72116bbd7ee479b984367dc60.
Tree-SHA512: 85b6ace7bcc4dee030c63461bef1ded1a9581d4fa249c59f6fcd5d33d89c4357a6b8b35888ce0960f276d397b5e38a21e6c5d4b7b79544827a28c950e097b36d
fac0c8db9f634ea7231fb0b6dc139ae0bda4a051 appveyor: Remove clcache (MarcoFalke)
Pull request description:
The build time without cache seems to be 47 minutes and with cache 46 minutes. Maybe we can save more time by not installing clcache.
Top commit has no ACKs.
Tree-SHA512: ce503641a465f5e49deb7aa6b566aaab5e567148c001704451891c49ba7ade3219ca788bc0d8e55565febb9aadd579e26894f6c03b99755eb8b5d1498acf4dc9
fa176e253fb473767c61d4d8cd2d93e87d71a015 test: Avoid accessing free'd memory in validation_chainstatemanager_tests (MarcoFalke)
Pull request description:
ACKs for top commit:
ryanofsky:
Code review ACK fa176e253fb473767c61d4d8cd2d93e87d71a015, though if you have to update this again, would suggest separating txindex test cleanup and the chainstatemanager test fix in separate commits, or identifying which part of the change is the bugfix fix in the commit description. Also to clean up the txindex test it might make sense to call SyncWithValidationInterfaceQueue in the test destructor to prevent nondeterminism in other tests
Tree-SHA512: 34c5dca283a7c205cd42b6aa59f00a71fd1bd980bc3d6640a18b280be11470bfabb2fd8c93fadde6fb8e084bcf96c80ec3aa72bbccccfde8a8260d173eaad08f
c47adf8df435831c26ca25813fb7272176bd4eb7 Added my fingerprint Stephan Oeste (Emzy) (Stephan Oeste)
Pull request description:
By request from laanwj added my PGP fingerprint.
See: https://github.com/bitcoin-core/gitian.sigs/pull/1220#issuecomment-612778442
ACKs for top commit:
Sjors:
ACK c47adf8. Fingerprint matches Twitter profile: https://twitter.com/emzy (haven't verified it in any other way)
fanquake:
ACK c47adf8df435831c26ca25813fb7272176bd4eb7
Tree-SHA512: 3e39ae88f507a12f11fb2d5c779eba79ee2daeddecd0dc3f1fddfa29ce963d0e9af3fa5a10357157812597c10205a6beae31cc70af9471a782da23d8753b7cbd
f2b5b0a3b48f9241971c14238040048f8b630811 build: add linker optimization flags to guix (fanquake)
b8b050a8d642e38c81d1e451750c2c8db92fee5e build: add linker optimization flags to gitian descriptors (fanquake)
Pull request description:
This PR adds `-Wl,O2` to our gitian and guix LDFLAGS. This makes the linker perform certain optimisations (and is different from LTO).
Any -O argument will enable optimizations in GNU ld. We can use -O2 here, as this matches our compile flags. Note that this would also enable additional optimizations if using the lld or gold linkers, when compared to -O0.
A nice writeup + diagrams of some of these optimizations is available here: http://lwn.net/Articles/192624/.
#### master
```bash
# bitcoind
Histogram for `.gnu.hash' bucket list length (total of 3 buckets)
Length Number % of total Coverage
0 1 ( 33.3%) 0.0%
1 0 ( 0.0%) 0.0%
2 1 ( 33.3%) 40.0%
3 1 ( 33.3%) 100.0%
```
```bash
# bitcoin-qt
Histogram for `.gnu.hash' bucket list length (total of 3 buckets)
Length Number % of total Coverage
0 0 ( 0.0%) 0.0%
1 1 ( 33.3%) 10.0%
2 0 ( 0.0%) 10.0%
3 0 ( 0.0%) 10.0%
4 1 ( 33.3%) 50.0%
5 1 ( 33.3%) 100.0%
```
#### this PR:
```bash
# bitcoind
Histogram for `.gnu.hash' bucket list length (total of 8 buckets)
Length Number % of total Coverage
0 3 ( 37.5%) 0.0%
1 5 ( 62.5%) 100.0%
```
```bash
# bitcoin-qt
Histogram for `.gnu.hash' bucket list length (total of 19 buckets)
Length Number % of total Coverage
0 9 ( 47.4%) 0.0%
1 10 ( 52.6%) 100.0%
```
#### GNU ld -O
> If level is a numeric values greater than zero ld optimizes the output. This might take significantly longer and therefore probably should only be enabled for the final binary. At the moment this option only affects ELF shared library generation. Future releases of the linker may make more use of this option. Also currently there is no difference in the linker’s behaviour for different non-zero values of this option. Again this may change with future releases.
#### lld -O
> Optimize output file size
ACKs for top commit:
dongcarl:
ACK f2b5b0a3b48f9241971c14238040048f8b630811
laanwj:
ACK f2b5b0a3b48f9241971c14238040048f8b630811
Tree-SHA512: e53f3a4338317dbec65d3a93b57b5a6204aabdf9ac82d99447847a3c8627facc53c58c2cf947376f13edd979fc8129a80f18d9ebeccd191a576c83f1dad5c513
fa6cb00ce535d9be2c91465200674a07fc13c7d9 doc: Fix macos comments in release-notes (MarcoFalke)
Pull request description:
ACKs for top commit:
hebasto:
ACK fa6cb00ce535d9be2c91465200674a07fc13c7d9.
fanquake:
ACK fa6cb00ce535d9be2c91465200674a07fc13c7d9 - this has also been done in the release notes in the wiki.
Tree-SHA512: ce325a9a32e28923ad34bdc7c1c69e9d14636a18d05604158e99f80683686bd2ba5550e84f85479336658b84c40dbe2940840e47d69c10ae3d41180dad49c8f3
fa50d11dd117eb6f91cbeac7476797f3885a6169 appveyor: Disable functional tests for now (MarcoFalke)
Pull request description:
ACKs for top commit:
jonasschnelli:
ACK fa50d11dd117eb6f91cbeac7476797f3885a6169
Tree-SHA512: 86ea9bc0ff47dc309120748627cd83b19aa61fb1db3cca5e04a9e0d56f34c248cd2bec20b037c663ab0f21f7ee6cd0ff8266a36fe79c773de95e5032eb48b118
When a wallet uses avoid_reuse and has a large number of outputs in
a single destination, it groups these outputs in OutputGroups that
are no larger than OUTPUT_GROUP_MAX_ENTRIES. The goal is to spend
as many outputs as possible from the destination while not breaking
consensus due to a huge number of inputs and also not surprise the
use with high fees. If there are n outputs in a destination and
n > OUTPUT_GROUP_MAX_ENTRIES then this results in one or many groups
of size OUTPUT_GROUP_MAX_ENTRIES and possibly one group of size
< OUTPUT_GROUP_MAX_ENTRIES.
Prior to this commit the coin selection in the case where
n > OUTPUT_GROUP_MAX_ENTRIES was skewed towards the one group of
size < OUTPUT_GROUP_MAX_ENTRIES if it exists and the amount to be
spent by the transaction is smaller than the aggregate of those
of the group size < OUTPUT_GROUP_MAX_ENTRIES. The reason is that
the coin selection decides between the different groups based on
fees and mostly the smaller group will cause smaller fees.
The behavior that users of the avoid_reuse flag seek is that the
full groups of size OUTPUT_GROUP_MAX_ENTRIES get used first. This
commit implements this by pretending that the small group has
a large number of ancestors (one smallet than the maximum allowed
for this wallet). This dumps the small group to the bottom of the
list of priorities in the coin selection algorithm.
48973402d8bccb673eaeb68b7aa86faa39d3cb8a wallet: Avoid use of Chain::Lock in CWallet::GetKeyBirthTimes (Russell Yanofsky)
e958ff9ab5607da2cd321f29fc785a6d359e44f4 wallet: Avoid use of Chain::Lock in CWallet::CreateTransaction (Russell Yanofsky)
c0d07dc4cba7634cde4e8bf586557772f3248a42 wallet: Avoid use of Chain::Lock in CWallet::ScanForWalletTransactions (Russell Yanofsky)
1be8ff280c78c30baabae9429c53c0bebb89c44d wallet: Avoid use of Chain::Lock in rescanblockchain (Russell Yanofsky)
3cb85ac594f115db99f96b0a0f4bfdcd69ef0590 wallet refactor: Avoid use of Chain::Lock in CWallet::RescanFromTime (Russell Yanofsky)
f7ba881bc669451a60fedac58a449794702a3e23 wallet: Avoid use of Chain::Lock in listsinceblock (Russell Yanofsky)
bc96a9bfc61afdb696fb92cb644ed5fc3d1793f1 wallet: Avoid use of Chain::Lock in importmulti (Russell Yanofsky)
25a9fcf9e53bfa94e8f8b19a4abfda0f444f6b2a wallet: Avoid use of Chain::Lock in importwallet and dumpwallet (Russell Yanofsky)
c1694ce6bb7e19a8722d5583cd85ad17da40bb67 wallet: Avoid use of Chain::Lock in importprunedfunds (Russell Yanofsky)
ade5f87971211bc67753f14a0d49e020142efc7c wallet refactor: Avoid use of Chain::Lock in qt wallettests (Russell Yanofsky)
f6da44ccce4cfff53433e665305a6fe0a01364e4 wallet: Avoid use of Chain::Lock in tryGetTxStatus and tryGetBalances (Russell Yanofsky)
bf30cd4922ea62577d7bf63f5029e8be62665d45 refactor: Add interfaces::FoundBlock class to selectively return block data (Russell Yanofsky)
Pull request description:
This is a set of changes updating wallet code to make fewer calls to `Chain::Lock` methods, so the `Chain::Lock` class will be easier to remove in #16426 with fewer code changes and small changes to behavior.
ACKs for top commit:
MarcoFalke:
re-ACK 48973402d8, only change is fixing bug 📀
fjahr:
re-ACK 48973402d8bccb673eaeb68b7aa86faa39d3cb8a, reviewed rebase and changes since last review, built and ran tests locally
ariard:
Coce Review ACK 4897340, only changes are one suggested by last review on more accurate variable naming, human-readable output, args comments in `findCommonAncestor`
Tree-SHA512: cfd2f559f976b6faaa032794c40c9659191d5597b013abcb6c7968d36b2abb2b14d4e596f8ed8b9a077e96522365261299a241a939b3111eaf729ba0c3ef519b
0306d78cb49d1684cc96ba3512b582a1fdaf78cc Use getbalances in wallet_address_types tests (Jon Atack)
7eacdc5167c8db94df84e206db85817bc64e4921 Shift coverage from getunconfirmedbalance to getbalances in wallet_abandonconflict tests (Jon Atack)
3e6f7377f600e47e5e3d439fc5d6ccf3db210038 Improve getbalances coverage in wallet_balance tests (Jon Atack)
Pull request description:
<strike>This PR updates several tests and then removes the `getunconfirmedbalance` RPC which was deprecated in facfb4111d14a3b06c46690a2cca7ca91cea8a96 a year ago.
Next steps: remove the deprecated `getwalletinfo` fields and the `getbalance` RPC in follow-ups, if there seems to be consensus on those removals.</strike>
Update:
`getunconfirmedbalance` RPC was deprecated in facfb4111d14a3b06c46690a2cca7ca91cea8a96 a year ago, but following the review comments below, this PR now only updates the test coverage to use `getbalances` while still leaving basic coverage for `getunconfirmedbalance` in wallet_balance.py.
That said, I've seen 3 regular contributors confused in the past 10 days by "DEPRECATED" warnings in the code that are not following the deprecation policy in [JSON-RPC-interface.md#versioning](https://github.com/bitcoin/bitcoin/blob/master/doc/JSON-RPC-interface.md#versioning).
ISTM these warnings should either be removed, or the calls deprecated (`-deprecatedrpc`), or the policy updated to describe these warnings as a pre-deprecation practice.
ACKs for top commit:
jnewbery:
utACK 0306d78cb
Tree-SHA512: 692e43e9bed5afa97d905740666e365f0b64e559e1c75a6a398236d9e943894e3477947fc11324f420a6feaffa0c0c1532aa983c50090ca39d06551399e6ddd1
