0ccb3addf68067200892963521a92713c4667a63 tests: Remove no longer needed UBSan suppression (float-divide-by-zero in validation.cpp) (practicalswift)
Pull request description:
Remove no longer needed UBSan suppression.
The float divide-by-zero in `validation.cpp` was fixed by instagibbs in ec30a79f1c430cc7fbda37e5d747b0b31b262fa5 (#15283).
ACKs for top commit:
MarcoFalke:
ACK 0ccb3addf68067200892963521a92713c4667a63
Tree-SHA512: 89a4f4b7371fa5725d9f801cee7ebbd17523f66017c9acfa813657dcb8d837f42209eff44ce9e5d48296a630bab9599d75f10024a0c7da7defb228f4eae3392a
38ada892ed0ed9aaa46b1791db12a371a3c0c419 addrman: ensure old versions don't parse peers.dat (Vasil Dimov)
Pull request description:
Even though the format of `peers.dat` was changed in a backwards
incompatible way, it is not guaranteed that old versions will fail to
parse it. There is a chance that old versions parse its contents as
garbage and use it.
Old versions expect the "key size" field to be 32 and fail the parsing
if it is not. Thus, we put something other than 32 in it. This will make
versions between 0.11.0 and 0.20.1 deterministically fail on the new
format. Versions prior to https://github.com/bitcoin/bitcoin/pull/5941
will still parse it as garbage.
Also, introduce a way to increment the `peers.dat` format in a way that
does not necessary make older versions refuse to read it.
ACKs for top commit:
jnewbery:
ACK 38ada892ed0ed9aaa46b1791db12a371a3c0c419
laanwj:
Code review ACK 38ada892ed0ed9aaa46b1791db12a371a3c0c419
MarcoFalke:
re-ACK 38ada892ed0ed9aaa46b1791db12a371a3c0c419 🥐
Tree-SHA512: 550bd660c5019dba0f9c334aca8a11c4a0463cfddf11efe7a4a5585ffb05549c82b95066fba5d073ae37893e0eccc158a7ffea9b33ea031d9be4a39e44f6face
241434200ec2067673d8522fee4f1228abfd8247 refactor: qt: Use vQueueNotifications.clear() (João Barbosa)
989e579d07bb5031639060b717f7a0be15d10e29 qt: Make transaction notification queue wallet specific (João Barbosa)
7b3b2303f44031c3545651858f697a495c3ea37a move-only: Define TransactionNotification before TransactionTablePriv (João Barbosa)
Pull request description:
Currently `vQueueNotifications` holds transactions of any wallet, but the queue is dispatched on a given wallet and it assumes notifications are of that wallet.
This means that some transactions can be missed if multiple wallets are loaded.
Fix this by having a queue for each wallet.
ACKs for top commit:
jonasschnelli:
utACK 241434200ec2067673d8522fee4f1228abfd8247
hebasto:
ACK 241434200ec2067673d8522fee4f1228abfd8247, I have reviewed the code and it looks OK, I agree it can be merged.
ryanofsky:
Code review ACK 241434200ec2067673d8522fee4f1228abfd8247. Only change is dropping one commit
Tree-SHA512: 61beac5a16ed659e3a25ad145dbceafcef963aaf8f9838355298949ec2324e2bd760f59353cd251d30cf0334d8dc1642a1f3821d8a9eec092533b581f6ce86db
79ef8324d4c85ed16a304e98805724b8a59022ac tests: Add fuzzing harness for CConnman (practicalswift)
Pull request description:
Add fuzzing harness for `CConnman`.
See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).
Happy fuzzing :)
ACKs for top commit:
MarcoFalke:
review ACK 79ef8324d4c85ed16a304e98805724b8a59022ac
Tree-SHA512: eb9ffae20e939b818f8b9def064544b9a8fcd127ca22d1a54af1afedf1d24143be42419f3a03d684be59a5ff07b29d8bfa34ef2aaf1d9f9f75c4c1aaa90a29a8
3c77b8009de9457c356c0bf4362d11bb99a17bb7 fuzz: Improve coverage for CPartialMerkleTree fuzzing harness (practicalswift)
Pull request description:
Improve coverage for `CPartialMerkleTree` fuzzing harness.
See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).
Happy fuzzing :)
ACKs for top commit:
MarcoFalke:
ACK 3c77b8009de9457c356c0bf4362d11bb99a17bb7
Tree-SHA512: a1fa0f7650a5ee5ff83f35e41b9faf6c34671fc304b9af00e5b83073f21d50bcbe91c2428fa64d05dc42a7c521bfd24031e307c7f4abf9ded469d69a55c5d64a
ee11a412a537f62aa46e8862678ce2069a2df5b7 Avoid signed integer overflow when loading a mempool.dat file with a malformed time field (practicalswift)
Pull request description:
Avoid signed integer overflow when loading a `mempool.dat` file with a malformed time field.
Avoid the following signed integer overflow:
```
$ xxd -p -r > mempool.dat-crash-1 <<EOF
0100000000000000000000000004000000000000000000000000ffffffff
ffffff7f00000000000000000000000000
EOF
$ cp mempool.dat-crash-1 ~/.bitcoin/regtest/mempool.dat
$ UBSAN_OPTIONS="print_stacktrace=1:halt_on_error=1:report_error_type=1" src/bitcoind -regtest
validation.cpp:5079:23: runtime error: signed integer overflow: 9223372036854775807 + 1209600 cannot be represented in type 'long'
#0 0x5618d335197f in LoadMempool(CTxMemPool&) src/validation.cpp:5079:23
#1 0x5618d3350df3 in CChainState::LoadMempool(ArgsManager const&) src/validation.cpp:4217:9
#2 0x5618d2b9345f in ThreadImport(ChainstateManager&, std::vector<boost::filesystem::path, std::allocator<boost::filesystem::path> >, ArgsManager const&) src/init.cpp:762:33
#3 0x5618d2b92162 in AppInitMain(util::Ref const&, NodeContext&, interfaces::BlockAndHeaderTipInfo*)::$_14::operator()() const src/init.cpp:1881:9
```
This PR was broken out from PR #20089. Hopefully this PR is trivial to review.
Fixes a subset of #19278.
ACKs for top commit:
MarcoFalke:
review ACK ee11a412a537f62aa46e8862678ce2069a2df5b7
Crypt-iQ:
crACK ee11a412a537f62aa46e8862678ce2069a2df5b7
Tree-SHA512: 227ab95cd7d22f62f3191693b455eacfa8e36534961bee12c622fc9090957cfb29992eabafa74d806a336e03385aa8f98b7ce734f04b0b400e33aa187d353337
c82336c493b112160d781974d4066fcb956b85f6 Remove references to CreateWalletFromFile (fanquake)
Pull request description:
`CWallet::CreateWalletFromFile()` was removed in 8b5e7297c02f3100a9cb27bfe206e3fc617ec173 but these references remain.
ACKs for top commit:
hebasto:
ACK c82336c493b112160d781974d4066fcb956b85f6
Tree-SHA512: 3dd50fe0cd5a60bbc96d265107d4739f3e08f943435f3772038963ac4be9e4a87a863412ac0d571226ea66d71550b17b52f01b9d46a6282d49feae1508fd682e
24d2d3341d07509ad3f37bb6f130446ad20ac807 QA: wallet_multiwallet: Check that recursive symlink directory and wallet.dat loops are ignored (Luke Dashjr)
69f59af54d15ee9800d5df86bcdb0e962c71e7c3 Bugfix: Wallet: Soft-fail exceptions within ListWalletDir file checks (Luke Dashjr)
Pull request description:
Previously, an exception would be thrown, which could kill the node in some circumstances.
Includes test changes to cause failure.
Review with `?w=1`
ACKs for top commit:
hebasto:
re-ACK 24d2d3341d07509ad3f37bb6f130446ad20ac807, rebased only since my [previous](https://github.com/bitcoin/bitcoin/pull/19502#pullrequestreview-520552944) review.
promag:
Tested ACK 24d2d3341d07509ad3f37bb6f130446ad20ac807, test change fails on master.
meshcollider:
utACK 24d2d3341d07509ad3f37bb6f130446ad20ac807
Tree-SHA512: f701f81b3aa3d3e15cee52ac9e7c31a73c0d8166e56bf077235294507cbcee099829fedc432a1c4b6d8780885f4e37897b44b980b08125771de3c849c000499e
5e146022daa4336de94447e5b8e5418296286927 wallet: fix scanning progress calculation for single block range (Sebastian Falbesoner)
Pull request description:
If the blockchain is rescanned for a single block (i.e. start and stop hashes are equal, and with that also the estimated start/stop verification progress values) the progress calculation could lead to a NaN value caused by a division by zero (0.0/0.0), resulting in an invalid JSON result for the `getwalletinfo` RPC. This PR fixes this behaviour by setting the progress to zero in that special case. Fixes#20297.
The behaviour can easily be reproduced by continuously running single block rescans in an endless loop, e.g. via
```bash
#!/bin/bash
while true
do
bitcoin-cli rescanblockchain $(bitcoin-cli getblockcount)
done
```
and at the same time perform some `getwalletinfo` RPCs.
On the master branch, this leads to frequent invalid responses (tested on mainchain):
```
$ bitcoin-cli getwalletinfo
error: couldn't parse reply from server
$ curl --user `cat ~/.bitcoin/.cookie` --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "getwalletinfo", "params": []}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
{"result":{"walletname":"","walletversion":169900,"format":"bdb","balance":0.00000000,"unconfirmed_balance":0.00000000,"immature_balance":0.00000000,"txcount":0,"keypoololdest":1603677276,"keypoolsize":1000,"hdseedid":"3196e33ecb47c7130e6ca60f2f895f9259860dca","keypoolsize_hd_internal":1000,"paytxfee":0.00000000,"private_keys_enabled":true,"avoid_reuse":false,"scanning":{"duration":0,"progress":},"descriptors":false},"error":null,"id":"curltest"}
```
(note that missing value for "progress" in the JSON result).
On the PR branch, the behaviour doesn't occur anymore.
ACKs for top commit:
MarcoFalke:
review ACK 5e146022daa4336de94447e5b8e5418296286927
promag:
Core review ACK 5e146022daa4336de94447e5b8e5418296286927.
Tree-SHA512: f0e6aad5a6cd08b36c5fe820fff0ef26663229b39169a4dbe757f3c795a41cf5c69c9dc90efe7515675ae1059307f8971123781a0514d10704123a6f28b125ab
Even though the format of `peers.dat` was changed in an incompatible
way (old software versions <0.21 cannot understand the new file format),
it is not guaranteed that old versions will fail to parse it. There is a
chance that old versions parse its contents as garbage and use it.
Old versions expect the "key size" field to be 32 and fail the parsing
if it is not. Thus, we put something other than 32 in it. This will make
versions between 0.11.0 and 0.20.1 deterministically fail on the new
format. Versions prior to https://github.com/bitcoin/bitcoin/pull/5941
(<0.11.0) will still parse it as garbage.
Also, introduce a way to increment the `peers.dat` format in a way that
does not necessary make older versions refuse to read it.
Create a fee_rate (sat/vB) RPC param and replace overloading the conf_target and
estimate_mode params in the following 6 RPCs with it:
- sendtoaddress
- sendmany
- send
- fundrawtransaction
- walletcreatefundedpsbt
- bumpfee
In RPC bumpfee, the previously existing fee_rate remains but the unit is changed
from BTC/kvB to sat/vB. This is a breaking change, but it should not be an
overly risky one, as the units change by a factor of 1e5 and any fees specified
in BTC/kvB after this commit will either be too low and raise an error or be 1
sat/vB and can be RBFed.
Update the test coverage for each RPC.
Co-authored-by: Murch <murch@murch.one>
fa949b3c1325693ea7ecc5556b2de50d2a6c9ead test: Suppress epoll_ctl data race (MarcoFalke)
Pull request description:
Happens intermittently: https://cirrus-ci.com/task/5462892373868544?command=ci#L5385
ACKs for top commit:
hebasto:
ACK fa949b3c1325693ea7ecc5556b2de50d2a6c9ead, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: d5aa559fc105053da594531722f2a03d898eadeb4413c3a728fc5116cc4d1a2c16c49649a24c75ea810e4ec6bb9728b0bcd2ea991886bb9d206170218eddf6d2
fa92cf29d971811ff8dd844ee1a1e2e12b79e389 ci: Remove redundant valgrind fuzz task (MarcoFalke)
Pull request description:
This task has several issues:
* It slows down other tasks and times out: It needs a lot of resources (CPU, RAM, time), because it builds more than 100 fuzzers, clones a 2 GB repo with 100k seeds and pipes them all through valgrind
* It doesn't add a lot of value: Except for one issue in the boost time library, it hasn't found any issues that the existing fuzz,asan,ubsan fuzzer has already found
* It is redundant: It is already run in the bitcoin-core/qa-assets repo on every push of new seeds and once daily
Fix all issues by removing it here.
Top commit has no ACKs.
Tree-SHA512: 76d16a3e5afc79ba9d89cfeb915d7e66fd1ad4e2035ae4ccd30a21cc060bfba3fb6a904346b7c41606e69d972e86bf660df962673028689227cc26072bba24bd
If the blockchain is rescanned for a single block (i.e. start and stop hashes
are equal, and with that also the estimated verification progress) the progress
calculation could lead to a NaN value caused by a division by zero, resulting in
an invalid JSON result for the getwalletinfo RPC. Fixed by setting the progress
to zero in that special case.
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
native_ds_store now takes advantage of Pythons ability to decode binary
plists (since 3.4), so we can drop its biplist dependency.
The call to biplist.Data() in custom_dsstore doesn't seem to do anything,
and from what I can tell can just be removed.
fa4234d877ea3193bfd0e18ff68dcb8fb84b47b5 test: Mock IBD in net_processing fuzzers (MarcoFalke)
Pull request description:
Without this the fuzzers fail to detect trivial crasher bugs, such as https://github.com/bitcoin/bitcoin/pull/20317#issuecomment-723047111
ACKs for top commit:
practicalswift:
Tested ACK fa4234d877ea3193bfd0e18ff68dcb8fb84b47b5
Tree-SHA512: ce5da5c0a604b7559805a98ffdde882b44ca4f91b003b493d6e1be230714ce4cccb11dbfc1fc175f9d8fc779551c0a4103ceb4b473552928207d7d78ae329e10
79b8f8d5749fc676d9eeca0296172b7cdbbf2238 fuzz: Assert roundtrip equality for both addrv1 and addrv2 versions of CService (practicalswift)
0e3a78a8ab7ab5da71bd5e2f428ec3a2c9ad0901 fuzz: Check for addrv1 compatibility before using addrv1 serializer/deserializer on CSubNet (practicalswift)
Pull request description:
Check for `addrv1` compatibility before using `addrv1` serializer/deserializer on `CSubNet`. As requested by MarcoFalke in https://github.com/bitcoin/bitcoin/pull/20289#issuecomment-724012969.
Assert roundtrip equality for both `addrv1` and `addrv2` versions of `CService`.
ACKs for top commit:
MarcoFalke:
review ACK 79b8f8d5749fc676d9eeca0296172b7cdbbf2238
Tree-SHA512: 3f758aa89ab0c253b593fbe8fe9adc5c6db9afec8856facfe635053a32b4feb438c951323ae0c9e27f1d7e89d12a9b62d81f094dc96159233c12f64d4b95c290
Previously, when running ./configure:
1. With CONFIG_SITE pointed to our depends config.site.in, and
2. PYTHONPATH was not set either in the environment or by the user
The configure would output something like:
PYTHONPATH='depends/x86_64-pc-linux-gnu/share/../native/lib/python3/dist-packages:'
When we really mean:
PYTHONPATH='depends/x86_64-pc-linux-gnu/share/../native/lib/python3/dist-packages'
...without the colon
This change makes sure that:
1. There's no trailing colon, and
2. We use the $PATH_SEPARATOR variable instead of a colon
Files like config.site.in are not referenced by any other script in our
tree, so we need to mark it manually with a "shellcheck shell="
directive and make sure that shellcheck is run on them.
Previously, if ./configure was invoked with:
```
$ env CONFIG_SITE=depends/x86_64-pc-linux-gnu/share/config.site ./configure
```
Where $CONFIG_SITE was a relative path, ./configure would fail with the
following misleading output:
```
checking for boostlib >= 1.58.0 (105800)... yes
checking whether the Boost::System library is available... yes
configure: error: Could not find a version of the Boost::System library!
```
Fully resolving depends_prefix in config.site.in fixes this. To make
sure that there are no other side effects I ran a diff on the
config.status generated by:
1. The scripts prior to this change with CONFIG_SITE set to a full path:
env CONFIG_SITE=$PWD/depends/x86_64-pc-linux-gnu/share/config.site ./configure
2. The scripts after this change with CONFIG_SITE set to a relative path:
env CONFIG_SITE=depends/x86_64-pc-linux-gnu/share/config.site ./configure
And it looks good!
Diff: https://paste.sr.ht/~dongcarl/95b469fbc555c128046e85723d87a9082a754f6b
Check for HAVE_STRONG_GETAUXVAL or HAVE_WEAK_GETAUXVAL before using
getauxval to avoid a build failure on uclibc
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
538be4219ae7e65862e4aff540af88c9421e6061 wallet: fix importdescriptor silent fail (Ivan Metlushko)
Pull request description:
Currently `importdescriptor` command will successfully import a descriptor with hardened derivations into a watch-only wallet while silently failing to expand the descriptor to fill the cache. This leads to a broken wallet state and failure to load such wallet due to missing cache on subsequent restart.
ACKs for top commit:
laanwj:
Code review ACK 538be4219ae7e65862e4aff540af88c9421e6061
achow101:
ACK 538be4219ae7e65862e4aff540af88c9421e6061
meshcollider:
utACK 538be4219ae7e65862e4aff540af88c9421e6061
Tree-SHA512: 4bdd0ab4437d55b3f1a79c3a300a0b186089155c020fe220a73d0cce274de47d90371d88918d39fd795f9fccf8db328f1e322d29a6062f9ce94a1c254398f004
77777c8b5e3d0edbe1178d982a778b7cc9964b70 ci: Run windows ci config on cirrus (MarcoFalke)
3333d6942e8c5d823b97ea51ef88c8a0791742c6 ci: Run macos ci config on cirrus (MarcoFalke)
fa8b1114e60100f503673ccc7ef5618851c0c9e3 ci: Run arm ci config on cirrus (MarcoFalke)
fa0795f54dc5268bacc6acdfdbe23f9952a0e7d8 ci: Replace TRAVIS_OS_NAME with CI_OS_NAME (MarcoFalke)
fafce1a13a65cc3ae0cd5351648bb0290f9bfe8c ci: Move documentation to correct config file (MarcoFalke)
Pull request description:
ACKs for top commit:
hebasto:
ACK 77777c8b5e3d0edbe1178d982a778b7cc9964b70
Tree-SHA512: 581d3bdb2c3e3da20bd8492c6b23d90f3b4f7f9300ade384667e41de9fbbe42bbcef26dd99328839dc0b49be6e303c4c6d1b66ceb5d8009800c52c97685f9080
